How a software update from cyber firm CrowdStrike caused one of the world’s biggest IT blackouts

Technology

In this article

George Kurtz, co-founder and CEO of CrowdStrike Inc., speaks during the Montgomery Summit in Santa Monica, California.
Patrick T. Fallon | Bloomberg | Getty Images

A fault with an update issued by cybersecurity company CrowdStrike led to a cascade effect among global IT systems Friday, with industries ranging from banking to airlines facing outages.

Banks and health care providers saw their services disrupted and TV broadcasters went offline as businesses worldwide grappled with the ongoing outage. Air travel has been hit hard, too, with planes grounded and services delayed.

At the heart of the issue is Texas-based cybersecurity vendor CrowdStrike. On Friday, the cybersecurity firm experienced a major disruption following an issue with a software update.

So what happened, exactly? CNBC takes a look.

What is CrowdStrike and what does it do?

CrowdStrike is a cybersecurity vendor that develops software to help companies detect and block hacks. It is used by many of the world’s Fortune 500 companies, including major global banks, healthcare and energy companies.

CrowdStrike is what’s known as an “endpoint security” firm as it uses cloud technology to apply cyber protections to devices that are connected to the internet.

This differs from alternative approaches used by other cyber firms, which involve applying protection directly to backend server systems.

What happened on Friday?

On Friday, people around the world began encountering an error screen known as the “blue screen of death.”

This issue — a common problem among PCs, for example if a machine overheats — was the result of an update from cybersecurity firm CrowdStrike concerning its Falcon product. 

Falcon is a platform developed by the company that’s designed to stop cyber breaches using cloud technology — it is at the heart of the firm’s focus on endpoints. CrowdStrike said Friday it is in the process of rolling back the update globally.

CrowdStrike’s software requires deep access to a computer’s operating system to scan for threats. In the case of Friday’s outage, machines running Microsoft’s Windows operating system crashed due to a fault in the way a software update issued by CrowdStrike interacted with Windows.

“We have been made aware of an issue impacting Virtual Machines running Windows Client and Windows Server, running the CrowdStrike Falcon agent, which may encounter a bug check (BSOD [blue screen of death]) and get stuck in a restarting state. We approximate impact started around 19:00 UTC on the 18th of July,” Microsoft said in an update at 5:40 a.m. ET.

“We can confirm the affected update has been pulled by CrowdStrike. Customers that are continuing to experience issues should reach out to CrowdStrike for additional assistance,” the company added.

A fix has been issued

Earlier, Microsoft said its cloud services had been restored after an outage that affected its Azure services and Microsoft 365 suite of apps in the central U.S. region. A company spokesperson said these are two different and non-related issues — one issue relates to Azure, the other is linked to CrowdStrike.

They added that they “anticipate a resolution is forthcoming,” in respect to the CrowdStrike problem.

CrowdStrike is “actively working with customers impacted by a defect found in a single content update for Windows hosts,” CEO George Kurtz said Friday in a update on social media platform X. He added that Mac and Linux hosts are not affected.

“This is not a security incident or cyberattack. The issue has been identified, isolated and a fix has been deployed,” Kurtz said.

That fix could be hard to implement, though. Andy Grayland, chief information and security officer at threat intelligence firm Silobreaker, said that in order to implement a fix, engineers would have to go into each individual data center running windows.

They’d then have to log in, navigate to a certain CrowdStrike file, delete it, and then reboot the entire system, he said.

“Where machines are encrypted, complex encryption keys also need to be entered manually. Unless Microsoft and CrowdStrike (if they are involved) pull something miraculous out of the bag, this could be painful to recover from.”

Beyond the Valley
Get a weekly round up of the top tech stories from around the world in your inbox every Friday.

Subscribe

Articles You May Like

SpaceX Successfully Launches Rapid Response Trailblazer-1 Mission from Florida
Sources: Georgia preparing for CFP without Beck
Russia should have invaded Ukraine ‘earlier’ says Putin – as he addresses possible peace talks
Tesla reverses losses to turn higher in a volatile week for the EV stock
Video shows arrest of suspect after attack at German Christmas market