Beyond 'Pointing Fingers:' Partners React to CrowdStrike Outage

In the wake of the world's largest IT outage, channel partners say resilience and disaster-readiness should play a bigger role for business customers going forward.

The recent CrowdStrike outage captured the attention of the world over the weekend. Windows machines across the world put up the blue screen of death on Friday following a content update on the endpoint detection and response (EDR) provider's Falcon platform. The faulty channel file 291 – now reverted – manages "named pipes" in Windows systems.

The incident, CrowdStrike stressed, did not come at the hands of cyber attackers. But it proved costly nonetheless. The outage impacted 8.5 million devices, grounded flights across multiple airlines and airports and could generate an estimated $1 billion in losses, according to CNN. It is widely described as the largest IT outage in history.

And many channel partners worked overtime alongside IT teams over the weekend.

White Oak's Daniel Beckworth

"Friday was a hectic day. I received my first text from an end user at 2:30 a.m. CST. Over 800 virtual machines were impacted in their environment. Approximately 40% of my clients consume CrowdStrike and felt some level of pain," said Daniel Beckworth, founder of the MSP and tech advisory company White Oak Solutions. "I think it’s safe to say this is the world’s largest outage to date. At the end of the day, this required a relatively simple fix, but it was quite manual. Huge miss on CrowdStrike’s part."

Related:Global IT Outage Impacts 8.5 Million Devices

Jeff Hathcote, security solution architect at tech services distributor Telarus, said the outage caused a massive trickle-down effect. Even for technology advisors and resellers who don't dabble in cybersecurity or even disaster recovery are finding themselves involved.

“While this wasn’t a cyberattack, it was a highly disruptive event that negatively impacted the market, resulting in massive business productivity and financial losses," Hathcote told Channel Futures. "This event shows just how connected technology products are within our ecosystem, and one failure can cause a devastating impact on an entire infrastructure."

While the outage did not come from cybercriminals, threat actors are chomping at the bit. Different threat intelligence providers have highlighted phishing and malware initiatives aimed at exploiting distressed end users.

"While both CrowdStrike and Microsoft are working diligently to assist in recovery efforts, we all need to remain on high alert to the cybercriminals that are taking advantage of the current situation by offering 'assistance,' which results in further harm to businesses," Hathcote said. "We all need to be aware of the potential for secondary attacks based on information collected via a phishing attack."

Related:CrowdStrike Update Disables Windows PCs Around the World

CrowdStrike Outage: The Importance of Backup

Many members of the technology advisor (agent) community have raised the topic of disaster recovery and backup services in the wake of the outage. About 75% of technology advisors have sold disaster recovery or backup as a service, according to Channel Futures' Q1 partner survey. And 10% saw DRaaS/backup deals increase in Q1 over the year prior.

The CrowdStrike outage has put such discussions back into the forefront.

Sandler Partners' Eric Beller

"This has opened the door to having disaster recovery conversations with our partners," said Eric Beller, senior vice president of sales and complex solutions for the technology solutions distributor Sandler Partners. "Between this and the CDK instance, we see all the IT dependencies that underpin our operations. IT systems are highly interdependent, and we can't always control how these systems and dependencies function."

But for Beller, disaster recovery is not just a technology platform but a "well-thought-out, tested plan."

"While Crowdstrike and Microsoft have enhanced their processes and collaboration to prevent future incidents, the key takeaway is ensuring your IT team can respond effectively with minimal impact," Beller told Channel Futures. "Being prepared and tested is essential. If you haven't already, it's vital to plan, test, and be ready, so when the time comes – not if, but when – your team can respond efficiently or have the necessary resources to do so."

Related:CF20: 2024's Top Backup/Disaster Recovery Providers

Koby Phillips, vice president of advanced solutions, cloud, for Telarus, agreed.

“This high-impact event emphasizes the urgency to keep resiliency plans current, communicated, and understood within the organization to avoid the types of customer disruptions experienced since July 19," Phillips said. "Like other industry disruptions this year, these events create opportunities for conversations with technology advisors about how to best prepare their customers for these inevitabilities. As more details on this specific event emerge, we will continue to engage with our advisors and their clients on the best next steps for growth and safety.”

Time to Fire CrowdStrike?

Beckworth said he doubts that customers will "shed Crowdstrike en masse," regardless of the size of the outage.

"Cybersecurity experts and technical resources tend to appreciate the product and Crowdstrike has historically executed on their value proposition. This will surely impact their stock and near-term pipeline, but I believe they’ll overcome this hurdle and continue to be a dominant force in the cybersecurity ecosystem," he said.

Chris Ichelson, who runs the the Arizona-based MSSP 360 SOC warned users about rushing to a new EDR vendor.

"Everyone, this is not a time to go buy SentinelOne (or another vendor) because of the CrowdStrike issue. SentinelOne will most likely get this issue as well," Ichelson wrote on LinkedIn. "There is a kernel issue with the endpoint vendors that we have observed. This is the third vendor in four months to have the blue screen of death issue. Seems to all be related to touching the kernel."

However, Beckworth said he hopes the incident will raise the standards for software vendors.

"This should cause the community to place greater scrutiny on vendors/manufacturers' quality assurance processes," he told Channel Futures. "It’s all too common for security vendors to push bad code that leads to compromised systems and mass outages."

For Stephen Semmelroth, senior director of security at Avant, the CrowdStrike outage showed just how dependent the tech industry is on "a small set of software providers.

"While CrowdStrike has had a fairly clean deployment history, trusting one vendor across an entire enterprise ecosystem is a risk. Many companies put all their eggs in one basket with Microsoft and CrowdStrike, which exposed them to a perfect storm," Semmelroth said.

Semmelroth encouraged partners to not use the blame game as a sales tactic.

"I would caution trusted advisors against pointing fingers at any of the security software providers," he said.

The alternative is pursuing "resilience" with customers, he said.

"Trusted advisors who adopt a resilient mindset can help their clients recover from a range of disasters like data center outages, cyber criminals, malicious employees, and faulty software updates," he said. "Having a resilient mindset brings more stakeholders to the table and elevates the discussion to drive very real, company-wide outcomes. Partnering with service providers that allow clients to identify, protect, detect, respond, and recover from disasters is why the resilience conversation should include business leaders and not be limited to the security organization.”