Black Hat USA: Crowdstrike-Microsoft Outage Lessons Learned

BLACK HAT USA — Last month's global Microsoft outage spurred by a CrowdStrike update remained a hot topic during this week’s Black Hat USA 2024.

Channel Futures caught up with Danny Jenkins, CEO of ThreatLocker, at Black Hat in Las Vegas, to talk about it.

The global IT outage impacted 8.5 million Windows devices. U.S. Fortune 500 companies are facing a $5.4 billion direct financial loss from the outage, excluding Microsoft.

“Of course there are lessons to be learned, and as a CEO of a similar type company with similar risks, it's something that keeps me up at night all the time. It probably could have been avoided,” Jenkins said. “But hindsight is always 20/20, so it's easy to say it could have been avoided. And when you can see what happened, I think the big thing to be learned, and I think every vendor needs to make sure they're doing this, is complete isolation of roles with checks and balances that makes it impossible to put something live without it going through three roles.”

In ThreatLocker’s environment, whenever a software update gets pushed, it has to be validated by the development team, the quality assurance (QA) team — and then it gets deployed by infrastructure, he said.

ThreatLocker's Danny Jenkins

“And if something were to go wrong, if they copied the wrong file to the server or something went wrong with the file, it would literally just do nothing,” Jenkins said. “I think there are a lot of lessons to be learned around that in general, that companies need to be very, very careful when pushing updates to their clients.

I think there's also another problem, fundamentally," he added, "with the way that we're relying on security systems, on servers that need 12 updates a day. I think CrowdStrike updates files like somewhere between 10 and 12 times per day. If you're relying on security updates for a critical piece of infrastructure to detect threats − and that's why they're pushing them so they can detect the latest threats − better solutions would be to actually harden the server in the first place. So there are lessons to be learned there that maybe with servers we shouldn't be running so frequent updates, and we should just be focusing on hardening and locking that server to begin with.”

Jenkins Talks Big Growth at Black Hat

In the meantime, the Orlando Business Journal reported that ThreatLocker is planning to add more than 1,000 jobs and is looking for a larger facility.

“So at the moment, we're adding 40-50 employees per month,” Jenkins said. “And if we go back to the beginning of 2021, we had 25 employees. We're nearly at 500 now. It’s because of the need for our services and because we offer such good support, we have to make sure we stack that team and we stack the teams really well. It's really important. We've grown five times in the last two years, and we probably expect to see similar continued growth.”

In June, ThreatLocker announced the launch of its free cyber software report to provide visibility into IT environments to analyze and diagnose the risks occurring within their systems, including some foreign software. This coincided with the Biden administration’s ban on Russia-based Kaspersky’s software.

“It wasn't necessarily in response to that because it just happened to come at the same time, so we'd been working on it for a few years,” Jenkins said. “What we realized is we were giving customers lists of software that was running on their computers, and the customers would [say], 'OK, what am I supposed to do with this? What's Cooper Software?' They'd get this list of software, 500 apps on their machine, and they didn't know what risks there were. So probably about a year before we released that report, we started building out a research team. And the research team's job was to find every piece of software in the world and catalog it to find out everything we could about the company, where they were based, where the developers were based. Did they have influence from the Kremlin or from Beijing, or anything like that? Our job wasn't to tell them they should or shouldn't run it, but we wanted to arm them with information so they can make their own decisions. So now you can go into ThreatLocker and you can see every application, what country it was developed in, what data it has access to on your machines, and you can make good decisions. And those decisions could be to not use it, or they could be to limit what it can access without ring fencing.”

Palo Alto Networks

Also at Black Hat, we spoke with Yotam Ben-Ezra, Palo Alto Networks’ director of product management for Prisma Cloud. He said his company is engaged with the partner ecosystem around the Prisma Cloud code-to-cloud platform, and now data security.

“We've just announced both data security posture management and AI security posture management for general availability with the Prisma code-to-cloud platform,” he said. “That doesn't only help partners essentially keep up with how AI is changing the environment, and how the variety and the explosion of data is changing the environment, but also the context that data security brings into the overall cloud-to-cloud application security today that is relevant in all of the other models.”

Palo Alto Networks' Yotam Ben-Ezra_2024.jpg

One of the challenges in security is prioritization, Ben-Ezra said.

“We have loads of information that our tools are generating. Where do we start and how do we invest our time given the shortage in personnel and qualified personnel? How do we invest our time in the best places and the most effective places?” he said. “So if you think about a vulnerability that you have in a virtual machine, for example, which is related to workload security or to cloud security posture management, now you have the context of whether that machine impacts sensitive data or not. That considerably helps with the prioritization. And that is relevant for all of the different types of models we've also integrated into the attack path analysis within Prisma Cloud. So the ability to understand what the most effective attack paths in the environment are and how they are impacting my sensitive data is a key part of the Prisma Cloud platform.”

Partners need to get up to speed with how AI and data are impacting cybersecurity, and Palo Alto Networks provides training around helping partners with AI adoption, Ben-Ezra said.

“Gartner published a few months back that by next year, 70% of organizations will have operationalized an AI architecture in their environment,” he said. “Such a short time and such a high percentage is super aggressive to me. And it's in line with what we see in the marketplace. So organizations will be at different stages of adoption, but everyone is just racing toward adopting AI into their environment. This is partly why we selected to address AI security and data security posture.

"The problem is that AI essentially is a data security problem," he continued. "The concern we hear from customers is how [their] data is impacted, [are they] exposing data as a result of adopting AI in \[their] environment and [are they] inadvertently exposing data? So the ability to keep track of that first and understand all of the different business units of your organization, and how they are adopting different types of AI technologies, and then how are those impacting [their] data, [their] security posture, [their] compliance adherence is key to essentially being able to adopt AI security.”

HackerOne

Also at Black Hat, we caught up with John Addeo, HackerOne’s new vice president of global channels. In June, HackerOne launched its PartnerOne program to build out a partner network around its cybersecurity products and services. PartnerOne enables partners to sell HackerOne through their channels.

“This is very exciting because this is very new for us as an organization, so this is a big deal, launching the program,” he said. “It's a shift in relationship to understanding the maturity of the markets, caught up to a point where we believe that the customer and the partner community can come together and actually create value with each other. What we do is really important to that journey, but we don't do everything. So when we look at the partner community and understand that they bring things like application security skills, they bring things like vulnerability management programs, they bring things like remediation services and other things that wrap around what we do and create value, that, to me, is really where the opportunity lies.”

HackerOne has moved to a channel-first approach with its partner program, Addeo said.

“So the big piece right now is we just launched a program. We're just bringing in partners, formally inviting them into the program as we start to really expand outward from there," he said. "There's a combination of being selective and intentional about the types of relationships we want in the markets. When we think about the different markets in Europe and Asia, etc., we're looking for a particular type of skill mainly around ... partners who can create that value around the platform, not just be a reseller and move forward. I'm not saying there's no value in that, but the ultimate value for the customer is when a partner can really help them through that whole journey and really bring all the services to the market.”

HackerOne has 2 million researchers in its bug bounty program community that help and support bug bounty and vulnerability disclosure programs, Addeo said.

“That's a unique approach to solving this one problem, and it's almost impossible to go build that yourself and or as a customer, so really bringing that to their customer community," he said. "I think the other thing when we look at the program is what we're bringing in is the ability to engage with us early, monetize that from an early standpoint, but be part of that journey throughout the customer life cycle. It's not just one and done, see you, bye. We want our partners to be part of that journey constantly through that life cycle, helping their customers solve those problems.”

Through the remainder of 2024, HackerOne partners will see the company continue to evolve its partner program with enablement programs to help partners “lean into it,” Addeo said.

“Also, opening up the research community where partners can take advantage of that as well so they're a part of that community and journey,” he said. “I think you're going to see us more visible in the world of … complementing technologies in application security, complementing technologies in the vulnerability management space, like how we integrate and bring those things into a much more cohesive manner. And then as we look to the future, how do we enable partners with a service provider model where they can be the owner and service their customers, and work with us in that kind of manner as well.”

Keeper Security

Also at Black Hat, Keeper Security displayed its new Zero-Trust KeeperPAM, its privileged access management (PAM) platform that unifies what would otherwise be four disparate, isolated solutions in the identity and access management (IAM) industry into one platform.

It unifies password management, secrets management for protecting infrastructure, zero trust network access and remote browser isolation.

Darren Guccione, Keeper Security’s CEO and co-founder, tells us Zero-Trust KeeperPAM is providing Keeper Security’s partners with new opportunities.

Keeper Security's Darren Guccione

“The thing that makes us quite unique just from a product perspective is that platforms drive larger annual contract values (ACVs) than conventional isolated solutions," he said. "It also enhances partners' business, whether it be on implementation and adding, or layering in services on top of that, and then also bolstering the technology stack that they would typically sell right to their end customers. So it's a huge value add.”

Keeper Security recently released a report that found people are overwhelmed by too many passwords. Guccione said every company of every size across every major industry suffers from this issue.

“So if you look at data breaches, roughly 70% of all breaches are a result of either weak or stolen login credentials, passwords or secrets,” he said. “In addition to that, the question then becomes, how do you mitigate the risk of a data breach by adding what we call pervasive visibility, security and control across every user, on every device from every location? We back that up with additional reporting around compliance reports, advanced reporting analytics and security scores. We have a risk score and then AI-based threat analytics. So we pull all of that together in a single control plane, which greatly benefits not just the reseller, but their end customer.”