Sophos Survey: IT Leaders Worry About Generative AI RisksSophos Survey: IT Leaders Worry About Generative AI Risks

A new Sophos survey shows 89% of IT leaders worry that flaws in generative AI cybersecurity tools could put their organization at risk.

Sophos surveyed 400 IT leaders on their use of AI in security. Sixty-five percent have adopted generative AI.

And according to new Sophos X-Ops research, also released Tuesday, there has been a slight, but noteworthy shift in the way cybercriminals use AI. Some bad actors are using it to automate mundane tasks, such as crafting bulk emails and analyzing data. Others are incorporating it into spam and social engineering toolkits.

What’s Surprising About Latest Sophos Survey

Chester Wisniewski, director and global field CTO at Sophos, said he’s a bit surprised that 99% of organizations surveyed said they assess the caliber of the cybersecurity processes and controls used in the development of generative AI.

Sophos' Chester Wisniewski

“This is an incredibly complex field of study requiring hard-to-find data scientists, so I am a bit concerned they may be overconfident,” he said. “We have been effectively using deep learning AI for malware detection, URL categorization and email filtering for nearly a decade, and have dramatically improved our ability to proactively detect new malicious content. When it comes to generative AI, there is room for concern as we aren’t entirely sure how these models draw their conclusions and have demonstrated what others have deemed ‘hallucinations,’ but there are no two ways about it; sometimes they just make things up. This is why it is so important for a human in the loop when using generative technologies.”

Related:Quorum Cyber Unveils New Threat Business Unit

With some form of AI embedded in the cybersecurity infrastructure of 98% of organizations surveyed, IT leaders expressed concern about potential over-reliance on AI, according to Sophos. Eighty-seven percent said they were concerned about a resulting lack of cybersecurity accountability.

Differing Priorities for Generative AI

In terms of priorities for utilizing generative AI, larger organizations prioritized improved protection, while smaller ones prioritized reducing burnout. However, organizations of all sizes worry about pressure to reduce cybersecurity professional headcount due to unrealistic expectations about AI’s abilities to replace human operators.

“It comes down to well-thought-out policies on where and when it is appropriate to use them, and when trying to benefit from automation, ensuring there is a human overseer,” Wisniewski said. “We need to remember these tools don’t think, they just help us to think faster. Security operations center (SOC) alert triage is a great application with low risks. AI can digest a set of alerts as a group and provide a quick opinion on the severity. The human can read the digest version and choose to override or not based on experience and context more quickly than gathering all the details on their own, and quickly agree or disagree with the tool. Over time, the humans get pretty good at knowing when the machine is right and wrong, but together they can process twice the alerts a single human can.”

Related:Commvault Launches New Integration with CrowdStrike

Other key findings in the Sophos survey include:

In terms of encouraging signs in the survey, the awareness of the risks is a “fantastic” start, Wisniewski said.

“Too often we read about AI as some God-like intelligence that we ought to bow down before, rather than soberly assessing its capabilities and how they might fit into our processes,” he said. “The potential of these technologies, well-applied, is enormous, but we should not rush to jump on the bandwagon before there is clarity on both the risks and benefits. This being top of mind by leaders is a healthy indicator.”  

Related:The Gately Report: Mounting Ransomware Attacks Fuel Halcyon's Expansion