Humans More Effective than ChatGPT at Phishing

New Hoxhunt research shows humans still outperform ChatGPT in perpetrating successful phishing attacks.

The study analyzed more than 53,000 email users in over 100 countries. It compares the win rate on simulated phishing attacks created by human social engineers and artificial intelligence (AI) tools.

While ChatGPTs potential for malicious phishing activity continues to capture everyone’s imagination, Hoxhunt’s research highlights that human social engineers still outdo AI in terms of inducing clicks on malicious links.

Mika Aalto is Hoxhunt’s co-founder and CEO. Hoxhunt provides enterprise security awareness solutions.

Hoxhunt’s Mika Aalto

“ChatGPT allows criminals to launch perfectly worded phishing campaigns at scale,” he said. “And while that removes a key indicator of a phishing attack, bad grammar, other indicators are readily observable to the trained eye. We now know from the results of our study that effective, existing security awareness and behavior change programs protect against AI-augmented phishing attacks. Within your holistic cybersecurity strategy, be sure to focus on your people and their email behavior, because that is what our adversaries are doing with their new AI tools. Embed security as a shared responsibility throughout the organization with ongoing training that enables users to spot suspicious messages and rewards them for reporting threats until human threat detection becomes a habit.”

Higher Phishing Click Rates with Humans Vs. ChatGPT

The study revealed that professional red teamers – security professionals who test systems by breaking down their defenses and evaluating their vulnerabilities – induced a 4.2% click rate. That compared to a 2.9% click rate by ChatGPT. Humans remained clearly better at hoodwinking other humans, outperforming AI by 69%.

The study also revealed that users with more experience in a security awareness and behavior change program displayed significant protection against phishing attacks by both human and AI-generated emails. Failure rates dropped from over 14% with less trained users to between 2% and 4% with experienced users.

The human layer is by far the highest attack surface and the greatest source of data breaches, according to Hoxhunt. At least 82% of beaches involve humans. Large language model-augmented phishing attacks do not yet perform as well as human social engineering. However, AI will likely close that gap. And attackers are already using AI.

Fighting AI Cyber Threats with AI Cyber Technology

Patrick Harr is CEO of SlashNext. He said it’s important to fight AI cyber threats with AI cybersecurity technology.

SlashNext’s Patrick Harr

“When cybercriminals launch successful attacks, the results are massively disruptive to people, organizations and the economy,” he said. “The No. 1 cyber challenge organizations face globally is human-focused attacks. Generative AI technology, which makes ChatGPT possible, will be used to develop cyber defenses capable of stopping malware and business email compromise (BEC) threats developed with ChatGPT.”

While many organizations already use AI-based cybersecurity products to manage detection and response, AI technologies using advanced AI, like generative AI, will become essential technology to stop hackers and breaches, Harr said.

“When new technologies become available, hackers and cybersecurity vendors will use it to perpetrate and stop cybercrime,” he said.