Finding Security Awareness Training Balance

Earlier this year, the National Institute for Standards and Technology (NIST) published updated recommendations for phishing simulations in security awareness training programs. We discussed it on our Community page soon after the updated standards were released, but the substance of the change bears repeating.

“Practical exercises include no-notice social engineering attempts to collect information, gain unauthorized access, or simulate the adverse impact of opening malicious email attachments or invoking, via spear-phishing attacks, malicious web links.” – NIST SP 800-53, Rev. 5, Section 5.3 (pg. 60)

This update includes a recommendation for “no-notice” phishing simulations to be delivered at the beginning of security awareness training programs to more accurately gauge the readiness of a set of users to recognize a phishing attempt. The thinking is obviously that letting users in on the phishing simulation game will heighten suspicion of their inbox and skew baseline results. This concern can be thought as a spin-off of the well-studied “observer effect” known in many scientific fields: Observing the behavior of something necessarily changes that behavior.

While it might be tempting for a chief information security officer (CISO) or other IT professional to take high grades on a phishing simulation as a sign of a job well done, that can be a dangerous conclusion to draw. Phishing tests that are too easy do little to address a problem that’s become one of the most common methods of entry for ransomware attacks. If IT professionals grade on a curve here, they’re doing very little to improve their organization’s overall cyber resilience.

Combatting this false sense of confidence about users’ ability to spot phishing attacks requires making sure simulations aren’t too easy to spot.

What Makes a Phishing Simulation Too Easy?

After putting some thought into that question, NIST researchers published a paper last year in the Journal of Cybersecurity citing three key criteria for determining if a phishing simulation makes for good training.

According to the authors, “low click rates do not necessarily indicate training effectiveness and may instead mean the phishing emails” were:

On the other hand, a phishing simulation is convincing if it does the following to some degree:

Tip: NIST has devised a weighted version of this scale, “the phish scale,” that you can use to determine the difficulty of your simulations. A phishing simulation that has all of the above characteristics would be considered extremely difficult. That’s good, right?  Click on Page 2 to continue reading…

Too Much Difficulty Can Be Dangerous, Too

Any security awareness training program that’s too difficult is liable to leave learners feeling put off, resigned to failure or, worse, coming away without any practical security learnings. This is especially true if users are punished too harshly for failing to spot a difficult phishing simulation.

Any program that’s both difficult and relying on a stick rather than a carrot for motivation runs the risk of:

For security awareness training to be successful, it must be collaborative. Learners should feel like they’re part of something constructive, rather than just subjected to another type of performance review.

Hitting the Sweet Spot

Finding the appropriate difficulty level for phishing simulations is one of the reasons the initial, no-notice NIST recommendation is so important. It helps administrators establish baseline results that most accurately reflect users’ real understanding of phishing attacks. But we don’t recommend a training program be hidden from employees forever.

Instead, after initial results have been established, it’s better to announce the program publicly along with its goals, evaluation criteria and a point of contact for those interested in learning more. Once users are in the know, subsequent phishing simulations can focus on incremental improvements over the baseline results. As scores rise across the board, the difficulty can be gradually increased over time.

One essential recommendation: Always report publicly on positive results. Let users know they’re managing to catch more and more difficult simulations. Be as specific as possible, as in, “Click-through rates dropped from A to B in this exercise.” This will help establish a sense of shared responsibility for organizational security and “gamify” the experience.

Calibrating your security awareness training is an ongoing experience. Don’t be afraid to adjust your simulations based on results. Happy learning.

Ready to establish your own successful security awareness training? Try us out free for 30 days.

Kyle Fiehler is a writer and brand journalist for Webroot. For over five years he’s written and published custom content for the tech, industrial and service sectors. He now focuses on articulating the Webroot brand story through collaboration with customers, partners and internal subject matter experts.

 This guest blog is part of a Channel Futures sponsorship.