Phishing simulation click rates are a useful metric for assessing employee risk and measuring the effectiveness of one’s security awareness training. But they are hardly the complete picture.
Too many companies are using phishing test metrics as a crutch, said Drew Rose, founder and chief security officer of Living Security, in a presentation at InfoSec World this week. If they truly want to give executive leadership a more complete picture of employee security posture, they should be taking a more mature, comprehensive look at internal behaviors and trends, he added.
“Don't get me wrong. Phishing is an insanely important part of helping to ensure that end users know how to react to real life threats,” said Rose. However, “it's not the only threat to report on. Phishing is always going to be part of a bigger picture.”
“It's just one metric… It's not the entire formula.”
By themselves, phishing simulation statistics certainly offer at least some perspective on how workers are faring with their security awareness. But there are flaws, said Rose. For starters, phishing tests can be designed to be anywhere from very easy to very hard – and the lures used in simulated emails might appeal to certain employees more than others, depending on their personal circumstances. However, the results of standard phishing tests generally don’t factor in such nuances or context.
Phishing tests alone also doesn’t account for all the other kinds of human error-based risks and threats facing organizations, including bad password management, unsafe web browsing and data loss. “We all recognize that between 83 and 90-something percent of breaches are caused by human behavior – and only 25 percent of those are caused by phishing. That means there are 75 percent more threats out there that we should be worried about and reporting on whether our end users are ready to respond to them,” said Rose.
The good news, Rose continued, is that companies can gain a window into these other threats through “some simple data correlation” that allows them to “find patterns.”
“There's a lot of behavioral information in the devices in your technology that we can leverage to be more predictive around human risk management,” said Rose. “Are the threats increasing or decreasing? Are the responses good or bad? Are they risky or vigilant?”
For instance, companies can introduce more context into their anti-phishing training metrics by correlating fake phishing click rates with the click rates of actual phishing emails. (“There are… platforms out there that… [can] report whether a user clicked on an actual phishing email that platform accidentally let through,” Rose noted.)
Beyond phishing, companies can also assess employees’ endpoint security hygiene by tracking whether workers are regularly administering software, browser and anti-virus updates, or if they are introducing risk by procrastinating on their patching.
“Whether they have good update hygiene, whether they have malware on their computer, whether they're downloading software from unknown sources, or their antivirus is out-of-date, or they're plugging in things they shouldn't: these are all data points that you're receiving in your point solutions or your SIEMs that you can leverage to start to create a human risk management dashboard – something that you could track over time to see if your intervention efforts are successful,” Rose said.
Businesses can also measure web security hygiene, by looking if employees are going on blocked, unauthorized or uncategorized websites, or downloading anomalous plug-ins. “These are all human risky human decisions where the metrics are within your platform. You just got to do a little bit of digging to get them, and presenting them back to yourself in a way that's understandable,” said Rose.
Companies can analyze data management metrics as well. “A lot of compliant and heavily regulated industries have data loss prevention add-ons” that track data leaks via email, the web, endpoints and mobile devices, said Rose. “And those violations… are events that are associated with a user that we can track over time.”
Once you begin collecting these various event-based metrics, it’s then important to track any notable deviations from established baselines of normal activity. “We want to see when the behavior is trending upwards or downwards,” said Rose. That will allow you to then take the proper intervention or action to inspire more vigilant behaviors.
“You can… do this on your own with any training or assessment platform with just a bit of data manipulation,” said Rose. “The output is worth the investment of time.”
Moreover, multiple metrics and data points can be jointly analyzed to create even deeper insights. For instance, Rose said companies can compare behavioral data against qualitative data to “look for insights to figure out which groups of users are most at risk in one area,” and how best to approach these various groups.
For instance, companies can administer assessment tests asking employees to rate their confidence their ability spot a phish, and then compare that against how they actually scored on phishing simulation tests. Users who scored poorly and also had low confidence would likely welcome assistance.
“These are the people that raised their hand and said, ‘I don't know how to assess whether an email is malicious or not. I need help’… So as a human risk manager, or security manager or CISO, I can go to this group of people and say, ‘I'm gonna give you training. We're gonna work through this together,’” said Rose.
On the other hand, users who score poorly on the tests, but are overly confident that they can spot might bristle at the idea of needing help, so they may need more hand-holding and coaxing before they’ll accept assistance, Rose explained.
After the training or retraining takes place, you can measure again to see if there’s improvement.
And that’s just one example. Rose said he has a list of “another 100 types of these metrics where we're correlating data across multiple platforms plus using qualitative information.” Bottom line:
“Metrics around human behavior exist,” said Rose. You just need to think outside the box. ‘Creativity goes a long way.”