Measuring success: Performance metrics
Measuring success: Performance metrics

Gauging performance does not necessarily result in enhanced security, but quantifying risk is still vital, reports Stephen Lawton.

These days, it seems everything has a metric. Some questions with quantitative results can be answered easily – such as how a network is performing – and key performance indicators (KPI) can be useful in that environment, says Gartner research director Anton Chuvakin. 

KPIs are a strategy to measure the effectiveness of an enterprise, a division or individual employees. However, there are other questions around basic data security that cannot be so easily assuaged. For instance, a company that spends a lot of money on security might identify attacks, but that does not mean it is well protected, Chuvakin says. 

It could mean that even with a substantial security budget, the organization's security team is not identifying and defending against the attacks. Likewise, a company with a small security budget might have an effective security plan in place that protects its corporate assets. 

Clearly, Chuvakin says, a security budget alone is no real indicator of how well an entity can protect its intellectual property. He maintains that because each company has its own security profile, there can be no standard set of KPIs to determine security.

A whitepaper produced by U.K.-based Iris Accountancy Solutions, a software and services company, says KPIs vary by department, so it is important for each to outline and work to its own set of measurements. The paper describes them as objectives or goals that can be used to measure the performance of each department. 

KPIs generally have five qualities: they are specific, measureable, achievable, realistic and timely, according to Iris. They need to quantify issues that will make a difference, and are thus focused on areas for which the department or employee can have an impact. Creating a goal that is outside the scope of an individual or department is unachievable, and should not be considered a KPI.

As with so many aspects of managing information security, administrating data overload can be an issue, the whitepaper explains. Understanding what needs to be analyzed and putting that data into a comprehensible format is critical. Because Iris has a large number of financial services clients, a popular format is the spreadsheet, which can incorporate both current and historical data.

This leads to the inevitable question: Can one create KPIs for security? The short answer is: yes and no, Chuvakin says. If one sets their standards too high, they will never reach them. Too many variables and an ever-changing security landscape mean that KPIs that are too general cannot be met.

On the other hand, if the KPIs are set at the tactical, instead of a more strategic, level, it is possible to meet those levels. The result, however, might not meet the real security needs for the network, he says. Today, setting network security KPIs is an inexact science with too many variables, Chuvakin says. “There is no silver bullet.”

With a given amount of money allotted for security, a company may spend that on specialty products based on price, he says. However, if a company takes a more holistic view of its security profile, they might find more efficient ways to protect their network – with the budget being built to meet the network's needs rather than those of a preset spending plan.