Product Group Tests

Risk management

Group Summary

For this month’s review we looked at products that could measure, analyze and report risk within an enterprise.

Scroll To Full Group Summary Below

Click for a side by side comparison of products
Click for a side by side comparison of products

Full Group Summary

Risk management tools need to take massive amounts of information, correlate it to industry regulatory requirements, and then assess the impact on business, says Michael Lipinski.


Risk is defined by the International Organization for Standardization in ISO 31000 as the effect of uncertainty on objectives. The effects of uncertainty can be positive or negative and we'll call these effects "impact." It follows then that risk management is the identification and assessment of the elements of risk, those being threats and vulnerabilities and the prioritization and remediation of these elements to prevent or mitigate impact. Risk management needs to take massive amounts of information from the entire organization, correlate it to industry regulatory requirements, identify areas where threats and vulnerabilities have the possibility of coming together to provide and impact, and then provide a usable measure of the value of that risk to the business.

Risk management is a challenge for most organizations. Periodic review of configurations, vulnerabilities, patches, server, user, network and security rules is also a challenge. As enterprises operate in a constant state of change, even a diligent IT staff will be challenged to validate every configuration to corporate policy, test and deploy all necessary patches in a timely fashion and validate end-to-end accuracy of all the security controls deployed.

Risk management is an enterprise initiative and is not limited to information technology. Business risk exists where adherence to corporate policies and regulatory requirements is not maintained. Operational risk exists when controls are not deployed to support business policies or regulatory requirements or when those deployed controls are either not effective or can be circumvented. 

For this month's review we looked at products that could measure, analyze and report risk within an enterprise. We looked for these products to report within the formats and frameworks of multiple regulatory requirements (e.g., SOX, GLBA, PCI, ISO, and more.). We were looking for products that were network-centric and centrally managed, that centrally collected and stored data, that centralized analysis and reporting, and that focused exclusively on risk management. Additional functionality we were looking for included the ability to collect data across the network, including threats and vulnerabilities, and the option to report associated risk, provide remediation options (beyond what traditional patch management systems deliver), and report based on regulatory requirements and local policies.

Our testing methodology for this month's review used vendor-provided web reviews. Vendors were allowed to run through a short presentation highlighting the value proposition of their company and product offerings and describe the implementation process that a typical end-user would experience.

We then ran through a full demonstration of all the products using our usual evaluation criteria: ease of use, features and functionality, reporting and alerting, documentation and support. The solutions reviewed consisted of client-side software deployments (usually server software and agents), appliance-based solutions and hosted SaaS offerings. We reviewed products that were focused on the business side of risk, the creation of policy and adherence to those policies and to multiple regulatory policies. We reviewed products that were operationally focused and collected information from the various deployed security controls and network systems to validate those policies against corporate policy and industry regulatory compliance. A number of the solutions collected vulnerability information from industry standard scanning tools and correlated that vulnerability data to the operational threat data it analyzed.

Just as "risk" has become an overused term today and applied to many security, policy management and vulnerability analysis tools, these solutions looked to address risk in very different ways. Regardless if the solutions focused on the business policy and adherence side or the operational controls side, we focused our review on the product's ability to identify and analyze risk, measure and report on the risk, easily compare to industry compliance regulations and provide remediation options in an easy-to-use interface.

We were very impressed with all the products tested for this review. The maturity level of these products really showed. The ease of use and the user interfaces have really been designed to present very large amounts of data in very neat and easy-to-follow formats. Reporting and alerting were very strong. All the solutions came with a substantial amount of content out of the gate covering all the standard regulatory bodies, and offered extensive reporting templates and numerous sets of questions for developing assessments.

When making your purchasing decision, as always, choose carefully as all of these solutions use content for various decision-making and analysis. The ability of the vendors to keep that content updated and fresh is very important. Some of the operational-focused solutions deployed rapidly. Other solutions required several months to fully deploy and implement within the organization.

All Products In This Group Test