Product Group Tests

Vulnerability assessment

Group Summary

The VA market is evolving into three segments: pure-play vulnerability assessment, combined VA and pen testing and vulnerability management.

Scroll To Full Group Summary Below

Click for a side by side comparison of products
Click for a side by side comparison of products

Full Group Summary

Vulnerability assessment has been one of my favorite groups for as long as I've been writing this section. I have enjoyed watching the market grow and change for several years. This year we are seeing a more mature market than in previous years. The products that we saw this year are the most capable ever. That's the good news. The bad news is that it was extremely difficult to select a Best Buy and a Recommended product.

All of the products we ran through the lab performed beautifully, did exactly what they claimed to do, and were extraordinarily easy to use. With that in mind, we selected one Best Buy and three Recommended products, along with an SC Lab Approved product. The Recommended products are in pure-play vulnerability assessment (VA), hybrid VA and penetration testing and vulnerability management.

The VA market is evolving into three segments: pure-play vulnerability assessment, combined VA and pen testing and vulnerability management. I have taken the position in the past that most VA products would, eventually, evolve into vulnerability management tools. I missed that one, but only by a little.

There are some vendors that have focused on making their products the best they can be within their domain. These vendors have no interest in moving their pure-play products into the vulnerability management domain. Looking at these tools, we find that they are beginning to improve beyond simply adding more vulnerabilities.

In a VA of a large distributed enterprise, there are numerous challenges. Some of those include accessing the network, selecting device candidates and maintaining currency with exploits. Today's pure-play VA tools focus on ease of use, VA functionality and certainty that they have the latest vulnerabilities covered. Almost all serious VA tools have references for common vulnerabilities and exposures (CVE), Bugtraq and other vulnerability sources. This allows a closer inspection of potential remediation beyond the short suggestions given by the tool.

More and more, we also are seeing references to CVSS - Common Vulnerability Scoring System. This is a standardized scoring system that helps one determine the real level of seriousness of the hole. This is vastly superior to the vendor-specific scoring systems that are inconsistent from vendor to vendor. Just remember that the CVSS as shown usually is only the base score. That is because the full score adds the dimension of the environment.

This takes us to the concept of risk. I have ranted about the misuse of this term by VA vendors for years. Vulnerabilities are vulnerabilities. They are not risks. In order for a risk to exist, there must also be a threat and an impact. That is where the CVSS comes in. The National Institute of Standards and Technology (NIST) has a CVSS calculator at http://nvd.nist.gov/cvss.cfm?calculator. I highly recommend using it. There are three components to a full CVSS score: base, environmental and temporal. Loosely, we can think of the base as the level of the vulnerability, the environmental as the impact, and the temporal as the threat.

When we talk about vulnerability management, we generally expect to see some form of VA scan automation, automatic analysis and reporting - much as with a pure-play VA tool - some form of patch management or other type of remediation management and, on the most complete tools, some way of automatically retesting to make sure that the remediation took. A capable vulnerability management tool needs solid, detailed reporting to meet regulatory reporting requirements.

In a full-fledged vulnerability management program, one will want either a VA tool or a vulnerability management tool - depending on the enterprise - as well as a penetration test tool. Pen testing is critical to validating vulnerabilities for reachability, exploitability and whether they need to be part of a chained exploit or may be exploited alone.

All Products In This Group Test