Product Group Tests

Group test: Vulnerability assessment

Group Summary

The security assessment field is maturing, converging and gaining depth rapidly. Tools are addressing real user needs instead of addressing point solutions. The buzz-phrase going forward must be “vulnerability management” instead of, simply, “vulnerability assessment” or “penetration testing.”

Scroll To Full Group Summary Below

Click for a side by side comparison of products
Click for a side by side comparison of products

Full Group Summary

This was an interesting year for our vulnerability assessment Group Test. Last year, we separated application vulnerability assessment from network vulnerability assessment. This year, we grouped them together. This revealed a few interesting differences.

The primary difference, strategically, is that network vulnerability assessment tools are converging with penetration testing tools to provide both capabilities in the same tool. This is very important in my view because, properly used, penetration testing is an extension of vulnerability assessment. In a proper network security assessment, one begins with the large view and progresses toward the specific. Two years ago, there were no solid combination tools. Last year, we had a couple that got pretty close. This year, we had solid entries that are really single security assessment tools.

I make a distinction between vulnerability assessment, penetration testing and security assessment. Vulnerability assessment (VA) reveals the global picture of possible vulnerabilities. I say "possible vulnerabilities" because VA tools can give false positives and, sometimes, the existence of a vulnerability does not constitute a risk. In order to have a risk, the vulnerability must be reachable by a threat and there must be a threat to exploit it.

We have learned that when there is a vulnerability we should pay attention to it. With SQLSlammer and now Conficker, we have exploits of vulnerabilities that were announced and patches provided months before an exploit appeared. So, when we identify a vulnerability, we need to test it thoroughly to determine if it can contribute to a risk. We do that by focusing on potential vulnerabilities with a tool to attempt to exploit the vulnerability. That means penetration testing.

Having all the tools in one network security assessment tool - from discovery and footprinting through VA to pen testing - is very useful. This offers consolidated reporting, simplified point testing where appropriate, and easier compliance testing and reporting. Unfortunately, application VA tools have not progressed quite to that maturity yet.

However, given what we saw this year, that maturing process is not far off. Perhaps we'll see the same kind of consolidated testing in applications as we do in networks. Of course, we'll know that security assessment has come to full maturity when all the tools are converged into a single product.

Even though substantive changes were few, one thing we did observe was that as the pack chases the perennial leaders the gaps at the front are narrowing.

The second important trend we are seeing is a slow move to vulnerability management. That means that some products are focusing more on remediation than in the past. Products in this class always have offered remediation recommendations, but now we are beginning to see products that actually help by providing assistance in patching. While VA products have yet to reach the level of patch management tools, some are getting close and we expect to see this whole approach to vulnerability management evolve over the next two years.

Compliance is probably the number one driver in information assurance currently. VA tools are responding by providing scan and testing templates that address specific regulatory requirements. This is an interesting twist on many other kinds of products that test or manage systems generally and then create specialized compliance reports.
The jury is out on this approach, because if you need to look at a comprehensive set of possible vulnerabilities, it's good to test completely and refine at the reporting stage. There is the potential to support what I call the "tick-in-the-box" syndrome. This happens when an organization opts to make sure that they can defend the checkmarks on the audit report, instead of making the enterprise truly secure.

The security assessment field is maturing, converging and gaining depth rapidly. Tools are addressing real user needs instead of addressing point solutions. The buzz-phrase going forward must be "vulnerability management" instead of, simply, "vulnerability assessment" or "penetration testing." Security assessment emphasizes the holistic nature of understanding the weaknesses in the enterprise. It goes beyond testing to include security assessment, VA, pen testing and patch management all in a single tool. This new class of tools is going to become an integral part of the overall enterprise management infrastructure.

All Products In This Group Test