Product Group Tests

Application vulnerability assessment tools

Group Summary

Application security has continued its momentum as one of the primary issues on the minds of information security professionals everywhere. Due in large part to the increased exposure to horror stories in the media, application security has come under scrutiny for being a favorite target for malicious users.

Scroll To Full Group Summary Below

Click for a side by side comparison of products
Click for a side by side comparison of products

Full Group Summary

Application security has continued its momentum as one of the primary issues on the minds of information security professionals everywhere. Due in large part to the increased exposure to horror stories in the media, application security has come under scrutiny for being a favorite target for malicious users.

As reliance on information assets continues to grow exponentially, the protection mechanisms at the application layer have created a void between the bad guys and the stakeholders who are tasked with protecting the infrastructure. As focus has long shifted away from traditional network perimeter attacks, client-side attacks and specialized application layer vulnerabilities now have some personnel scrambling to play catch-up. Regulatory compliance mandates, such as PCI-DSS (Payment Card Industry – Data Security Standard), and even greater industry awareness have created an elevated level of interest in taking the proper steps to secure external facing applications and critical databases.

The often publicized gap between developers and infrastructure stakeholders has created a prime opportunity for vendors. Unsecure coding practices and database misconfigurations can introduce vulnerabilities into critical application infrastructures that tend to cost more money to remediate after they are sent to production. What we’re seeing in 2008 is a maturing of the space of application vulnerability assessment products, and it’s no surprise. Take a trip to the Open Web Application Security Project (OWASP), or the Application Security Consortium (WASC) – two worldwide groups of experts, industry practitioners and organizational reps focused on securing the safety of web applications – and the writing is on the wall for information security programs to step up their protection of critical app infrastructures.

We examined three general classes of products for this Group Test: web application vulnerability assessment tools, source code analysis tools and a database security assessment tool. All three classes of software have their places in the software development lifecycle and can help security stakeholders make intelligent risk-based decisions. It’s up to the individual business to decide the appropriate level of investment within its own information security program by complementing internal skill sets and resource needs versus the criticality of the assets they are protecting.

The tools in this group all performed above and beyond uncovering the traditional SQL injection, XSS and other vulnerabilities that can be found in the OWASP Top 10. Scanning engines, while important from a performance perspective, are maturing to the point where catch rates are impressive, and the convergence of features and integration into the security program becomes the differentiator.

All of the products in this Group Test were installed on either Windows XP Professional SP2 host machines or Windows 2003 SP2 servers, per the requirements of each vendor, with ample horsepower to run all of our tools. We featured instances of four popular database backends to ensure testing remained varied.
We ran the source code analysis tools against two separate sets of extensive and vulnerable test code. Our database vulnerability scanner was run against all four of our vulnerable database instances, and the web application vulnerability tools were run against three sample web applications, which contained a myriad of popular critical vulnerabilities.

All of the products were scored on our typical criteria of support, documentation and price. But we also considered ease of administration and configuration, timed performance of the application to cycle through its targets, compliance templates and report offerings, the inclusion of remediation reports, and the ability to provide role-based access control.

All in all, the majority of our products tested well. Pricing considerations should be given to ancillary features that introduce real value beyond catch rates. We found most of the products to be priced as expected, with some in the higher range.

We also saw that some product brands are still in transition because of a merger or acquisition. While they remained strong products, support suffers a tiny bit because of the confusion of branding and website content that has yet to make it over to the parent company.



Nathan Ouellette is consulting director of Viopoint.

All Products In This Group Test