A pet peeve I've always had relates to consultants who insist on producing client reports by mindlessly copying tool-discovered vulnerability information without any reflection on the environmental context of the security assessment. By failing to incorporate this information, they can cause confusion and might even weaken an organization's security when the client diverts valuable resources to address incorrectly prioritized risks.
The output of these automated tools, while often providing extremely detailed information about each and every vulnerability uncovered, should only be used as a guide for remediation – not for prioritization. Even though the descriptions invariably include a "risk" value, it is made without any contextual understanding and really only represents the impact of exploitation.
This tool-based "risk" value, while not necessarily accurate enough for prioritization, still forms a solid basis for understanding the significance of a security finding – assuming it comes from a reliable source. The source caveat is important. Each vulnerability assessment tool will enumerate a vulnerability's risk differently, with its evaluation dependant on the original source of the information, the research that went into its evaluation, and the quantization of the ranking system (e.g., three-tier; high, medium, low, or four-tier; critical, high, medium, low).
If one tool evaluates the risk of a vulnerability as high in a three-tier ranking system, while another evaluates the same vulnerability as critical based upon their four-tier system, is it high or critical? The same confusion arises when reading the original vulnerability advisories – it is not uncommon for the original discoverer of the vulnerability to rank its "risk" higher than that claimed on the affected vendor's advisory publication.
All this could soon change. There is growing momentum behind the adoption of a new, more consistent vulnerability scoring mechanism -- the Common Vulnerability Scoring System (CVSS).
CVSS is a framework designed to be used by vendors, consultants and clients alike to calculate a composite score for a vulnerability based upon severity and risk. Using 12 evaluation metrics split into three groups, CVSS aims to provide a consistent platform for calculation and incorporates temporal as well as environmental data to arrive at a score.
Once security tools support CVSS, it is likely we will see a change in the way in which an organization manages vulnerability prioritization and remediation. Vulnerability assessment tools will then be able to provide the seven metrics that make up the base group score -- this includes static information, such as access complexity and vectors, authentication requirements and traditional risk management CIA impact values.
Temporal data, such as whether exploit material or proof of concept code is loose and whether vendor patches or work-around processes are available, is used to formulate the Temporal Metric Group. This factors events that might affect the urgency of the threat posed by the vulnerability. This information will need to be supplied by trusted vulnerability research teams and evaluated on an almost daily basis to accurately reflect the threat.
Environmental -- the last metric group -- must be evaluated in the context of each organization, as it factors in collateral damage potential and target distribution.
While it is likely that CVSS will increase the effort required to evaluate a threat, if used properly, clients will benefit from more accurate assessments and remediation prioritization.
As for those consultants who copy-paste risk values, they're either going to have to change their business practices or their occupation.
Gunter Ollmann is director of X-Force, Internet Security Systems
