Knowing where your network is vulnerable is half the battle. Illena Armstrong looks at the need for penetration test assessments
Depending on the expert who's talking, penetration tests and vulnerability assessments are simply one and the same, subsets of each other, or they merely serve to bolster one or the other's ultimate goals. Despite philosophical differences and changeable marketing spins, the winning consensus among infosec specialists is that both these proactive security practices are important components of an organization's overall security plan.
Penetration testing and vulnerability assessments really have the same end goal - to help organizations find and fix vulnerabilities plaguing devices on networks that are critical to business operations. The difference lies in how the two exercises go about reaching this end and the frequency with which you need to undertake them. There are also contrasting opinions on how to optimize the information found in reports gained from the individual tests.
"Many people are confused between the terms vulnerability assessment and penetration testing. The goal of vulnerability assessment is to identify what vulnerabilities a system may have, whereas penetration testing aims to exploit vulnerabilities," says Naveed Hamid, CEO of Sintelli, a London-based company with development facilities in India.
Often any confusion about the value of one test or another really happens as a result of marketing language disseminated by vendors and service providers, further explains Jeff Cassidy, director of business development for Core Technologies, a developer of penetration testing technology. Normally, a vulnerability assessment would simply be part of a penetration test, where the tool or service would identify vulnerabilities across a network and provide information on how to plug them.
Frequently, the actual penetration test focuses on exploiting vulnerabilities, both technical and non-technical, to leverage access to critical data through a certain subset of the network under some kind of time deadline, he notes. Vulnerability assessments are a bit more comprehensive and penetration tests engage more manual processes to find the path of least resistance in getting to that all-important intellectual property.
And, the most helpful information can be found after a vulnerability assessment is conducted, says Firas Raouf, COO of eEye Digital Security, a provider of various infosec tools, including a vulnerability assessment product.
He added that whether such an assessment is done in-house with the aid of one of approximately 35 scanning tools now on the market, or by a consultant or managed security service provider, it proactively locates all the holes on a network and then offers up pointers on how to fix them.
A penetration test, on the other hand, regularly takes the view of a malicious attacker, simulating any number of tactics a hacker might take to get access to critical information on a given system. And these simulations can take the perspective of either an insider or outside attacker, Raouf adds.
The objective is really to see how much damage can be caused by leveraging unpatched holes on servers or machines, as well as user name/password problems or social engineering tactics, for instance. These tests can be done each quarter or annually, when an IT project is deployed, or to support ongoing vulnerability assessments.
A chilling picture of vulnerabilities
While the importance of being proactive in plugging critical system holes has been proven time and again, most organizations are not doing it enough, adds Scott Blake, head of Bindview's Razor Team. He suggests that vulnerability assessments especially should be integrated into an organization's routine security practices, if not weekly or daily, at least monthly. Their usefulness will completely escape companies only undergoing them annually since so many vulnerabilities crop up daily.
As it stands now, vulnerabilities and attacks are rising fast with each passing year, helping malicious attackers in their efforts to access mission-critical data housed in private and government sectors all over the globe. According to CERT, a research center operated by Carnegie Mellon University, close to 99 percent of successful intrusions have occurred when hackers exploit known vulnerabilities or misconfigurations. The federally funded research center also notes that computer security vulnerabilities have doubled annually, with some 2,500 unique exposures found in 2001 alone.
Bindview's Blake says the average rate of new vulnerabilities for 2002 ran to approximately 10 a day. Between 1992 and 2002, these rates have risen annually by 90 percent. Because of the rates of change in vulnerabilities, he says companies should be conducting vulnerability assessments daily or weekly.
The main reason many companies might balk at such frequency is in the belief that they do not have enough resources to follow through with remedial efforts. But Blake says it is just a matter of prioritizing. How detailed and informative reports are from assessments can help in this regard. And, while there is "always more work than there are hands to do it, there is such a thing as too much security. You can spend too much [and] you can have too onerous security measures rather than risk," he warns, explaining that "installing the patches is no silver bullet."
Prioritizing system holes
While both penetration testing and vulnerability assessments work to provide information on exposures a company may have, Sintelli's Hamid further explains that, depending on the organization's critical business applications and security policies, it may not always be necessary to patch or reconfigure them all. According to his company's research, out of the known vulnerabilities that have existed since 1993 "only a handful of high risk vulnerabilities" popped up to require fixes within days of their being made public.
"Most other vulnerabilities can be fixed up to three months or more after the vulnerability has been identified because of the low risk it poses to an organization," he adds.
Additionally, depending on a company's risk appetite and business processes, simply moving forward with these tests' recommendations on remediation without considering the way various business applications are run may ultimately cause more harm than good, says Symantec's Ronald Van Geijn, director of product marketing for the vendor's penetration testing and vulnerability assessment products and services.
When accounting for the way systems are set up to support business projects, some patching or reconfiguration can be detrimental to a company's bottom line. If, in the name of security, the wrong holes are plugged or the wrong devices reconfigured, critical applications may not run any more, customer-facing web sites might crash or users might lose access to certain services, he notes.
Pointing out the risk
So, reports from penetration tests and vulnerability assessments must be detailed in showing the actual risk to the company based on what services and devices it has running. Also, reports should be based on an already established security policy, which will help in educating IT staff on what are acceptable practices and what are not. This will help to define what system components are important to what business applications and how these should be configured to maintain up-time.
On top of all this, when reports from such tests recommend certain modifications to a system, the company should have a 'change management process' in place that helps to assign duties, track their progress, inform on their effects, note their completion and more, he says.
Although Bindview's Blake believes that a company would be better spending its money on routine vulnerability assessments rather than on penetration tests, he concedes that pen tests have their place. But whatever the philosophical, marketing or methodology differences on penetration tests and vulnerability assessments, there are few dissimilarities in the end, says Steve Solomon, CEO of Citadel, a provider of a vulnerability assessment/remediation tool that, in the simplest language, nixes the need for manual exercises in the remediation process by providing automated action settings on a list of problems to fix that IT administrators simply pick and choose from.
With both tests aiding organizations in their efforts to proactively maintain the integrity of their system security, searches for exposures will become common practice to reduce liabilities and business risk - however organizations want to conduct them and whatever they want to call them.
Prioritize to mimimize the risk
Still, warns Core's Cassidy, vulnerability assessments alone are not enough to improve business postures. These scans for exposures, along with penetration tests, should both be parts of "a diligent security practice." The key is understanding that there will be some necessary follow-up to plug the holes that affect specific business practices, with the requirement to prioritize these tasks by risk to mission-critical applications.
It is simply up to the organization concerned to make this commitment from the outset or else be left with a business that proves more appealing to hackers than customers.
Illena Armstrong is U.S. editor of SC Magazine. Ron Condon, editor-in-chief, also contributed to this article.
Look at the non-technical issues too
Much has been written about the technical aspects of penetration testing and vulnerability assessments, but it is equally important to consider the non-technical issues, says Eric Jacksch.
Many vendors offer services labeled one or the other, but from a practical standpoint you need both. Attempting to penetrate a system without identifying, documenting and analyzing vulnerabilities has no risk-management value. In practice it is virtually impossible to detect vulnerabilities in a comprehensive manner without employing at least some intrusion techniques. Whether you perform in-house tests or contract out, there are important non-technical issues to consider.
- Purpose. The goal of the tests should be to identify all potential vulnerabilities, categorize them according to the business risk they present, and provide a report that allows management to understand their current risk exposure and technical staff to take appropriate action. Collecting unnecessary 'trophies' to prove that a system was compromised might be fun for the analyst, but it wastes money, poses unnecessary risk to production systems, and seldom makes a real contribution.
- Methodology. It is important to understand the methodology to be used. While many firms believe that providing too much information to the analyst gives him or her an 'unfair advantage' in conducting the test, too few consider the potential cost savings possible by providing basic information to analysts. Doing this lets them concentrate on identifying and analyzing vulnerabilities.
- Limits. Have the limits of the test and authorized activities been clearly defined? Safeguards to prevent accidental intrusion into unintended systems, especially those that you don't own, are important and critical.
- Ethics. Is your penetration test analyst a reputable security professional or a 'reformed' criminal? Don't end up falling for the 'hire a hacker' line. You wouldn't hire a child molester to help you protect your children, would you? Like a good locksmith, a skilled infosec professional knows everything the criminals know, and more. Remember, your firm's reputation is on the line.
- Independence. An in-house penetration test has advantages, especially in the area of cost, but is your test team independent of those who develop and maintain the system to be tested? Consultants may be more expensive, but equally they are also more likely to provide you with truly objective results. They also tend to have a much broader experience base.
- Report quality. Some firms add a cover page to output from an off-the-shelf tool and call it a report. If that's what you really want, you can buy or download the same tools and do it yourself. The report should be much more than a list of vulnerabilities. It should include an expert analysis, indicate the relative severities, provide practical recommendations, and relate them to your business in a format that both management and technical staff can understand.
Eric Jacksch is president of Tenebris Technologies Inc., an Ottawa-based information security and investigation firm (www.tenebris.ca).
