Vulnerability assessments, penetration testing and risk assessments are all key parts of an information security program. But how often should these tests be done? Is it better to do them in-house or bring in a consultant? And what should be done with all the data gleaned from these assessments?
There are no cut-and-dried answers to those questions – one size does not fit all when it comes to assessing vulnerabilities and risk, and performing pen tests. The approach a company takes usually depends on its size, amount of in-house expertise, and individual business needs.
Generally, though, security experts say more is better in terms of vulnerability assessments.
"Some people do vulnerability assessments once a year. You get a 600-page document and have to sort through what's important, who will fix it and why," says Chris Hoff, CISO at California-based Western Corporate Federal Credit Union (WesCorp). "By the time you've done all of that, it's out of date already."
WesCorp uses a web-based service from Qualys to assess vulnerabilities and track remediation. It runs scans weekly, monthly or every two months depending on asset criticality, configuration changes and the like. Servers are also scanned before they go online.
At Centillium Communications, a supplier of broadband access products, IS manager Chris Amos uses nCircle's appliance-based technology to run monthly vulnerability assessments of the company's entire network. He also runs weekly scans of mission-critical servers.
"We figure monthly full network scans are sufficient, unless there was a huge outbreak of something," he says. "Then we would run the scan immediately to make sure patches are up."
Firas Raouf, chief operating officer at eEye Digital Security, recommends more frequent scans – at least weekly – because of the regularity and number of software vulnerabilities cropping up.
"It's not just that vulnerabilities are emerging every day, but there are new devices being added to the network," he says. "A device that's recently added could be misconfigured or susceptible to an old vulnerability."
Many of Qualys's customers run scans frequently, with daily scans of critical servers, says Philippe Courtot, Qualys chairman and CEO. "There are many, many things you need to constantly check to make sure there are no vulnerabilities, and that people are properly configuring systems and following the procedures you've established," he says.
Courtot also believes it is critical for companies to conduct assessments on any outsourced elements of their networks.
Michelle Drolet, chief executive of security services company Conqwest, recommends monthly vulnerability scans of servers and desktops. But that can be overwhelming for some organizations, because they can never fix all the problems the scan turns up, she says.
"They don't have enough people power," explains Drolet, adding that a more realistic schedule is tri-annual or biannual assessments, which her firm can conduct, or the company can use the technology to do in-house.
Typically, having a consultant conduct a vulnerability assessment once a year will be far cheaper than using an internal process, says Dave Ostrowski, a product manager at Internet Security Systems.
"But the net deliverable from a consulting engagement is going to be a voluminous report that doesn't necessarily fix anything," he warns. For a consultant or managed service provider conducting assessments, implementing remediation requires some politicking and negotiating with the client, he adds.
An in-house security team can shift directly from finding the vulnerabilities to implementing a patch or other protection. But an internal team might bring subtle biases to an assessment and alter a scan to make a point, he adds.
ISS advocates quarterly vulnerability scans, with the same schedule for pen tests, which simulate attacks on a network. But Ostrowski says semi-annual pen testing is the normal procedure for many organizations.
WesCorp asks an external consultant to conduct an annual pen test, mostly to serve as a validation for its own pen testing, says Hoff. Much of the firm's pen testing is focused on web applications. For example, when a server with a web component is about to come onstream, the company first uses tools to perform deep pen testing.
Centillium also undertakes in-house pen testing, using freeware tools such as Satan (Security Administrator Tool for Analyzing Networks) and port scans. In the past, the firm hired consultants for pen tests, but they became too expensive, recalls Amos.
"They did a lot of port scanning, going from outside the network and trying to get into our web or mail servers, which is pretty much what we can do already."
For Hart InterCivic, on the other hand, having a third party is essential for pen tests. The company, which makes electronic voting systems, has a tiny IT staff and does not have the expertise to conduct such tests. Hart InterCivic has network pen tests done every six months.
"Attacks are getting better, more robust and more numerous," says Ed Smith, operations manager. "It just makes sense to have them done more frequently than annually, but I'm not sure if doing it every quarter would really be a value-add."
In the past Hart InterCivic also tapped consultants to help patch vulnerabilities, but now performs vulnerability assessments and remediation in-house with Citadel Security Software.
Standard Chartered Bank does not consider consultants to be cost-effective for vulnerability or risk assessments. But it does tap them for application-level pen testing, says John Meakin, group head of information security at the bank, which has over 500 international locations. Web-facing application pen testing is "still very much a black art," he says.
"The expertise to do that effectively and in a probing way is pretty rare," explains Meakin. "If you find a good consultancy offering that expertise, it's probably worth the money."
For many companies, vulnerability assessments and pen tests are part of a broader, overall vulnerability and risk-management program. They want to not only find vulnerabilities, but fix them according to which poses the most risk to their critical business assets. To that end, risks assessments play a crucial role.
"When organizations are trying to define a security program, they first need to understand what they're trying to protect – that's where we look at the risk assessment," says Conqwest's Drolet.
A company could implement a lot of sophisticated technology, but if it is not protecting the organization's crown jewels, it is not worthwhile, she adds.
Indeed, a vulnerability by itself is not a basis for action, says Bob Jacobson, president of consulting company International Security Technology. Firms need to look at the vulnerability in the context of business assets and functions, the potential for loss to those assets, and then evaluate the cost benefit of mitigation measures.
"What I don't want to do is spend a million to achieve a $10 reduction in expected loss," he says.
A risk assessment allows a company to focus its efforts on the assets that are most critical, agrees Hoff.
"It lets you focus on the precious few, because do you really want to spend a bunch of time patching a vulnerability on your print servers versus your e-commerce servers?" he asks. "The only way you can determine that is to let the business help drive and direct you in terms of telling the security folks what the most important assets are."
Standard Chartered Bank has an ongoing process of risk assessment, says Meakin. It uses its own proprietary tools to determine a system's value to the business, and to develop an overall risk inventory database. That risk data is combined with threat alerts from Cybertrust and Symantec, and vulnerability assessments from Qualys, to pinpoint network vulnerabilities and determine which would have the highest impact.
The bank applies technology from Skybox Security to produce more finely tuned data via threat modeling, explains Meakin. "Then we know we're going to patch not only that which is valuable, but that which is most valuable and most at risk because it's most exploitable."
The output from this overall vulnerability management system can be used to show compliance with internal security standards as well as government regulations, he adds.
"These systems help me patch in the right order, quickly, and they help my business demonstrate in a very measured way a state of achieved security."
