Mark Schaefer and Jim Carpenter, IT managers at Edwards Lifesciences and ALON USA, respectively, can tell you all about it.
After Schaefer and Carpenter's IT infrastructures were hit by distributed denial of service (DDoS) attacks, the assaults temporarily shut down network resources critical to running their firms' businesses. They both responded by not only deploying new security technologies, but also implementing policies that will help their systems fend off attacks in the future.
Those types of strategies are vital to ensuring the ongoing health and welfare of any enterprise infrastructure in this era of ever-present danger, according to security experts.
Yes, they admit, bad things are bound to happen. But how the IT and security personnel managing those systems respond to attacks and learn from the strikes are critical elements in successfully protecting their systems moving forward.
After Carpenter had tested, then rejected and even re-packaged TippingPoint's intrusion protection system (IPS), the device came to ALON USA's rescue during a network shutdown.
"We had a situation where nothing was working on our network because of overloaded traffic," says Carpenter, the IT manager for the Dallas-based marketing and oil refining company that operates more than 160 Fina convenience stores.
"We pulled [the TippingPoint IPS] back out of the box and re-deployed it," he says. "It figured out what was going on and eliminated the problem. It turned out to be a DDoS attack -- we never quite pinned down exactly what caused it."
Carpenter did learn a valuable lesson from the attack, though. "You have to keep up-to-date on the security patches from Microsoft for Exchange and your desktop operating systems."
He now has policies in place to ensure that every machine is patched, a key factor in keeping his network free of malware.
Similarly, when the Blaster worm entered the Edwards Lifesciences network via a overseas worker's laptop, Schaefer, the information security manager for the Irving, Calif.-based company, discovered that "corporate perimeter defenses" offer insufficient protection against threats. "You need to look at all entry points. Ultimately, the desktop is where you're compromised."
Quick response
The attack left Edwards, a developer of biomedical devices for treating advanced cardiovascular disease, without sufficient network resources to handle its manufacturing for almost two days. In response, Schaefer deployed two products, Check Point's Integrity desktop firewall software and a patch management solution from Altiris, to ensure Edwards' systems are updated when necessary.
He says his intrusion detection system (IDS) helped "mitigate some of the effects of the attack, but not all." Backtracking through the box's logs, Schaefer detected the source of the exploit -- that overseas worker's notebook -- and was able to point the way to a solution.
Not every enterprise infrastructure has an open hole that can be exploited, of course. But as the above examples illustrate, worms, trojans and other malware pose an ongoing threat to virtually any corporate network.
Security managers are increasingly turning to IPS, the IDS's more sophisticated cousin, to help protect against the many exploits making the rounds. Unlike the IDS, which is a passive device that merely collects security event data, the IPS can take proactive steps to halt attacks based on abnormal network activity or pre-defined signatures.
There are several reasons for the move to an IPS, not the least of which is the mind-boggling volume of data an IDS generates and the problems associated with decoding the information. John Loyd, vp and director of IT at Patton Harris Rust & Associates (PHR&A), a Chantilly, Va.-based consulting engineering firm, says he turned his company's two IDSs off in early 2004 "because they were generating too much information. They weren't telling us anything meaningful or actionable. They were just overwhelming us with information."
Loyd admits that if his network was larger he might well have opted for an IPS rather than the threat-mitigation tack he chose, SecureWave's Sanctuary Application Control product.
Sanctuary creates a "whitelist" of approved executables and stops others from executing. Loyd says it gives him "more bang for the buck" than an IPS.
Effective management
Loyd's experience notwithstanding, IPS and IDS both have their places in corporate security environments. IT managers must learn how to manage them effectively, according to experts.
Take the IPS, which is gradually supplanting the IDS as the perimeter security device of choice. Many enterprises first use them as an IDS, just collecting information about the status of their network. They then gradually turn on the IPS's intrusion-protection functionality after they have developed a baseline picture of what is "normal" on their network.
That is exactly the tack that Joe Adams, director of IT at Nuclear Fuels Corp. (NFC), took when he deployed StillSecure's StrataGuard IPS to monitor and protect the systems he manages for six divisions of General Atomics, NFC's parent company. He used the device in passive mode for two years before using its active deterrent capabilities in early 2004.
He waited, he says, because he was concerned that the box, if improperly configured, would degrade performance for his end-users or block partners' access to the IT resources of NFC, which markets uranium fuel rods used by electricity-generating nuclear reactors.Adams says he has not changed his attitude about waiting even after using it for nearly two years in IPS mode.
"Patience, especially in security products, is a virtue. Experience is supposed to teach you something, and my experience has been that if you deploy a new security product without full testing, the transition is rockier than when you spend time in the IT department properly testing prior to deployment."
Sanjay Beri, director of product management for Juniper Networks, believes many enterprises fail to use the IPS effectively. Many IPSs work at layer four in the OSI model, just looking for patterns in bit streams, he says.
An IPS must be application-aware or it will generate false positive intrusion alerts, he adds. In addition, Beri believes IT managers relying on an IPS must "look at the vulnerability as well as the attack. Make sure the IPS goes after the vulnerability itself -- the baseline issue."
Know thy network
One of the lesser-known issues for an IT staff in keeping its enterprise systems safe is developing in-depth knowledge about its own infrastructure, says Mike Paquette, vp of product management for Top Layer Networks.
"Keep in mind that every vendor has the same problem: We have to develop products good enough for 80 percent of the customer base."That means IT personnel "should internalize the knowledge gained from a break-in to pinpoint the precise nature of the malicious activity on their own systems," he explains.
If an attack hits an IIS web server, for instance, discover the specific nature of the vulnerability it exploited, then resolve those issues and configure the IPS to stop the "signature" of those attacks, he advises.
The days of "if it's not broke, don't fix it" are gone, says Andre Gold, director of information security at Continental Airlines, referring to the wide array of supposedly stable (read: impregnable) devices found opn an enterprise network. Although they may appear to be passive devices, network-based printers, in particular, are anything but dormant devices anymore, he warns.
"They're taking on more and more features, and are actually running applications that are vulnerable," Gold says, adding that this makes tham a potential liability risk. "You have to have a strategic remediation process of how to upgrade new versions of software for all devices. they are vulnerable now."
