And it doesn't matter if they're running a mature or undeveloped program, or whether they're at a large or small business. Every security officer must answer to someone when it comes to dollars spent.
In recent years, the acronym of the day was ROSI — return on security investment. Analysts and security managers alike were struggling to find ways to measure security return on investment (ROI) and offer it up as proof to their bosses and executive boards that their money was being maximized. But the magic method to do this has never appeared. And some, such as André Gold, Continental Airlines' information security director, doubt it ever will.
"There are a lot of people out there who want to turn the information security department into a profit and loss (P&L) entity and I don't think you can do it," Gold says. "I ran our ecommerce environment for almost seven years and it was really easy to do ROI-type of metrics there. In my opinion you just don't have that in security."
Gold isn't alone. Increasingly, security professionals are dropping the goal of searching for ROI in favor of looking for better ways to communicate how security is making the most of its budget.
"I truly believe there is no real ROI," says Kevin Mandia, CEO of the security consultant firm Mandiant. "A lot of smart people have sat around trying to think about this for the last 10 years and nobody has come up with anything."
All you can do, he says, is detail the proactive things you've done to protect the company from identified threats, and when those thresholds are breached, discuss how fast you reacted to them.
Gold's philosophy is that as a risk management division, security is akin to insurance.
"Risk management is, I think, about insurance," he says. "Insurance doesn't have a P&L associated with it. Insurance is what it is."
Though Gold doesn't believe there is a clear way to calculate a return on security investment, it doesn't mean he can't communicate his department's value to the organization. It just means he has to work harder.
"How do I justify that I have maximized my dollars? I communicate what it costs to be risk averse. I say, ‘By doing these things we can avoid these types of risks.' The organization and I need to come to an agreement on what risks we are willing to assume and what risks we'd like to offset," he says.
In addition, Gold constantly holds court with senior managers as well as stakeholders throughout Continental's lines of business to educate them on the importance of shoring up information security across the organization. His work with these partners not only helps him garner budgetary support for future security projects that might have no measurable ROI, it also helps non-security stakeholders finish their projects securely — an accomplishment that can lend itself to better overall business ROIs.
Paradox of security
Security professionals such as Gold have been pressured to find a way to measure success for a long time. The problem, says George Kurtz, senior vice president of risk management at McAfee, is that they very quickly run into the paradox of security spending.
"The security paradox is the more nothing happens the better you are at security," he says. "The problem is a CSO may walk into the office of their CIO and say, ‘Look, I have five new projects I need to fund this year. I need more money.' And the CIO will sit back and say, ‘Why do you need more money? Nothing happened last year.'"
Mandia agrees: "The analogy I always use is if there's 20 people protecting the president and the president doesn't get killed, then you turn it down to 16 and the president doesn't get killed. Then you turn it down to 12 and the president gets shot. What was the right number of people protecting the
president? You still don't even know the answer."
This paradox has made plenty of people search for some metrics — any metrics that can help win buy-in from those with control of the purse strings. But some of the measurables people use today may be doing more harm than good when approaching executives, Gold says.
"Part of the issue from an industry perspective is that you may have people who don't have a business background who are trying to develop this business justification and ROI," Gold says. "And I think that's why you get these confusing metrics from a business perspective that I can't necessarily take to my senior management. My CEO is our former CFO. He's going to look at me like I'm crazy if I bring those numbers."
Instead of focusing on the technical minutiae of what his security products are doing every day, Gold says that it is more beneficial to focus on how projects and purchases mitigate risk to the business.
If Gold is asking for budget to implement a new project, he doesn't necessarily come at them with a request for the technology. He'll instead present the risk at hand and offer options for neutralizing that risk — along with potential price tags for the solutions. In some cases, leaders may choose to assume the risk. In others, they'll give the green light to move forward.
Once he gets the go-ahead on a project, Gold is constantly on the lookout for ways to maximize the budget he's given. Whenever he's evaluating products, he searches for those that offer additional features and functions beyond the ones that he's looking for.
"Then what we do after we've procured and implemented the technology, we go back to the business," he explains, "and we say, ‘Hey you know what? Because of this $2 million to $3 million you've given us, we were able to not only mitigate this risk that we both agreed needed to be addressed, but we were also able to do this other stuff.'"
By looking creatively for ways to make the most of the capital he's given and by communicating his efforts to the higher-ups, Gold has created what he calls a feedback loop that has strengthened senior leadership's trust in his judgment.
"What that does for us is two things," he says. "One, it helps us if, in the future, we underspec or underscope a project. Two, it helps us address new security concerns that have been created via legislation or industry, and allows us to go and secure funds that are outside the capital budget."
Security fan club
Gold doesn't just justify his expenses to his bosses, he also takes the word out to the rest of the organization.
"One of the things I've had to do is become a politician and chief campaigner for the information security program," Gold says.
Gold is a bit of a security populist, constantly rallying support for security activities from those within other divisions at Continental Airlines.
"Ultimately in order to have an effective information security program you must get buy-in," he says. "So what I have to do a lot of times is really meet and have discussions with HR, internal audit, legal and various business and technology stakeholders to really grasp what their business objectives are and how, from a security perspective, we can aid them in accomplishing their goals."
By doing this, he's managed to create an environment where he's not the only one clamoring to upper management for security dollars. He's got a legion of non-technical supporters doing it, too.
"The upside of that is now we've had general councils over here who have given presentations to our board that say, ‘OK, this is the importance of information security,' and have communicated it to our CEO and chairmen as one of our top 10 [business] concerns," Gold says. "So we are getting other people on board outside of security who are saying ‘Hey, this is important.'"
Building a security fan club across the organization has not only helped Gold rally support for future security projects. It has also created a cultural shift that has been radical enough to pave the way for integrating security into IT procurement and project management on the frontend.
The key benefit is that with just a little bit of investigation into products or a slight bit of added time in application development lifecycles, the security team can prevent vulnerabilities that would cost millions of dollars to fix later.
"If I go through and I build an unsecure network or unsecure application, and then I go in at the end and try to figure out how can I hook into that to provide a secure environment, it's going to cost a lot more than if I just integrate security into that development lifecycle," he says.
This method may be just the key to finding a way to measure security's usefulness in the long run. Though security projects don't always have demonstrable ROI, Gold believes injecting security into everyday IT projects may have measurable affects on their overall return.
Gold illustrates his point with this example: If he were to fund an ecommerce infrastructure that was going to allow him to make three to five times more transactions, that translates to a set number of millions of dollars in return. If that project is built unsecurely, the risk is either a bad report from internal audit, or worse, a compromise.
"The internal audit is going to say you should add these firewalls, you should add IPS, and everything else," he says. "So now I have to go back and say, well what's the cost of procuring this technology? And then the ROI metric that I had initially assumed is going to be diminished by that number. That's because I didn't include all of these security primers to begin with."
Gold is not alone in his beliefs on this matter. In fact, the Application Security Industry Consortium (AppSIC), a newly formed organization made up of security luminaries — such as Mary Ann Davidson, Amit Yoran and Serge Moreno — has targeted this method as one of the keys to improving security metrics.
"If I'm building a piece of software, I think of it as this infinite wall of light switches that I can turn on. Each of them costs me a different amount to flip," says Herbert Thompson, chief security strategist for Security Innovations, and the chair of AppSIC.
He says that each of the light switches involved represent different things a company can do to improve application security on the frontend of the lifecycle. This could include training developers on security, putting secure requirements processes in place, or putting penetration testing in place.
"The cool thing is we know how much each of those things cost, but we don't know how much leverage we get by flipping that switch," he said. "One of the metrics that we will produce is providing the insight of what that leverage means." n
We welcome your comments. Email us at [email protected].
A brief bio
Prior to taking his spot as director of information security at Continental Airlines, André Gold served the company as technical director of internet services. There he ran the airline's billion dollar ecommerce site for over six years. When Continental created a position specifically in charge of information security, he was hand-picked for that role. The shift in technical focus and mentality was dramatic, Gold says, and he had to spend a lot of time in his first months educating himself on the job.
Since then, he has been recognized across the industry as a leading security practitioner. On top of his duties at Continental, Gold makes time to serve on the Microsoft Chief Security Officer Council, the Skyteam Data Privacy and Security Subcommittee, and the eEye Digital Security Executive Advisory Council.
André Gold, information security director at Continental Airlines, isn't partial to telling his bosses at how many attacks his firewalls prevented or how many spam messages his gateway solution blocked. To him those kinds of numbers aren't beneficial.
Instead he prefers to prove he's earning his keep by finding creative ways to minimize the costs associated with security and then reporting his accomplishments back up the chain. One tool he's used to do this is Core Impact from Core Security Technologies.
Gold initially purchased the product as a penetration testing tool for existing infrastructure. But he's managed to leverage the technology by using it to test new products before acquisitions as well. By testing these products comprehensively before making a purchase, Gold feels safe opting for less expensive options that he sees performing as well or better under Core's battery of tests.
The first time Gold put Core to work in this capacity, he was able to save Continental between $200,000 and $500,000 on an intrusion prevention system purchase.
This type of heads-up product assessment and fiscal responsibility is what helps foster a higher level of trust from his superiors, he says, making it easier to ask for more money in the long run.
Experts weigh in
Not every business has developed the type of organizational-wide buy-in of security seen at Continental Airlines. And some bosses absolutely demand some kind of security measurables to justify the department's spending.
Before presenting the readily available statistics often used by your peers — such as the reams of reports offered by your IPS system — experts advise that you think first about the audience you are speaking to. If it is senior management, they aren't going to want to hear about "technically myopic" security measurables, says Frank Bernhard, a technology economist from OMNI Research Group.
"If you are going to do security investment strategy, think about it from a purposeful economic benefits statement," he says. "If I can't set down in a statement to my CEO and justify that I spent $15 million and I've curtailed a loss of x amount related to this type of activity or relative to the year over year succession of business, I haven't done a very good job in justifying why I've just spent $15 million of the company's scarce resources."
If your business is in a heavily regulated environment, Kevin Mandia of Mandiant says that statistics will also lean heavily toward how compliant you are and whether you can sustain that status. This is going to be of key interest to those in upper management and on the board.
"You've only got about one minute to convince them that you are doing all of the right things with your IT security program," he says. One of the things you've got to brief is that dashboard that says we have the structure here, [and] there is the accountability to maintain compliance."
Things can be tricky in situations where you are practicing within a business that hasn't had any recent major events and isn't dealing with compliance, Mandia says. In those cases, it is sometimes a necessary evil to use the sometimes inflated numbers offered by your security vendors.
No matter which metrics you do use, though, the real key is consistency, says George Kurtz, senior vice president of risk management at McAfee.
"You don't have to be right, you just have to be consistent," he says. "You should be able to articulate the same consistent results over time."