Lies, Damned Lies and Statistics


Analysts and marketers love surveys.

They like impressive market numbers that show what people are doing, why they're doing it and whether they'll keep on doing it. Stock markets thrive on these, as do marketing campaigns. Unfortunately, they're often dead wrong. Not for nothing did Mark Twain famously say, "There are three kinds of lies: lies, damned lies, and statistics."

The last few weeks have had some good ones here in the U.K. I should point out in advance that a lot of studies do have real value. The devil is in the detail - usually in the form of doubtful extrapolation and interpretation. Even if the numbers are trustworthy, the Twain effect usually comes along the moment it becomes part of someone's marketing spiel. Most of the surveys I'll mention here have their counterparts in the U.S. and elsewhere; I'd advise similar skepticism of any of them.

The U.K.'s Department of Trade and Industry (DTI) commissioned PricewaterhouseCoopers to figure out just how much hacking costs U.K. businesses each year. That study, released at Infosecurity Europe, was an interesting collection of figures, one that we'll see quoted a great deal in the coming months I expect.

Taking fraud, hacking and viruses into account, the PwC analysts reckon the damage comes in at around £10 billion annually. PwC offers a number of good reasons for this, such as increased exposure through e-commerce, employees using the Internet for personal tasks, and so on, but it's fraud and outright penetration that cost the most - around £30,000 each incident.

Wow. £10 billion is being hacked away in the U.K. each year? The newspapers loved it, but let's keep the numbers in perspective. £10 billion is a big scary number, but the DTI does point out that this is the same as "giving everyone ... an extra day's paid holiday a year." For employers, that's not great news. Most people working in the U.K. get 20-odd days holiday a year - you can do the math easily enough to see what each piece of the £10b is worth.

But so what? If you spend just ten minutes a day dealing with spam email, that'll add up to around four or five lost days - whole working days - over the course of a working year. Even without factoring in a cost for the network overhead and admin bother of dealing with junk mail, that's already five times the problem caused by security breaches.

And yet somehow companies are exhorted to spend! spend! spend! on intrusion detection, firewalls, anti-virus and managed services, because look, the DTI says it's costing you billions. Oh, and squeeze out a couple of pennies for mail filtering if you can afford it. When you consider how easy spam is to stop - when you put your mind to it - this is just nuts. No slur intended to the fine folks at the DTI; it's those who misquote them I take to task. Let's keep some perspective here. Obviously security is important, and worthy of its slice of budget pie, but blowing it out of all proportion is not the way to get board level buy-in.

PwC wasn't the only one with a security survey. A couple of studies of U.K. users have shown that two-thirds are happy to give passwords to colleagues and friends, and that 80 percent thought 'banana' would be a good password. There was more to it than that, of course, but that was the general idea, earning another ringing "so what?"

I mean, any ethical hacker worth his white hat will tell you that socially engineering a user's password out of them is the easiest way into a system. "Hi, I'm Bill from support and we're migrating Exchange servers. I need your password to import your mail..." And there are good reasons why operating systems ought to be configured to reject weak passwords. So what has the survey shown us? A bunch of stuff we already knew, that's what. But some analysts got a free lunch out of it, so it can't be all bad.

It gets better. Webscreen estimated recently that denial-of-service (DoS) attacks will cost U.K. businesses £54million this year, and more than £270m by 2005. These numbers look rather worrying, but you have to take them with a pinch of salt. Webscreen arrived at those figures by repeatedly extrapolating a number of other analysts' estimates: Forrester predicts U.K. online revenues to be £26b in 2002, and £300b in 2005, the CSI and FBI estimate that 38 percent of sites will suffer distributed denial-of-service (DDoS) attacks, and Webscreen themselves claim that each attack will cause at least two days downtime.


From the top. First, let's note that the analysts' track records of predicting online revenues accurately have been less than stellar to date. Second, no listed company on earth will attempt to predict revenue for three years from now, so I have to wonder how reliable these figures are. Third, the CSI/FBI figures are U.S.-based, and not necessarily applicable to the U.K. market. Fourth, this assumes no one will find a way to reduce the numbers of DDoS attacks in the next three years, and with the number of anti-DoS products on show at Infosec Europe, that seems implausible. Fifth, two days is a lot of downtime. It doesn't take an ISP long to do some filtering at their router interface and block an attack - any attack that takes a server down for more than a few hours is going to be unusually serious.

So, we have five questionable premises which, when you look at them in the right light, add up to a scary sort of estimate. But is it one you can trust? Certainly DoS attacks are a serious proposition, and can cost a lot of money. But this sort of scaremongering is doing no one any favors.

Security is serious business. It deserves better and more authoritative market research if CSOs and security specialists are expected to make informed decisions. A consultant specializing in denial-of-service commented to me: "My fear is that many executives have stopped listening to security reasoning because they have heard too many times that the sky is falling."

As a footnote, it's amusing to note that one anagram of "lies, damned lies and statistics" is "data scientist's mindless ideal." Mark Twain would be proud.

Jon Tullett is U.K. editor of SC Magazine (

Get daily email updates

SC Media's daily must-read of the most current and pressing daily news

By clicking the Subscribe button below, you agree to SC Media Terms and Conditions and Privacy Policy.