Way back in 2003, Scott Sidel, senior security manager at Computer Sciences Corporation, gave a presentation on security information management (SIM) and security event management (SEM) at the Information Security Decisions conference. The presentation was called "The Real Deal with SIM/SEM." It was amazingly prophetic. The things Sidel was looking for in 2003 were, mostly, the things we want to know now. They also are consistent with the things Gartner, for example, is looking for today.
For example, he noted the key tasks of SIM/SEM systems are to gather data; normalize data; correlate events (eliminate duplicates and check for patterns); respond appropriately; and learn. He also presented some typical needs he expected SIM/SEM systems to address:
As we looked at the SIM/SEM products featured in this group test, we observed that some provide these services better than others. So what distinguishes these products from each other? It turns out that there are several things that can make a SIM/SEM offering unique.
The first, and perhaps most important differentiator for many users is the ease with which the product can be deployed and used. On the surface, these products are very straightforward to implement. However, we found that, generally, the appliances went in faster and more easily, and gave us information we could use more quickly, than the software products.
Another differentiator is price, which varies enormously. Some of the software products are priced deceptively low. I say "deceptively," because you need to take into account the cost of hardware, which can include multiple platforms, an external database if the product does not accept a free one such as MySQL, and the additional expense of deployment resources. Software products take a lot longer to deploy than the appliances.
A final differentiator is performance. We found that while the appliances gave us a lot of good information, the software products were a lot more versatile. That flexibility comes with a downside, of course. They are more laborious to implement than the appliances.
Generally, we found that the products we tested realized Sidel’s vision for what a SIM/SEM should do. What we also found was that the entire genre is still not well understood. There is a tendency to mix in multi-purpose appliance and universal threat managers.
However, SIM/SEM products are distinct in that they are designed to be information and event managers, not device managers. Their job is to report, not act. While a few do provide action under certain circumstances, most do nothing more than observe and report. However, if they do that well, they are worth their, sometimes hefty, weight in gold to an over-burdened security analyst in the middle of a crisis.
We conducted the review by using pre-created data to test the products, so that we could control the collection of data and keep the testing fair for all. We would like to point out that all products were able to take input from a wide variety of sensors or, in SIM/SEM terms, "collectors." We were interested in how easy it was to deploy the product and start getting useful data. We looked for a rich correlation feature set and the ability to present a useful and easily configurable dashboard.
For the software products, we were concerned with how complicated installation was and how much flexibility we were offered in terms of deployment across the network. We evaluated the products’ value for money in terms of what is really necessary to implement the product in a typical environment. Is any extra hardware or software required? How much effort does it take?
The bottom line for SIM/SEM products is that they are coming of age. They are useful if the data you put into them is useful. Their displays, while quite busy sometimes, tell a lot. They are most beneficial in large, heavily segmented enterprises with a lot of security data and nobody with sufficient time to analyze it.
One key vendor missing from this round-up is Sensage, which performed well in a standalone test in March. But with a new release in development, the company was not able to participate.