For example, although all of the products we tested had the same core capabilities - correlating data from multiple sources, giving a consolidated report, and more - some products had a strong forensics focus. However, as we know, any good SIEM is a very useful network forensics tool in the context of incident response. There are some purely forensic requirements when legal issues come into play, though. Not all SIEMs can cover those. We saw some in our crop of products that covered all of the forensics bases well. So, even though we considered them SIEMs, these had a strong focus on forensics.
As well, all SIEMs do some log management. Some products we looked at have a legacy of log management, and that legacy showed strongly. On the topic of log correlation, there are a couple of ways to handle that. One is to cull metadata or some important pieces of data from the original logs and throw the rest away or store it in an archive. Some keep a lot of the logs - how many usually depends on storage - online. These products generally offer robust drill-down capabilities, often to the level that the contents of the original log can be analyzed in context with other correlated logs.
An extension of log correlation and deep diving into raw log content is the ability to replay events that the SIEM captures. Part of that includes how well the SIEM creates graphical network maps and representations of attack paths. That fits well with forensics capabilities.
What makes a good SIEM?
SIEMs, as with just about any network tool, should fit your needs. In this case, there are a number of important considerations and, happily, it is pretty certain that you can find a solid product in this crop to fit your needs. Your first step is to determine how you are going to feed the SIEM. Generally, the SIEM does not originate data logging - it simply analyzes what you send it. So you will need to decide what the data sources are going to be. And, of course, more always is better than less.
The extension of that requirement is what types of data your SIEM will need to handle. At least you should be able to handle syslogs, your firewall logs, any IDS logs that you are generating and if your infrastructure provides it, net flow data. Beyond that, Windows server logs, database logs and web server logs are very useful. The third level includes other types of application logs.
Your next question should have to do with your network architecture. If you are widely distributed - multiple data centers, for example - you need to consider how you are going to (or if you are going to) feed a centralized SIEM. Some products lend themselves easily to distributed environments with separate data collection boxes - sometimes called receivers or collectors. These boxes put the large volumes of data collected in a production environment into a format that allows efficient transfer to the central point without overloading the wide area network.
Finally, what kinds of reports do you need? Do you need to get regulatory reports off of your SIEM? Although it's nice to be able to do that, it is not always necessary. SIEMs are most valuable for trouble-shooting, investigating, correlating and alerting. Their big value is in the correlation. Distributed attacks, complicated attack paths, insider abuse and a raft of other events that take place across the enterprise instead of in a single place, cry out for the correlation capabilities of a robust SIEM. And these events are not limited to security events. Normal network performance failures often can be spotted with a good SIEM and a capable analyst.
How we tested
SIEM testing in our labs is a huge challenge. We have 350 words to give you the meat of the review, and if we tested each SIEM fully our reviews would be pages long for each one. That in mind, we were heavy on how easy the product is to deploy, the range of data sources it can support, how well you can drill down to raw log data, and how easy the dashboards and analysis screens are to populate and read. Then we looked at reporting to ensure that it was consistent with the personality of the SIEM. We also cared about how complete the reporting was in that context. For example, who is/are the audience(s) for the reports? Is the audience well-supported?
At the end of the day, these reviews scratch the surface of a very interesting and competent crop of SIEM products.