We tested the SenSage Enterprise Security Analytics (ESA) and admired its speed, scalability, and built-in intelligence. The product satisfies several regulatory and industry compliance standards through Sarbanes-Oxley, HIPPA, FFIEC and other analytic packages. It is available now for RedHat 3.0 servers and shortly for SUSE Enterprise 9 servers.
Pricing starts at $70,000 for the base package (single server,
Additional analytics packages are priced at $30,000 each, and clustering and support can help to move the typical total starting price nearer to $200,000.
We were impressed with the design from the start. Built without relying on any relational database, SenSage nevertheless creates an SQL-compatible repository that combines speed, optimised storage, built-in reports and dynamic drill-down investigation capability. Graphs of our scalability test data were impressively flat curves, so you can add processing resources incrementally as log data and queries grow.
We tested SenSage ESA on ten IBM eServer HS20 Blade servers with older dual 2.4GHz processors running hardened RedHat 3.0 Linux, arranged in clusters of one, three, five and ten servers. Since data collection and queries are automatically load-balanced across cluster members, overall performance depends on the speed of the slowest processor.
Using real-life log data from a CheckPoint firewall, a Blue Coat web proxy, and a web server, and in sizes varying from 200Mb to 10Gb, we first checked the data load rate. SenSage pumped the data across cluster members as a B-tree, indexing and compressing each server’s allotment.
Load rates for the three varieties of logs varied from 17,000 to 61,000 records per second on a five-server cluster, and were about 70 per cent faster on a ten-server cluster. Using the recommended 3GHz processors should boost
this speed. SenSage told us that Windows event logs (which we did not test) would load at up to 90,000 records per second on a five-server cluster. After loading the initial set of logs, ESA adds new log entries on schedule, on demand, or continuously.
The compression ratio was essentially identical to a manual gzip of the same data and, besides archiving the original log for compliance, the data also became fully searchable at top speed in its compressed state.
SenSage ESA also provides automatic failover on a cluster, storing two segments of the data on each member – one to query plus a duplicate of the next server’s allocation.
If a server dies, the unanswered query will be immediately and automatically picked up by the rest of the cluster. Query speed will drop, since fewer servers now have to process the workload, but that is a huge advantage over sending techs to recover or redistribute data and reconfigure mechanisms.
SQL queries brought forth great search performance without any database tuning or pre-processing. Creating an aggregated report of network user time spent online took only 67 seconds for 10.7Gb (30 million records, 441,000 records per second) of data on the five-system cluster, for example. A simpler report of blocked website categories took 22 seconds (1.285 million records per second).
We liked the Java-based Console interface, despite finding a couple of minor bugs. It let us use pre-built and custom reports as well as launch real-time investigations.
Drill-down mechanisms were automatically provided based on relationships between events. For example, a report of user activity outside normal business hours could correlate data from multiple devices and systems, and drill-downs can open up the events for further on-the-spot queries.
Exceptions with batch data as well as triggered rules in real-time correlation can be sent to the console, email or SNMP. The user can easily investigate all activity at the source or destination, via IP address or user ID.
Built-in graphing capability aids visualisation of the tabular results, and we also visually replayed a series of disparate log events to help expose the perpetrator.
Reports can be created ad hoc or run as scheduled standard reports, and sent to users as PDF, HTML or CSV files. Access permissions secure data even to the field level.
Installing the application software and log adaptors is not hard, but SenSage routinely assists its customers with implementation. Most common log files are supported out of the box, and an SDK is supplied to customers who want to write a log adaptor.
Adaptors include both real-time (such as SNMP, HL7, LEA and syslog) and batch-mode (such as FTP, SCP) collectors. After mastering the core product, expanding data collections and imagining new uses for ESA will consume the most thought time.
