Increasing security in an internet-connected organization means you must monitor security event logs and collect massive amounts of data. It is a challenge to sort through all these logs to identify system attacks or exploits, but the data must be analyzed and it has made threat detection a huge challenge. Security event management (SEM) products have emerged to respond to it, and they help in collecting security events, analyzing the collected data, applying intelligent event analysis and responding to the identified threats.

Security event collection

There are two approaches to security event collection: explicit event collection and event log consolidation. Explicit event collection assumes organizations know exactly what events will later be relevant to understanding a security breach, and only the information related to those events is retained. The limited information can then be synthesized and presented in high-level summary reports. In contrast, event log consolidation archives all collected data. Both techniques are critical to gaining a clear understanding of a company's threats while also providing the data it needs for comprehensive event analysis.

Security event analysis

Three typical methods of security event analysis are real-time analysis, scheduled analysis and a hybrid of these.

Real-time analysis recognizes significant events as they occur by evaluating them against pre-defined criteria. This immediately detects known threats, so organizations can quickly respond and reduce the threat exposure. Real-time analysis is useful because it immediately alerts the administrator to a potential threat, but if there is no context for interpreting single events, then real-time analysis can result in alerts requiring a human response, only to discover there is no real threat.

Scheduled analysis collects and evaluates data over longer time intervals (days and weeks, against seconds for real-time analysis). The collected data is analyzed to determine security weaknesses and identify ways to mitigate risks. Scheduled analysis provides more context for events, but does not provide alerts for security breaches taking place.

Although scheduled analysis summarizes and aggregates data to make it more manageable, summarized data can limit insight into the past events. A hybrid approach provides the benefits of both techniques, alerting the infosec professional in real-time of specified threats, but also preserving event information for long-term analysis.

Event correlation

To add more insight to security events evaluation, it can be useful to correlate two or more events to provide further context. Two or more data points are correlated to prove or disprove the existence, severity and priority of an identified threat or vulnerability. Organizations can correlate security event data in many ways.

Identifying specific events that occur within a specific time interval is a common example of set correlation. Using set correlation, infosec professionals define a specific group of events, all of which can share common properties, such as a source IP address or a destination port. When these events occur within a specified time period, they constitute an attack or threat. Sequence correlation builds on set correlation by specifying the order in which the defined events must occur to indicate an attack or exploit.

Weighted analysis of events (the threshold correlation) is another approach. This enables the administrator to define events that must occur before raising an alert. For example, if several failed logon events occur on the same computer a specified number of times, the correlation threshold can be reached, resulting in an alert.

Responding to the results

When a threat is identified, someone must respond. Staff can respond manually, design computer-based responses, or combine the techniques. To respond manually, staff must know what to do or have in-depth security knowledge at their fingertips.

Facing a specific threat, computer-based responses can automatically execute a pre-defined series of steps. Computer-based responses can include running applications, batch files or scripts or blocking ports. These automated responses attempt to contain the incident, but the automated response might not be the optimal solution. Experience shows that neither manual nor automated responses alone provide an optimal solution. Both types of responses must be leveraged.

Making the most of the four phases

SEM focuses on collecting, analyzing, notification of, and responding to security events that occur daily. When used effectively, SEM tools can help reduce threats to an organization by providing the essential resources to distill actual threats and attacks from a chaotic collection of security event data. The net result identifies threats more quickly and combines SEM product response with human intervention to improve the overall efficiency and security of the organization.

Jim McGrath is senior director for security products at NetIQ