The Secret of Agents

A generation ago, the world of espionage and counter-intelligence was populated by agents who operated in the shadows of eavesdropping, codes and hollowed-out books to collect and transmit important information.

If you believe the spy novels, they had their own code of ethics and in their spare time drank martinis, shaken not stirred.

In today's more utilitarian and electronically leveraged information security environment, agents are now programmed conduits between an information generator and an information consumer, and their world is comprised of systems, networks and protocols instead of dark alleys and smoky cafes. Many types of information management products utilize agents, including network managers, device and configuration managers, and the rapidly growing category of security management products. In fact, agents have become so popular and ubiquitous that the challenge now is to deploy them so that they actually support and enhance the application on whose behalf they are installed, while preserving the integrity and resources of the operating environment. This is particularly true for security management applications.

As organizations struggle to integrate and optimize their security infrastructure, they are increasingly turning to security management products to monitor, consolidate, correlate, investigate and report on the literally millions of security events that are received each day. With the proliferation of firewalls, intrusion detection systems (IDSes), virtual private networks (VPNs), etc., there is no longer the opportunity to observe and understand the output of individual or even groups of devices. Moreover, with today's sophisticated multi-source, multi-target attacks occurring in mere seconds, without an electronic 'force multiplier' there is a significant gap between the level of threat an organization faces and its ability to respond.

Agents are the point of entry for security data in a security management system and are deployed through the network infrastructure to collect and transmit information. Agents are necessary because there are no standards or common protocols to communicate information about a perceived threat or attack. Each outpost or security sentry (firewall, IDS, VPN, anti-virus program, etc.) generates alarms and alerts in a unique format and language. Moreover, the devices were not designed with the idea that they are part of a security ecosystem that requires a common language plus very efficient communication mechanisms to make the data available to a consolidated global analysis and investigation system.

Agents are part of a multi-tier security management system that also includes a manager that acquires and stores events from across the network, analyzes the traffic for correlated patterns of behavior, displays events for review and analysis, and manages the database of collected events. Therefore, agents in a robust security management system have three important roles:

1. Capture and securely transmit the information to the manager for
    storage and subsequent processing.
2. Translate raw security data into a common format.
3. Optimize resource utilization.

Role 1. Capture and Transmit

There are two sides to the communication process. One is device facing and the other is manager facing. If a security management application does not provide data-collection agents, the burden falls on the organization to collect and transmit security events to initiate the management process.

Because of the diversity of networking architectures, communication protocols and organization policies, it is extremely important that agents can both accept and transmit data in a variety of deployment scenarios. Security devices offer interfaces that range from syslog and database collection to real-time SNMP traps and proprietary interfaces such as OPSEC and Post Office. There is no single interface strategy that will satisfy an organization since inevitably more than one of these interfaces exists in the security infrastructure. A simplified diagram of the typical agent deployment options is shown in Figure 1

Figure 1. Security management agent deployment options

Once bandwidth is used in the best possible way, then the next step is to secure the data transmission with a carefully considered range of encryption options.

Keys to success:

  • All data paths within the security management system are protected.
  • A full range of agent deployment options and protocols are supported.
  • Agents can physically reside wherever the organization requires them.

Role 2. Translate and Normalize

Agents are the synchronizing interface between the raw alarms and alerts that come from security-relevant sources and the analysis and incident response functions that comprise the defense mechanisms for the organization. For example, in order to infer a relationship between an event coming from the firewall and another coming from an IDS, the automated correlation engine looking for such connections must see the data in a common format. This means that the timestamp, attack signature, level of severity, source address, target address, etc. are always in the same place in the message. Hence, a key task for the agent is to translate unique data streams into a common schema.

Keys to success:

  • All agents share a pre-defined schema.
  • 100 percent of the raw event data is captured 100 percent of the time.
  • The normalization is done as close to the source as possible.

Role 3. Optimize Resources

In addition to data translation, if the agent is designed with enough intelligence, it is a very effective mechanism for interpreting and managing event flow to preserve network bandwidth and downstream database utilization. Since an efficient deployment allows an agent to see each event at the source, there is a great opportunity to improve the efficiency of the downstream communication and processing.

A primary objective is to reduce network traffic. This is done by providing options in the agent setup to do intelligent batching and aggregation of events. Instead of immediately communicating events as they are received, sophisticated agents can be set up to send them out in batches at pre-determined time intervals. Agents that feature aggregation will count the number of identical events that occur over a period of time and send only one complete message with a count. An agent filtering function can eliminate useless noise, which further reduces network traffic.

Keys to success:

  • The agents have enough intelligence to optimize network usage.
  • Event analysis is begun immediately.
  • The agent itself consumes minimal resources.


Until the highly unlikely event occurs that there are a standard set of interfaces, communication protocols and deployment options defined, agreed to and implemented, agents will be an indispensable element to any enterprise security management strategy and implementation. However, not all agents are created equal and not all security management solutions use them to maximum advantage. Unlike the starring roles of James Bond and Austin Powers, today's best security agents toil unobtrusively yet intelligently in the background in support of the higher profile security analysis, investigation, resolution and support functions. Agents may not lead the exciting lives they once did, but their function is no less important.

Larry Lunetta is vice president of marketing for ArcSight ( a provider of enterprise software solutions.


Get daily email updates

SC Media's daily must-read of the most current and pressing daily news

By clicking the Subscribe button below, you agree to SC Media Terms and Conditions and Privacy Policy.