As the industry increasingly moves towards the adoption of new preventative security products, an interesting debate has arisen as a consequence.
This concerns the best method of detecting and preventing malicious activity - behavioral rules or signatures.
Prevention is increasingly becoming the new buzzword in the security industry. Many security products detect malicious attacks but very few actually take preventative action to deal with them. However, within the field of preventative security products, opinion is divided over the best way to detect and then prevent hacking activity: signatures or behavioral rules. Each approach has advantages and disadvantages, but it is only by combining the two approaches that companies can ensure that their servers are fully protected.
Signatures
A signature is a specific description of a known attack - a pattern of characters that can be matched against a data stream. A simple example might be a signature that looks for the string '../' in an http request to a web server. If the signature matches the characters in the request, the signature triggers a response from the security product. The signature-based approach accounts for the vast majority of detection schemes currently in use by anti-virus (AV) and intrusion detection system (IDS) vendors. Once a new threat has been discovered, a specific response or signature is written to ensure that the system can recognize and deal with the attack if it occurs again.
The advantage of a signature-based approach is that it enables the security administrator to specifically identify an attack. Without this exact information, it is difficult for a security administrator to know how to mitigate threats associated with the attack. The administrator needs this forensic information in order to put layered preventative measures in place to block future attacks.
Yet one important problem is that signatures have the same limitations as a patch - it is not possible to write the signature until the hack has materialized and been successful. By that time, it can be too late, as early attack targets will have already been compromised.
This presents security managers with a significant problem - it's almost impossible to predict what hacks or viruses will occur next. On the one hand, it is possible to say that one particular piece of software protects against all known vulnerabilities. On the other, true security requires software that can deal with the unexpected - and it is entirely possible there are an infinite number of vulnerabilities out there, it's just that they haven't all been discovered yet. By using signatures on their own, companies can't hope to prevent against something which has not yet been identified.
If security systems rely on signatures alone, it is vital that the library of signatures contains a comprehensive and up-to-date list of malicious attacks. We are constantly reminded of the flaw in this approach as new viruses and attack methods take down systems worldwide before IDS and AV vendors investigate and deliver a workable patch.
Rules
A behavioral rule defines a profile of legitimate activity. Any activity that does not match the profile is considered anomalous and will cause the security product to be triggered. (The desired response can be pre-set by the security administrator.) As rules are not specific to a particular type of attack, they can block malicious behavior without having to recognize the precise attack used. The security administrator therefore, has additional protection against new attacks as they emerge. Unlike a signature, a rule is not concerned with identification; it is only concerned with intent. A rule, for example, could set up a barrier to prevent any changes being made to the registry. This would effectively prevent installation of any unauthorized applications, including malicious code and Trojans.
A rules-based approach allows companies to place what is effectively a shield around an operating system or application. It could also be used to shield a web server. In this case it would mean that no one could access the web server to change the files - automatically limiting the risk of a hack. The benefit is that the risk of web site defacement practically disappears. But the potential downside is that no one, not even administrative staff, can access it to make legitimate changes.
But for every rule, there is always an exception...
In order to allow the web server content to be changed, it is possible to create an exception. This will allow a known user access to the web server as long as the conditions within the exception are met: the user must be 1) logged on as administrator, 2) using defined software, 3) from a specific IP address.
As the rule is not specific to a particular type of attack, it can identify malicious behavior without having to know what specific attack was used. The weakness inherent in the rules based approach is that it is not able to detect and report the specific forensics information about an individual attacker or threat. This data is important in identifying the attack and allowing the administrator to implement corrective patching and countermeasures.
Strength in combination
We have already seen that signatures on their own are not a satisfactory defense - a signature can only be written after the attack has been deployed. The other school of thought dictates that a rules-based approach is satisfactory. After all, the rules themselves (and their exceptions) should be capable of stopping all attacks, keeping servers secure.
However, for true intrusion prevention, companies need to adopt the defense in-depth method of protection, which uses a mix of signatures and rules. The behavioral rules allow the network to be protected from new, previously unknown attacks, and for full forensics capability, the signature is critical in identifying known attacks so that security managers know what sort of a hack was being directed at their system.
To be taken seriously in the security market, it is imperative that security software can deliver full and accurate information on the attack to prevent it happening again. Prevention is seriously devalued and, one might go so far as to say next to useless, unless the security manager has full information on attacks and why they have been blocked. Regular signature updates provide managers with that information so that they have the choice and option to go back and harden the system further. So, rules and signatures in combination with an effective prevention engine working at OS level can effectively prevent attacks before they execute.
Iain Franklin is European vice president of Entercept Security Technologies, intrusion prevention specialists (www.entercept.com).
