Product Group Tests

Patch management (2006)

Group Summary

It is difficult to choose between the products reviewed in this test, as none stands out head and shoulders above the rest. All opf them will do a job depending on your particular circumstances. However, for our Best Buy, it is difficult to look beyond the enterprise capabilities of PatchLink Update, while the simplicity of Shavlik NetChk Protect, when combined with its thoroughness and attention to detail, earns it our Recommended award.

Scroll To Full Group Summary Below

Click for a side by side comparison of products
Click for a side by side comparison of products

Full Group Summary

The recent WMF Windows vulnerability emphasised just how important the concept of software patching has become and underscored the need for all firms to manage this process in a systematic and timely way. Keeping up with software security and other patches is now a top priority for firms of all sizes.

When the exploit was announced, organisations were faced with an immediate problem: whether to use the first patch out, from a software developer called Ilfak Guilfanov, or wait for Microsoft’s official response. In the end, it all came down to the individual judgement of organisations.

Luckily, the developer provided the source code and an easy means of uninstalling the patch, and several security groups analysed the patch to make sure it actually worked and was safe, but it shows just how much patch management is down to judging what to do from the information available.

Patch management products are more than just a handy application for automating patching across the enterprise. All the products on test have behind them armies of researchers that look at each patch and how it affects the systems upon which it is to be installed.

Of course, the likes of Microsoft have their own facilities where they test patches across a wide range of different servers, desktops, appliances and operating systems. When a patch gets the all clear, the next stop is for them to put it into the monthly update cycle.

Because the WMF flaw was so critical, Microsoft had to release the patch out of this cycle, when most people had just gone back to work from the holidays. This, as well as the news of exploits circulating the internet, meant that security professionals had to put in overtime immediately to test patches and make sure, not only that they would not disrupt their systems, but that they wouldn’t cause damage to any bespoke applications running on them.

The patch management firms were busy doing their own testing before streaming this information into their products, and this is one of the many things that we looked at in this group test.

We also tested how the products worked with different systems. Patch management products need to work across different OSs from a variety of vendors. Threats are not restricted to the Microsoft environment, and many organisations have embraced other OSs. Products scored higher if they could cope with a range of environments, as this better reflects the heterogenous nature of many real-world organisations.

As always, we spent a lot of time scrutinising the ease of installation and configuration. We needed to be sure that any of the products under analysis could be integrated into existing security structures and work with enforced policies in those infrastructures. If, for some reason, it has to be installed outside that infrastructure, we tested how the product’s own security stood up under testing.

We also examined where the products used agents. We looked at how easy it was to install agents, how well they could be managed, and how well they ran themselves. Ideally, they should cause as little overhead as possible.

This is very important in larger networks, and patch management applications should take up as little bandwidth as possible in order to perform their functions as efficiently as possible. Agentless products need to be bandwidth efficient in the way they scan and remediate target systems, and we were pleased to see that CPU usage flexibility was evident.

The products should also provide the administrator with as much information as possible about what the patch fixed and, when possible, how.

Also, we looked for products that could manage the patch process in order to minimise downtime. This is of particular importance to environments where downtime is measured in terms of lost revenue.

We also tested how the software coped when new devices connected to the network, particularly how a product integrated with endpoint security systems – although with endpoint security still relatively immature, we were not expecting too much from the products in this particular. In future tests, we will not be so lenient on this.

Patching cannot be the be-all and end-all of security. It can only help mitigate against known threats. As a result, it should be used as part of an overall strategy that encompasses securing the endpoint and detecting anomalous behaviour before it manifests itself in a full-blown attack.

Any product that shines a light on your corporate infrastructure can help show up weaknesses. So use them not only to catch up, but also to spot problems and fix them before the next hit.

All Products In This Group Test