In an organization with a few routers or switches (switches have approximately the same requirements), that is not a particularly big deal. But most of today's enterprises have dozens, if not hundreds, of internetworking devices. And switches or routers are not the only devices on the enterprises with configuration maintenance challenges.
Add very rigid regulatory requirements - and the heavy penalties attached for lack of compliance - and you have a very serious set of business and technology drivers. The only reasonable answer to the challenges of compliance, security and configuration management is to automate the tasks. That is what this Group Test is all about.
That said, we saw a very wide variety of capabilities. Some of the products we looked at did very little, but performed very well. Because we do not compare products to each other in our group reviews, that posed a real challenge. What we do, generally, is to compare a product against its own claims and against the general expectations in the marketplace. That is difficult because even vendors don't agree on a common definition of policy management. More confusing yet is that there is a difference between policy management and policy enforcement. Some products do one or the other and some do both.
Generally speaking, we agree that policy management, at minimum, should manage the security aspects of the network consistently with written policies. Those policies could be explicit (specifically dictated by a written policy document) or implicit (derived from the configuration standards themselves). So a product that is not comprehensive - e.g., manages only one aspect of policy, such as being restricted to internetworking devices, or manages only a single or suite of applications - would be slightly deficient in our view since comprehensive enterprise policy management is the goal.
We also would expect that a complete policy management product would address compliance reporting in some manner. The question as to whether a policy management tool also includes policy enforcement is an interesting one. While these are different functions, in the context of compliance they probably should go together in the same tool. So we gave extra consideration to products that did both. However, true policy enforcement really is not common since it requires some explicit enforcement mechanism. For example, a port found to be misconfigured on a router should be reconfigured or, at least, should be reported to a trouble-ticketing system.
Buying policy management tools
This, like most other products, is a function of what you are trying to do and what your enterprise looks like. If you have an enterprise that is heavily based on hundreds of internetworking devices, that is where you should put your emphasis. However, there are some consistencies you should look for no matter what your system looks like.
The product you buy should have robust reporting, preferably in the context of reports designed specifically for compliance reporting. It should facilitate remediation, in some manner or other, and the method should be manageable within the context of your enterprise architecture. Finally, it should address the entire enterprise. That means that it should be easy to manage centrally, should have a robust, easy-to-use policy engine and, if there are agents, they should be easily deployable.
This is a difficult category of security tools because the functionality is not universally well-defined. That does not mean that there are not some great products. In fact, in some cases, we found that several products were so close together in capability that picking a clear winner was very tough.
The best advice I can give you is: Look closely at your network and compliance-reporting requirements and select the two or three products that fit the best and are within your budget. Then test them in your environment and make your selection.