Our digital forensics tools test this month was full of surprises. One of the pleasures of doing this job is that we can share these "voyages of discovery" with the SC readership. The first surprise was that in the realm of computer forensic tools very little has changed since we tested the same category last year. After those reviews, some readers attacked our results with religious fervour. Those we favoured fought back with equal zeal, leaving me to believe that beauty really is in the eye of the beholder and these core computer forensics tools are all equally competent. It's just the users who are different. The users become enamoured of a particular tool and will fight for that choice. What amazed me, however, was that the vendors have done very little to put some distance between themselves and their competitors.
Here, again, however, there were some unexpected discoveries. As I was struggling to come up with a theme for digital forensics tools, an idea fell into my lap, almost literally. A colleague brought a bunch of manuals for this month's reviews into my office, dropped them on my desk and asked: "Have you ever heard of these?" Well, of course I had, but each one of the products he brought me had a unique purpose beyond simply analysing a computer's hard disk.
That gave me an idea. When we looked at a collection of forensics tools that went beyond straightforward computer forensics last year, we did it in the context of incident response. The common theme in our current batch of products is that each is unique and innovative. So, in addition to looking at the traditional products, this month we also examine specialised digital forensics tools. So, the theme this year is uniqueness and specialised capabilities.
However. that can make it a little hard to review since no two products are exactly alike. What they have in common is that they belong in the digital investigator's tool kit. So, and this is a good time to make this point, each tool in this group is scored on its own merits. This is the way we do things here at the SC Labs. We do not compare products. We have two sets of standards against which we test. One is fairly generic, while the other is product-specific.
Finally, for those of you in the UK vendor community, please remember that admin for all group tests is now handled out of the US SC offices.
HOW WE TEST AND SCORE THE PRODUCTS
Our testing team includes SC Magazine Labs staff, as well as external experts. In our group tests, we look at several products around a common theme.
Generally, we do not compare products to each other. We test and review them within the group based on a predetermined set of standards, which have been compiled from several sources.
The general test process is a set of criteria built around the six review areas (performance, ease of use, features, documentation, support and value for money) and comprises roughly 50 individual criteria in the overall process.
We develop the second set of standards specifically for the group under test and use the Common Criteria (ISO 1548) as a basis for the test plan. Given that we need to give a good picture in 350 words, reviews focus on operational characteristics.
Once the testing is completed, we rate each product according to the results, assign star ratings and, if appropriate "Best Buy" and "Recommended" awards.
Our final conclusions and ratings are subject to the judgement and interpretation of the tester and are validated by the reviewer.
All reviews and tests are reviewed for consistency, correctness and completeness by the technology editor prior to being submitted for publication. Even so, errors, though rare, are possible. If you believe that an error of fact has affected a review of your product, please contact the technology editor directly.
WHAT THE STARS MEAN
Our star ratings indicate how well the product has performed against
each of our test criteria.
These are marked as follows:
- Seriously deficient
- Fails to complete certain basic functions
- Carries out all basic functions to a satisfactory level
- Carries out all basic functions very well
WHAT THE AWARDS MEAN
Best Buy goes to products the SC Lab rates as outstanding. Recommended means the product has shone in a specific area. Lab Approved is awarded to those tools that are extraordinary stand-outs that fit into the SC Lab environment.