As of a couple of weeks ago, I ceased to be a director of the Anti-Malware Testing Standards Organization (AMTSO). There's nothing sinister about that. It just happens that ESET recently recruited researcher Righard Zwienenberg, who is president of AMTSO, and one of the bylaws of AMTSO is that a single entity member can't be represented by two directors. Since I wasn't planning to stand for re-election this year, it wasn't difficult to come to a decision as to who would step down from the board. But that wasn't the main event, and nor was the fact that two documents were approved by the membership, and will appear on the AMTSO web site shortly.
Part way through the second day of the workshop, a major shift in direction was proposed. It's no secret that AMTSO's attempts to make testers and reviewers more accountable for the accuracy of their tests and test reports through a "review of reviews" analysis, attempting to assess whether a review was compliant with the organization's Fundamental Principles of Testing, attracted a great deal of attention. Most of that attention was on the whole negative and mistrustful of the motives of the anti-virus industry, of which AMTSO is seen as a mouthpiece (not altogether accurately, but the fact is that vendors do outnumber testers in AMTSO, and the organization's efforts to compensate for possible bias in its voting procedures did not stop two well-known testing organizations from quitting as members, though they remain at present as €25 subscribers). The proposal covers too much ground to summarize in a short article, but a key component is the revival of the idea of tester accountability in a different form. If I understand it correctly, it is a more general review of the testing landscape commissioned from academia.
This is just a proposal. I expect it to excite a great deal of debate at the next AMTSO meeting in May, and I'm not going to attempt to predict what the final outcome will be. Personally, I have no problem with the principle of tester accountability. It seems to me that there is an undercurrent of admission here that AMTSO has failed to convince the world that it's an impartial commentator on testing issues, and needs to channel the undoubted expertise of its participants (vendors and testers) via a credible, trusted third party. The success of this proposal, if adopted, may well depend on how consistently both testers and vendors within the AMTSO community (both members and subscribers) can put the well-being of the community ahead of their own vested interests as commercial organizations. That's not a small ask, but AMTSO cannot afford too many more mistakes.
