Since 2018, MITRE has been testing top endpoint security products by simulating how well each defends against some of the world's top threat actors. Using the MITRE ATT&CK framework, the performance of each product is analyzed at every step of a well-understood attack kill chain, and the results are publicly posted on the MITRE website for anyone to view and use at no cost.
These evaluations, formally called the MITRE Engenuity ATT&CK Evaluations but referred to as "Evals," let any organization considering new security products see how well some of the top vendors did. Nearly 40 vendors have participated in the Evals, and 30 did so in the most recent round, which included Cisco, CrowdStrike, McAfee, Microsoft and Symantec. Their products were pitted against simulated attacks by the Wizard Spider financially motivated group and the Sandworm state-sponsored group.
MITRE does not declare winners or losers in the evals. There is no ranking system. MITRE wants potential endpoint-security-product customers not to see how well a particular product works, but how it works.
This is useful because a good CISO will know their organization’s security posture as mapped against the ATT&CK framework. That CISO can then use each product's evaluation result to discern which product might be the best fit for their organization.
As one of the world's leading security providers, Cisco has participated in the last two rounds of MITRE Engenuity evals and plans to continue doing so.
Its reasons for participating are "not altogether altruistic," said Cisco Secure Endpoint Lead Technical Engineer Eric Howard. As the Engenuity evals grow in scope and influence, it becomes more necessary for a top-ranked vendor to take part.
"Participation alone gets us a seat at the table," said Project Manager for Cisco Secure Endpoint Shyue Hong Chuang. But, he added, it "then allows us to articulate the values that a product brings to the customer."
Those values hint at the second reason Cisco takes part in the MITRE Engenuity Evals. The MITRE ATT&CK framework has created a common set of guidelines and references through which security practitioners can analyze attacks, threats, detections and responses. It also created a common language through which these practitioners can communicate with each other across organizations.
Perhaps just as importantly, the MITRE ATT&CK framework provides CISOs and security managers with tangible, unbiased data that they can show to their company C-suites.
"Most CISOs will ask for investments and increases in budget to respond to either current events or long-standing security concerns, but they don't have sufficient data points to support the ask," said Dr. Joel Fulton, co-founder and CEO of asset-discovery company Lucidum. "By using the MITRE ATT&CK framework as a guide for these conversations, CISOs will be able to effectively explain the severity of threats and the actions to mitigate them while allowing CIOs to be active participants."
For Cisco, the Engenuity evals extend the reach of that common language. Even if a CISO's organization is not considering any new security vendors, the CISO can use the detailed ATT&CK-based analyses of well-known threat actors' kill chains that are part of the Evals results to see how well their security team would have handled the threat.
"Here is a true-to-form attack in sequence with the kill chain," said Cisco Secure Endpoint Senior Product Marketing Manager Adam Tomeo. "At this point, regardless of where you can potentially stop it on the kill chain, you can leverage each one of these sub-steps to help strengthen your security posture in your organization."
To Cisco's Eric Howard, the MITRE Engenuity ATT&CK evaluation results provide an essential tool for any security team to learn from and train against.
The Evals create "a common language between both those that know how to test an environment and those that are tasked with defending against the things that are thrown at an environment," Howard says. "Red and blue teams can now speak the same language, reversing the power of the Babel effect so that we can get to the same goal."
To learn more about how your team can benefit from the Evals results, read our report, "MITRE Engenuity ATT&CK: What it is and how to use it for stronger security posture.”
