Coverage and Context: The Key Measures of MITRE ATT&CK 2020

The Quick Read

MITRE has become the common language of EDR and is the de facto way to evaluate a product’s ability to provide actionable information to the SOC. MITRE ATT&CK’s use of APT29, the notorious threat actor that evaded the DNC, shows us that many of today’s EDR tools fail to cope with advanced techniques. CISOs should carefully evaluate which technologies capture the most information and provide context at each stage in MITRE’s simulation. In this post, we discuss SentinelOne’s performance in MITRE’s ATT&CK Round 2 with the following takeaways.

• SentinelOne had the lowest number of missed detections. SentinelOne has proved it provides the widest coverage of the MITRE ATT&CK framework – an EDR is primarily measured by its ability to see, analyze, and react – SentinelOne saw more, and provided more insight and context than any other vendor.
  
• SentinelOne achieved the highest number of combined high-quality detections and the highest number of correlated detections. SentinelOne delivered the highest number of actionable detections – MITRE have identified the successful attribution of activity to their tactics (good) and their techniques (best) as a significant measure of an EDR’s value and ROI. SentinelOne delivered better MITRE attribution than any other vendor.
  
• SentinelOne automatically grouped hundreds of data points over the 3-day test into 11 correlated console alerts. SentinelOne automatically correlates related activity into unified alerts that provide Campaign Level Insight. This reduces the amount of manual effort needed, helps with alert fatigue and significantly lowers the skillset barrier of responding to alerts. SentinelOne aggregates the same amount of visibility into a fraction of the alerts.
• Human-powered MSSP scores must not be a crutch for failures in a software’s ability to detect. SentinelOne had the highest number of product-only detections and – in parallel – the highest number of human-only “MSSP” detections. Having the top scores in both is a good thing and means that the technology itself is robust and can stand alone without a Managed Detection & Response (MDR) service; for SentinelOne, worldclass Vigilance MDR service is optional – providing verification and actioning should customers desire or require such a service.

Great products will catch the eye of CISOs and SecOps professionals if they provide the ability to do more with less, if they make work easier and more interesting for analysts, and if they can be operationalized without adversely affecting the production environment.

Now Read The Full Story…

The latest MITRE ATT&CK results were released Tuesday, April 21, 2020 and as expected interpreting them is an exercise. Consulting MITRE testing is one component an organization can employ to help evaluate cyberthreat preparedness – specifically how well cybersecurity solutions perform in the face of adversaries. 

SentinelOne’s April 20, 2020 blog delves into the rationale and methodology behind MITRE testing and is a helpful compendium for understanding MITRE ATT&CK. It is important to understand that this MITRE test does not test everything in the MITRE framework, but instead focuses on two specific attack flows over three days in a lab. All told there were 135 substeps. 

High-level Remarks

We caution you out of the gate to not believe all of the claims you will read in relation to this test – question everything and check the data. What you can expect from SentinelOne is that we will present the data and be as helpful as we can to enable you to help yourself to interpret what it all means. We will also articulate our value claims and design principles. Thus by intersecting these two realms, we will provide fertile ground upon which you may draw your own conclusions.

1. Seeing Is Believing: Coverage is Table Stakes

The foundation of a superior EDR solution lies in its ability to consume and correlate data at scale in an economic way harnessing the power of the cloud. Every piece of pertinent data should be captured to provide breadth of visibility for the operator. Data, specifically capturing all events, is the building block of EDR.

As the graphs below show, SentinelOne had the fewest misses of all the participants in Round 2. 

2. Context is the Key to Operationalizing Data

We believe that our MITRE results clearly support the SentinelOne platform’s effectiveness in not only identifying and stopping malicious code early and often but also illustrating the extent to which SentinelOne goes in solving the data overload problem. In the MITRE evaluation, “Techniques” and “Tactics” are the key measures of data precision. 

1. Technique: this is the epitome of relevant and actionable data – fully contextualized data points that tell a story, indicating what happened, why it happened, and crucially, how it happened. 

2. Tactic: this is the next level down in the hierarchy, representing categories of techniques that tell us the actor’s steps in achieving their ultimate goals (persistence, data egress, evasions, etc). In short, the ‘what’ and the ‘why’.

These two detection classifications are the core of the MITRE framework and are of the highest value in creating context. According to MITRE’s published results, out of all participants in the Round 2 evaluation, SentinelOne recorded the highest number of “Techniques” and “Tactics” awards. 

To paraphrase Paul Webber, Gartner Endpoint Protection analyst, today’s security products must turn weak signals into strong detections. This is exactly what SentinelOne does as proven by MITRE’s ATT&CK evaluation. SentinelOne as a standalone product is effective in identifying and placing actionable context in the attack. 

As we explained in our MITRE primer post on Monday, correlation is one of the detection modifiers applied to Technique and Tactic detections. Correlation represents the act of building relationships between data, completed at machine speed, so an analyst doesn’t have to manually stitch data together and waste precious time. SentinelOne had the most correlation modifiers in the MITRE ATT&CK Round 2:

Defenders are past the stage where more data is good. Instead, they need context from related data. We believe that the ideal state for a SOC is articulated stories that have all data pre-indexed and assigned to actionable storyline alerts. The entire MITRE ATT&CK Round 2 testing battery is captured in the SentinelOne console with just 11 alerts. 

Our customers don’t want 3,000 pieces of a Do-It-Yourself telemetry jigsaw puzzle, and they don’t want 135 uncontextualized alerts for every attack. What they want is the jigsaw puzzle ready-assembled into a tidy package that makes it easy for analysts to discern what is happening at first glance. 

Our data ingestion capabilities capture data that, coupled with our patented AI engines, assembles storylines with vivid pictures and rich context to apply autonomous actions at machine speed.

3. Great Products Solve Complex Problems – Services Should Always Be Optional

For an extra layer of completeness, or for those who seek to outsource SOC operations, SentinelOne offers optional Vigilance Managed Detect and Respond (MDR) services. The MITRE data as related to SentinelOne proves unequivocally that our technology paired with our global expert MDR analysts provides absolute coverage across the board. 

Join our MITRE webinar to see SentinelOne’s victorious performance against APT29.

Wednesday, April 29th @ 9 am PST

Register Now

To summarize conversations with Josh Zelonis, Forrester’s EDR Wave analyst, buyers should ask and ascertain if MDR is a crutch for the product or if it’s value additive. 

MITRE’s ATT&CK data shows that SentinelOne had the highest level of detections delivered exclusively by the product (without any MDR services); in addition, Vigilance MDR services, operating the SentinelOne product, had the most MSSP detections. This is possible because our agent captures a richer set of events/signals that can be analyzed by our Level 4 Vigilance Ninjas to identify targeted attacks.

The experience for a SentinelOne Vigilance MDR customer would have been a single proactive email or phone call (depending on customer preference) with updates across the 11 alerts.  Even in a detection-only policy with the product taking no actions – which is the MITRE ATT&CK simulation – Vigilance would have stopped and remediated the attack in under 20 minutes. 

It is important to note that SentinelOne’s platform operated independently of its MSSP scoring proving that the tool stands alone but that if extra depth is desired, the tool + MDR provides a deep solution.

MITRE 2020 Results Take Away

Great products, and in this case great security products, will catch the eye of CISOs and SecOps professionals if they do these things:

• Shift work from mundane brain-killing activities to more interesting initiatives
• Integrate with other parts of the security stack
• Automate more of the work
• Defeat adversaries in real time
• Include granular remediation capabilities for automated cleanup and recovery
• Encompass preventative measures that handle garden-variety up to advanced attacks 

SentinelOne’s performance in MITRE ATT&CK Round 2 is a strong statement that visibility and AI when coupled together create a powerful EDR solution. As evidenced in the data output of the simulation, SentinelOne excels at detection, and even more importantly, the autonomous mapping and correlating of data into fully indexed and correlated stories. SentinelOne’s storyline technology, powered by our patented Behavior AI, sets us apart from every other vendor on the market. 

Every CISO we speak with tells us the value they fundamentally seek is a solution that not only sees more but also does more – without adding friction, complexity, or cost. Cyber attackers move quickly, especially advanced adversaries. With SentinelOne, CISOs and their team can trust a performant, cloud-native EDR platform that is proven to stay one step ahead with a product-driven approach to understanding, organizing, and actioning data at machine speed.

To learn more about SentinelOne’s performance in MITRE ATT&CK APT29, join us in the webinar on Wednesday, April 29 at 9AM PST.

Like this article? Follow us on LinkedIn, Twitter, YouTube or Facebook to see the content we post.

Read more about Cyber Security

• The Complete Guide To Understanding MITRE’s 2020 ATT&CK Evaluation
• Return to Base | The CISO’s Guide to Preparing A COVID-19 Exit Strategy
• MBRLocker Wiper Malware | Destructive Pranks Are No Joke For Victims
• Dashboards & Business Intelligence – Feature Spotlight
• The Great Transition: Transforming Your Business To Survive COVID-19
• How Will Cybercriminals Take Advantage of AI in the Future?