I frequently get questions about my concept of threat hunting so in this blog I'll take a look at that topic. We'll return to the analysis of particular events in our next outing, but for now let's examine how we do threat hunting, interdiction of malicious actions and post-event forensics.
For our purposes - not a generalized approach, by the way - we will look at three threat hunting tools: enterprise threat hunting tools such as Sqrrl, endpoint protection tools, and threat analysis tools such as Packetsled. These, of course, aren't the only tools available to the threat hunter - I probably use around 20 in various combinations - but they make a good focus for this blog because Sqrrl for example, is great for internal threats and Packetsled is great for external threats.
Before you get too focused on that statement, both overlap a lot. Sqrrl can find the kinds of outliers that suggest external behavior - such as command and control interactions - and if you have Packetsled you really must have internal sensors. Where Packetsled excels is hunting on the network and where Sqrrl excels is mapping threat-related activity in the enterprise. While we're at it, we'll add one more tool, generic this time, to view activity on the end-points.
Our premise - and the way I do threat hunting - is that no single tool is enough and, while endpoint protection is critical, it does not give you the depth of information that allows you to proactively address threats, especially those about which you know nothing until it is - maybe - too late.
One of my favorite analysis tools is the SIEM and one of my favorite SIEMS lets me integrate behavioral analysis my way. Remember, SIEMS don't capture data. They correlate and analyze data that other tools have captured and then give up their assessment under a single pane of glass. So, here in the lab we bring everything except Packetsled into the SIEM and because we are a lab and not a production environment, we do not have Sqrrl deployed. But if we did, we would have three separate visualizations: Sqrrl, the SIEM and Packetsled.
It used to be that the SOC engineer was the focal point of threat management in the enterprise. That is no longer true. Now, larger organizations also have threat analysts that do exactly what their job title suggests: they analyze threats. That leaves the SOC engineers free to respond to threats and to conduct incident response with intelligence they gain from the analysts. Network engineers coordinate with both teams.
And, that brings up another point that really is of critical importance if you want to succeed: these three teams - SOC, analysts and network management - must work closely together. In fact, I recommend that the wall between the SOC and the NOC be torn down and the SOC and NOC teams work together in a common environment. The analysts should be in the same room as well because the enrichment provided by their analysis and intelligence feeds is the glue that holds the pieces together in a good threat hunt.
Back to how I threat hunt in the hope that you will get a hint or two that can help you. I agree with Sqrrl that most threat hunting takes place in the enterprise. Note that I said, "...takes place in..." not, "...should take place in...". This simply is a matter of perspective. I view the Internet as a huge enterprise. That means that smaller enterprises - such as organizational ones - are branches off of the larger whole. Those branches are, presumably, protected from intrusions. However, there are many factors that impact that protection. Some are internal vulnerabilities, some are user error and some are revealed by external monitoring and intelligence.
This is where my combination of tools comes into play. The tools in my kit work together to do two things: protect the internal enterprise (in my case, it's a deception network), and enable analysis and understanding of the threats coming at me (or, potentially coming at me).
Sqrrl defines the threat hunting process quite well in their Hunting Loop: Create a Hypothesis -> Investigate Via Tools and Techniques -> Uncover New Patterns and TTPs -> Inform and Enrich Analytics. This loop then feeds back to itself (create, or, perhaps, modify a hypothesis) and continues the cycle. Unfortunately, endpoint protection does not provide this depth of analysis and, usually, is largely concerned with anomalous behavior. On occasion, endpoint tools do some enrichment using threat feeds and machine learning, but their mission is identifying anomalous behavior and taking protective action. Thus, we can say that they are necessary but not sufficient.
Tools such as the SIEM and Packetsled provide the enrichment that triggers early warnings and proactive action to prevent, rather that react to, breaches. Are there other tools that do what Packetsled does? Of course. It so happens that in my lab I have found that the combination of Packetsled with my other tools offers the optimum mix of threat hunting and anomaly detection and response.
The point is that endpoint protection, no matter how complete, is limited it its ability to be proactive by its mission which is to interdict threats and take protective action. Along with that part of its mission, most endpoint tools also product a significant collection of forensic data that is necessary to understand either a successful attack or one that was stopped in process before it could be successful. There is a bottom line here, of course.
In any event - we learn this from successful criminal investigators - there are three stages: pre-event, event, and post-event. We need tools that can describe each of these stages in detail. It's important that these tools have overlapping functionality so that we can see a continuum of steps, often called the chain of evidence. It is this chain of events leading up to and following the main event that enable us to develop appropriate protection.
--Dr.S
