How to Hunt Threats with Rare-Event Analysis

TL;DR

Hunt rare events by filtering out behaviors that are common across a large corporate environment, where most network and endpoint activity is expected to be benign. This approach uses the law of large numbers to surface unusual artifacts and machine behaviors, helping analysts work efficiently across millions of IP addresses and billions of daily connections without depending solely on attack-specific indicators.

Transcript

Glad you guys made it out. Um, I'm Vernon Habersetzer. I, uh, lead the hunt team within the incident response portion of Walmart. Um, it's a very humbling job to have. Uh, I wanted to kind of give you a background of myself real quick, and then we'll jump into the material. And, uh, the slides are online, so, you know, don't worry about taking note... Read More

Key Insights

• Most activity in an average private corporate network is expected to be benign, so analyzing more network or endpoint events should produce a baseline increasingly dominated by normal behavior. This assumption allows threat hunters to use the scale of enterprise data as an analytical advantage.
• Rare-event hunting works by deprioritizing highly common activity and investigating behaviors or artifacts that appear infrequently across the environment. The method identifies anomalies before analysts necessarily know which exact threat, malware family, or attack procedure they are seeking.
• Most attacks introduce an artifact that is new or uncommon within the victim's environment. Examples named in the presentation include domains, IP addresses, file hashes, registry values, services, and scheduled jobs, all of which can become useful inputs for rarity analysis.
• Scalable threat hunting must account for limited analyst time as well as enormous data volume. A hunter may have only three to five hours on a good day, despite needing to analyze an environment containing millions of addresses and billions of daily connections.
• Attack-specific detection is an incomplete long-term strategy because adversaries and their individual technical traits continually change. Analysts can spend years writing and tuning detailed logic without completing the task, particularly when attempting to maintain command-and-control detections.
• TTP-based detection is broader than relying solely on indicators such as IP addresses and domains, but hunters should also examine patterns above the level of individual techniques. Broad environmental comparison can reveal suspicious differences without requiring a predefined signature for every attack.
• The law of large numbers provides the conceptual basis for establishing an expected benign baseline. Repeated observations should move the average result closer to its expected value, which the presentation adapts from mathematical experiments to large collections of network and endpoint events.
• Anomaly identification resembles finding the one item that differs from the others in a Sesame Street exercise. The important analytical feature is that a person can recognize an unusual item through comparison without first receiving detailed instructions about the exact anomaly to find.

Questions & Answers

Q: How can rare-event analysis improve threat hunting?

Rare-event analysis improves threat hunting by using common activity as a benign baseline and directing attention toward artifacts or machine behaviors that occur infrequently. Since most events in an average private corporate network should be benign, hunters can deprioritize widespread patterns and investigate unusual domains, IP addresses, hashes, registry values, services, scheduled jobs, and other uncommon observations.

Q: Why should threat hunters focus on uncommon behavior?

Threat hunters should focus on uncommon behavior because most attacks introduce something that the victim's environment has not frequently seen. The unusual element may be a domain, IP address, file hash, registry value, service, scheduled job, or machine behavior. Searching for these differences can expose suspicious activity without requiring analysts to know the exact attack or malware beforehand.

Q: How does the law of large numbers apply to threat hunting?

The law of large numbers says that the average result from many repeated experiments should approach the expected value as more experiments are performed. The presentation adapts this idea to security data: as hunters analyze more network and endpoint events, the overall pattern should increasingly reflect the expected benign behavior of an average private corporate network, making rare observations more visible.

Q: What makes threat hunting difficult in a massive enterprise?

Threat hunting becomes difficult when enormous data volume must be examined with limited analyst time. The environment discussed contains three million IP addresses, 2.4 million employees, and eight billion network connections per day. Meanwhile, meetings, email, and projects may leave a hunter only three to five hours on a good day, creating a strong need for automation and scalable methods.

Q: Why is attack-specific detection difficult to sustain?

Attack-specific detection is difficult to sustain because attacks continually change and each one may introduce different technical traits. Analysts can spend every day writing logic for individual threats without ever finishing. The presenter describes spending three or four years refining command-and-control detection logic and still needing to tune it, illustrating the maintenance burden of highly specific approaches.

Q: Are IP addresses and domains sufficient for threat hunting?

IP addresses and domains are not sufficient as the sole basis for threat hunting because they are granular indicators associated with particular activity and can require constant updating. The presentation recommends stepping back toward techniques and broader environmental patterns. Indicators still have value, but scalable hunting should also compare behavior across systems and identify uncommon activity that transcends specific attacks.

Q: What qualities should a scalable hunting method have?

A scalable hunting method should be sustainable, usable across very large datasets, and capable of transcending the details of individual attacks. It should help analysts automate comparisons, reduce the volume of common benign activity requiring attention, and force adversaries into a smaller set of unusual behaviors where their activity is more likely to become visible during investigation.

Q: How is the Sesame Street comparison relevant to security analysis?

The Sesame Street comparison illustrates anomaly detection through the exercise of identifying which item is unlike the others. A viewer can recognize the differing object without being told its exact characteristics in advance. Threat hunters can apply the same principle by comparing machines, events, or artifacts across an environment and investigating observations that differ from the dominant pattern.

Summary & Key Takeaways

• Large enterprise threat hunting is constrained by scale and limited analyst time. Walmart's environment includes three million IP addresses, 2.4 million employees, and eight billion network connections each day, while a hunter may have only three to five hours available for hunting after meetings, email, and project work.

• Detection based only on IP addresses, domains, malware traits, or specific attack techniques creates an endless maintenance burden. A scalable and sustainable hunting method should transcend individual attacks, examine the environment at a broader level, and make adversaries reveal themselves through behavior that differs from the dominant benign baseline.

• The proposed method adapts the law of large numbers to security events. Because most activity in an average private corporate network should be benign, frequently observed behaviors can be deprioritized while rare domains, hashes, registry values, services, scheduled jobs, IP addresses, and unusual machine behaviors become candidates for investigation.