← Back to list
Security & Privacy
#위협헌팅#Pyramid of Pain#MITRE ATT&CK#PEAK#탐지엔지니어링
Last updated · 2026-11-10

Threat Hunting and Hypothesis-Driven Proactive Detection

1. Overview

Definition: Threat Hunting is a human-led, proactive and iterative detection activity in which analysts assume that an undetected compromise—one that has not yet triggered an alert—already exists inside the network (Assume Breach), form a hypothesis, and actively sift through telemetry data to find the attacker's traces (TTPs and artifacts).

Threat hunting emerged from two structural problems: the "limits of detection" and "dwell time." Traditional security stacks such as SIEM, IDS, and antivirus assume "known-bad"—an alert fires only when there is an already-confirmed malicious hash, IP, or signature. However, targeted attacks (APTs), fileless attacks, and Living-off-the-Land (LotL) techniques that abuse legitimate tools leave no signatures and blend into normal traffic, slipping past automated detection. As a result, dwell time—the weeks or months it takes to recognize a compromise after it occurs—arises. According to Mandiant M-Trends 2024, the global median dwell time for 2023 improved markedly to 10 days, yet the fact that a substantial share of intrusions are still discovered via external (third-party) notification exposes the gap in internal detection.

The LotL technique shows the need for threat hunting most dramatically. When an attacker performs reconnaissance, persistence, and exfiltration using only OS built-in tools such as powershell.exe, wmic.exe, and certutil.exe, these are all legitimately signed programs, so antivirus and signature-based detection do not react at all. Ultimately one must judge not "was this tool executed" but "was this tool executed in an unusual context," which inherently requires human analysis that reads context and intent. This is precisely the boundary where automated detection and threat hunting diverge.

The paradigm shift that fills this gap is threat hunting. If traditional detection is a reactive model—"respond when an alert comes"—threat hunting is a proactive model: "assume they have already breached even without an alert, and go find them first." The core premises are threefold. First, there is no perfect prevention—perimeter defenses will inevitably be breached. Second, a compromise leaves traces—an attacker inevitably leaves abnormal patterns in process, network, and authentication logs. Third, human intuition is required—judgment that reads context and intent is not fully replaceable by automation alone. In other words, threat hunting is an operating model that draws on the analyst's ability to form hypotheses and the organization's visibility as its resources.

Threat hunting also differs from incident response (IR). IR is a post-event activity that cleans up an already-confirmed incident, whereas threat hunting is a preemptive activity that finds latent threats "before an incident is declared." The output of a successful hunt is not merely finding one compromise; it lies in converting the detection gaps discovered along the way into new automated detection rules (Detection Engineering) so that machines catch them from then on. In this sense, threat hunting should be understood as a feedback engine that continuously raises the SOC's detection capability.

2. The Overall Structure of Threat Hunting and the Hunting Loop

Threat hunting is not improvised exploration but an iterative cycle of hypothesis → data investigation → pattern discovery → automation. The diagram below shows the overall structure from inputs (threat intelligence, understanding the environment) to outputs (new detections, response linkage).

flowchart TB
  subgraph IN["Input (Triggers)"]
    TI["Threat Intelligence (TTP)"]
    BL["Baseline / Anomaly"]
    NEW["New Vuln / Incident"]
  end
  subgraph LOOP["Hunting Loop (Analysis)"]
    HYP["1.Hypothesis"]
    DATA["2.Data Investigation (query)"]
    TECH["3.Techniques (stacking/clustering)"]
    FIND["4.Pattern / Breach Found"]
  end
  subgraph OUT["Output (Value)"]
    DET["New Detection (Sigma/EDR)"]
    IR["Incident Response (IR)"]
    KB["Knowledge (ATT&CK coverage)"]
  end
  TI --> HYP
  BL --> HYP
  NEW --> HYP
  HYP --> DATA --> TECH --> FIND
  FIND -->|"breach confirmed"| IR
  FIND -->|"repeatable"| DET
  FIND --> KB
  KB -.->|"improve next hunt"| HYP
  DET -.->|"hand to automation"| BL

The starting point of this structure is the hypothesis. A hypothesis is the blueprint of a hunt, defining "what, where, and how to look." A good hypothesis must be testable, grounded in the organization's environment and data, and specifically tied to a particular attacker behavior (TTP). For example, the hypothesis "an attacker will load an unsigned DLL via rundll32.exe to evade defenses" can lead to concrete investigation querying abnormal parent-child relationships and network connections of rundll32 in process-creation logs. By contrast, a vague hypothesis such as "there is probably a hack in our network" cannot be verified and thus cannot serve as a hunt's starting point.

Once a hypothesis is set, the process moves to the data investigation stage. Visibility is decisive here, because even an excellent hypothesis cannot be verified if the relevant logs are not collected. Key data sources include endpoint process-creation logs (e.g., Windows Sysmon Event ID 1), network flow (NetFlow) and DNS query logs, authentication logs, PowerShell script-block logs, and cloud audit logs. Next, in the technique application stage, analysis methods such as stacking, clustering, and grouping (described later) narrow outliers down from a massive dataset. Finally, in the pattern discovery stage, if the hypothesis is true (a compromise is confirmed) it proceeds to incident response; if false it is recorded as a valuable result confirming "that threat is absent," and if a repeatable detection condition is found it is always handed over to an automated detection rule.

A design principle worth noting in this loop is feedback (the dotted arrows). The knowledge outcome improves the next hunt's hypothesis, and new detection rules refine the baseline's definition of "normal." That is, each turn of the hunting loop is a cumulative virtuous cycle that raises the organization's detection level by one step. For instance, if a financial institution hunts on the hypothesis "abnormal lateral movement via Remote Desktop (RDP)" and discovers the misuse of a specific jump server, that condition is immediately codified into an automated detection, while the "normal RDP usage pattern" baseline is updated so that even finer deviations are caught next time. In this way, the output of threat hunting should accumulate not as a one-off catch but as a permanent increment of detection assets.

3. Hunting Types and the Pyramid of Pain

Threat hunting is divided into three types depending on what triggers the hunt. The PEAK framework (Prepare, Execute, and Act with Knowledge) proposed by Splunk SURGe in 2023 explicitly distinguishes these three types. The table below summarizes each type's trigger and characteristics, while the practical implications of each type are described in the paragraph below the table.

Type Trigger (starting point) Representative method Suitable situation
Hypothesis-driven TTP from threat intelligence ATT&CK mapping, IOC expansion Tracking a specific actor/campaign
Baseline Defining "normal" and deviations Frequency analysis, anomaly detection Unknown threats, insiders
Model-Assisted (M-ATH) Clusters/scores from an ML model Clustering, unsupervised learning High-volume, high-dimensional data

Hypothesis-driven hunting starts from threat intelligence. For example, upon receiving a report that a specific APT group achieves persistence via a scheduled task, the hunter searches for similar task-scheduler creation logs in its own environment. This approach is efficient because the target is clear, but being dependent on known TTPs, it is weak against entirely novel attacks. Baseline hunting first defines a "normal state" and chases deviations from it. For example, if a baseline is set that a particular server account logs in only during business hours, a successful login at 3 a.m. immediately becomes an investigation target. This approach is strong against unknown and insider threats but produces many false positives in environments where the range of "normal" is broad. Model-Assisted hunting (M-ATH) uses machine learning to cluster millions of events and automatically surface rare groups, which a human then interprets. It is effective as data grows vast, but model explainability and label acquisition are challenges.

The theoretical compass for prioritizing hunts is the Pyramid of Pain, proposed by David Bianco in 2013. It is a model that expresses, as a hierarchy, the size of the "pain (cost of evasion)" an attacker feels when a given indicator is detected or blocked. The diagram below shows the pyramid's layered structure.

flowchart TB
  TTP["TTPs (Tactics/Techniques/Procedures) — max pain"]
  TOOL["Tools"]
  ART["Network/Host Artifacts"]
  DOM["Domain Names"]
  IP["IP Addresses"]
  HASH["Hash Values — trivial pain"]
  TTP --> TOOL --> ART --> DOM --> IP --> HASH

This pyramid explains "why IOC sharing alone is insufficient." The malicious hash and IP lists provided by threat intelligence feeds grow stale the moment they are received, because attackers swap dozens of C2 domains even within a single day. Thus an organization that relies only on the bottom of the pyramid is always chasing "yesterday's attack." By contrast, an organization that has accumulated detection assets at the TTP level catches the same behavioral pattern even when the attack infrastructure changes, so the detection's lifespan is fundamentally longer.

The hash value at the very bottom of the pyramid is instantly neutralized if the attacker changes even a single bit of the file, so detecting it causes the attacker almost no pain. The IP/domain above it also cause little pain, since attackers can easily swap infrastructure. By contrast, the TTP at the top is the attacker's "behavioral pattern" itself, so being detected on it forces the attacker to change their entire way of operating. For example, if an organization stably detects the technique "inject code into a legitimate process," the attacker must pay the large cost of abandoning the injection technique itself. Therefore mature threat hunting does not stop at tracking easily-changed hashes and IPs (IOCs) but must climb to TTP-centric (IOA, Indicator of Attack) detection. Here the MITRE ATT&CK matrix serves as the common language that systematically classifies and maps TTPs.

4. Core Analysis Techniques and the Hunting Maturity Model

Several formalized data-analysis techniques are used for a hunter to find a few malicious traces in terabyte-scale logs. Searching is the most basic technique, querying artifacts that meet specific conditions. Stacking (frequency analysis) aggregates the occurrence frequency of a specific attribute value to find the "Least Frequency of Occurrence"—for example, a rare service executed on only 2 out of 10,000 company PCs becomes a strong investigation candidate. Grouping bundles whether several weak indicators appear simultaneously on a particular time/host, and clustering automatically groups similar behaviors with statistics or ML. The common strategy of these techniques is to narrow "the statistically outstanding minority in a sea of normal" down to the analyst's review scope.

These techniques are easier to grasp through concrete cases. In baseline hunting to find C2 (command-and-control) communication, a normal user's web access has irregular request intervals and varied session lengths, whereas malware beaconing shows mechanical regularity, such as "transmitting an identical-size packet outbound every 60 seconds with ±2-second jitter." Statistically stacking this subtle regularity (low jitter, constant byte count) can pick out a few beacon candidates from millions of normal-traffic records. As another example, if aggregating the hash of a specific service binary across 10,000 company endpoints yields 9,998 with an identical hash and only 2 with a different hash, those 2 might be a legitimate update but could also be supply-chain tampering or a backdoor, so they inevitably become investigation targets. Thus "rarity" and "abnormal regularity" are the two clues a hunter trusts most.

The level of an organization's threat-hunting capability is again gauged by David Bianco's Hunting Maturity Model (HMM). The table below summarizes the characteristics of each stage from HMM0 to HMM4, while the practical conditions for advancing a stage are explained below the table.

Stage Name Data collection Hunting approach
HMM0 Initial Almost none Relies only on automated alerts
HMM1 Minimal Some IOC-based searching
HMM2 Procedural Much Applies procedures made by others
HMM3 Innovative Much Invents its own new analysis procedures
HMM4 Leading Much Fully automates successful hunts

HMM0–HMM1 organizations are effectively at an alert-response level, and true hunting begins at HMM2. It is worth noting that the essence of the top stage, HMM4, is "automation." That is, a mature organization does not repeat hunting outcomes manually but immediately converts them into automated detection, always investing human creativity in "new unknown threats." This shows that threat hunting and detection engineering are not separate activities but a single virtuous cycle. A successful hunt is codified into Sigma rules or EDR detection logic, and that automated detection in turn contributes to refining the "normal" baseline of baseline hunting.

5. Comparison with Similar Concepts

Threat hunting is often confused with adjacent security activities such as penetration testing, red teaming, BAS, and IR, but its perspective and output are clearly different. The core difference lies in "whether one plays the attacker's role or finds hidden compromises from a defender's perspective." The table below organizes those differences, while the reasons the differences arise are described in the paragraph below the table.

Activity Perspective Purpose Main output
Threat hunting Defender (proactive) Preemptively find latent compromise New detection rules, breach report
Penetration testing Attacker (one-off) Prove exploitability of a vulnerability Vulnerabilities, attack paths
Red teaming Attacker (sustained) Test detection/response capability Detection-gap assessment
BAS Attacker (automated) Continuously validate control effectiveness Control-gap report
Incident response (IR) Defender (post-event) Clean up a confirmed incident Recovery/eradication report

Why this distinction matters in practice becomes clear in budget and staffing allocation. If an organization outsources "security checks" and only repeats penetration testing, vulnerability lists pile up but "attackers already inside" are never discovered. Conversely, if it only hunts and never performs attacker-perspective validation, it cannot systematically grasp the blind spots of its own detection system, and hunting hypotheses become biased. Therefore a mature security organization combines the four activities in a complementary way.

Penetration testing and red teaming are activities that play the attacker's role and actively probe the defense system. Penetration testing is a one-off project proving "whether this vulnerability is actually exploitable," and red teaming more broadly tests the organization's entire detection/response capability over a long period. By contrast, threat hunting does not perform attacks but finds the traces of an attacker who may already be inside, from a defender's perspective. BAS (Breach and Attack Simulation) is close to an automated version of red teaming in that it automatically reproduces known TTPs to validate that controls work. If these "attacker-perspective" activities aim to reveal the holes in the defense system, threat hunting is complementary in that it finds the intruder who has already entered through those holes. In practice, the three activities form a single validation ecosystem: hunters take the detection gaps found by the red team as hypotheses to hunt on, and the results are continuously validated by BAS.

6. Deep Dive — The PEAK Framework and Convergence into Detection Engineering

The most conspicuous change in the threat-hunting field recently is the "industrialization and knowledge-building of ad-hoc hunting." In the past, hunting relied on the tacit knowledge of skilled analysts, so reproducibility was low and performance was hard to measure. To overcome this, Splunk SURGe's David Bianco and Ryan Fetterman published the PEAK framework in 2023. PEAK turns hunting into a repeatable process through three stages: Prepare (topic selection, research, scoping) → Execute (data investigation and hypothesis testing) → Act with Knowledge (automating detections from findings, documenting, and sharing). The final "knowledge" stage is especially important in that it forces every hunt, regardless of its result (whether or not a compromise was found), to leave behind an organizational asset—new detection rules, an ATT&CK coverage map, baseline updates.

PEAK also turns hunting into a measurable activity. In the Prepare stage one specifies the hunt's scope, data sources, and the ATT&CK techniques to validate; in the Execute stage one records elapsed time and query results; and in the knowledge stage one tallies the number of new detections and coverage increase. As this data accumulates, one can statistically grasp "which tactics have the most detection gaps" and "which hypotheses show a high hit rate," enabling the next quarter's hunting investment priorities to be set on an evidence basis. In other words, adopting a framework is the turning point that elevates hunting from an artisan's art to manageable engineering.

Another trend is the combination of threat hunting with Detection Engineering and the introduction of generative AI. The hypotheses and findings an analyst produces are codified into Sigma (a platform-neutral detection-rule standard) and deployed across multiple SIEMs/EDRs, and this process is evolving into "Detection-as-Code," version-controlled and tested like CI/CD. Since 2024, AI-assisted hunting—using LLMs to convert natural-language hypotheses into queries (e.g., SPL, KQL) or to summarize and prioritize suspicious patterns from massive logs—has been spreading rapidly. However, because of LLM hallucination and data-leakage risks, AI is positioned not as a replacement for the analyst's judgment but as an assistive tool that augments hypothesis generation and data summarization. In addition, within the CTEM (Continuous Threat Exposure Management) paradigm proposed by Gartner, threat hunting is increasingly repositioned as a core component responsible for "validation" and securing visibility.

7. Considerations and Implications

From a Professional Engineer's perspective, the following strategic considerations are needed when designing a threat-hunting deployment.

  • Up-front investment in visibility (data) determines success or failure: Threat hunting is only possible "to the extent the available logs allow." Since no hypothesis can be verified without high-quality telemetry such as EDR, Sysmon, and cloud audit logs, before launching a hunting program one must prioritize investment in the scope, retention period, and normalization (schema standardization) of log sources. The trade-off between data-storage cost and detection coverage should be explicitly designed.

  • Define performance metrics as "capability improvement," not "number of breaches": A hunt can be a success that confirms "no threat" even if it finds no compromise. Therefore KPIs should be set not as the number of catches but as ▲ the number of hypotheses tested ▲ the number of new detection rules created ▲ the growth rate of ATT&CK matrix coverage ▲ reduction in dwell time, so operations are not distorted. Getting the metric design wrong makes hunters cling to "easy catches" only.

  • Balance between automation and human expertise: Repeatable hunts should be quickly handed to automated detection (HMM4) to reinvest human time in unknown threats. However, judgment of context and intent remains the human's domain, so over-reliance on AI/automation misses new attack patterns. A layered division of roles—filtering by automation first, then in-depth human analysis—is desirable.

  • Organizational/process establishment and framework adoption: Threat hunting must be a continuous program, not a one-off event. By adopting process frameworks such as PEAK and TaHiTI and the MITRE ATT&CK common language, one secures reproducibility and the ability to hand over, and should institutionalize linkage processes with red teaming, BAS, IR, and CTEM. Small and mid-sized organizations realistically need a hybrid strategy of complementing capability via MDR (Managed Detection and Response) outsourcing while accumulating knowledge of their own environment internally.

  • Legal/ethical boundaries and privacy balance: Threat hunting extensively looks into employees' endpoint, network, and authentication activity, so excessive monitoring can conflict with personal-data-protection and worker-surveillance regulations. One must minimize the scope of data collection to be proportionate to the security purpose, separate and audit log-access rights, and secure legitimacy through a Privacy Impact Assessment (PIA) and internal policies. Since the approach "collect anything for the sake of security" can become another insider threat, the balance design between expanding visibility and protecting privacy is a core task at the Professional Engineer level.

References


In one line: Threat hunting is a proactive, iterative defensive operating model in which, under the premise that "we are already breached," analysts form hypotheses to preemptively find latent compromises and convert those findings into TTP-centric automated detection, continuously raising SOC capability.