Two detections for the same behavior
Impossible travel. Two successful authentications for one identity, from locations whose separation exceeds feasible travel time for the interval between them.
Detection A is a written condition. You can read it. The interval is there, the distance threshold is there, the source fields are named. A reviewer can disagree with it — argue that the interval is too short, that VPN egress addresses will generate this constantly, that the location data source is too noisy to support the claim. The reviewer is arguing with the logic because the logic is present.
Detection B scores authentication sequences and flags the improbable ones. It catches cases Detection A misses, including ones nobody wrote a condition for — gradual session migration, credential reuse timed to fall inside the interval, access patterns that deviate from an identity's own history rather than from an estate-wide threshold. A reviewer opens it and finds a model reference, an input list, and a threshold. The model fires. The reviewer cannot argue with it the same way, because the claim the model is making is not located in that artifact.
Both detections are legitimate. Written rules and model-based detections can operate on the same behavior and produce complementary signal. The asymmetry is not about which one is better. It is about where the intent lives — and whether anyone has written it down.
What a model-based detection actually asserts
A model-based detection makes a specific, real claim: that on data resembling the training distribution, these inputs separate the behavior labeled as interesting from the behavior labeled as ordinary. Every part of that claim can be wrong. The labels can be wrong. The training distribution can stop resembling production. The inputs can be unavailable at inference time.
None of that makes the claim illegitimate. It makes it a claim — one with a specific structure, specific failure modes, and specific evidence requirements. The problem is not that the claim is unverifiable. The problem is that the claim is not located in the artifact the reviewer is reading.
That has a practical consequence: the claim can be written down deliberately, as its own artifact, before review. In many environments, it is not. The reviewer is left with a model reference and a threshold, and the review gate silently becomes a performance check — did the recall figure pass? — rather than a judgment about whether the detection is doing what it was built to do.
NIST SP 800-53 Rev. 5 SI-4 goes beyond simply requiring that monitoring objectives be defined: it mandates that organizations actively monitor information systems, define the scope and purpose of that monitoring, and ensure the resulting capability can be evaluated against stated objectives — not merely that it produces signal (Source: NIST SP 800-53 Rev. 5, https://csrc.nist.gov/pubs/sp/800/53/r5/final). A detection capability that cannot be evaluated against a stated objective is accountable to nothing but its own output volume.
What each gate can still test
A structured detection engineering lifecycle governs how any detection moves from initial hypothesis to production. Typically, this follows a promotion path across six distinct gates — hypothesis, feasibility, draft, review, test, and observation — each establishing a specific property of the detection before it advances. Those gates do not become obsolete when the artifact being evaluated is a model rather than a written rule.
Feasibility barely changes. Observation becomes more important, because it measures output against specification rather than reading logic. The gate that genuinely breaks is review, and it breaks for a specific reason: review is comparison against a stated intent, and when the intent lives in training data and a labeling decision rather than in the artifact, the reviewer has nothing to compare against unless the intent has been authored separately.
The substitution at each gate is concrete — it is not a principle; it is an artifact that somebody produces.
| Gate | What it establishes for a written rule | What it can still establish for a model | What has to be substituted |
|---|---|---|---|
| Hypothesis | The falsifiable claim about the environment is written and reviewable | The claim exists but lives in the training objective and label definition, not in the artifact | An explicitly authored intent statement: what behavior this model is meant to separate, and from what |
| Feasibility | The required fields, scope, and retention are confirmed present | Identical and unchanged, plus the feature inputs must exist at inference time, not only in training | A documented inference-time data contract distinct from the training-time one |
| Draft | The logic can be read against the hypothesis | The logic cannot be read; the training data, labels, and objective can be | A recorded account of label provenance and what counted as the positive class |
| Review | An independent judgment agrees the logic implements the hypothesis | An independent judgment can assess the intent statement, the label definition, and the evaluation method — but not the decision boundary | Review of the evaluation design rather than of the decision logic |
| Test | The detection fires on generated behavior and its fields exist in production schema | The same, plus performance measured on data the model did not train on | A held-out evaluation with its population and period stated explicitly — not an accuracy figure |
| Observation | The false positive rate against real traffic matches the specification | The same — and this is the gate that changes least, because it measures output not logic | Nothing; this gate carries more weight than it did |
The review row deserves emphasis. An independent reviewer looking at a written rule is comparing logic to intent. An independent reviewer looking at a model-based detection, without the authored intent statement, is looking at a threshold and a performance figure and has no basis for the comparison the gate requires. Writing the intent statement is not a workaround — it is what makes the gate function as designed.
The tuning difference
Tuning a written rule is editing a condition. The edit is legible in a diff. If the interval changes from four hours to six, a reviewer can see the change, understand its scope, and argue about whether it is right. The detection's behavior against specific cases is predictable from reading the change.
Tuning a model-based detection means moving a threshold on a score. A threshold change affects the detection's output across every case simultaneously — the diff records that a number changed, but the behavioral consequence of that change is not readable from the diff. Retraining is a larger version of the same problem: the detection's decision boundary shifts, the artifact looks similar, and the behavioral change is not captured by comparing the before and after files.
This creates a documentation requirement that does not exist for written rules: the threshold value and the model version should be recorded in the case record at the time a detection fires, so that a past decision can be evaluated against the detection that actually made it, not against whatever version is current. Without that record, a retrospective — why did this fire, why did this not — cannot be answered with confidence. While broader operational workflows govern routine tuning cadence and ownership, the critical factor here is what tuning actually means when the parameter is a score threshold rather than an explicit condition, because that distinction dictates what the change record must contain.
Where model-based detection earns its place
Written rules require someone to have anticipated the behavior and expressed it as a condition that holds across the estate. For a large class of behaviors, that is feasible. For another class, it is structurally impossible — not because the behavior is exotic, but because the condition cannot be written without a per-entity baseline that moves.
Consider a service account that has accessed the same three systems for eighteen months and then begins querying a domain controller it has never touched. The behavior is anomalous. But the condition that makes it anomalous is not "a service account queried a domain controller" — that fires on half the estate. The condition is "this account, given its own history, accessed something outside its established pattern." That condition cannot be written in advance because the baseline is per-entity and the baseline itself is a moving target. A rule cannot encode it without becoming either a per-entity rule factory or too broad to be useful.
That is the specific class of work model-based detection does that rules structurally cannot. The deciding condition is plain: if the behavior can be stated as a condition that holds across the estate, a written rule is preferable because it is reviewable. If the behavior can only be stated relative to a per-entity moving baseline, a model is performing work a rule cannot replicate. That condition should drive the choice at the hypothesis gate — not capability availability, tooling familiarity, or throughput.
What this does not cover
Model-based detection does not reduce the telemetry dependency. A model cannot infer an event from data that was never collected. The feasibility gate is unchanged — if the required fields are not present at inference time, the detection does not function, regardless of how well it performed in training. A documented inference-time data contract, as the table above requires, makes this failure mode visible before deployment rather than after.
Explainability features describe a decision after it has been made. That output is not the same artifact as a stated intent authored before deployment. An explanation of why the model scored a given session highly is useful for triage. It does not substitute for the intent statement that the review gate requires, because it describes the output of the model, not the goal the model was built to meet.
None of this is a maturity ranking. A detection programme running written rules with functioning review gates — logic readable, intent stated, evaluation designed — is in better operational condition than one running models whose evaluation nobody has inspected and whose intent was never authored. The measure of a detection programme is not whether it uses models; it is whether a reviewer can establish what any given detection was built to do and evaluate whether it is doing it.
One property specific to model-based detection warrants a reference rather than a section: a trained model's decision boundary is itself a documented attack surface. NIST AI 100-2 E2025 classifies evasion attacks — in which an adversary modifies inputs at inference time to produce an incorrect output — as a distinct category of attack against deployed AI systems (Source: NIST AI 100-2 E2025 Adversarial Machine Learning taxonomy, https://csrc.nist.gov/pubs/ai/100/2/e2025/final).
That property changes what the test gate can establish: a detection that performs well against benign variation in test data may perform differently against inputs crafted to cross the boundary. The consequence for detection engineering is that the observation gate's weight increases, because output measurement against real traffic is the only gate that observes the model's behavior under production conditions.
The detection review test
Take the most recently deployed model-based detection in your environment. Ask for the sentence — one sentence — that states what it was built to separate, and from what.
If the answer describes the model's inputs, cites its accuracy, or refers to its performance on the training set, the review gate was passed against a performance figure rather than an intent. A detection evaluated only by whether it fires has no basis for review, investigation prioritization, or retirement.
The sentence does not have to be in the model artifact. It has to exist somewhere, authored by a person, before review. If it does not exist, that is the gap the next gate cycle closes — not by replacing the model, but by writing down what the model was built to do.
Sources
- NIST SP 800-53 Rev. 5: https://csrc.nist.gov/pubs/sp/800/53/r5/final
- NIST AI 100-2 E2025 (Adversarial Machine Learning taxonomy): https://csrc.nist.gov/pubs/ai/100/2/e2025/final
