Closure codes are chosen to close the case

The rule retired on its own closure data

A detection rule was retired last quarter. The stated reason: seventy percent false-positive rate. The evidence: disposition fields, closed as "false positive" by analysts over six months. Re-reading the closure notes afterward revealed something the aggregate rate did not — four of those closures described activity nobody had been able to attribute. Not benign activity. Unattributed activity, recorded as false positive because that was the field value that ended the task.

The rule is gone. The activity it was detecting may not be.

That sequence is the hypothesis this article defends: the disposition field is produced under a closure incentive and consumed as evidence by three downstream processes — detection tuning, coverage reporting, and staffing decisions — none of which have visibility into the method used to produce it. When the incentive and the consumption are both present, and no verification loop sits between them, the field carries less information than the processes using it assume.

The argument does not require analyst failure. It requires only that the incentive exists and that the field is used as though it does not.

What a disposition field is used for

Disposition data has three concrete consumers, and each one inherits whatever error rate the field carries without seeing it.

Detection tuning uses disposition counts to decide which rules to modify, suppress, or retire. A rule accumulating false-positive closures gets deprioritized or removed. That decision is structurally correct if the closures are accurate. It is structurally invisible if they are not.

Coverage reporting aggregates dispositions to characterize detection effectiveness — how much of the environment is generating actionable signal versus noise. Executives and auditors receive that characterization as a quality metric. When the metric is derived from shallow closures, it reflects closure throughput, not detection quality. The distinction matters when the metric is used to argue that coverage is adequate.

Staffing decisions use disposition distributions to estimate queue load and analyst capacity. A team closing eighty percent of its alerts as false positives looks like a team carrying unnecessary volume. That framing supports arguments to suppress rules or reduce headcount. It also supports arguments to increase headcount, depending on which direction the data is pointed. Either way, the quality of the underlying closures is not visible in the distribution. Detection engineering receives feedback through this same channel.

How the value gets chosen

The analyst is at the end of the task. The queue is visible. The list of available disposition values is short. No value they select will be audited, questioned, or revisited unless the case is escalated — and a case closed as false positive is not escalated. Choosing the value that closes the case is the rational act under those conditions. Framing it as an error misidentifies where the problem sits.

The problem sits in what the disposition value is asked to carry. A disposition is a conclusion. A conclusion has a method behind it. The method has depth and coverage — how many data sources were consulted, how completely the behavior was characterized, whether attribution was attempted or assumed. NIST SP 800-53A Rev. 5 decomposes assessment into determination statements precisely because the existence of a record does not confirm that the underlying work was performed — it confirms that a field was filled (Source: NIST SP 800-53A Rev. 5, https://csrc.nist.gov/pubs/sp/800/53/a/r5/final). The same logic holds in reverse for disposition: a closed case confirms that a field was selected, not that the selection was grounded.

When the method is shallow — a single log source, thirty seconds of review, no cross-referencing — the disposition carries a shallow conclusion wearing a deep label. "False positive" is a claim about a detection's logic. "Benign" is a claim about behavior. "Unattributed" is an honest statement about analyst confidence. Most case-management systems offer one or two of these and collapse the rest into whichever value closes the case fastest.

A better disposition taxonomy does not fix this. A richer list under the same incentive fails the same way — analysts choose the closest available value that ends the task. The incentive is upstream of the vocabulary.

What changes the result is whether the analyst has the data to make the distinction in the first place. That is a data-access problem and a tooling problem before it is a process problem. Disposition quality cannot exceed data quality. If attribution requires correlation across three log sources and the analyst can see one, "false positive" is not a misuse of the field — it is the only honest answer available given the constraints.

The four things false positive absorbs

The field "false positive" is doing more work than its label implies. In practice, it absorbs at least four distinct outcomes — and the table below names them concretely, because the distinction between them determines what the correct downstream action is.

The first row justifies retiring a rule. The fourth row is an incident. The disposition field records them identically.

The operational consequence is direct: any tuning decision made from aggregated false-positive counts is implicitly weighted by the ratio of row one to row four in that population — and that ratio is unknown until someone looks. Detection coverage gaps created by the fourth row will not appear in coverage reports because the cases are closed, not open.

The verification loop that makes the field evidence

The disposition field can be evidence. It is not evidence by default.

The principle that ongoing awareness of security state depends on data that is current and complete applies broadly across security operations environments — whether the program operates in a commercial, private, or federal context. Monitoring decisions made on stale or incomplete data reflect a prior state, not the present one. Disposition data is subject to exactly this dependency: a tuning decision made from six months of closures reflects the accuracy of those closures. If the closures were systematically shallow, the decision reflects a systematic gap, not the detection environment.

This principle is codified for federal systems and organizations in NIST SP 800-137, which establishes continuous monitoring requirements specifically within that scope (Source: NIST SP 800-137 Information Security Continuous Monitoring, https://csrc.nist.gov/pubs/sp/800/137/final). Operators of federal systems should treat that document as directly applicable guidance.

Operators of commercial or private systems will not be subject to its requirements, but the underlying dependency — that monitoring quality is bounded by data quality — holds regardless of regulatory environment. The mechanism described in this article is not federal-specific; the cited standard is.

The correction is a sampling loop, specified concretely. A fixed small sample of closed cases — the appropriate size depends on closure volume, risk tolerance, and available reviewer capacity, but even a modest sample meaningfully exceeds zero — is re-adjudicated weekly by someone who did not close them. The reviewer works from the same available data the original analyst had, reaches an independent disposition, and records whether they agree. The disagreement rate is published — not to blame analysts, but to serve as the confidence interval on any tuning or coverage decision made from the underlying data.

That disagreement rate is the bound the field currently lacks. A tuning decision made from disposition data with a known disagreement rate of, say, fifteen percent carries a different confidence level than one made from data with a known rate of two percent — and both carry more credibility than one made from data with no rate at all, which is the current default.

The cost is honest: reviewer time, a defined sampling method, a place to record disagreement, and someone accountable for publishing the rate. Teams that already sample-audit closures informally are running this loop without formalizing it. The claim is not that no team does this — it is that without a formal feedback path, the disagreement rate is invisible to the downstream consumers of the data.

The loop does not require new tooling. It requires that someone with authority over the detection program treat disagreement rate as a metric with the same standing as detection rate. Where that authority exists and is exercised, the field becomes evidence. Where it does not, the field remains a closure mechanism — useful for ending tasks, not for characterizing the environment.

The hypothesis holds where the feedback path is absent. It breaks where re-adjudication is routine, and the disagreement rate is already factored into downstream decisions. Both states are achievable. The difference is not the disposition taxonomy — it is whether the method behind the conclusion is ever examined.

Sources