Proving your MDR works: testing detection coverage and response effectiveness

The absence of alerts is not evidence of safety. In managed detection and response engagements, silence can mean the provider detected nothing of consequence — or it can mean the provider's sensors are not seeing the traffic, the log sources are incomplete, or the detection logic was never tuned for your environment. Without deliberate validation, you cannot tell the difference from the dashboard.

This is distinct from measuring SLA performance or reviewing escalation metrics. Efficacy validation asks a narrower question: does the detection machinery actually fire against the threats that matter to you?

This question cannot be answered by reading the provider's coverage matrix or counting closed tickets. It requires controlled stimulus, traced response, and evidence you can inspect — and it requires running that cycle repeatedly, not once at contract start.

Security leaders tend to underestimate this gap because MDR contracts describe coverage in terms of technology scope — endpoint, cloud, identity, network — rather than detection fidelity against specific behaviors. A provider can legitimately claim endpoint coverage while running detections that miss lateral movement executed entirely within WMI. Scope and fidelity are not the same thing, and contracts rarely distinguish them.

Controlled detection testing: safe methods

Controlled detection testing sends a known, observable signal through your environment and measures whether the MDR provider generates the expected alert. The signal must be representative of actual adversary behavior to be meaningful — benign traffic labeled as malicious is not a detection test, it is a false-positive exercise.

NIST SP 800-115 requires that any security test be preceded by explicit scope definition and rules of engagement, distinguishing controlled examination from live testing to prevent production impact (Source: NIST SP 800-115, Technical Guide to Information Security Testing and Assessment, https://csrc.nist.gov/pubs/sp/800/115/final). That requirement is not procedural boilerplate. Running an undeclared test that mimics credential dumping or lateral movement can trigger incident response, damage trust with the provider, and — depending on your contract — put you in breach. Get scope authorization in writing before anything runs.

Safe testing methods include atomic simulation (executing discrete technique-level behaviors without full exploitation), replay of sanitized log data from known-bad sources, and tabletop-driven trace exercises where you confirm the provider can articulate how a specific technique would traverse their detection stack.

Each method has limits: atomic simulation confirms the rule fires but not whether the analyst interprets it correctly; replay testing is safe but may not reflect live telemetry fidelity; tabletop exercises confirm knowledge but not operational behavior. Choosing between them depends on your risk tolerance for production disruption, the provider's contract terms, and whether you need to test detection logic, analyst judgment, or both.

Critically, these tests should not be one-time events. Scheduling controlled detection tests on a recurring basis — at minimum semi-annually, and after any material environment change — ensures that coverage claims remain accurate as your architecture evolves and as the provider updates their detection stack. A test that passed twelve months ago does not confirm that detections are working today.

Coverage mapping against a threat model

Coverage mapping asks whether your MDR provider's detections address the techniques most likely to be used against your organization, not the techniques most commonly seen in aggregate threat intelligence.

CISA's guidance on ATT&CK mapping warns explicitly that technique mapping performed without incident context produces mislabeled results — a coverage claim that does not reflect how an adversary would actually operate in your specific environment (Source: CISA — Best Practices for MITRE ATT&CK Mapping,https://www.cisa.gov/sites/default/files/publications/Best%20Practices%20for%20MITRE%20ATTCK%20Mapping.pdf). Applied here: a provider who maps their detections to MITRE ATT&CK without reference to your industry, your architecture, or your known threat actors is producing a coverage document that may be accurate in the abstract and irrelevant in practice.

The MITRE ATT&CK framework gives a shared taxonomy for this conversation, but the mapping exercise must be driven by your threat model, not the provider's marketing sheet. Concretely: identify the five to eight technique clusters most relevant to your sector and architecture — for identity-heavy environments, that likely includes credential access (T1003, T1558), privilege escalation via misconfigured delegation, and lateral movement through trusted relationships. Then ask the provider to show you which specific detections cover each, what log sources they depend on, and what the detection fires on, not just what technique it is labeled under.

Coverage maps also decay. After a cloud migration, a new SaaS rollout, or an identity platform change, a previously accurate map may no longer reflect what is actually detectable given your current telemetry. Treating coverage mapping as a repeatable exercise — revisited after architecture changes and reviewed on a defined schedule — is what keeps it useful as a validation instrument rather than a static document.

The tradeoff: building a precise threat-model-driven coverage map takes time and requires your team to have a defensible threat model to begin with. A generic ATT&CK coverage checklist is faster but tells you less about your actual exposure.

Validating the investigation not just the alert

An alert that fires but produces an investigation report too thin to act on is a failed detection outcome, not a successful one. Validating the provider's investigation product means reviewing whether the escalated finding includes sufficient context to make a triage decision without returning to the raw logs yourself.

A sufficient investigation artifact from an MDR provider should include: the initial indicator and the detection rule that fired, a reconstruction of the activity chain the indicator sits within, any enrichment applied (asset context, user context, prior activity), a confidence assessment with reasoning, and a clear recommended action. If any of those elements are absent, the alert has been escalated, not investigated.

When reviewing investigation quality, comparing two or three escalated findings against your own analysts' reconstruction of the same events — where you have internal capability to do so — can surface systematic gaps in the provider's enrichment or reasoning. This review should be a standing periodic activity, not something triggered only by a visible failure. This is not a replication of investigation craft; it is a quality check on the investigation product you are paying for.

Evidence the provider should produce on request

A provider who cannot produce structured evidence of detection coverage on request gives you no basis for assessing whether your contract is delivering. The following evidence types are reasonable to request, though the specific form varies by provider and contract:

Detection rule inventory mapped to the techniques relevant to your threat model, with the log sources each rule depends on
Telemetry completeness confirmation showing which sources are actively ingesting, with the most recent successful log receipt timestamp
Alert-to-escalation trace for a sample of closed findings, showing the decision path from raw signal to analyst disposition
False-negative acknowledgment process — documentation of how the provider handles a case where your team identifies a missed detection

Requesting this evidence quarterly, or after any significant environment change, tends to be more operationally useful than waiting for an annual review — but the right cadence depends on your risk profile and the rate of change in your environment. The key discipline is that evidence requests happen on a schedule, not only when something goes wrong.

Cadence and ownership of validation

NIST CSF 2.0 treats detection improvement as a continuous governance activity, not a point-in-time assessment — the Detect function is explicitly framed around ongoing monitoring and iterative refinement (Source: NIST Cybersecurity Framework CSF 2.0, https://www.nist.gov/cyberframework). This framing applies directly to MDR efficacy validation: a single validation exercise at contract start decays in value as your environment and the threat landscape both change.

Ownership of validation must be explicit. In many MDR engagements, the provider runs detections and the client assumes they are working. When neither party has formal accountability for periodic validation, it does not happen. Designate an internal owner — typically the security operations lead or the MDR program manager — with authority to schedule tests, request evidence, and escalate gaps to the provider's service leadership.

A defined review cadence makes validation a routine operational discipline rather than a reactive response to a missed detection. A reasonable structure: controlled detection testing semi-annually, coverage map review after any material architecture change, and evidence review quarterly. This specific cadence is illustrative — your actual schedule should reflect your threat exposure, the volatility of your environment, and the operational capacity of the team running validation. The point is that the cadence is defined, documented, and owned, so that validation happens predictably rather than sporadically.

Validation failure modes

Telemetry gap masking a coverage gap. The provider's detection rule exists but the log source it depends on is not ingesting. The rule never fires; the coverage claim remains on paper. Root cause: log sources are often confirmed at onboarding and not re-validated after infrastructure changes. Remediation: include telemetry completeness as a standing agenda item in provider reviews, not just a setup step.

Test signal not reaching the detection stack. A controlled test fires on the endpoint but the provider reports nothing because network segmentation or agent policy prevented the telemetry from reaching the SIEM. This looks like a missed detection but is actually a telemetry architecture problem. Remediation: trace the expected data path before testing and confirm signal receipt independently before concluding detection failed.

Alert without investigation. The provider escalates a finding with a technique label and a raw log excerpt but no analytical narrative. This passes automated SLA metrics while delivering no investigative value. Remediation: add minimum investigation artifact standards to the contract — not as a general quality clause, but as specific required fields in the escalation template.

Coverage map drift. The provider's ATT&CK coverage document is updated on their release cycle, not yours. After a cloud migration, a new SaaS rollout, or an identity platform change, the coverage map may no longer reflect the techniques that are actually detectable given your current telemetry. Remediation: trigger a coverage re-mapping review on any architecture change that alters log sources or authentication paths.

Validation treated as a one-time exercise. A validation program that runs at contract start and is not repeated gives false assurance. Detection rules change, log sources drift, and adversary techniques evolve — a result that was accurate at onboarding may be meaningless a year later. Remediation: build validation into a formal recurring schedule with documented outputs so that the absence of a recent validation is itself a visible signal.

Efficacy validation checklist

Methods vary by environment — treat this as a negotiation framework, not a fixed standard.

Validation activity	What it proves	Typical owner	Evidence produced	Failure signal
Controlled detection test	Detection logic fires on representative technique behavior	Internal security ops or designated test owner	Alert generated, analyst disposition recorded	No alert within agreed detection window; alert lacking technique context
Coverage-vs-threat-model map	Provider detections address your relevant technique clusters	Internal security ops + provider service manager	Mapped coverage document tied to your threat model	Coverage document references generic ATT&CK, not your threat profile
Alert-to-investigation trace	Escalated findings include actionable investigation context	Internal MDR program owner	Annotated escalation sample with completeness rating	Findings contain raw logs but no analytical narrative or recommended action
Telemetry completeness check	All expected log sources are actively ingesting	Provider (with client verification)	Source inventory with most-recent-receipt timestamps	Sources missing or last-seen timestamps older than agreed monitoring window
Escalation drill	Provider escalation path functions end-to-end under realistic conditions	Both parties jointly	Drill record with timing, contacts reached, response actions taken	Escalation delayed; wrong contacts reached; no documented outcome
Periodic re-validation	Coverage and detection fidelity remain accurate after environment changes	Internal MDR program owner	Dated re-validation record compared against prior baseline	Validation not triggered after architecture changes; prior results treated as current

Each activity in this checklist should appear on a defined schedule, not be performed on demand or only after a visible incident. The value of validation as a discipline comes from its regularity — gaps surfaced by routine testing are far cheaper to remediate than gaps discovered during an actual intrusion.