How to Evaluate EDR, XDR, and MDR Based on Response Outcomes

Evaluation Frame

Platforms prove their value during active incidents, not in vendor demos. The evaluation question is: when an alert fires and escalates to confirmed compromise, can this platform give the investigation team what it needs to establish scope and make containment decisions? Platforms that cannot answer this question amplify alert volume without improving incident outcomes.

Detection accuracy is not the evaluation criterion. An endpoint platform can have excellent detection rates while being unable to support scope determination, timeline reconstruction, or containment decisions during an incident. CISA guidance on endpoint security identifies behavioral detection and incident response capability as core requirements for effective endpoint security programs — establishing that platform evaluation should prioritize response outcome capability over detection volume metrics.

The test is whether an analyst using only this platform's data can answer the three investigation questions in the readiness test. This evaluation approach exposes platforms that provide alerts without investigation support — precisely the gap that extends incident response timelines under operational pressure.

Telemetry Depth and Retention Criteria

Test what the platform collects and how long it keeps it. Key questions: does full-fidelity telemetry require a premium tier or additional configuration? What is the default retention period and what is the maximum? Can you query process execution history for a specific host across 90 days?

Platforms with short retention windows or alert-only collection cannot support reconstruction of incidents discovered after the initial evidence window closes. This creates a critical gap when investigating slow-moving threats that establish persistence weeks before detection.

Test telemetry completeness by requesting demonstration of: full process trees including command lines and parent-child relationships, network connection logs with destination details and timing, file operation records including creation, modification, and deletion events, and registry modification tracking. Platforms that summarize these events into alert metadata lose the granular data investigation requires.

Retention testing reveals budget versus capability tradeoffs. Default retention periods below 30 days indicate cost optimization over investigation readiness. Maximum retention periods reveal platform architecture constraints — some platforms cannot extend retention regardless of customer requirements.

Detection Quality Criteria

Focus on behavioral detection coverage for post-compromise techniques — the living-off-the-land behaviors that attackers use after they have credentials and are moving laterally. Test: present the platform with a scenario involving PowerShell execution, WMI lateral movement, and scheduled task persistence — does the platform surface these as technique-level detections or require prior malware signatures?

Platforms that rely on signature-based detection produce gaps precisely where investigation readiness matters most. Post-compromise activity typically uses legitimate system tools in suspicious patterns, not malware that signature engines can identify.

Evaluation criteria should test detection granularity. Can the platform distinguish between legitimate administrative PowerShell use and suspicious encoded command execution? Does it identify WMI process creation as a lateral movement technique, not just process execution? Can it correlate scheduled task creation with network authentication events to identify persistence establishment?

The tradeoff is detection specificity versus alert volume. Platforms with high behavioral detection coverage often require tuning to reduce false positives in specific environments. This tuning capability becomes part of the evaluation — can the platform suppress false positives without creating detection gaps?

Investigation Support Criteria

Test pivot capability, timeline reconstruction, and cross-host correlation. Starting from one alert, can an analyst identify all other hosts the compromised system communicated with? Can they reconstruct the attacker's process execution timeline across multiple hosts? Can they search for specific indicator patterns across the entire endpoint estate?

Platforms that surface individual alerts without supporting investigation workflow require analysts to manually correlate evidence — extending investigation timelines under incident pressure. This manual correlation becomes the bottleneck when investigating lateral movement across multiple systems.

Pivot testing should verify: host-to-host relationship mapping through network connections and authentication events, process timeline reconstruction showing parent-child relationships and execution sequences, and indicator search capability across the full endpoint estate. Each capability addresses a specific investigation question that analysts must answer under time pressure.

Cross-host correlation capability determines whether the platform can support enterprise-scale incident response. Platforms limited to single-host analysis require external tools for lateral movement investigation — adding complexity and time to response efforts.

Response and Containment Criteria

Test isolation capability and IR integration. Can the platform isolate a host from the network while preserving telemetry collection? Can it terminate a specific process remotely? Does it integrate with identity systems to trigger credential revocation on compromise?

Response capability gaps force incident teams to use multiple tools during active containment — creating delays when containment speed determines incident impact. Platforms that require separate tools for host isolation, process termination, or credential response extend the window of attacker access.

Integration testing should verify: network isolation that maintains platform connectivity for continued monitoring, remote process termination with confirmation of execution, and identity system integration for automated credential response. Each integration point represents a potential failure mode during high-pressure incident response.

The platform should deliver investigation output to SIEM or SOAR with enough context for the IR team to act without separately logging into the endpoint platform. This integration determines whether the platform supports coordinated incident response or requires dedicated endpoint analysts.

MDR Evaluation Criteria

Maintain the critical distinction between MDR services that deliver investigation findings and those that deliver alert notifications. These are different products at different price points with different incident outcomes.

The questions that reveal the difference: what is the SLA for producing a scope assessment — not for alerting, but for telling the customer which systems are affected and what the attacker did? What does the deliverable look like — is it a notification of a confirmed incident, or a documented investigation with timeline, affected systems, and recommended containment actions?

Can the MDR service support the customer's IR team during a declared incident with endpoint investigation capability, or does its responsibility end at alert handoff? This question separates investigation-grade MDR from managed monitoring services.

Investigation-grade MDR evaluation should test: incident scope determination capability within defined SLAs, investigation deliverable completeness including timeline reconstruction and affected system identification, and ongoing investigation support during customer-declared incidents. Services that cannot demonstrate these capabilities provide alert triage, not investigation support.

Proof of Concept Design

Simulate an active investigation scenario rather than a detection benchmark. Establish a known attack sequence on a test endpoint: initial execution, lateral movement to a second host, persistence mechanism establishment, and data staging.

Evaluate the platform against three tests: can an analyst reconstruct the full attack timeline from platform data alone? Can they identify the second host without prior knowledge of which host was accessed? Can they search the entire estate for the persistence mechanism?

The PoC passes when the platform answers all three questions without requiring external forensic tools. This test reveals whether the platform provides investigation capability or requires supplemental tools for incident response.

PoC design should avoid detection-focused scenarios that platforms can pass through signature recognition. Focus on behavioral analysis and investigation workflow — the capabilities that determine incident response effectiveness.

Evaluation Questions

Frame each question so that an inadequate answer is immediately recognizable:

"Show me how an analyst identifies all hosts a compromised system communicated with using only your platform data."

Inadequate answer: requires correlation with external network monitoring tools.

"Demonstrate timeline reconstruction for a multi-host lateral movement attack."

Inadequate answer: provides individual host timelines without cross-system correlation.

"Walk through remote isolation of a compromised endpoint while maintaining investigation capability."

Inadequate answer: isolation requires separate tools or breaks telemetry collection.

"Show scope determination for a credential theft incident affecting multiple systems."

Inadequate answer: identifies individual alerts without mapping attacker progression.

These questions force demonstration of investigation workflow, not feature description. Platforms that cannot demonstrate these capabilities cannot support enterprise incident response requirements.

Sources