Cloud incident response: what changes when you do not own the host

The instance that was gone before anyone looked

The alert arrived at 02:40. Unusual outbound traffic, one instance, flagged by network telemetry. The responder opened the IR plan. Step one: isolate the host. The instance was already gone — terminated twenty minutes earlier by an autoscaling group that had replaced it with a clean image before the alert even fired.

This is not an edge case. It is a common operating condition in cloud-native environments. Autoscaling and immutable deployment patterns can terminate and replace instances faster than human response cycles, and an IR plan that assumes a persistent host will fail at its first step in these architectures. The host is not missing because something went wrong. It is missing because the platform behaved exactly as designed.

The decision the responder now faces is not how to isolate a host. It is whether any evidence from that instance still exists, where it is, and whether it was preserved in a form that can support investigation. Those answers were determined — or not determined — long before the alert fired.

What acquisition assumed

Traditional host forensics assumes you can reach the artifact. You image the disk, capture memory, pull network state, and work from a complete copy of the system as it existed at a point in time. NIST SP 800-61 Rev. 3 frames incident handling as a continuous cycle of detection, analysis, and improvement aligned to the CSF 2.0 functions rather than a discrete lifecycle a team completes — and analysis, wherever it sits in that cycle, still depends on having something to analyse (Source: NIST SP 800-61 Rev. 3, https://csrc.nist.gov/pubs/sp/800/61/r3/final). In an ephemeral cloud environment, the artifact that analysis depends on can be gone before detection completes.

The lifecycle assumption that breaks first is not collection methodology. It is availability. Cloud workloads in autoscaled or serverless architectures do not wait for responders. The forensic window — the period during which the artifact is reachable — can close in minutes. What survives that window, and in what form, depends almost entirely on decisions made before the incident began.

Three assumptions that break

Ephemerality is not a failure. Treating instance termination as an obstacle to investigation misframes the problem. Autoscaling and immutable deployment exist because they improve reliability and reduce configuration drift. The response capability has to be designed around what survives termination, not around preventing it.

Assuming you can freeze a workload long enough to image it may be valid in some environments — dedicated instances, non-autoscaled workloads — but in many cloud architectures it is not a viable default. Where isolation of a specific instance is required, some providers allow autoscaling group detachment or instance protection settings that can prevent termination during investigation; whether these are viable depends on the architecture and must be evaluated in advance.

Acquisition is observable. Calling the API to snapshot a disk volume, export a log stream, or pull instance metadata is an action recorded in the provider's audit trail. In AWS environments, for example, CloudTrail logs control-plane API calls including snapshot creation; in GCP, Cloud Audit Logs record administrative and data access activity. If an adversary retains access to credentials with audit-log read permissions, your acquisition activity may be visible to them in the same trail you are using to reconstruct their activity.

In an active intrusion where containment depends on not alerting the adversary, the order of operations — snapshot before revocation, or revocation before snapshot — is a decision with real consequences, and it should be made in advance rather than under pressure at 02:40. The right order depends on the specific threat model: if evidence preservation is the priority and the adversary is unlikely to act on detection, snapshot first; if continued access represents active harm, revocation takes precedence.

Observability of the workload is not guaranteed. In traditional environments, incident responders can attach a debugger, examine process state, or access local logs even after an incident has begun. Cloud workloads present a different challenge: what responders can investigate depends largely on the monitoring and evidence collection systems configured beforehand.

If logs were not sent to durable storage, snapshot policies were not enabled, or cloud provider audit logs were not retained beyond their default retention periods, critical evidence may be permanently lost. Some provider-level data, such as network flow logs and managed service logs, may still be available depending on the configuration. However, once a cloud instance is terminated, its memory state and any local logs not preserved elsewhere generally cannot be recovered. This is primarily an architectural limitation rather than a procedural failure.

What a provider-rendered copy is and is not

When a responder calls the API to snapshot a volume, they do not get a forensic image in the traditional sense. They get a point-in-time copy of persistent storage as the provider's infrastructure renders it — block-level, consistent within the provider's consistency model, but mediated by a software layer the responder does not control and cannot fully inspect.

This matters for evidence quality in several ways. First, the snapshot captures what was on the volume at the moment of the call, which may be minutes or hours after the relevant activity occurred. Memory state, running process state, and network connection tables are not included. If the workload was terminated before the snapshot was taken, the snapshot reflects the terminated state of the volume — potentially after the operating system had already written shutdown artifacts.

Second, the chain of custody for a provider-rendered copy runs through the provider's infrastructure. For many internal investigations and operational purposes, this is sufficient. For regulatory proceedings or litigation, the evidentiary weight of a provider-rendered copy depends on jurisdiction, the specific proceeding, and whether the provider can supply supporting attestation such as integrity hashes or API call logs confirming when the snapshot was initiated.

Some providers offer mechanisms — such as AWS CloudTrail log file integrity validation or GCP's audit log export to immutable storage — that can support chain-of-custody arguments; these should be evaluated and configured before they are needed. Assuming provider copies are forensically equivalent to directly-acquired images in all contexts is a gap worth closing before it matters.

Third, when a volume snapshot can be taken from a running or recently terminated instance, it may still contain recoverable artifacts in unallocated space or application data directories that support the investigation — this is not equivalent to a live acquisition, but it is not valueless either. The investigative utility depends on what was written to persistent storage before termination.

Network evidence has a distinct acquisition path — captured by provider flow logging and perimeter monitoring rather than by anything on the workload itself — and that path has its own retention constraints and observability limits.

The evidence that has to be preserved by default

NIST SP 800-92 establishes log management as a set of defined functions — collection, storage, monitoring, analysis, and reporting — with retention requirements determined by the threat exposure and operational context of the environment (Source: csrc.nist.gov, https://csrc.nist.gov/publications/detail/sp/800-92/final).

Applied to cloud workloads, that framework means retention is a design decision, not a platform default. Provider-managed tamper-resistant logging — such as routing CloudTrail or Cloud Audit Logs to write-protected, separately-permissioned storage — is part of that design decision and should be treated as a baseline, not an enhancement.

The table below names what survives and what does not across common cloud architectures. Behaviors vary by provider, service type, and configuration; the table reflects common patterns rather than universal facts. The column "Must Be Preserved Before The Incident" identifies evidence that typically cannot be recovered after the fact regardless of provider capability — though some entries are conditional on specific architectural choices noted in the table.

Identity and permission data can create some of the biggest challenges during an investigation. IAM policies, role bindings, and group memberships can change during an incident, including as a result of containment actions. Revoking credentials or removing roles may erase the original permission state that investigators need to examine. Some cloud providers offer tools to track these changes. For example, AWS Config can record IAM configuration changes, while Google Cloud Policy Intelligence provides related visibility.

However, these capabilities depend on prior configuration and are subject to retention limits. In environments where IAM permissions change frequently, determining what access a user or service account had at a specific time can be difficult without audit records that capture the actual policy state, not just the changes made. To preserve this evidence, containment runbooks should explicitly require responders to capture relevant IAM configurations before modifying permissions, whenever operationally feasible.

The decision to make before the next incident

The six fields in that table are not the only evidence cloud incidents generate, but they represent the set where the decision to preserve or not preserve has already been made before the alert fires. Every row marked "Must Be Preserved Before The Incident" is a gap that no IR procedure, however fast, can substantially close after the fact.

Three decisions should be made now, before the next incident:

Decide the preservation set explicitly. Which log sources are shipped off the workload to a write-protected, separately-permissioned destination? Which snapshot policies are active on which volumes? Is the provider audit trail exported to durable storage and retained longer than the expected time-to-discovery for this environment? Is tamper-resistant logging — routing audit logs to storage that the compromised principal cannot modify or delete — part of the baseline configuration? The answer to each question should be a named, documented configuration — not an assumption.

Decide the retention floor. Retention requirements vary by regulatory obligation, threat model, and operational context. A useful starting point is to ask: how long does it typically take this environment to detect an intrusion that began as a credential compromise? The retention floor for the provider audit trail and workload logs should exceed that gap, with margin for the investigation itself. Let risk tolerance, regulatory obligations, and actual detection timelines drive the number — not a provider default that was set for operational convenience rather than investigative need.

Decide the acquisition order before an active intrusion. Snapshot before revocation, or revocation before snapshot — the answer depends on the threat model, the evidence already preserved, and the risk of continued access. Where possible, capture IAM state, active session information, and any available instance metadata before executing containment actions that will overwrite them. Document the decision criteria now so the responder at 02:40 is executing a pre-authorized procedure, not improvising a tradeoff under pressure.

The instance that was gone before anyone looked did not fail the investigation. The absence of a preservation policy did. The acquisition step is not available after termination. The decision about what to preserve automatically is available right now.

Sources