The boundary where your evidence stops
Your failover design carries an assumption. Somewhere in that assumption sits assurance evidence — a SOC 2 Type II report, an ISO 27001 certificate, a shared responsibility matrix — that covered the provider's internal controls and stopped precisely at the integration boundary your architecture depends on. The evidence is real. The gap is real. The assumption is that the interaction behaves the way the design requires it to.
That interaction is what nobody has tested.
Cross-domain assumptions are where architecture validation separates from standard control assurance. Control testing simply verifies that an individual safeguard functions within its designated environment. Architecture validation, by contrast, determines whether an assumption bridging multiple environments actually holds under live operational demands. The vendor boundary makes this especially difficult, as you cannot independently inspect or test a third party's internal infrastructure and controls on demand.
What you can do is decompose the assumption into its load-bearing pieces, match each piece to whatever evidence exists, name the pieces for which no evidence will ever reach you, and decide consciously how to carry the gap.
What the assumption actually depends on inside the vendor
Most cross-vendor architecture assumptions are not single claims. They are stacks. A typical assumption — "token issuance fails over to the secondary region within the service-level commitment" — actually depends on: the enforcement point behaving identically in both regions; the key material being synchronized before a failure, not during one; the failover trigger being automatic and not requiring human acknowledgment; the vendor's own upstream dependencies being available to the secondary region; and telemetry and audit evidence covering the switchover so you can verify it happened.
You own none of those sub-components. The vendor's assurance evidence may cover the control environment around some of them. It covers the interaction between them and your system only where the vendor chose to define that interaction as in-scope.
Break each assumption down before running any tests. If you cannot pinpoint the underlying sub-components, you cannot evaluate existing evidence or identify where the visibility gaps sit. Build directly on your existing dependency mapping and degraded-mode logic to isolate these elements rather than trying to recreate the model from scratch.
Reading a vendor attestation for what it does not cover
A SOC 2 Type II report describes controls as the vendor's auditor scoped them, over a period the vendor selected, against criteria the vendor agreed with the auditor to test. The scope boundary is in the report. Read it before you read anything else.
What to look for specifically:
Scope exclusions. The description of the system under examination names what is in scope. Shared infrastructure, CDN layers, and upstream identity providers can sit outside the scope the report declares. If your architecture assumption depends on one of those components, the report does not cover your assumption.
The distinction between existence and effectiveness. A Type II report attests that a control operated effectively over the covered period. It does not attest that the control configuration matches the configuration your integration requires. An IdP that logs authentication events effectively for its own audit purposes may not emit the event format your SIEM ingests, at the retention depth your detection logic requires.
Subservice organizations. An attestation may treat subservice organizations — the vendor's own vendors — by the carve-out method. The report tells you those dependencies exist. It does not give you the subservice organization's attestation. If your failover assumption depends on a subservice organization behaving predictably, you are one level deeper into unvalidated territory.
NIST SP 800-37 Rev. 2 places the burden on the relying organization to determine whether evidence a provider supplies is sufficient to support the relying organization's own risk determination — not sufficient for the provider's purposes (Source: NIST SP 800-37 Rev. 2, https://csrc.nist.gov/publications/detail/sp/800-37/rev-2/final). That distinction matters operationally. The provider's auditor answered the provider's question. You have to answer yours.
Testing your side of the boundary properly
You cannot independently test inside the vendor's internal infrastructure at will. You can test customer-accessible APIs, run interface tests, and in some cases jointly perform failover and disaster recovery exercises with the vendor. You can also test everything at and behind the integration point you control, and test your side of the failure mode without the vendor's cooperation.
Step 1: Map the observable surface. Identify every signal the vendor exposes to you — token responses, webhook events, error codes, status page mechanisms, health endpoints — and confirm that your logging captures each one. If you cannot observe the signal in your own telemetry, you cannot validate the assumption even partially.
Step 2: Inject failure at your boundary. Block or delay the vendor's responses at your ingress — using a proxy, a firewall rule, or a test network policy — and observe whether your system degrades, fails, or fails over as designed. This tests your side of the assumption: that your system behaves correctly when the vendor does not respond. It does not test whether the vendor behaves correctly.
Step 3: Verify the observable signals match your detection logic. Run your detection logic against the signals you collected during Step 2 and confirm that an alert fires within the time window your incident response plan requires. Mis-timed or mis-formatted vendor signals are a common root cause of silent failover failures.
Logic pattern / pseudocode — validate for your platform
```
// Detect vendor authentication endpoint unresponsiveness
// at your boundary — not a production-ready querysource: proxy_access_logs
filter: destination_host = "vendor-auth-endpoint"
AND (response_code = null OR response_latency_ms > threshold)
AND time_window = "last 5 minutes"
group_by: destination_host, source_service
alert_if: event_count >= N within rolling_window// Validate: does this fire before your failover SLA expires?
// Validate: does it suppress correctly when vendor recovers?
```
Step 4: Confirm token and session revocation behavior at your boundary. Depending on platform, token type, and your IdP's policy configuration, revocation behavior varies — access tokens issued before a failover event may remain valid at your resource servers until expiration, regardless of what happens at the vendor. Test for this explicitly; do not assume revocation propagates on the vendor's clock.
If boundary testing uncovers an exposure path stemming from an unresolved vulnerability or unaddressed flaw, route that finding directly to your standard vulnerability and exposure management workflows rather than treating it as an architecture validation issue.
Contractual routes to evidence you cannot gather
Some evidence gaps can be partially closed through contract terms, even when they cannot be closed through technical testing.
The levers that can matter:
- Right-to-audit clauses, scoped specifically to the integration behaviors your architecture depends on, not the vendor's general control environment. General audit rights rarely produce the evidence you actually need.
- Incident notification SLAs that name the specific event types relevant to your assumption — failover triggers, key rotation events, upstream dependency failures — rather than generic "security incidents." Vague notification terms produce notifications after the fact, not during the window your recovery sequence requires.
- Change notification requirements with a defined lead time for changes that could affect your integration boundary, including subservice organization changes. Without this, the vendor can change the behavior your assumption depends on with no obligation to tell you.
The CIS Critical Security Controls include a service provider management control that requires inventorying service providers and reviewing them against defined security requirements (Source: CIS Critical Security Controls, https://www.cisecurity.org/controls/cis-controls-list). The practical application here is that the review must be scoped to your security requirements — specifically the assumptions your architecture carries across the boundary — not a generic checklist.
The assumption you have to carry untested
This section has to be written honestly.
After decomposing the assumption, testing your boundary, reading the attestation, and working through available contractual routes, there will be a remainder. The remainder is the portion of the assumption that depends entirely on vendor behavior you cannot observe, test, or contractually compel evidence of. It is not a gap you failed to close. It is a structural consequence of operating a design that crosses a boundary you do not own.
The honest practice is to name it exactly:
- What the assumption requires the vendor to do
- What evidence exists that the vendor does it
- What evidence does not exist and cannot exist given the boundary
- What your architecture does if the assumption is wrong
That final point dictates the true severity of the risk. When an architecture lacks a degraded mode for vendor downtime—such as local fallback routines, cached states, or time-limited autonomous operations—an unverified assumption leaves the entire blast radius exposed. Conversely, building graceful degradation into the design contains and bounds that impact. Deciding how to handle system degradation is a broader architectural responsibility; explicitly documenting the risk profile when that dependency fails is what makes up the validation finding.
The boundary evidence test
Run this against each cross-vendor assumption in your register before marking it validated.
| Component of the assumption | Side of the boundary | Evidence available to you | Its limit | What to do when no evidence is obtainable |
|---|---|---|---|---|
| Enforcement point behavior (authz, policy decision) | Vendor-side | SOC 2 scope inclusion, vendor runbook excerpts | Covers control existence, not configuration parity with your integration | Document as untested; define your degraded-mode behavior explicitly |
| Identity or key material synchronization | Vendor-side | Attestation controls around key management | Does not confirm synchronization timing relative to your failover window | Test observable effects at your boundary; note timing gap in assumption register |
| Telemetry and audit evidence emission and retention | Shared (emission vendor, retention may be configurable) | Vendor documentation, API log schema | Format and retention depth may not match your SIEM or detection requirements | Validate log ingestion and query fidelity on your side; flag retention gaps |
| Failover trigger and timing | Vendor-side | SLA commitments in contract | SLA defines the commitment, not the mechanism; subservice dependency behavior excluded | Negotiate incident notification with named trigger events; test your side for degraded handling |
| Vendor's own upstream dependencies | Vendor's vendor | Subservice organization carve-out in SOC 2 | Assurance evidence stops at the subservice boundary | Request vendor's subservice SOC 2 inclusion or accept and name the gap |
| Change notification for boundary-affecting changes | Contractual | Change notification clause (if negotiated) | Clause quality varies; generic terms rarely cover subservice changes | Scope clause to your integration-relevant change types; monitor vendor status and changelog signals |
A cross-vendor assumption is validated only when each row has a completed entry, and the gaps in the final column are named explicitly in your assumption register. Rows with no evidence and no named gap are the failure mode — not a test result, an unexamined load-bearing dependency.
Sources
- NIST SP 800-37 Rev. 2: https://csrc.nist.gov/publications/detail/sp/800-37/rev-2/final
- CIS Critical Security Controls: https://www.cisecurity.org/controls/cis-controls-list
