The blind spot in AI security: Resilience for model and training data

COMMENTARY: While there is plenty of hype about AI, the fact is that this powerful technology is having a positive economic impact. According to analysis from the Federal Reserve Bank of St. Louis, AI contributed nearly a full percentage point to U.S. GDP in 2025. 

But with this momentum has come pressure. Consider the so-called “SaaSpocalypse,” which is Wall Street’s fear that AI-native applications will disrupt traditional SaaS companies. The result has been a major sell-off of their shares. 

But the fear is not just about tech companies. Many sectors are facing the prospect of AI-driven disruption.

[SC Media Perspectives columns are written by a trusted community of SC Media cybersecurity subject matter experts. Read more Perspectives here.]

Yet with the scramble to innovate and invest in AI, companies are taking on new risks. The focus on speed means that systems can become more vulnerable. AI models can be compromised, suffer from drift, and act unpredictably. 

What’s needed is a shift in mindset. The companies that win won’t just be the fastest to deploy, but the ones best prepared to recover when something inevitably goes wrong. That’s why it’s critical to have a well-thought-out resilience strategy. 

The new attack vector: Your proprietary models 

AI development teams are innovating at a breathtaking pace. They need to deal with the flood of new models, as well as techniques and approaches. 

In light of this, it’s understandable that resilience gets scant attention. Besides, this is not the core skillset for AI development teams. 

However, internal AI models are particularly vulnerable to bad actors. These systems are grounded in proprietary and confidential data, such as personal files, sales forecasts, customer information, and upstream/downstream supplier profiles.

Related reading:

The extent of the potential damage can easily be more severe than a traditional breach. An attacker does not necessarily need to exfiltrate a database. They can simply query the model to extract the sensitive information. 

Recovering from a breach can also be highly complex and time-consuming. Reverting to a storage backup snapshot is usually not enough. Retraining proprietary AI models can be  expensive, running from five to seven figures because of the advanced compute, token usage, and GPU resources. 

Then there is a loss of valuable time from the AI development team. Instead of innovating, they will need to rebuild systems from scratch. 

Best practices 

Managing the resilience gap does not mean sacrificing innovation. Rather, the strategy should be to treat resilience as a core capability. 

Here are some best practices to consider: 

Integrate resilience into the development workflow. 

Just as DevSecOps integrated security into the software development lifecycle, resilience must also be built into AI pipelines. They should not be treated as separate processes. Model versioning, training data protection, and recovery procedures need to be embedded into the same workflows where models are developed, tested, and deployed. 

Introduce ResOps as a complement to DevOps. 

DevOps is about shipping software reliably, while DevSecOps is about securing it. Yet these frameworks were not designed to manage the risks of AI systems. 

This is where ResOps comes in. This involves ensuring AI systems can be recovered quickly, safely, and completely after failure or compromise. This includes restoring training datasets, fine-tuned models, prompt/retrieval pipelines, and vector databases/embeddings. 

Phase resilience controls gradually. 

Implementing comprehensive governance all at once will likely fail. A more practical approach is incremental adoption, such as the following: 

The goal is to build resilience progressively without disrupting the velocity that makes AI development teams effective. 

Strengthen collaboration between security and development teams. 

Many CISOs are still developing familiarity with AI development environments, while AI development teams often lack experience in cyber resilience planning. To close this gap, there needs to be active collaboration. For example, CISOs can spend more time with engineering teams. Security leaders can also focus on understanding training pipelines and data flows. 

Treat AI systems as critical infrastructure. 

AI systems are quickly becoming central to business operations. This means that they should be managed with the same rigor as other mission-critical systems. This requires: 

Ultimately, resilience is what allows AI to scale safely. The enterprises that invest in it now will be far better positioned to innovate with confidence and recover when it matters most. 

Conclusion 

AI won’t fail because models are inaccurate. It will fail because enterprises are unprepared to recover when those models are compromised. The winners in AI won’t just be the fastest builders; they’ll be the fastest to effectively restore systems and operations when something goes wrong.