Artificial intelligence has moved from experimental labs into the core of business operations. From financial forecasting to customer service and supply chain management, AI systems are now woven into the fabric of corporate infrastructure. But that deep integration has created an unintended consequence: a growing attack surface for cybercriminals.
Industry researchers warn that adversarial attacks on AI models are on the rise. Unlike conventional cyberattacks,
these exploits manipulate the AI itself — feeding it poisoned data, corrupting its inputs, or leveraging its built-in biases. The result can be reputational damage, regulatory penalties, or operational disruption.
[Editor's Note: This is part SC Media's partnership to unpack OWASP's Top 10 for LLM Applications.] How cybercriminals target AI
Business leaders should understand that AI vulnerabilities are not theoretical. Several real-world incidents have already underscored the risks.
- Manipulated prompts: Researchers have shown that “prompt injection” attacks can override safeguards in large language models (LLMs) like ChatGPT. Multi-modal versions hide malicious commands in images, causing the model to generate harmful outputs.
- Exposure of sensitive data: Employees sometimes upload proprietary code, financial records, or client details into AI tools for analysis. In 2023, Samsung suffered a breach when internal source code was exposed through a chatbot.
- Data poisoning: Malicious actors can seed doctored information online — fake reviews, misleading websites, or biased datasets. If an AI system ingests this data, it can mislead decision-makers or redirect users to fraudulent sites.
- Model confusion: Minor manipulations can cause models to misinterpret data. In critical environments like autonomous vehicles or industrial control systems, such errors could cause serious physical or financial harm.
- Exploitation of hallucinations: AI models are prone to inventing answers that sound credible but are false. Attackers can exploit these “hallucinations” by slipping malicious code or fraudulent packages into the gaps.
Five defensive priorities for business
Executives looking to
mitigate AI risks should treat model protection as a board-level issue. Security experts recommend five clear steps:
- Invest in employee training: Human error is often the weakest link. Educate staff on the risks of feeding sensitive data into AI tools and train them to recognize phishing or adversarial prompts.
- Establish AI governance policies: Define ethical and responsible use of AI within the company. Policies should cover acceptable data inputs, privacy protections, and compliance with evolving regulations.
- Secure the infrastructure: Apply zero-trust principles, strict access controls, and continuous monitoring to the servers and systems hosting AI models. These assets should be treated as critical infrastructure.
- Validate and sanitize inputs: All inputs must be screened before reaching the model. This is especially important for businesses relying on LLMs, where prompt injection attacks are difficult to detect.
- Minimize and anonymize data: Restrict models to the minimum necessary data. Use anonymization or encryption to reduce the risk of exposing sensitive details in the event of a compromise.
An arms race with high stakes
New threats are already emerging. Researchers at Vulcan Cyber explained AI package hallucination, a technique where LLMs recommend nonexistent software libraries. Attackers can exploit this by uploading malicious “replacement” packages that unsuspecting developers install.
For enterprises, the lesson is clear: AI security cannot be an afterthought. It requires ongoing investment, clear governance, and a culture of skepticism toward AI-generated outputs.
The bottom line for executives
AI offers powerful advantages, but unmanaged, it also poses serious liabilities. Regulatory bodies are beginning to scrutinize how organizations deploy AI, and investors increasingly see cybersecurity lapses as governance failures.
Business leaders must ensure their organizations adopt not only the latest technologies but also the safeguards that make them sustainable. The companies that succeed will be those that treat AI not as a novelty but as a core enterprise system — one that demands the same diligence as financial reporting, compliance, or risk management.
This article is part of SC Media’s 10-part editorial series on the OWASP Top 10 for LLM Applications 2025. Produced in partnership with the OWASP Generative AI Security Project, the series highlights actionable steps for secure, transparent GenAI application development.
