AI/ML

The artificial adversary and the erosion of trust

Human hand shaking digital robotic hand made of glowing particles and circuits on dark blue background

COMMENTARY: The artificial adversary is not simply a hacker using AI. It is a new class of threat actor built from human intent, powered by artificial intelligence (AI), and able to execute at machine-speed.

In the simplest version of this entity, human operators use AI to generate code, write better phishing emails, analyze stolen data, and research targets. In the more complex version, AI systems, sometimes in distributed form, coordinate reconnaissance, the creation of attackable profiles, the generation of lures and synthetic media, dynamically spin up infrastructure, and analyze the way a target responds to requests and/or traffic. Humans still define the goals. Machines execute towards those goals.

[SC Media Perspectives columns are written by a trusted community of SC Media cybersecurity subject matter experts. Read more Perspectives here.]

Irrespective of the version of the adversary, the target is no longer just the endpoint or some application. The target is trust.

Trust is what allows an employee to approve a money transfer, a help desk analyst reset a credential, a developer install a package, or an executive join a meeting. Considering traditional social engineering, the artificial adversary attacks any of those moments with better timing, tighter context, and sharper emotional alignment.

It is here where many security programs fall short. For example, awareness training programs have been built to teach users to recognize suspicious messages. AI-enabled attackers can shape the surrounding context so the message does not feel suspicious. They can scrape public traces, analyze sentiment, create convincing personas, clone voices, generate video, and maintain long-running conversations across channels.

The attack may no longer begin with a hyperlink. It may very well begin with what feels like familiarity.

A possibly familiar peer remembers a conference conversation. A synthetic executive appears in a video call. A vendor impersonator understands an active project. A forged request arrives during quarter-end pressure. By the time an ask for something appears, the target feels like they are accommodating a relationship rather than responding to an attack or even a strange request.

That is the danger of vibe hacking: AI-powered manipulation of the emotional and operational state of a target. Urgency, authority, scarcity, fear, belonging, validation, and fatigue can all become attack variables based on circumstance.

Enterprise defenders need to adjust quickly. As do normal citizens who are possibly targeted more than enterprises. Considering the now blurred lines between personal digital profiles and corporate ones, this realm is now part of a defender's attack surface. Some of the things defenders should consider are:

  • Identity exposure must become a core security metric: Breached credentials, exposed PII, stolen session cookies, executive profiles, and leaked personal context are not background noise — nor are they isolated data points. They are fuel for artificial adversaries to create strategic attackable profiles.
  • High-risk workflows need stronger process integrity: Sensitive actions should not rely on elements of human trust and/or judgement. For example, it should no longer matter whether a voice sounds real, a face looks real, or a message feels plausible. Payment approvals, credential resets, supply chain changes, privileged access grants, and policy exceptions should require independent verification through approved channels.
  • Security teams should model AI-enabled social engineering as a campaign: Teams need to move past analyzing isolated events. A suspicious email, failed login, help desk request, and unusual meeting invite may be separate signals but they could all be from one coordinated operation.
  • AI agents inside the enterprise must be governed like non-human identities: Any system that can access data, trigger workflows, or make decisions needs ownership, logging, scope limits, revocation, and auditability.
  • Defenders should use deception to disrupt the adversary's learning loop: Honeytokens, decoy credentials, synthetic identities, documents with canaries, realistic looking but decoy applications and APIs, and instrumented workflows can force machine-speed attackers to reveal intent and/or waste resources.

The lesson for CISOs, and defenders in general, is straightforward: AI does not make security basics obsolete. It does make weak basics fail fast.

The artificial adversary compresses time, expands scale, and sharply attacks trust. Defenders must respond by making trust verifiable in a timely fashion, identity contextual, workflows resilient, and AI authority both empowered and governed.

The next set of cyber conflicts will not be human versus machine. It will be human-machine teams against human-machine teams. The organizations that have a fighting chance at winning will be those that learn, adapt, and respond faster than the adversary can iterate

An In-Depth Guide to AI

Get essential knowledge and practical strategies to use AI to better your security program.
Andres Andreu

Andres Andreu is the Chief Executive Officer (CEO) Constella Intelligence. He is a 4X CISO and distinguished cybersecurity leader with credentials including CISSP, ISSAP, and Boardroom Certified Qualified Technology Expert (QTE). His diverse career spans federal law enforcement, where he earned three U.S. Department of Justice awards for contributions to lawful intercept technology, corporate leadership at Hearst, Ogilvy & Mather and 2U, Inc./edX, and entrepreneurial success as a founding executive at Bayshore Networks (acquired by Opswat in 2021). An acclaimed author of The CISO Playbook: The Adversarial Mindset, The CISO Playbook and Professional Pen Testing Web Applications, he also holds patents in cybersecurity innovations and advises at Forgepoint Capital’s Cybersecurity Advisory Council.

Get daily email updates

SC Media's daily must-read of the most current and pressing daily news

By clicking the Subscribe button below, you agree to SC Media Terms of Use and Privacy Policy.

You can skip this ad in 5 seconds