COMMENTARY: Artificial intelligence (AI) has delivered undeniable gains in productivity. Employees move faster. Workflows are automated. Creativity gets scaled. But the same generative AI capabilities reshaping businesses are also transforming cybercrime — and the imbalance has already had considerable real-world impact.The Osterman Research Group reports that 98% of security leaders say AI has already been widely used in cyberattacks against their organizations. Meanwhile, the FBI reported $2.7 billion in losses from business email compromise (BEC) attacks in 2024 alone. These are not projections. They are proof that the threat landscape has fundamentally shifted.[SC Media Perspectives columns are written by a trusted community of SC Media cybersecurity subject matter experts. Read more Perspectives here.]For years, email security followed a familiar pattern. Early phishing campaigns, often called “Nigerian Prince” scams, were crude and riddled with grammatical errors. They relied on volume and luck. Users were trained to look for typos, awkward phrasing, and generic greetings.As defenses improved, attackers evolved. Spear phishing emerged with carefully researched, socially engineered emails that were crafted for a specific individual. These messages were sophisticated and far more convincing, but they were also time-consuming to produce. Scale was limited by human effort.That paradigm broke in November 2022, when ChatGPT went mainstream.Suddenly, anyone — not just skilled social engineers — could generate polished, persuasive, hyper-personalized messages in seconds. Everything security teams had taught users to watch for over the past two decades became far less reliable. Perfect grammar was no longer a sign of legitimacy. In fact, it could now signal the opposite.In one demonstration, a simple generative AI prompt identified vendors associated with a target organization and drafted a realistic invoice email for $120,000 per month. The message authenticated successfully, came from a legitimate-looking domain, contained no malicious attachments, no suspicious links, and no known-bad sender IP addresses. There were no traditional indicators of compromise (IOCs) to flag.In today’s new reality, we now have attacks at scale that are polished, persuasive, and personalized.AI enables instant volume, letting attackers send thousands of tailored messages in seconds. It allows for endless variation, dynamically rewriting content to evade static filters. And it operates continuously, 24/7, without fatigue.Humans simply cannot keep pace with that level of automation on their own.Legacy email security was built around detection of known threats: blocklists, signatures, sandboxing, and static policy rules. These controls depend on identifying something that looks malicious based on prior evidence.But generative AI has changed the equation. Today’s attacks often contain no malware, no payload, and no previously observed infrastructure. They succeed by manipulating human trust, not by exploiting software vulnerabilities.Every AI-generated phishing email has effectively become a zero-day attack, unique in wording, structure, and delivery. If our defenses rely on spotting known bad indicators, we are already behind.In this model, every attack gets treated as a potential zero-day. Protection does not depend on prior signatures or known infrastructure. It depends on understanding human behavior at scale and recognizing when something is off, even if it appears technically clean.AI itself is a tool, capable of creation or destruction depending on how it’s used in cybersecurity. Today, it already exists on both sides of the fight.As attackers weaponize generative AI to scale deception and automate fraud, defenders must respond with equally advanced capabilities. In an era defined by AI-powered threats, only AI-powered defense can restore balance in the cybersecurity arms race.Mick Leach, Field CISO, Abnormal AISC Media Perspectives columns are written by a trusted community of SC Media cybersecurity subject matter experts. Each contribution has a goal of bringing a unique voice to important cybersecurity topics. Content strives to be of the highest quality, objective and non-commercial.
The rise of malicious AI models
Attackers aren’t just experimenting with general-purpose tools. They are leveraging purpose-built malicious large language models (LLMs) such as GhostGPT and WormGPT platforms, which are stripped of safety guardrails and optimized for abuse.These tools can:- Generate localized language that mirrors regional speech patterns — from Boston to Louisiana to Los Angeles — making impersonations more believable.
- Scrape publicly available data from LinkedIn and corporate websites to convincingly impersonate vendors, executives, or partners.
- Rapidly iterate through A/B testing of attack variants to determine which phrasing, tone, or urgency drives the highest response rate.
The only path forward: AI vs. AI
The cybersecurity arms race has entered a new phase. The only viable path forward is AI defending against AI.Defensive AI represents a shift from reactive controls to active, autonomous decision-making. Instead of relying solely on predefined rules or known patterns, defensive AI understands context. It learns how real employees communicate, how relationships between vendors and executives normally function, and what typical workflows look like inside an organization.By ingesting behavioral signals directly through APIs and continuously learning from human communication patterns, defensive AI can identify subtle deviations that indicate fraud — even when the email itself looks flawless.This approach has the following features:- Contextual: It understands who normally pays whom, how requests are phrased, and what typical timing looks like.
- Dynamic: It adapts as communication patterns evolve.
- Autonomous: It makes real-time decisions to stop malicious messages without adding friction to end users.
