COMMENTARY: When it comes to AI we’re not concerned that it has opened the door to an advanced wave of cyberattacks. The more pressing concern: it has made familiar attacks cheaper, more potent, more convincing, and far easier to scale than before.But AI does not just enable attacks: it’s also being adopted within an organization's own security operations, software development practices, and business workflows, adding a new layer of cyber risk.[SC Media Perspectives columns are written by a trusted community of SC Media cybersecurity subject matter experts. Read more Perspectives here.]Discussions around AI-enabled attacks often revolve around AI-generated malware and sophisticated deepfake videos, largely because they bring about an element of novelty to existing attacks.However, they aren’t always the most immediate security issues. AI has made familiar attack methods such as phishing, impersonation, fraud, and supplier detection harder to detect and easier to run at scale. That’s the problem that has been knocking on the door of most organizations. An analysis of breaches across 600 organizations globally found that 16% of incidents involved the attacker's use of AI, with AI-generated phishing or other AI-generated communication accounting for 37% of AI-driven attacks.Offensive AI typically shows up in three different ways:In a post-AI cybersecurity world, security awareness has to transition from “spotting fakes” to ensuring employees leverage clearer verification processes for high-risk workflows. Communication that involves payment changes and executive requests should have guardrails such as trusted channels, callbacks, an approval hierarchy, and other checks that the organization deems appropriate.If attackers use AI, why would defenders fall behind? Sixty-nine percent of cybersecurity professionals already use, test, or evaluate AI security tools. AI has been adopted for alert triage, log analysis, threat detection, monitoring, testing, and investigation. AI has lightened the load of security teams stretched thin, reducing manual work and accelerating pattern recognition.But there’s a problem. AI adoption creates its own risk, with sensitive data being shared across poorly governed AI systems. AI tools are often integrated into security workflows before safe use policies are put in place and ownership is assigned. Very little attempt gets made to monitor them continuously.This means while AI adoption happens at a breakneck speed, defensive AI needs these type of guardrails to keep up with the pace of adoption:We need to ensure AI adoption doesn’t come at the expense of risk exposure.The rise of agentic AI adds another layer of potential risk. These AI systems don’t just answer questions but are capable of completing multi-step tasks, integrating with connected business systems, accessing internal data, and acting across workflows while pursuing a pre-determined goal. It’s important to see agentic AI as a privileged actor whose scope demands identity controls, access limits, monitoring, audit trails, and boundary-setting to avoid over-agency.
- It becomes difficult to sift the fake from the real when it comes to phishing attacks. Spelling errors, awkward language, and poorly written messages no longer signal fake emails and messages. In the age of AI, these cues have become redundant. A fraudulent email today can be as polished, unique, specific, and personalized as the real deal.
- Social engineering attacks are increasingly personal, aligning messages with a person’s role, specific responsibilities in the organization, and recent activities. AI can design messages that can look credible because they are contextual and drive a one-to-one dialogue.
- Voice-based impersonation has become a popular ruse for attackers because it can be tailored to ongoing processes. They include payment approvals, executive instructions, vendor onboarding, and urgent requests for sensitive information.
- Comprehensive visibility into which AI tools are allowed, who can use them, and for what purpose.
- Set data boundaries to prohibit information that should never be entered into public or insufficiently governed AI tools.
- Integrate human-in-the-loop reviews to ensure close oversight for high-impact decisions.
- Set clear guidelines for the software review of AI-generated code, including secure coding, dependency checks, and accountability.
- Integrate a process of monitoring that helps explain AI output and actions, the data it accesses, and the people responsible.
