Rolling the dice on secure code: A look at AI and enterprise security

COMMENTARY: Imagine receiving a dozen alerts for new security incidents all at once. Which one do you address first?

You decide to leverage a new AI-powered tool that quickly searches and prioritizes alerts. What if it gets things wrong and incorrectly triages the tickets? While extremely annoying and wasting time, no new security concerns are introduced.

[SC Media Perspectives columns are written by a trusted community of SC Media cybersecurity subject matter experts. Read more Perspectives here.]

Now imagine you're working on one of those issues and you run into a problem that you're really not sure how to solve. You turn to your large language model (LLM) powered assistant in your integrated development environment (IDE) and ask it what to do. If the AI gets it wrong this time, and you blindly trust the results, you could potentially introduce a breaking change or new vulnerability. If that happens, it's not just a matter of productivity and annoyance; it's a security issue.

The above scenarios are not imaginary, and the risks they bring are very real. Modern enterprises have widely adopted artificial intelligence into their workflows and toolchains.

But not everyone has the same understanding of how these tools work and what kind of risks are present. If we demystify how AI works, it will help everyone work more securely in a world where "probably" is driving a lot of our systems.

What probably looks like in AI

When you roll a pair of dice, the outcome is unknowable before the dice settle. The outcome is never guaranteed. That’s the nature of probability. It’s the same principle behind LLMs.

In predictive systems, like classifiers, estimates of likely outcomes are based on historical data, answering questions like "Does this picture contain a dog?"

Predictive models are commonly used in security for risk scoring, fraud detection, and forecasting. Because they operate within fixed boundaries and have measurable outputs, they are generally easier to test and validate. As it rolls the dice, things might slip through or priorities get ordered poorly, but for the most part, companies have decided that the productivity gains outweigh the risks.

Generative AI (GenAI), by contrast, creates entirely new arrangements of text, images, or other output by making token-by-token predictions based on the likelihood of satisfactorily answering a prompt.

GenAI is incredibly useful for accelerating development and the documentation process. At the same time, it is also unpredictable, inconsistent, and prone to hallucinations, especially when performing complex tasks. When it gets it wrong, new vulnerabilities and dangers can easily be introduced. A generated code suggestion might look correct while quietly introducing insecure practices, such as hardcoded secrets or outdated libraries. The productivity gains are great, but the risk GenAI introduces is also potentially very high. Still, companies are mostly charging ahead with adoption, expecting that we will improve ways to lower the risks it brings.

What can we do to make sure we stay safe? The answer (probably) lies in combining traditional deterministic approaches with smart applications of predictive AI, and establishing guardrails around any systems where GenAI is in use.

Best of all probable worlds

When AI can suggest changes, it's critical that the outcome passes through policy-enforcing controls that never rely on assumptions. Those controls, in the form of scripts and rule sets, need to be deterministic. This means they always produce the same outcomes from the same inputs.

In AI-assisted development workflows, this might take the form of pre-commit Git hooks that leverage security tools to automatically scan every change before the commit occurs, ensuring that LLM output doesn't introduce new incidents, such as hardcoded secrets or dependencies with known vulnerabilities. By enforcing these guardrails at the point of commit, or even at the pull request stage, teams can help ensure that any productivity gains from AI do not come at the expense of security or stability in production.

AI absolutely does have a role to play in security, especially in speeding up detection and prioritization. For example, using LLMs to reduce false positives through contextual analysis of a string is used for authentication, vs just a placeholder for a vault path to a real secret. Assigning severity scores to leaked secrets to help with triage is another valuable use case for AI.

However, even these good use cases still require deterministic validation using regular expressions, heuristics, and context-aware rules to ensure the desired outcomes. This hybrid approach pairs AI’s speed and pattern recognition with the reliability of provable enforcement.

It’s how to stay fast, secure, and audit-ready in a world powered by probability.