AI/ML, AI benefits/risks, Threat Intelligence

Security researchers highlight supply-chain risks in AI model distribution

(Adobe Stock)

Bleeping Computer reports that security researchers successfully gained control of the Kokoro text-to-speech model on the Hugging Face platform, exposing significant vulnerabilities in how AI models are managed and distributed.

The researchers demonstrated that by registering an expired domain associated with the model's publisher, they could reset the account password and gain full control. This allowed them to modify the model's neural weights to produce altered audio output without changing the underlying application code. Because the malicious changes exist only within the weights, traditional malware scanners fail to detect the compromise. The Kokoro model is widely used in projects supported by major companies including Google, NVIDIA, AMD, Intel and Amazon Web Services. The researchers noted that this attack vector poses a severe supply-chain risk, as compromised accounts can push poisoned updates that appear legitimate to automated systems. Despite the potential for widespread impact, the researchers reported that Hugging Face does not currently verify publisher ownership or vet uploaded models. This lack of oversight allows attackers to distribute manipulated AI components that flow through various mirrors, registries and deployment guides. The team highlighted these risks to emphasize the broader trust issues within the open-model ecosystem, noting that even non-hijacked accounts could be used to distribute harmful content.

Source: Bleeping Computer

An In-Depth Guide to AI

Get essential knowledge and practical strategies to use AI to better your security program.

Get daily email updates

SC Media's daily must-read of the most current and pressing daily news

By clicking the Subscribe button below, you agree to SC Media Terms of Use and Privacy Policy.

You can skip this ad in 5 seconds