Google fixes 2 Vertex AI flaws that could lead to privilege escalation, model leaks

Google fixed two vulnerabilities in its Vertex AI platform that could have led to privilege escalation and exfiltration of fine-tuned machine learning (ML) models and large language models (LLMs), Palo Alto Networks’ Unit 42 reported Tuesday.

Vertex AI provides tools and features for developers to train and deploy AI models leveraging Google Cloud resources. One of these features is Vertex AI Pipelines, which developers can use to create custom training jobs in order to flexibly tune their models.

Since running a custom job is similar to running code, it creates the potential for malicious actors to misuse this feature to execute unauthorized commands, the researchers found. They also noted that custom jobs are executed with “service agent” permissions, which grant much greater access than needed to launch the job to begin with.

Seeking to leverage the elevated privileges of the custom job’s service agent identity, the researchers created a custom image that opens a reverse shell when the custom job is executed, ultimately escalating the user’s privileges to that of an AI Platform Custom Code Service Agent.

With this role, the researchers could access the metadata service and all BigQuery tables, acquire service credentials, extract the user-data script, list all service accounts, create, delete, read and write all storage buckets, and more. They further escalated their access by using the user-data script to gain visibility into virtual machine creation and obtain metadata on Google Cloud Platform internal Artifactory repositories, the researchers wrote.

“We used the metadata to access the internal GCP repositories and downloaded images that we didn’t have permissions for with our original service account. Although we gained access to restricted internal GCP repositories, we could not understand the extent of the vulnerability that we discovered, since permissions on the repository are granted at the repository level,” the report authors explained.

How deploying the wrong model could lead to vast model theft

The second vulnerability discovered by Unit 42 and fixed by Google could have led to LLM and ML model exfiltration through deployment of a malicious model designed to create a reverse shell. An attacker could alter a legitimate model and publish it on an open-source repository to be imported and deployed by an unsuspecting user.

Unit 42 tested the potential consequences of such a scenario on a test Vertex AI environment and found that their malicious model gained them read-only access, but that they could achieve lateral movement from the Google Cloud Platform to the Kubernetes environment by gathering information about project resources and ultimately obtaining cluster credentials.

By leveraging their limited access to gradually scope and scout out the resources, assets and architecture of the infiltrated environment, the researchers were able to locate, identify and extract images of all the models stored in the environment. They were also able to access buckets containing adapter files, which can contain potentially sensitive and proprietary information for LLM fine-tuning.

The researchers’ work emphasizes the risk of deploying untrusted AI models on platforms such as Vertex AI, and the report authors urged users to ensure that the ability to deploy new models in sensitive projects is properly restricted. They also recommend that test environments for deploying new models be better isolated from live production environments, and that third-party models be properly vetted and validated prior to deployment.