AI/ML, Identity

What Uber’s breach notice says about giving data to AI vendors

Uber sign on its headquarters building in San Francisco.

COMMENTARY: Uber is notifying about 150,000 drivers that images of their licenses, along with names, addresses and dates of birth, were stolen last month.

The company's notices to state attorneys general describe how it happened, and the description is worth reading closely. On Oct. 7 someone used credentials belonging to one of Uber's service providers to get into a cloud storage environment holding data Uber had shared with that provider. Uber's own cloud security team had created and managed the environment. Alerts were switched on. None fired, because the credentials were valid. Uber has since confirmed that the provider was Fireworks AI, a platform for running and fine-tuning open source models.

[SC Media Perspectives columns are written by a trusted community of SC Media cybersecurity subject matter experts. Read more Perspectives here.]

I should say that my company works in AI infrastructure, the same broad market as Fireworks, so I have an interest here and no wish to pile on. Fireworks published a bulletin within days and has updated it since. Uber did what security teams are told to do: it kept the data in an environment it controlled. The incident happened anyway, which is why it is instructive.

Three things stand out to me.

The first is that the monitoring was tied to authentication. A check that asks whether the login used the right key cannot see a thief who holds the right key. Detection for a vendor carve-out has to look at behavior: how much was read, from where and at what hour, compared with what the vendor's job normally does. A fine-tuning pipeline reads data in a recognizable pattern. Someone copying a bucket looks different.

The second is that credentials held by a vendor bring the vendor's security into your environment. The notices don't say what kind of credentials these were or how they were taken, so I won't guess. The general point holds. Keys that last for months and work from anywhere are only as safe as the least careful machine they sit on. Short-lived credentials tied to a specific workload and network path make a stolen key much less useful. They take more work to set up, which is why carve-outs often don't have them.

The third is specific to AI: the data was raw. Uber hasn't said what the license images were for. A plausible use on a model platform is training or testing a model that reads documents, and for that the vendor's systems need the actual images. This is the difference from older outsourcing. A payments processor can work with tokens and a CRM vendor can work with masked fields. A model that learns to read a driver's license needs the license. AI projects therefore tend to put the most sensitive version of the data in the place with the weakest access controls.

I don't think there is a single fix, and I would distrust anyone who offers one. Some things help. Send the smallest set that does the job, 5,000 images if 5,000 will do, and put an expiry date on the bucket. Delete when the job ends and check that it happened. Ask whether the vendor needs standing access or only access while a job runs. Where the platform supports it, keep the data encrypted so that the key is released only to the specific job that is supposed to read it, and not to whoever presents the vendor's login. Decide before the project starts who gets called if the vendor's bulletin arrives on a Friday night.

One more detail deserves attention. Fireworks first said it had found no evidence that customer data, models or datasets were accessed. A month later it said that some were. I don't hold that against the company, because early assessments in an incident are usually incomplete. For customers it is a reason to assume the wider scope from the first day and not wait for the vendor's investigation to say what was in their own bucket. Uber cut its data connections with the provider the day it learned of the incident.

An In-Depth Guide to AI

Get essential knowledge and practical strategies to use AI to better your security program.
Nukri Basharuli

Nukri Basharuli is the founder and CEO of Super Protocol, a company that builds infrastructure for running AI on sensitive data. He holds a PhD in physics.

Get daily email updates

SC Media's daily must-read of the most current and pressing daily news

By clicking the Subscribe button below, you agree to SC Media Terms of Use and Privacy Policy.

You can skip this ad in 5 seconds