New Agent Workspace feature comes with security warning from Microsoft

An experimental new Windows feature that gives Microsoft Copilot access to local files comes with a warning about potential security risks.  

The feature, which became available to Windows Insiders last week and is turned off by default, allows Copilot agents to work on apps and files in a dedicated space separate from the human user’s desktop. This dedicated space is called the Agent Workspace, while the agentic AI component is called Copilot Actions.

Turning on this feature creates an Agent Workspace and an agent account distinct from the user’s account, which can request access to six commonly used folders: Documents, Downloads, Desktop, Music, Pictures and Videos.

The Copilot agent can work directly with files in these folders to complete tasks such as resizing photos, renaming files or filling out forms, according to Microsoft. These tasks run in the background, isolated from the user’s main session, but can be monitored and paused by the user, allowing the user to take control as needed.

Risks associated with agentic AI

Windows documentation warns of the unique security risks associated with agentic AI, including cross-prompt injection (XPIA), where malicious instructions can be planted in documents or applications to trick the agent into performing unwanted actions like data exfiltration.

“Copilot agents’ access to files and applications greatly expands not only the scope of data that can be exfiltrated, but also the surface for an attacker to introduce an indirect prompt injection,” Shankar Krishnan, co-founder of PromptArmor, told SC Media.

Microsoft’s documentation about AI agent security emphasizes user supervision of agents’ actions, the use of least privilege principles when granting access to agent accounts and the fact that Copilot will request user approval before performing certain actions.

While Microsoft’s agentic security and privacy principles state that agents “are susceptible to attack in the same ways any other user or software components are,” Krishnan noted that the company provides “very little meaningful recommendations for customers” to address this risk when using Copilot Actions.

“One of the folders this allows agents access to is the ‘Downloads’ folder, which is a high-risk source of indirect prompt injections as it largely contains data from third parties. This greatly increases the likelihood that an agent could be manipulated into exfiltrating data or crafting a personalized phishing attack,” Krishnan said.

The use of indirect prompt injection to extract sensitive data from files was recently demonstrated by PromptArmor in a proof-of-concept exploit of Anthropic’s Claude for Excel, which used hidden instructions in spreadsheet data to trick Claude into generating a formula that exfiltrated other sheet contents to an attacker.

Previously, another Microsoft AI feature called Recall, which takes snapshots of a user’s active screen every few seconds, was postponed due to security concerns over storing screenshots containing potentially sensitive information.

Copilot Actions also takes screenshots of the agent’s desktop, which are retained for up to 30 days unless manually deleted, although these do not capture the user’s own desktop, which is isolated from the Agent Workspace.