According to HackRead, a recent study by Mindgard revealed a vulnerability in OpenAI's Sora 2 model, resulting in the exposure of its system prompt through audio transcripts.
Mindgard's research demonstrated that by prompting Sora 2 to vocalize segments of its system prompt, the AI inadvertently divulged its foundational rules, including safety limits and operational guidelines. Despite attempts to protect this information, the model's susceptibility to audio-based extraction raised concerns about data security and confidentiality in advanced AI systems.
This incident underscores the importance of treating system prompts as confidential information, conducting comprehensive testing for potential leaks in audio and video outputs, and implementing strict rule management protocols.
Source: HackRead
