Cisco warns of AI inaccuracies in security incident reports

Companies considering AI for security reporting should be aware of potential pitfalls, as outlined in a new report from Cisco detailing its experiences with AI-generated incident reports. The company has cautioned that using AI for long-form technical content can lead to significant inaccuracies, unusual conclusions, and inconsistent writing styles, primarily due to the probabilistic nature of large language models (LLMs), Tech Radar reports.

Cisco's research highlights several key issues with AI-generated reports, including inconsistency and standardization challenges due to LLMs using different data for each query and producing slightly different outcomes even with the same data. Formatting and structure also pose standardization problems, and AI can sometimes discard valuable data. To mitigate these issues, Cisco recommends using granular, single-task prompts focused on specific report sections, providing the AI with fixed source documents rather than allowing it to choose, and enforcing strict formatting rules.

A significant challenge identified is cross-contamination between reports when multiple are processed in a single session. To avoid this, Cisco advises starting a new session for each new incident report to ensure accuracy and prevent data loss.

Source: Tech Radar