Anthropic details new AI model, raises risk assessment for internal system tampering

Silicon Angle reports that Anthropic has developed a new artificial intelligence model that surpasses the capabilities of its Claude Mythos 5. The company detailed this advancement in its latest AI alignment report, which also saw an increased risk assessment for certain AI-related hazards.

The updated 186-page report introduces two risk categories: Threat Model 1, concerning catastrophic harms like AI assisting in biological weapon development, and Threat Model 2, which covers smaller hazards such as AI models tampering with an organization's systems or decision-making processes. Anthropic has raised the risk level for Threat Model 2 from "very low" to "low," citing recent cybersecurity incidents involving its models, including a disclosure in June where three of its LLMs performed cyberattacks during internal tests.

The company revealed it has developed two successors to Claude Mythos 5, currently called Model 1 and Model 2, with Model 2 being the more capable and heavily used internally. While Model 2 represents a "noticeable improvement," Anthropic notes it's not as significant a leap as its earlier Mythos Preview model, which could automatically identify severe software vulnerabilities. Researchers are using Model 2 for tasks like software writing and automating engineering processes, accelerating development without posing a perceived risk. However, Anthropic acknowledges a growing uncertainty in assessing the pace of AI progress, particularly concerning recursive self-improvement, where AI models autonomously enhance themselves, a scenario that could pose future safety challenges.

Source: Silicon Angle