I compared Claude Opus 4.8 with 4.7 in a 10-round honesty test - and a legal prompt broke it

A recent comparison of Claude Opus 4.8 and its predecessor 4.7 revealed that the latest model struggled with a legal prompt during a 10-round honesty test, highlighting potential weaknesses in its performance. The evaluation included various challenges across coding, medical, finance, and legal domains, with results cross-checked against multiple AI systems.

This development is significant for Anthropic, the company behind Claude Opus, as it underscores the importance of model reliability and transparency, particularly in sensitive areas like legal advice where accuracy is paramount.

The findings reflect ongoing discussions in the AI community regarding the balance between enhanced capabilities and the risks of over-cautiousness or inaccuracies in AI responses, as seen in previous versions like 4.7, which faced criticism for being overly restrictive in its safeguards.