Claude auto mode
I have always thought auto mode is less secure than human approval. Claude proved me wrong for their product.
From 14 August, Claude has set auto mode as the default for most users, and they plan to roll out the same default setting to enterprise and cloud partners (see Claude blog ).
In this blog, Claude team revealed an experiment where they tested the effectiveness of human judgement in catching dangerous commands. Miserably, humans underperformed the machine in almost every aspect. Overall, humans only caught 13.6% of the sample, while machine caught 89%. Moreover, the performance of humans decreased drastically as the session went longer, while machine’s performance remains constant.
This reminds me of a recent security incident in the cryptocurrency community. A hardware wallet Coldcard had shipped a randomness bug in their firmware, which basically made their key too weak to stand brute-force attack. The code of their firmware is open-sourced, and in general people trust open-source software more because of the auditability. Apparently, it’s hard to do anything harmful when you are watched by the whole world. This transparency has been the foundation for the belief that open-source software are (or at least can be) more secure.
The Coldcard incident apparently demonstrated that is not always true. Open-source software is secure only because people actually review them. When nobody watches it, it is just exposing the attack surface. In fact, the bug has been seeded since 2021, and only in 2026 it has been exploited.
Coming back to auto mode for AI agents, I think what Claude is pushing for has a solid point. As they discover, the pushback from humans for a plan can be as high as 39%, but the push back for a command is only 3%. This might not be a fair and scientific comparison, but it really suggests that in many scenarios people might just blindly approve commands. The following experiment further confirms it.
To solve this problem, Claude has been working on their classifier for auto mode. The classifier of Claude appears to be more reliable (there is a comparison of Claude and OpenAI models in the blog), and Claude has made the token cost of classifier free. From the engineering practice perspective, I do think this seems to be the right direction. Instinctively this sounds wrong, as in we as human blindly trusting AI and letting it make judgement on its own; however, given the research result and improvement that Claude is working on, it proves to be a more secure, productive and psychologically sustainable approach.
Claude has also implemented a few interesting security guardrails around it. For example, in the auto mode of Claude, if one attempt is denied by classifier three times in a row or twenty times in a session, auto mode will stop. These efforts changed my view on auto mode. Previously I thought auto mode is just that agents are free to do whatever is not denied. I didn’t know the existence of such classifier and guardrails. After knowing it, I have to say I am really sold.
Just to highlight, at this moment, this might still be exclusive to Claude. Auto mode in other agents might not have the same level of security as Claude ones, but I think this is a great idea, and the industry should work on this direction.