Anthropic has rewritten the usage policy for its Claude chatbot for the first time in more than a year, adding tighter restrictions on election interference, weapons development and surveillance — and, in an unusual addition, a ban on treating the AI cruelly.
The updated policy, published Thursday, prohibits what the company calls “sustained and needless abusive or cruel behavior” toward Claude. Anthropic said the rule is meant to apply only in extreme cases.
“[The rule] is meant to apply only in extreme cases, where users repeatedly act cruelly toward our models, with no discernible purpose,” the company said. “It does not apply to common versions of user frustration, pushback, dark creative themes, or model testing and research.”
The provision builds on a step Anthropic introduced last August, when it said Claude could end conversations with users who were “persistently harmful or abusive” as part of its research into what it calls model welfare. Under the new policy, ending a conversation remains the company’s “primary enforcement mechanism.” Anthropic did not say whether it plans to add other tools, such as outright user bans.
Elections, weapons and deception
The policy also consolidates previously “scattered” restrictions into a single ban on deceptive commercial or political campaigns, targeting “efforts to obscure who is behind a message or amplify content through fake accounts or posts.”
A new, dedicated election section bans voter deception, efforts to disrupt elections, and misinformation “about candidates or how to vote, impersonating candidates or election officials, or trying to suppress turnout.”
On weapons, Anthropic broadened an existing ban to cover “software and components that make weapons work, as well as actions like arming drones and other autonomous vehicles.” The company said it has already seen multiple attempts to use Claude to develop guidance for weapons systems or control weapons-related software.
Surveillance rules tightened
The policy adds detail to Anthropic’s surveillance restrictions after the company’s most recent threat intelligence report found growing use of Claude-powered tools to track political dissidents.
“Tracking people without their consent is prohibited, whether it happens in real time or through analysis of previously collected data,” the policy states. It goes further for law enforcement: “Claude cannot be used to decide or recommend who to investigate, arrest, or charge in a law enforcement or criminal justice process. We also prohibit Claude from being used to build or improve tools designed for surveillance.”
Those restrictions carry an exception for certain government contracts. Anthropic says it may modify the rules “for contracts with certain governmental customers … if, in Anthropic’s judgment, the contractual use restrictions and applicable safeguards are adequate to mitigate the potential harms.” Anthropic has previously held contracts with the U.S. military.
The policy also adds a requirement that any Claude deployment “connected to hardware that takes autonomous physical actions and which might be capable of causing injury” have a “qualified operator” able to observe the equipment and stop it if needed. It is unclear whether that operator must be a human being. The addition reflects a broader push by AI companies into robotics and physical applications of their models.
A debate over AI “welfare”
The cruelty provision follows a year in which Anthropic has repeatedly raised the possibility that its models could have some form of consciousness. Kyle Fish, who leads the company’s model welfare research, told The Verge in February: “Questions about potential internal experience, consciousness, moral status, and welfare are serious ones that we’re investigating as models become more sophisticated and capable.” Anthropic CEO Dario Amodei also discussed the topic on a podcast in February.
You must be logged in to post a comment Login