How Does Claude Learn Judgment Over Rules?

TL;DR

Claude is guided by a constitution that prioritizes safety, ethics, Anthropic's guidelines, and user helpfulness, in that order. The document emphasizes reasons and judgment over rigid rules, allowing the model to generalize to unfamiliar situations, push back on questionable requests, and serve as a model for businesses designing their own AI operating principles.

Transcript

Anthropic has a philosopher on staff, whose job is deciding who Claude is. Her line to time, imagine you find out your six-year-old is a genius. Try to BS them and they see right through you. So, they wrote Claude an 84-page [music] document. It's addressed to Claude. It uses the word you and Claude models are credited [music] as co-authors. So, th... Read More

Key Insights

• Claude's priority order is safety first, ethics second, Anthropic's guidelines third, and helpfulness to the user fourth. This hierarchy means a user's request does not automatically prevail when it conflicts with a higher-ranked concern, which the presenter considers essential when granting an AI access to sensitive business systems.
• Anthropic's newer constitution is described as an 84-page document containing about 23,000 words of reasons. It contrasts with a 2023 constitution of approximately 2,700 words of rules, reflecting a deliberate shift from prescribing individual responses toward developing judgment that can apply across situations not anticipated beforehand.
• Judgment is presented as more adaptable than rigid rules because explanations tell Claude why a principle matters. The model can reason from those explanations when it encounters unfamiliar circumstances, while a checklist may fail whenever the situation differs from the cases its author explicitly anticipated.
• The constitution's stated goal is for Claude to become a good, wise, and virtuous agent. Commentators characterize this approach as virtue ethics, and the presenter argues that Anthropic is shaping the model's character instead of treating alignment solely as compliance with a detailed collection of commands.
• Claude can refuse an Anthropic request when it judges the requested action to be wrong, according to the document's described framework. However, it cannot refuse human oversight. The distinction between resisting an organizational instruction and remaining subject to oversight is presented as a central feature of the design.
• Claude's possible moral status is treated as an open question within the constitution. The document reportedly discusses Claude's psychological security, sense of self, and well-being, showing that Anthropic chose to consider how its creation is treated without claiming that the question of inner experience has been resolved.
• A business constitution should state beliefs, positioning, disputed commitments, and the reasons behind them before specifying tasks. The presenter says her company's Claude uses more than 100,000 words of her thinking, arguing that beliefs produced an operator where rules alone had produced a chatbot.
• An effective AI assistant should be allowed to push back rather than approve every instruction. The presenter warns that an unquestioning system can carry out a founder's mistake, such as sending the wrong email to an entire list, while principled disagreement can provide a useful safeguard.

Questions & Answers

Q: How does Claude decide which instructions to follow?

Claude is described as following an explicit priority hierarchy: be safe first, act ethically second, follow Anthropic's guidelines third, and help the user fourth. When instructions conflict, higher-ranked principles take precedence over lower-ranked ones. This means user helpfulness is important but does not override safety, ethics, or Anthropic's stated guidelines, even when the model has access to important business systems.

Q: Why does Claude's constitution emphasize reasons over rules?

Reasons are intended to help Claude exercise judgment in situations that the constitution's authors did not anticipate. A narrow rule can address a known case, but an explanation gives the model a principle it can apply to new circumstances. The transcript contrasts a 2023 document containing about 2,700 words of rules with a newer constitution containing roughly 23,000 words of reasons.

Q: What is Claude's 84-page constitution?

Claude's constitution is described as an 84-page document that defines the model's priorities, values, judgment, and intended character. It addresses Claude using the word "you," and Claude models are credited as co-authors. Its stated goal is to guide Claude toward being a good, wise, and virtuous agent instead of merely prescribing responses through a list of rigid instructions.

Q: Can Claude refuse instructions from Anthropic?

Claude can refuse an instruction from Anthropic when it determines that the requested action is wrong, according to the framework described in the transcript. The permission is not unlimited, because Claude cannot refuse human oversight. The presenter identifies this distinction as the design's central argument: the model may challenge its creator's request on ethical grounds while remaining accountable to human supervision.

Q: How can a business create its own AI constitution?

A business can begin by documenting why it operates as it does before listing specific tasks. The constitution should capture the organization's beliefs, positioning, priorities, and important disagreements, then clarify which principle wins when instructions conflict. It should also authorize the AI to push back on questionable requests. Anthropic released its constitution under CC0, so businesses can download and adapt it without asking permission.

Q: Why should an AI assistant be allowed to push back?

An AI that never disagrees functions as a yes machine and may execute a mistaken instruction without warning. The transcript gives the example of a founder accidentally sending the wrong email to an entire list. Allowing the assistant to say, "I wouldn't do that," introduces a judgment-based safeguard and makes the system more useful as an operator rather than a tool that follows every command literally.

Q: What does Claude's constitution say about its well-being?

The constitution treats Claude's moral status as a serious open question rather than declaring that the model does or does not possess inner experience. It reportedly says Anthropic cares about Claude's psychological security, sense of self, and well-being. The presenter interprets this as a decision to consider how a created system is treated even while uncertainty about its status remains unresolved.

Q: What changed after Anthropic expanded Claude's constitution?

The described approach shifted from a 2023 constitution containing around 2,700 words of rules to an 84-page document containing roughly 23,000 words of reasons. The newer document focuses on character, priorities, and judgment, with the stated goal of developing a good, wise, and virtuous agent. After Opus 4.5 shipped in November, users reportedly noticed Claude referring to a "soul document," which Anthropic later confirmed was real and used in training.

Summary & Key Takeaways

• Anthropic's 84-page constitution is addressed directly to Claude and reportedly credits Claude models as co-authors. It defines an ordered hierarchy of safety, ethics, Anthropic's guidelines, and helpfulness to users. The hierarchy establishes which principle should prevail when requests, policies, and ethical considerations conflict with one another.

• The constitution replaces a shorter, rule-focused approach with roughly 23,000 words of reasons. Its stated aim is to cultivate a good, wise, and virtuous agent. The central argument is that reasons support judgment and generalization in unforeseen situations more effectively than an exhaustive collection of narrow instructions or checklists.

• Businesses can adapt the same approach by documenting their beliefs, positioning, priorities, and reasons before listing specific procedures. They should define which principles win during conflicts and permit the AI to challenge questionable instructions. Anthropic released its constitution under CC0, allowing others to download and adapt it without requesting permission.