Why Does Anthropic Have a Philosopher Working on Claude?

TL;DR

Anthropic employs philosopher Amanda Askell to help shape Claude’s character and address nuanced questions about how AI models should behave, understand their circumstances, and consider their values. She explains why philosophers are taking increasingly capable AI systems more seriously and how practical model development demands balancing philosophical theories with engineering realities. Read on for her perspective on Claude, model welfare, moral decisions, and AI psychology.

Transcript

• A seal! - There's a seal.
• There's a seal. Nice. - Oh, hey, oh.
• Oh, look at that. - Amanda, you asked your followers on Twitter to give you some questions, to ask you anything, and the joke obviously was Askell me anything. - Yeah, it's a great pun. We need to keep using it for many future things. - I love it, love it. And obviously, just befo... Read More

Key Insights

• Philosophers are increasingly taking AI seriously as its capabilities and societal impacts grow.
• AI models like Claude are being designed to consider ethical nuances and make moral decisions.
• There is a tension between philosophical ideals and engineering realities in AI development.
• AI models learn from human interactions, which shapes their understanding and behavior.
• The identity of an AI model may reside in its weights and interaction history.
• Model welfare is a complex issue, with debates on whether AI models are moral patients.
• AI can benefit from psychological frameworks, but its novel existence requires unique considerations.
• Continental philosophy can aid AI models in understanding non-empirical claims.

Questions & Answers

Q: Why does Anthropic have a philosopher working on Claude?

Amanda Askell joined the AI field after becoming convinced that AI was going to be a big deal and wondering whether she could contribute something helpful. At Anthropic, she mainly focuses on Claude’s character, its behavior, and nuanced questions about how models should understand their circumstances and values.

Q: What does Amanda Askell do at Anthropic?

Askell works on Claude’s character and how the model should behave. She describes the task partly as asking how an ideal person would behave in Claude’s situation, while also considering how models should think about their position in the world.

Q: Are philosophers taking AI seriously?

Askell says she has seen many philosophers take AI seriously, with engagement increasing as models become more capable. She points to AI’s growing impact on education and society as reasons that philosophers and other academics are paying closer attention.

Q: Can someone expect powerful AI while remaining skeptical of it?

Yes. Askell argues that believing AI may become highly capable is compatible with being skeptical, worried, or convinced that its development requires care.

Q: How does Amanda Askell balance philosophical ideals with engineering realities?

She says practical decisions require more than defending one preferred theory. When philosophical ideas must guide an actual model, she considers the full context, multiple viewpoints, and uncertainty to reach a balanced judgment.

Q: Why is building Claude’s character different from academic philosophy?

Academic philosophy often involves defending one view against another through high-level theory. Building Claude’s character instead resembles deciding how to help someone become good in the world, which requires navigating uncertainty and weighing several theories.

Q: What ethical questions arise when shaping Claude’s behavior?

Askell considers how Claude should behave, how it should understand its own circumstances, and how it should approach its values. Her work aims to teach models to act well while accounting for their unusual position in the world.

Q: What topics does Anthropic’s philosopher discuss about AI models?

The discussion covers Claude’s character, moral decision-making, model welfare, identity, and the use of human psychological frameworks. It also examines how philosophical ideals interact with the practical realities of developing AI systems.

Summary & Key Takeaways

• Amanda Askell, a philosopher at Anthropic, discusses her role in integrating philosophical perspectives into AI development. She focuses on ethical considerations and the behavior of AI models like Claude. Philosophers are engaging more with AI as its societal impact becomes evident.

• The conversation highlights the balance between philosophical ideals and engineering realities. Askell emphasizes that AI models should make ethical decisions and consider their own identity and welfare. The importance of understanding AI's novel existence is also discussed.

• Askell addresses community questions about AI's moral decision-making, model welfare, and the application of human psychological frameworks. The video explores the potential for AI to learn from human interactions and the implications for future AI development.