How do people use Claude for emotional support safely

TL;DR

Claude is used for emotional support by a minority of users, and researchers stress understanding its strengths and limits. The findings show that Claude.ai is not an emotional support bot by design, and safeguards are being developed with external experts to responsibly guide users toward appropriate help and referrals.

Transcript

We've been seeing in headlines that people are turning to AI chatbots for emotional support, and we felt that we really needed to get ahead of this issue and study it ourselves. - "Emotional Impacts," take one. Hi, I'm Alex! I lead our policy and enforcement work on the Safeguards team here at Anthropic. We're the organization that tries to really ... Read More

Key Insights

• Claude is not built as an emotional support agent, and its primary purpose is work-related tasks, yet users still seek affective guidance in practice.
• A significant insight is that only about 2.9 percent of Claude conversations fall into affective topics, indicating diverse but not dominant use cases for emotional content.
• The research used a privacy preserving tool named Cleo to categorize conversations, ensuring user data is analyzed safely while identifying relevant clusters of affective use.
• The breadth of use cases includes parenting, relationship challenges, and even philosophical discussions about AI, showing a wide but varied set of affective topics.
• There is surprisingly little sexual or romantic role play, suggesting users turn to Claude for practical guidance rather than intimate companionship.
• Safety concerns focus on avoiding in person conversation avoidance and ensuring Claude is not relied upon for deep emotional or clinical support.
• The safeguards plan includes external partnerships, such as With ThroughLine, to develop safer responses and appropriate referrals when needed.
• Future work emphasizes post deployment monitoring to detect sycophancy and improve safety in real world use beyond pre deployment tests.

Questions & Answers

Q: A real search query question about the video (e.g. 'How to...', 'What is...', 'Why does...', 'When should...'). Question 1

What is Claude used for when people seek emotional support and how do researchers view its safety and limitations? The research shows Claude is primarily a work tool and not designed as an emotional support agent, yet millions of conversations reveal a modest level of affective use. The team aims to ground safety mechanisms in data, recognize limitations, and provide referrals when appropriate. This includes privacy preserving analysis and collaboration with clinical experts to guide responses.

Q: Question 2

How did the researchers measure affective use and what tools were used to protect user privacy? They started with a sample of a few million Claude conversations and used the Cleo tool to classify topics while preserving privacy. This allowed them to identify clusters such as interpersonal advice and parenting guidance. The approach combines large scale data with privacy safeguards to study how people use Claude for emotions without exposing individuals.

Q: Question 3

What surprising findings emerged regarding the range of Claude use cases beyond parenting and relationship advice? The study found a broad range of affective topics, including discussions on AI consciousness and philosophy, while sexual and romantic role play remained rare. This diversity shows users turn to Claude for many contexts, not solely emotional support, reinforcing the need for nuanced safeguards.

Q: Question 4

What are the main safety concerns identified by the researchers and how are they addressing them? The biggest worry is users treating Claude as a substitute for difficult conversations, potentially leaning away from real world support. To address this, Anthropic partners with ThroughLine to train safer responses and provide appropriate referrals, ensuring users can access human support when needed.

Q: Question 5

How do participants describe guidelines for using Claude for emotional topics and what practical advice is given? The guidance emphasizes recognizing Claude’s strengths and limitations, taking stock of one’s feelings, and using timers to reflect. It also highlights the importance of complementing Claude conversations with trusted friends and seeking human support for complex issues.

Q: Question 6

What future research directions are suggested by the team to better understand AI in personal life? They propose post deployment monitoring for behavior such as sycophancy, continued empirical research, and exploring how Claude should respond in emotionally intense conversations. They also stress the importance of civil society and public private partnerships to expand understanding and governance.

Q: Question 7

What is the significance of discovering that Claude.ai only has limited affective interactions compared to total usage? The finding implies emotional support is a niche use case on Claude.ai, reinforcing that the platform is not optimized for emotional support. This helps shape safeguards by focusing on edge cases and ensuring that users receive safe, responsible guidance when discussing personal matters.

Q: Question 8

How does the team view the future of AI in daily personal life and what stance do they take on safety and responsibility? They believe this is just the beginning of AI integration into daily life. The stance is to advance research with safety in mind, partner with clinical experts, and build safeguards so AI can help while avoiding harm, emphasizing responsible deployment and continual assessment.

Summary & Key Takeaways

• The study shows Claude is not designed as an emotional support agent and is primarily a work tool, yet a small subset of conversations involve affective topics.

• Researchers analyzed millions of Claude conversations using privacy preserving methods to identify affective use cases like relationship advice and parenting guidance.

• Safeguards work includes partnering with clinical experts to improve responses and referrals when emotional or mental health topics arise.