How does Shopify use AI to guide decision making

TL;DR

AI guides Shopify’s decisions by using agents like River to assist in open channels, enabling transparent collaboration and rapid prototyping. Pruning and rebuilding are central to creating durable systems, while AI helps with pull requests and decision making through an AI council and distributed, memory-enabled assistants.

Transcript

Things need to be pruned. You cannot make things better and better by adding stuff. You can't. You must prune. You must rebuild. You must create an end for things. >> Toby, welcome back. >> Shane, it's so good to be back. I'm glad you're doing this again. How are you using AI internally in Shopify? >> We find some ways for it to be supportive. No, ... Read More

Key Insights

• AI agents supplement human work by participating in open channels, enabling scalable collaboration.
• Pruning and rebuilding are essential to durable software design, not just adding incremental features.
• Memory enabled AI like River can summarize conversations, propose tickets, and generate prototypes within Slack workflows.
• The skill of using AI effectively becomes a core capability for teams in distributed work environments.
• Historical lessons in computing, such as file systems, influence how Shopify thinks about AI architectures.
• A well designed AI needs personality and boundaries to be useful and trusted by engineers.
• Open channels and transparency reduce risk and improve adoption of AI tools across teams.
• Strategic decisions benefit from an AI council that surfaces diverse viewpoints and data-driven tradeoffs at scale.

Questions & Answers

Q: How does River differ from a traditional software assistant?

River is integrated into Slack with memory by channel and has a profile and personality, allowing it to participate in conversations, summarize discussions, and propose concrete actions like tickets or code changes. Unlike isolated tools, River operates in an open, collaborative environment where teams can see its reasoning and outputs, increasing trust and speed.

Q: What is meant by pruning in Shopify's AI philosophy?

Pruning refers to removing unnecessary components and simplifying systems rather than continuously adding features. This approach helps rebuild robust architectures and prevents bloat, ensuring that the AI infrastructure remains maintainable and focused on delivering lasting value rather than short term gains.

Q: Why is an AI council used for decision making?

An AI council aggregates diverse AI perspectives to examine hard decisions, augment human judgment with data-driven insights, and surface tradeoffs that might not be obvious. This structure helps ensure that strategic choices are robust, aligned with long term goals, and resistant to overfitting or overreliance on a single model.

Q: How does Shopify use AI to handle pull requests?

AI assists in generating and coordinating pull requests by participating in conversations, proposing changes, and automating parts of the workflow. This reduces manual effort and speeds up the integration process, while keeping the collaboration context visible to engineers who review and approve the changes.

Q: What role does transparency play in Shopify’s AI usage?

Transparency is achieved by keeping River and other AI agents working in open channels, where interactions, decisions, and outputs are visible to the team. This openness helps with accountability, trust, and better learning as engineers can observe how AI reasoning evolves and how it contributes to tasks.

Q: What is the significance of memory in Shopify’s AI agents?

Memory allows agents like River to retain context across conversations and tasks, enabling more cohesive guidance and multi step workflows. This continuity helps users rely on AI for sustained work, such as tracking a feature from discussion to ticket creation and implementation, rather than starting over on each interaction.

Q: How does AI influence the long term skill set for engineers at Shopify?

AI becomes a core capability for engineers, shifting from pure coding to collaborative problem solving with agents, managing complex systems, and using AI to surface insights. This new skill set includes knowing when to rely on AI, how to question outputs effectively, and how to design interfaces that leverage AI responsibly.

Q: What is the potential downside of AI in decision making and how is it mitigated?

The risk includes overreliance on AI, loss of human judgment, or derailed decisions if models misinterpret data. Mitigation comes from maintaining human oversight, using AI councils to challenge conclusions, and designing AI with safeguards, clear boundaries, and explainable reasoning to preserve responsibility and accountability.

Summary & Key Takeaways

• Shopify employs AI agents to support complex decisions and to prototype ideas within a large, open Slack environment.

• The company emphasizes pruning, rebuilding, and creating lasting systems rather than simply adding features.

• River, an AI colleague with memory and personality, participates in conversations and can propose changes or create tickets based on discussions.