How to give AI agents long term memory with Zep

118.7K views
•
July 14, 2025
by
Nate Herk | AI Automation
YouTube video player
How to give AI agents long term memory with Zep

TL;DR

Long term memory is built with a user relational graph using Zep memory, enabling AI agents to remember entities and relationships beyond the short memory window. The video shows setting up Zep memory, linking it to agents, and shows strategies to reduce token costs by filtering relevant facts instead of dumping the entire graph every time.

Transcript

Today I'm going to be talking about the next evolution of AI agents, which is knowledge graphs for long-term memory. When we set up these relational graphs, our agents are able to look through them, understand relations between different entities and different things that you've talked about in the past, and use them to actually be way more intelli... Read More

Key Insights

  • Zep memory creates a user relational graph that stores long term memory across sessions.
  • Long term memory in AI agents enables recalling user preferences, locations, and past conversations.
  • A simple memory stores past interactions but lacks true long term memory beyond a few chats.
  • As the graph grows, the amount of tokens used in responses increases, driving up costs.
  • Token cost can be reduced by filtering and retrieving only the most relevant facts from the graph.
  • HTTP requests offer more control over memory retrieval than native integrations.
  • Merging filtered graph facts with conversation history preserves context with fewer tokens.
  • A balance is needed between memory depth and token economy to maintain performance.

Explore YouTube Video Summarizer or Get YouTube Transcript Extractor

Questions & Answers

Q: How does Zep memory differ from simple memory in AI agents?

Zep memory uses a user relational graph to store long term memory about entities and their relationships, which allows the agent to infer connections and recall broader context across sessions. In contrast, simple memory typically stores only recent interactions and has a limited window for memory, restricting the agent to short term context and making it harder to personalize or maintain continuity over time.

Q: What is a practical drawback of expanding a knowledge graph too much?

As more memories and relationships accumulate in the graph, the agent has to process more context and tokens for each response. This increases API usage and computation, which can quickly raise costs and slow down responses if not managed properly. Implementing selective retrieval methods helps mitigate this issue.

Q: What strategy does the video propose to reduce token usage when querying memory?

The video proposes a method that uses HTTP requests to access Zep memory and filter for only the most relevant facts related to the current user input. By selecting a subset of facts, merging them with short term memory, and then querying the AI model, the system reduces the token load while preserving essential context and personalization.

Q: How is a new user graph initialized and populated?

A new user graph starts with a fresh user entity and sessions. The agent asks initial questions to gather preferences and basic data, then creates relationships such as interests, location, and activities. Over time, additional relations are added as conversations occur, expanding the graph and enriching the agent’s memory.

Q: What is the role of sessions in this memory system?

Sessions store pairs of human messages and AI responses, providing a short term memory of the dialogue. This context window helps the AI maintain continuity within a conversation and can be linked to the long term memory in the graph to improve future interactions.

Q: Why is token cost a concern when growing memory graphs?

Token cost grows with the amount of information being processed and included in each response. A larger graph means more data to scan for relevance, which increases token usage. Efficient retrieval strategies and selective memory loading help control costs without sacrificing usefulness.

Q: What does the video suggest about maintaining balance between memory depth and performance?

The video suggests monitoring both the depth of the memory graph and the performance implications of retrieving and processing it. It recommends strategies like filtering relevant facts, using HTTP access for better control, and merging memory with conversation history to maintain responsiveness while preserving important context.

Q: What is the purpose of linking a user graph to an AI agent in this tutorial?

Linking a user graph to an AI agent enables the agent to understand and recall who the user is, what they care about, and their past interactions. This results in more personalized and context-aware responses, allowing the agent to tailor suggestions, questions, and actions based on long term memory rather than only recent chats.

Summary & Key Takeaways

  • After introducing short term and long term memory concepts, the video explains how a relational graph with Zep allows an AI agent to remember user data and connections across sessions. It emphasizes the cost of large memory graphs and the need to manage tokens.

  • The creator demonstrates how to wire memory to Zep, create a new user graph, and build a growing knowledge graph that captures preferences, locations, and activities. The example includes updates as conversations evolve and the need for efficient retrieval strategies.

  • A practical method is presented to reduce token load by HTTP-based memory access that fetches only the most relevant facts from the user graph, merging them before sending to the chat model for response.


Read in Other Languages (beta)

Share This Summary 📚

Explore More Summaries from Nate Herk | AI Automation 📚