How to Build a Local RAG Agent with DeepSeek-R1

TL;DR

Build a local retrieval-augmented generation application by loading a PDF, splitting its text into chunks, converting those chunks into Ollama embeddings, and indexing them in an in-memory vector store. A Streamlit interface accepts questions, retrieves related document chunks through similarity search, and sends that context to a locally installed DeepSeek-R1 1.5B model for concise, factual answers.

Transcript

hello all my name is kushak and welcome to my YouTube channel so guys in this specific video we are going to create an endtoend rag application with the help of deep seek and this entire deep seek will be specifically installed in our local with the help of AMA I'm also going to use AMA embedding and uh try to show you completely end to end like ho... Read More

Key Insights

• A local RAG pipeline is built from document ingestion, text chunking, embedding generation, vector indexing, retrieval, and answer generation. Each stage is implemented as a separate function, making the flow from an uploaded PDF to a contextual answer explicit and easier to follow.
• PDFPlumberLoader is used to extract content from the uploaded PDF. The Streamlit file uploader accepts a single PDF in the demonstrated configuration, saves it under the local document store path, and then passes that saved file path into the document-loading function.
• RecursiveCharacterTextSplitter divides the raw PDF documents into smaller chunks before indexing. The configuration also enables the starting index, allowing the processed document chunks to retain information about where their text began within the original loaded content.
• Ollama embeddings convert document text chunks into vector representations on the local machine. Those vectors are stored in an InMemoryVectorStore, so the demonstration does not depend on a third-party vector database or a cloud-hosted storage service.
• The in-memory vector store supports similarity search for retrieving context related to a user's question. The retrieval function sends the query to the vector database and returns related documents, with the transcript describing the matching process as using cosine similarity.
• The answer prompt defines the model as an expert research assistant and directs it to use the provided document context. It also instructs the model to say that it does not know when uncertain, remain concise and factual, and limit its answer to two or three sentences.
• OllamaLLM connects the generation chain to the locally installed DeepSeek-R1 1.5B model. The prompt template, user query, and joined context text are invoked together, producing the response that the Streamlit application presents after the document has been analyzed.
• The Streamlit workflow runs the processing stages automatically after a PDF is uploaded. It saves the file, loads its contents, chunks the documents, indexes the chunks, retrieves related passages for each user message, generates an answer, and displays that answer in the interface.

Questions & Answers

Q: How do you build a local RAG agent with DeepSeek-R1?

Create a Streamlit application that accepts a PDF, saves it to a local document store, and loads its content with PDFPlumberLoader. Split the loaded documents with RecursiveCharacterTextSplitter, create vectors through Ollama embeddings, and add the chunks to an InMemoryVectorStore. For each question, retrieve related documents with similarity search and send their joined text, along with the query, to DeepSeek-R1 1.5B through OllamaLLM.

Q: What libraries and components are used in the RAG application?

The application uses Streamlit for the user interface, PDFPlumberLoader for reading PDF content, and RecursiveCharacterTextSplitter for creating document chunks. It uses Ollama embeddings to convert text into vectors, InMemoryVectorStore to index those vectors, ChatPromptTemplate to structure the instructions and inputs, and OllamaLLM to run the locally installed DeepSeek-R1 1.5B model for answer generation.

Q: How is an uploaded PDF processed by the application?

After the user uploads a PDF, the save function writes it to the configured local document store and returns its file path. PDFPlumberLoader then loads the document content. RecursiveCharacterTextSplitter divides the raw documents into chunks and records their starting indexes. Finally, the indexing function adds those chunks to the in-memory vector store using the configured Ollama embedding model.

Q: How does the application retrieve relevant PDF context?

The application defines a function that receives the user's question and calls similarity search on the in-memory document vector database. The transcript identifies this matching process as cosine similarity. The returned documents are treated as relevant context, and their content is joined into a context string before the language model is asked to generate a concise answer.

Q: How are Ollama embeddings used in this RAG pipeline?

Ollama embeddings transform the text chunks produced from the PDF into vector representations. The application initializes its InMemoryVectorStore with that embedding model, then adds the chunked documents to the store. This makes the chunks searchable by semantic similarity when a user submits a query, while keeping the embedding and vector-indexing workflow on the local machine.

Q: How does the prompt reduce unsupported RAG answers?

The prompt tells the model that it is an expert research assistant and explicitly directs it to answer from the supplied document context. It also says to state that it does not know when it is unsure, remain factual, and keep the response to a maximum of two or three sentences. These instructions encourage concise answers grounded in retrieved PDF passages.

Q: Why does the example use an in-memory vector store?

The in-memory vector store lets the example index and search document vectors without depending on a third-party vector database or a cloud platform. It is initialized directly with the Ollama embedding model and receives the processed document chunks. The presenter notes that cloud storage can also be used, but chooses local storage to demonstrate a self-contained implementation.

Q: How do you run and use the Streamlit RAG application?

Install the required libraries and ensure the specified DeepSeek-R1 1.5B model is available locally through Ollama. Activate the project environment, then launch the Python application with Streamlit. In the displayed interface, browse for a PDF and upload it. The application saves, loads, chunks, and indexes the document, after which a user message triggers retrieval and answer generation.

Summary & Key Takeaways

• The application uses Streamlit to provide a simple interface for uploading and analyzing one PDF. An uploaded file is saved in a local document store, loaded through PDFPlumberLoader, and passed into the processing pipeline. The example keeps the PDF and its derived vector representation local rather than requiring cloud storage or a third-party vector database.

• The document processing pipeline divides loaded PDF content with RecursiveCharacterTextSplitter and preserves the starting index. Ollama embeddings convert the resulting chunks into vectors, which are added to an InMemoryVectorStore. When a question arrives, similarity search retrieves related document chunks that can supply relevant context to the language model.

• Answer generation combines the user query with text joined from the retrieved documents. A ChatPromptTemplate instructs the research assistant to rely on the supplied context, admit uncertainty, remain factual, and answer in two or three sentences. OllamaLLM runs the locally installed DeepSeek-R1 1.5B model, while Streamlit displays the generated response.