What is context engineering vs prompt engineering?

TL;DR

Context engineering covers more than prompts, assembling everything the model sees, including memory, documents, and tools, to achieve accurate results. Prompt engineering is the craft of wording instructions and examples, while context engineering builds the entire inference environment. Together they enable agentic AI with memory, RAG, and tools for dynamic tasks.

Transcript

I think by now most of us are familiar with the term prompt engineering. It's the process of crafting the input text used to prompt a large language model, including instructions and examples and formatting cues. It's what steers the LLM's behavior and output. Now, context engineering, on the other hand, is the broader discipline of programmaticall... Read More

Key Insights

• Prompt engineering shapes output by instructions and examples, while context engineering structures the full inference environment.
• Context engineering integrates memory, long term and short term, to maintain relevant knowledge during multi step tasks.
• RAG connects agents to dynamic knowledge sources by selective retrieval of relevant sections rather than entire documents.
• Agents require tools to interact with databases, APIs, and live data to perform tasks beyond the model’s built in capabilities.
• Memory management has two forms: short term to summarize conversations and long term to recall user preferences.
• State management tracks progress across steps so tasks like booking a trip stay coherent.
• Tool interfaces specify what tools do, when to use them, and any constraints that apply.
• Dynamic prompts fuse static instructions with current context for flexible, accurate reasoning.
• More videos from this channel:
  • How does chunkless RAG use document structure to answer
  • How do word embeddings turn words into numbers?
  • What Is Explainable AI and How Does It Help?
• More from this channel: IBM Technology

Questions & Answers

Q: What is context engineering and how does it differ from prompt engineering?

Context engineering is the system level discipline of providing the model with everything it needs to plausibly accomplish a task, including prompts, retrieved documents, memory, and tools. Prompt engineering focuses on crafting the input instructions and examples to steer the model. Context engineering thus orchestrates multiple components to produce accurate, actionable results across complex tasks.

Q: What role does memory play in context engineering?

Memory management in context engineering has short term and long term forms. Short term memory summarizes long conversations to stay within context windows, while long term memory stores user preferences and past interactions using vector databases. Together they ensure the agent remembers essential details and can retrieve relevant information when needed.

Q: How does retrieval augmented generation (RAG) fit into the framework?

RAG connects an agent to dynamic knowledge sources by using hybrid search that combines semantic and keyword matching. It retrieves relevant sections of documents, not whole files, and returns only contextually relevant parts to the agent. This makes responses more accurate and reduces noise from irrelevant information.

Q: Why are tools important for agents according to the video?

Tools bridge the gap between LLMs and real world actions. They allow querying databases, fetching live pricing, or deploying infrastructure. Context engineering defines the interfaces for these tools, describes when to use them, and imposes constraints so the agent can operate safely and effectively within a task.

Q: What is the function of role assignment in prompt engineering?

Role assignment tells the LLM who it should be, such as saying you are a senior Python developer reviewing code for security vulnerabilities. This adoption of a persona shapes vocabulary, concerns, and output style, leading to output that better matches the desired task and quality.

Q: What is the purpose of few shot examples in prompt engineering?

Few shot examples demonstrate the exact input/output format and style required by the user. By showing 2 or 3 examples, the model learns the expected structure, such as a specific JSON schema or a particular response format, improving consistency and reducing misalignment.

Q: What is chain of thought prompting and when is it useful?

Chain of thought prompting, or asking the model to explain its reasoning, forces the model to reveal its step by step process. It helps with complex reasoning tasks by preventing premature conclusions, increasing transparency, and improving the reliability of the final answer when the task requires careful internal deliberation.

Q: How do constraints support better model outputs according to the video?

Constraint setting explicitly defines boundaries, such as word limits or information scope. By restricting outputs to the provided context and applying timing or content rules, constraints prevent tangents and ensure the response stays focused, relevant, and within acceptable limits for the task at hand.

Summary & Key Takeaways

• Context engineering expands beyond prompts to include memory, retrieval, and tools to guide LLM behavior. It emphasizes the system level setup that enables accurate, dynamic responses. The video contrasts this with traditional prompt engineering and introduces agentic AI concepts and practical techniques.

• The discussion explains techniques like role assignment, few shot examples, chain of thought prompting, and constraint setting as core tools for prompt engineering, while detailing how memory, state management, RAG, and tool interfaces enable context engineering.

• The speaker uses an example agent Graeme for travel booking to illustrate how proper context and tools prevent errors like wrong location or policy violations. It highlights the need for integrated systems for reliable AI workflows.