We’re really moving from a world where humans are authoring search queries and humans are executing those queries and humans are digesting the results to a world where AI is doing that for us.
Jeff Huber, CEO and co-founder of Chroma, joins Hugo to talk about how agentic search and retrieval are changing the very nature of search and software for builders and users alike.
We Discuss:
“Context engineering”, the strategic design and engineering of what context gets fed to the LLM (data, tools, memory, and more), which is now essential for building reliable, agentic AI systems;
Why simply stuffing large context windows is no longer feasible due to “context rot” as AI applications become more goal-oriented and capable of multi-step tasks
A framework for precisely curating and providing only the most relevant, high-precision information to ensure accurate and dependable AI systems;
The “agent harness”, the collection of tools and capabilities an agent can access, and how to construct these advanced systems;
Emerging best practices for builders, including hybrid search as a robust default, creating “golden datasets” for evaluation, and leveraging sub-agents to break down complex tasks
The major unsolved challenge of agent evaluation, emphasizing a shift towards iterative, data-centric approaches.
You can also find the full episode on Spotify, Apple Podcasts, and YouTube.
You can also interact directly with the transcript here in NotebookLM: If you do so, let us know anything you find in the comments!
👉 Want to learn more about Building AI-Powered Software? Check out our Building AI Applications course. It’s a live cohort with hands on exercises and office hours. Our final cohort is in Q1, 2206. Here is a 25% discount code for readers. 👈
Oh! One more thing: we’ve just announced a Vanishing Gradients livestream for January 21 that you may dig:
A Builder’s Guide to Agentic Search & Retrieval with Doug Turnbull and John Berryman (register to join live or get the recording afterwards).
Show notes
Context Rot: How Increasing Input Tokens Impacts LLM Performance by The Chroma Team
AI Agent Harness, 3 Principles for Context Engineering, and the Bitter Lesson Revisited
From Context Engineering to AI Agent Harnesses: The New Software Discipline
Generative Benchmarking by The Chroma Team
Effective context engineering for AI agents by The Anthropic Team
Making Sense of Millions of Conversations for AI Agents by Ivan Leo (Manus) and Hugo
How we built our multi-agent research system by The Anthropic Team
Join the final cohort of our Building AI Applications course in Q1, 2026 (25% off for listeners)
Timestamps
00:00 Introduction to AI-Driven Search
00:42 The Role of Context Engineering
01:56 Challenges in AI Search and Retrieval
04:03 Jeff’s Background and Chroma’s Mission
08:09 The Evolution of Search and AI’s Role
17:27 Agent Harness and Context Engineering
23:51 Harness Engineering and Future Directions
27:36 Relevance of Multi-Agent Design
28:06 Embracing Sub-Agents
29:39 Harness Engineering
30:36 Chroma’s Evolution and Context Windows
31:00 Challenges of Large Context Windows
32:52 Context Rot & Performance Degradation in LLMs
36:24 Best Practices for Context Engineering
38:43 Evaluation of Context Engineering
45:57 Adapting to Rapid Changes in AI
48:44 Focusing on What Matters in AI
50:23 Closing Remarks and Future Directions
👉 Want to learn more about Building AI-Powered Software? Check out our Building AI Applications course. It’s a live cohort with hands on exercises and office hours. Our final cohort is in Q1, 2026. Here is a 25% discount code for readers. 👈
