Long Live Context Engineering - with Jeff Huber of Chroma
Jeff Huber of Chroma joins us to talk about what actually matters in vector databases in 2025, why “modern search for AI” is different, and how to ship systems that don’t rot as context grows. Full show notes: https://www.latent.space/p/chroma 00:00 Introductions 00:48 Why Build Chroma 02:55 Information Retrieval vs. Search 04:29 Staying Focused in a Competitive AI Market 08:08 Building Chroma Cloud 12:15 Context Engineering and the Problems with RAG 16:11 Context Rot 21:49 Prioritizing Context Quality 27:02 Code Indexing and Retrieval Strategies 32:04 Chunk Rewriting and Query Optimization for Code 34:07 Transformer Architecture Evolution and Retrieval Systems 38:06 Memory as a Benefit of Context Engineering 40:13 Structuring AI Memory and Offline Compaction 45:46 Lessons from Previous Startups and Building with Purpose 47:32 Religion and Values in Silicon Valley 50:18 Company Culture, Design, and Brand Consistency 52:36 Hiring at Chroma: Designers, Researchers, and Engineers
Video Chapters
- 0:31 Meet Chroma: The Open-Source Vector Database for the AI Era.
- 2:42 Unveiling Chroma: A Modern Retrieval Engine for Production AI.
- 5:40 The Power of a Contrarian Vision: Chroma's Unwavering Focus.
- 8:33 The Magic of `pip install chromadb`: Instant AI Database Power.
- 11:00 Human vs. LLM Search: Why Modern AI Demands a New Approach.
- 13:00 Debunking "RAG": A Controversial Take on Retrieval-Augmented Generation.
- 14:00 Context Engineering: The High-Stakes Art of AI Development.
- 15:30 The Chroma Way: Patient Building & Unrivaled Developer Experience.
- 17:30 Beyond Hype: Tackling Deep Technical Challenges for Lasting Impact.
- 25:00 The Future of AI Tools: Why Developer Experience is Everything.
Original Output
0:31 Meet Chroma: The Open-Source Vector Database for the AI Era. 2:42 Unveiling Chroma: A Modern Retrieval Engine for Production AI. 5:40 The Power of a Contrarian Vision: Chroma's Unwavering Focus. 8:33 The Magic of `pip install chromadb`: Instant AI Database Power. 11:00 Human vs. LLM Search: Why Modern AI Demands a New Approach. 13:00 Debunking "RAG": A Controversial Take on Retrieval-Augmented Generation. 14:00 Context Engineering: The High-Stakes Art of AI Development. 15:30 The Chroma Way: Patient Building & Unrivaled Developer Experience. 17:30 Beyond Hype: Tackling Deep Technical Challenges for Lasting Impact. 25:00 The Future of AI Tools: Why Developer Experience is Everything. Timestamps by McCoder Douglas