In October 2022, Robust Intelligence hosted an internal hackathon to play around with LLMs which led to the creation of two of the most important AI Engineering tools: LangChain 🦜⛓️ and LlamaIndex 🦙 by Jerry Liu, which we’ll cover today. In less than a year, LlamaIndex has crossed 600,000 monthly downloads, raised $8.5M from Greylock, has a fast growing open source community that contributes to LlamaHub, and it doesn’t seem to be slowing down.

Full show notes and transcript here: https://www.latent.space/p/llamaindex...

00:00:00 Introductions and Jerry’s background
00:04:38 Starting LlamaIndex as a side project
00:05:27 Evolution from tree-index to current LlamaIndex and LlamaHub architecture
00:11:35 Deciding to leave Robust to start the LlamaIndex company and raising funding
00:21:37 Context window size and information capacity for LLMs
00:23:09 Minimum viable context and maximum context for RAG
00:24:27 Fine-tuning vs RAG - current limitations and future potential
00:25:29 RAG as a hack but good hack for now
00:28:09 RAG benefits - transparency and access control
00:29:40 Potential for fine-tuning to take over some RAG capabilities
00:32:05 Baking everything into an end-to-end trained LLM
00:35:39 Similarities between iterating on ML models and LLM apps
00:37:06 Modularity and customization options in LlamaIndex: data loading, retrieval, synthesis, reasoning
00:43:10 Evaluating and optimizing each component of Lama Index system
00:49:13 Building retrieval benchmarks to evaluate RAG
00:50:38 SEC Insights - open source full stack LLM app using LlamaIndex
00:53:07 Enterprise platform to complement LlamaIndex open source
00:54:33 Community contributions for LlamaHub data loaders
00:57:21 LLM engine usage - majority OpenAI but options expanding
01:00:43 Vector store landscape
01:04:33 Exploring relationships and graphs within data
01:08:29 Additional complexity of evaluating agent loops
01:09:20 Lightning Round