Recursive Language Models with Alex Zhang - Weaviate Podcast #142!

Alex Zhang, a PhD student at MIT, joins the Weaviate Podcast to discuss Recursive Language Models (RLMs), a new abstraction for designing agent harnesses. Instead of the standard ReAct-style loop that stuffs every tool observation into an ever-growing prompt, an RLM treats the prompt as a variable in a program. The model writes code that manipulates its own context and spawns recursive LLM calls over pieces of it. The published headline was long-context performance, but the deeper intention, inspired by how DSPy programmers decompose tasks, is letting the model do that decomposition itself, relieving context pressure so each call only sees a small, local, in-distribution problem. Stacking tool outputs into one giant sequence is terribly out of distribution for how models were trained. Frontier labs spend enormous resources making those long trajectories in-distribution, but RLMs aim to get more from the model without that.From there the conversation turns to PrimeAgent, Prime Intellect's production harness centered on an RLM, where the only tool is a persistent IPython REPL: search, compaction, sub-agents, and skills are all callable inside code, yielding strong results on ARC-AGI-3 and long-horizon benchmarks with big token savings. Alex shares why they moved from prompt tuning to RL on the weights of Qwen3, and why Claude Code's dynamic workflows are RLM-like but constrained by legacy tool-calling defaults.Alex then previews an unreleased library for speculative programmatic tool calling. Borrowing from speculative decoding and CPU speculative execution, a shadow executor queues up sub-agent calls while the main model is still generating code, delivering roughly 2x speedups and better GPU utilization for locally served agents. The discussion moves into running RLMs in the cloud with sandbox providers like Modal and Daytona, security concerns around code execution and prompt injection, and why serving engines should be redesigned around agents. It closes with RLMs and search, combining ColBERT-style retrievers as callable tools for problems like agent log analysis, and a theory that everything "long" is naturally decomposable. The podcast concludes with exciting ideas about where AI goes next from principled harness design to whether the transformer itself gets replaced with a newer neural architecture design.