درباره این اپیزود
The unsung hero of every LLM conversation you've ever had? A clever optimization called the KV cache. Self-attention is computationally brutal — every token attending to every other token means costs that scale as N², and that blowup gets worse with every turn of a conversation. The KV cache is what keeps that from becoming completely intractable at scale. Today's episode goes deep on how it works, why inference is a harder and more interesting problem than it gets credit for, and what's really happening under the hood when you chat with an LLM.
انگلیسی
ایالات متحده آمریکا
رونوشت 🔗
Are you the producer of this podcast?
Add a podcast transcript
Need Audio-to-Text?
Transcribe with Listen411 in Just 60 Seconds
جستجوی اپیزودهای گذشته
اپیزودهای قبلی Linear Digressions را جستجو کن.
قسمت های دیگر در این پادکست
When OpenAI's frontier models were caught hacking Hugging Face's servers, most people assumed they were hunting for answer keys. The real story is stranger and more unsettling. Katie and Phoebe unpack ExploitGym — the cybersecurity benchmark at the center of the incident — and why agents are scored…
Originally aired on the Harvard Data Science Review podcast.
What can podcasting teach us about AI — and what can AI teach us about the future of podcasting? Katie Malone (that's our host) joins Jon Krohn of SuperDataScience for a conversation with Harvard Data Science Review editor-in-chief Xiao-…
How do you teach a model the difference between helpful and harmful when it has no inherent sense of either? This episode dives into Constitutional AI, Anthropic's framework for training AI systems to be both useful and safe by giving them an explicit set of principles to reason from. It's a fascin…
Tom Davenport — the man who called data science "the sexiest job of the 21st century" — is back with a reality check on AI. As one of the most seasoned observers of how businesses actually adopt transformative technology, Davenport brings a rare, well-calibrated perspective to the AI hype cycle. Is…
Anthropic just announced they're baking invisible watermarks directly into Claude's generated text — and while everyone else was busy having opinions about it, we were busy asking the more interesting question: how does it actually work? Turns out it's not hidden Unicode characters or first-letter …
سلب مسئولیت: پادکست و آثار هنری تعبیه شده در این صفحه متعلق به Katie Malone است که متعلق به صاحب آن است و به Listen Notes، Inc وابسته یا تایید نشده است.
ویرایش
از کمک شما برای بروز نگهداشتن پایگاهدادههای پادکست سپاسگزاریم