De audio van deze aflevering is mogelijk defect.
How to fix it?
OVER DEZE AFLEVERING
We discuss Accurate KV Cache Quantization with Outlier Tokens Tracing, a deep dive into improving the efficiency of LLM inference. The authors enhance KV Cache quantization, a technique for reducing memory and compute costs during inference, by introducing a method to identify and exclude outlier tokens that hurt quantization accuracy, striking a better balance between efficiency and performance.
Learn more about AI observability and evaluation, join the Arize AI Slack community or get the latest on LinkedIn and X.
Engels
Verenigde Staten
OUDE AFLEVERINGEN ZOEKEN
Zoek naar eerdere afleveringen van Deep Papers.
ANDERE AFLEVERINGEN IN DEZE PODCAST
What if your LLM could think ahead—preparing answers before questions are even asked?
In this week's paper read, we dive into a groundbreaking new paper from researchers at Letta, introducing sleep-time compute: a novel technique that lets models do their heavy lifting offline, well before the…
The authors of the new paper *Self-Adapting Language Models (SEAL)* shared a behind-the-scenes look at their work, motivations, results, and future directions.
The paper introduces a novel method for enabling large language models (LLMs) to adapt their own weights using self-generated data and trai…
This week we discuss The Illusion of Thinking, a new paper from researchers at Apple that challenges today’s evaluation methods and introduces a new benchmark: synthetic puzzles with controllable complexity and clean logic.
Their findings? Large Reasoning Models (LRMs) show surprising failure mode…
In this week's episode, we talk about Elastic Reasoning, a novel framework designed to enhance the efficiency and scalability of large reasoning models by explicitly separating the reasoning process into two distinct phases: thinking and solution.
This separation allows for independent alloca…
Disclaimer: De podcast en de illustraties die op deze pagina zijn ingesloten, zijn afkomstig van Arize AI, eigendom van de eigenaar en niet gelieerd aan of goedgekeurd door Listen Notes, Inc.
BEWERKEN
Bedankt voor uw hulp bij het up-to-date te houden van de podcast-database .
