Efficient Streaming Language Models with Attention Sinks
Deploying Large Language Models (LLMs) in streaming applications such as multi-round dialogue, where long interactions are expected, is urgently needed but poses two major challenges. Firstly, during the decoding stage, caching previous tokens' Key and Value states (KV) consumes extensive memory. S…
Efficient Streaming Language Models with Attention Sinks (Paper Explained)
#llm #ai #chatgpt
How does one run inference for a generative autoregressive language model that has been trained with a fixed context size? Streaming LLMs combine the performance of windowed attention, but avoid the drop in performance by using attention sinks - an interesting phenomenon where t…
arxiv preprint - LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models
In this episode, we discuss LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models by Yukang Chen, Shengju Qian, Haotian Tang, Xin Lai, Zhijian Liu, Song Han, Jiaya Jia. The abstract describes "LongLoRA," a technique designed to efficiently expand the context size of large language m…
StreamingVLM: Real-Time Understanding for Infinite Video Streams
🤗 Upvotes: 26 | cs.CV, cs.AI, cs.CL
Authors:
Ruyi Xu, Guangxuan Xiao, Yukang Chen, Liuning He, Kelly Peng, Yao Lu, Song Han
Title:
StreamingVLM: Real-Time Understanding for Infinite Video Streams
Arxiv:
http://arxiv.org/a…
Inside Efficient AI: From GPUs to GPTs — Song Han
Song Han is an associate professor in Electrical Engineering and Computer Science whose research focuses on efficient AI computing. His work spans high-resolution computer vision for autonomous vehicles, more efficient image generation, improved GPT performance, and novel methods for training machi…
TriAttention: Efficient Long Reasoning with Trigonometric KV Compression
🤗 Upvotes: 70 | cs.CL, cs.CV
Authors:
Weian Mao, Xi Lin, Wei Huang, Yuxin Xie, Tianfu Fu, Bohan Zhuang, Song Han, Yukang Chen
Title:
TriAttention: Efficient Long Reasoning with Trigonometric KV Compression
Arxiv:
http://a…
AnyFlow: Any-Step Video Diffusion Model with On-Policy Flow Map Distillation
🤗 Upvotes: 79 | cs.CV, cs.AI
Authors:
Yuchao Gu, Guian Fang, Yuxin Jiang, Weijia Mao, Song Han, Han Cai, Mike Zheng Shou
Title:
AnyFlow: Any-Step Video Diffusion Model with On-Policy Flow Map Distillation
Arxiv:
http://ar…
SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformer
🤗 Upvotes: 53 | cs.CV
Authors:
Haoyi Zhu, Haozhe Liu, Yuyang Zhao, Tian Ye, Junsong Chen, Jincheng Yu, Tong He, Song Han, Enze Xie
Title:
SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformer
Arxiv:
…
VC-Attention: Value Smoothing and Softmax Casting for Low-bit Attention
🤗 Upvotes: 30 | cs.CV
Authors:
Xingyang Li, Dongyun Zou, Shining Zhang, Jiacheng Chen, Haocheng Xi, Lvmin Zhang, Jun-Yan Zhu, Song Han, Zhekai Zhang, Yujun Lin, Muyang Li
Title:
VC-Attention: Value Smoothing and Softmax Casting for Low-bit Attenti…
PODCASTIDE OTSINGUMOOTOR
Saate kasutada meie veebisaiti3,831,882taskuhäälingusaadete ja196,875,085osade otsimiseks inimeste, kohtade või teemade järgi.
Investeering teadmistesse annab parima tootluse, seega kuulan podcaste.
‐ Benjamin "Llamacorn" Franklin
MUUDA
Aitäh, et aitad hoida podcastide andmebaasi ajakohasena.
