Browser AI, WebGPU, and the Road to On-Device Agents With Nico Martin | React Universe On Air

Many AI features in web apps send the user's audio, images, or text to a server and wait for the result. That adds network latency, moves data off the device, and ties the experience to a connection. Transformers.js gives developers another option: run the model in the browser.
In this episode, Mike Grabowski speaks with Nico Martin, Open Source ML Engineer at Hugging Face, about what browser AI can do today. They break down the Transformers.js pipeline API, ONNX Runtime, WebGPU acceleration, model downloads, browser caching, CPU fallbacks, and performance across devices.
Nico explains when local inference can beat a server round trip, why model size and hardware variation shape the user experience, and how browser code can handle parts of an agent workflow. He also shares early work on structured output and a custom WebGPU inference engine, where current experiments point to 5x to 10x speedups.
For developers deciding which AI tasks belong in the browser and which need the cloud, this episode maps the tradeoffs through speech recognition, background removal, embeddings, local tool calling, and on-device agents.
Check out episode resources on our website ➡️ https://clstk.com/4zMwzq9
Catch more React Universe On Air episodes 🎧 https://clstk.com/4zNCV8W
Sign up for our newsletter ✉️ https://clstk.com/4h2b8Ks
Follow us on X 🐦 https://x.com/callstackio
Chapters:
0:00 Welcome to React Universe On Air
1:04 Nico Martin's work at Hugging Face
2:04 What Transformers.js is
3:30 Browser AI beyond large language models
6:00 Bringing Python ideas into a JavaScript API
10:08 How the Transformers.js pipeline API works
14:23 Choosing compatible models and architectures
15:46 Why Transformers.js uses ONNX Runtime
19:01 CPU inference, WebGPU, and browser coverage
22:57 Model downloads, browser caching, and UX
28:28 Startup time and performance across devices
30:57 Why Transformers.js focuses on the browser
34:07 What developers misunderstand about browser AI
38:44 Building AI agents in the browser
43:19 Local fallbacks and hybrid AI
44:28 The browser AI roadmap
47:04 Hugging Face's custom WebGPU inference engine
50:57 Why speedups matter on constrained hardware
52:07 Nico's favorite browser AI use cases