GIỚI THIỆU VỀ TẬP NÀY
What happens when your CI/CD pipeline can’t give you a simple yes or no? Tarek Ziadé, who leads CI/CD on Hugging Face’s Transformers library (and previously built CI at Mozilla), joins David Burns to talk about testing software where the maths changes every time you run it.
In this episode:
- Why green doesn’t mean working: non-determinism, GPU variance and out-of-memory leaks across 500+ model architectures
- Living with 0–500 daily failures out of 13,000 GPU tests, and triaging them with Grafana trends instead of by hand
- Serge, the overnight agent that reproduces failures, builds patches and opens PRs (170 runs, 19 PRs in two months)
- Catching reward hacking when the agent “fixes” a test by changing the assertion
- Cutting agent costs from $1.50 to under 50 cents a review by moving logic out of prompts and into deterministic Python
- A Ruff-style model linter with 58 rules, tiny overfitted models for CPU-only end-to-end checks, and the Model Integration Contract
- Sandboxing agents: separating instructions from data, Bubblewrap, network proxies and a human on the merge button
Plus the quickfire round: time-to-signal as the most underrated CI metric, why Tarek’s dream job is running coach, and whether AI will ever replace testing.
Follow Tarek’s writing on the Hugging Face blog.
Subscribe to BrowserStack Talks on Spotify, Apple Podcasts or YouTube so you don’t miss future episodes.
Tiếng Anh
Hoa Kỳ
BẢN CHÉP ÂM THANH 🔗
speaker-0: Every time you push something in main, we have 200,000 tests running. So you have this test, it's doing a bunch of assertion, and and the day after it started to fail for no reason, no code changed, nothing, nothing happened, and and then it starts to be red. Everyone experienced that. When you run LLM, you're gonna generate text at some point, and out of those 13,000 tests we do daily. we have between zero and five hundred reds. Which one are real problems?
speaker-1: said there that…
speaker-1: said there that…
TÌM KIẾM CÁC TẬP TRƯỚC
Tìm kiếm các tập trước của BrowserStack Talks.
Tuyên bố miễn trừ trách nhiệm: Podcast và tác phẩm nghệ thuật được nhúng trên trang này là của BrowserStack, là tài sản của chủ sở hữu và không được liên kết hoặc xác nhận bởi Listen Notes, Inc.
CHỈNH SỬA
Cảm ơn bạn đã giúp cơ sở dữ liệu podcast luôn cập nhật.