36: How the BBC Experiments at Scale: Learning Since 1922

How do you build a culture of experimentation inside a 100-year-old institution?

This episode is part of our special miniseries on product experimentation where we go beyond A/B testing basics to explore how teams build experimentation culture, move faster with data, and make smarter product bets in practice.

Hosting this series are two of Spotify’s finest experimentation nerds: Luke Frake, group product manager for subscriptions, and Mårten Schultzberg, staff data scientist for Spotify’s experimentation platform, Confidence (confidence.spotify.com).

In this second episode, we explore what it’s like when your streaming content provider is also a national institution. As the UK's national public service broadcaster, the BBC produces everything from news and sports to drama and education across major digital services used by millions every day. With products spanning iPlayer, Sounds, and over a dozen mobile apps, experimentation at the BBC means optimizing for vastly different formats, platforms, and audience needs — all without the subscription metrics that most digital companies rely on.

Luke Frake and Mårten Schultzberg are joined by Divya Isaiah, Head of Experimentation at the BBC. She walks us through how the BBC runs a decentralized experimentation model supported by a center of excellence that acts as an in-house consultancy, filling gaps across teams at different levels of maturity. The conversation also explores how Divya approaches experimentation velocity (both the number of experiments and the number of teams experimenting), why the BBC's editorial teams are now getting into A/B testing, and where AI is showing up in experimentation tools versus where it's actually adding value.

Hear what happens when a venerable institution like the BBC embraces a culture of experimentation in order to inform smarter product decisions.

Learn more about how we experiment at Spotify:

• Dive deep into experimentation nerdiness on the Confidence blog

• 👉 Here’s a good post to start with: “Beyond Winning: Spotify's Experiments with Learning Framework”

• Get started with Spotify’s product experimentation platform: Confidence 

• Become an experimentation expert with our free experimentation bootcamp

• Follow the Confidence team on LinkedIn for product updates, event announcements, experimentation insights, and more

• 👋 Follow Luke on Linkedin

• 👋 Follow Mårten on Linkedin

Read what else we’re nerding out about on the Spotify Engineering Blog: engineering.atspotify.com

You should follow us on Twitter @SpotifyEng, LinkedIn, and YouTube!