關於這一集
In Elixir Wizards S15E04, Charles Suggs and Emma Whamond are joined by Somtochi Onyekwere, a software engineer at Fly.io and contributor to the Corrosion distributed database project, to talk about distributed systems, infrastructure resilience, and the growing fragility of centralized cloud platforms.
We discuss what recent outages across major providers reveal about modern infrastructure and why more teams are starting to rethink assumptions around reliability, failover, and system design. Somtochi explains how Fly.io approaches geographic distribution, eventual consistency, and replication across nodes, along with the trade-offs that come with building systems this way.
The conversation explores CRDTs (Conflict-free Replicated Data Types), consensus, split-brain prevention, and what actually happens when distributed systems fail in production. We also talk about testing strategies, rollback planning, property-based testing tools, and how teams can reduce blast radius when things inevitably go wrong.
Along the way, we discuss AI infrastructure, sandboxing AI agents, and how newer workloads may add pressure to already centralized systems. The episode closes with practical advice for developers who want to build more resilient applications without over-complicating their architecture.
- Corrosion and distributed database replication
- Centralized cloud fragility and recent outage patterns
- Distributed systems versus traditional cloud architectures
- Multi-region deployment strategies for Phoenix applications
- CRDTs and conflict resolution in distributed systems
- Eventual consistency versus strict consistency tradeoffs
- Consensus, leader election, and split-brain prevention
- Testing failover and recovery scenarios
- Property-based testing and Antithesis
- Rollback planning for database schema migrations
- Reducing blast radius through system isolation
- Health checks and blue-green deployment strategies
- Fly Proxy request routing and replay behavior
- Cross-region synchronization and replication challenges
- Single points of failure inside “redundant” systems
- Backup restoration testing and disaster recovery planning
- Network partitions and failure handling in production
- Infrastructure monitoring and operational visibility
- AI infrastructure workloads and operational strain
- Sandboxing and securing AI agents
- Sprites and AI workflows at Fly.io
- Latency improvements from geographic distribution
- Distributed systems tradeoffs in real-world environments
- Transitive dependency failures across cloud providers
- Practical resilience strategies for modern engineering teams
https://fly.io
https://github.com/superfly/corrosion
https://docs.gitops.weaveworks.org/
FluxCD https://fluxcd.io/
Fly.io Stateful Sandbox Environments https://sprites.dev/
Cloudflare Workers AI Inference Platform https://www.cloudflare.com/products/workers-ai/
“An AI Agent Just Destroyed Our Production Data. It Confessed in Writing” Twitter post from PocketOS founder: https://x.com/lifeof_jer/status/2048103471019434248
Oct 2025 AWS Outage https://www.theguardian.com/technology/2025/oct/24/amazon-reveals-cause-of-aws-outage
Dec 2025 Cloudflare Outage https://www.theguardian.com/technology/2025/dec/05/another-cloudflare-outage-takes-down-websites-linkedin-zoom
July 2025 Crowdstrike Outage https://www.ibm.com/think/news/recent-crowdstrike-outage-what-you-should-know
March 2026 Stryker Cyber Attack https://www.stryker.com/us/en/about/news/2026/a-message-to-our-customers-03-2026.html
https://aws.amazon.com/
https://cloud.google.com/
https://azure.microsoft.com/en-us
https://fly.io/docs/elixir/
CRDTs!! https://smartlogic.io/podcast/elixir-wizards/s13-e03-local-first-liveview-svelte-pwa/
https://antithesis.com/docs/resources/property_based_testing/
https://hex.pm/packages/proper
在這一集裡
逐字稿 🔗
===
[00:00:33] Charles: Hey, everyone. I'm Charles Suggs, software developer at SmartLogic.
[00:00:36] Emma: And I'm Emma Whamond, a software developer at SmartLogic
[00:00:41] Charles: And we're your hosts for season 15, episode four. We're joined today by Somtochi Onyekwere, software engineer at Fly.io. Today, we're talking about the growing fragility of centralized cloud systems, recent large-scale outages, and how engin…