Why Memory Is AI's Most Underestimated Layer

Storage used to be an afterthought. In the era of trillion-parameter models and agent swarms consuming millions of tokens per session, it has become one of the most strategically critical layers in the AI stack. On this episode of MindMakers, John Kim is joined by Val Bercovici, Chief AI Officer at WEKA, the AI storage and memory systems company.

Val brings a rare vantage point of two decades across NetApp, SolidFire, and early Kubernetes development to explain why memory architecture is the defining challenge of this moment in AI. He breaks down the gap between what models can theoretically do and what infrastructure actually allows, introduces WEKA’s concept of a "token warehouse," and explains how the shift from prompt engineering to context engineering is changing the way teams build and scale AI systems.

Val also shares a candid take on the Chief AI Officer role and whether it's built to last, his read on the cultural and competitive differences between Anthropic and OpenAI, and why the most durable business model in AI may ultimately come down to who sells the tokens. 

For anyone building or leading at the infrastructure layer of AI, this episode offers both the technical depth and the strategic clarity to think several moves ahead.

—
Guest Bio

Valentin (Val) Bercovici is the chief AI officer at WEKA. He has extensive experience in the data infrastructure industry, having previously been the CTO at NetApp/SolidFire, where he drove innovation in cloud storage and data management solutions. Val co-authored the Windows Shadowcopy snapshots and has made significant contributions to the storage standards community. 

As co-chair of the Storage Networking Industry Association's (SNIA) Solid State Storage Initiative, Val helped to establish the first NAND Flash SSD storage standards. Additionally, Val served as the chair of the SNIA Cloud Storage Initiative (CSI), where he led the development of the international S3 standard CDMI (ISO 17826). He was also a founding member of the Kubernetes Cloud Native Computing Foundation’s Governing Board, helping to shape the global direction of container orchestration. 

Val holds patents in AI agent smart contracts, streaming data integrity, and augmented reality (AR) for data center maintenance. His work continues to push the boundaries of what’s possible at the intersection of AI, cloud, and emerging technologies.

—

Guest Quote

"The biggest value of memory today is without any additional expense, without more CAPEX or energy OPEX, you can have a high level of concurrency... Even at a modest scale, we're seeing 6.5x, another way of saying it is 550% more concurrent tokens without latency sacrifice, without any more GPU spend or any more energy spend." – Val Bercovici

—

Time Stamps  

[00:49] Welcoming Val to the show

[01:02] Val's background pushing innovations

[02:58] Are Chief AI Officers here to stay?

[08:40] How WEKA delivers for clients

[12:16] Unlocking memory architecture bottlenecks

[18:53] Where memory impacts the bottom line

[24:51] The future of reliably stored context

[28:29] Keeping up with AI research

[31:47] The power of being an early adopter

[34:19] Who is winning the AI code battle?

[39:15] Are we heading into a SaaS apocalypse?

[44:31] Val's Human Prompt

—

Links

• Connect with Val Bercovici on LinkedIn
• Check out WEKA
• Connect with John Kim on LinkedIn
• Check out Sendbird

Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.