Data Storage Steers AI Strategies

In this video interview, HyperFrame Analyst Don Gentile explains how data storage is shifting from a passive to an active participant in AI, raising a defining question: bring compute to the data, or data to the compute? He says months of supply chain pressure is pushing enterprises to squeeze more from existing infrastructure and lean on private cloud and neo-clouds.
Get tech leader insights to move faster and smarter.
Get more stories by subscribing to The Forecast.
Video transcript:
Don Gentile: With the advent with GPUs now, and we need to constantly feed those GPUs, the concept of performance and the AI data pipeline, it really a point of contention is in the storage. The bottleneck happens there. And so you have to start to think about how is that storage media, whatever that McFlash, for example, going to feed those GPUs to make sure that they're not starved, that they're constantly being fed the pipeline data. So that's a big shift.
Ken Kaplan: IT teams, are they having to use a variety of different types of storage technologies today or can they get it all done simply with one kind of storage?
Don Gentile: That is a question for every organization to deal with, right? In some cases, an organization might want to reuse their existing storage for cost performance reasons. They might have the ability to bring on new type of storage platforms. And so it's a little bit of a purpose built exercise there. Certainly you have three tiered environments where you've got your hot, your medium, your cold storage for cost and performance reasons as well. There can be offsite archives, data archives, which is the least expensive and also air gapped for data protection reasons. So there's a variety of different storage platforms that exist out there. The evolution then has been about thinking about how are we going to feed those AI pipelines and how does storage become an active participant in the AI process?
Ken Kaplan: Storage seems like it's a very dynamic environment. There's just a lot of evolution around that. Why did that happen and why do people want new types of storage capabilities?
Don Gentile: Maybe that started about 10 years ago. I could probably put a pin on it and say, if you start to think about where the workloads are going and you start to think about the advent of AI, which ChatGPT had not happened yet, but that moment was going to happen. So a number of companies started thinking about how do we need to design for those future workloads? And so you can rethink storage as more of a substrate that is managing and coordinating and governing across within a data center from cloud to on - prem and of course across geographies as well. And so things like global namespaces emerged where you have to start keeping track of where all that data sits. And then the ultimate question I think for companies is going to be, do you bring the compute to the data or do you bring the data to the compute?
And so that also will influence your storage decisions.
Ken Kaplan: How is storage evolving when we have more activity at the edge?
Don Gentile: Right. Well, if it's a real time inferencing kind of experience that you have to have, you don't have the time for that round trip. And so vehicles are part of that autonomous vehicles. And so if the decision has to happen in the moment, there's no round trip to the cloud for that inferencing to happen. So you have data collection, you have data analysis, and then you have the action from that happening all at the edge. And in some cases you can aggregate, you can collect data from the edge and you can bring it to a central location. It really is going to depend on the application.
Ken Kaplan: You have a perspective from HCI and now storage and security. How have you seen those things evolve?
Don Gentile: Well, I think where companies are looking now is more toward private cloud instances. It's a very rational way of thinking about how to evolve for flexibility and for cost reasons and for future proofing. We can't always predict where things are going to go, so you need to be able to add new technologies or ad scale. And so private cloud gives us that opportunity to be able to scale and add technology as it occurs. So it's more future-proofing.
Ken Kaplan: You're talking about people who do want to still manage some of their infrastructure. They might use cloud and what they own. What happens when they decide to migrate to the cloud or start all in the cloud?
Don Gentile: I think it'll be a combination of things. So there will be situations where you want to get more out of your existing infrastructure because you're not able to add the capacity at the pace that you plan to. It might also mean abandoning or postponing some projects where you say that we're just not going to be able to get to that right now because we don't have the capacity. Now you also have the neo-clouds where you have the ability to scale out to a third party to access that capacity in the cloud. That's why they exist. And they have the access to the current GPUs that are coming out of Nvidia and AMT. And so you can look to the NeoClouds as the next scalable option for enterprises that are not able to access that capacity on site. So that's another option.
Ken Kaplan: In some ways they're forced to do these things. The capacity is not there. It's an interesting time.
Don Gentile: Or they may need to postpone existing projects. Some may say we're getting more value out of this. We'll postpone that. We'll keep focusing on this. And if we've got existing infrastructure that we can repurpose, and vendors are helping with that too. There are programs, there are software solutions as well to help people to get more capacity.
Ken Kaplan: Is that driving people to get more out of what they have? Because they will need to keep adding storage. It seems to be a given.
Don Gentile: Well, from a storage perspective, we have a supply chain issue and that is really pressing for a lot of organizations. So that impacts performance, it impacts cost. And so a lot of organizations need to be planning for that. Not just the enterprises, but the vendors as well. And so there are vendors like Dell and others who have long-term contracts and access to NVIDIA GPUs and all that ad infinitum, but there are cost pressures on top of that too. And you see storage vendors and memory vendors that are starting to elevate their prices. And we have 18 to 24 months supply chain constraints. And so that really could change people's plans about how fast they're able to adopt AI, how fast they're able to extract value of AI because now they have to say, "Well, I have to get more value out of my existing resources." There's no magic store I can go to, to purchase more capacity like that. And so the memory that GPUs, all these assets are suddenly constrained and enterprises and vendors are rethinking their plans and saying, "How do I adapt to that supply chain issue?"