[Podcast] Why Companies Fail To Use Their Data and How To Change That
Welcome to The Data-led Professional Podcast: A podcast dedicated to helping folks become data-led to build better products and services.
In this week’s episode, join hosts Claudiu Murariu, CEO and Co-Founder of InnerTrends, Arpit Choudhury, Founder of Data-led Academy, and special guest Ruben Ugarte, Author of The Data Mirage, as they discuss the biggest misconceptions surrounding data, the impact that the pandemic has had on the data space in the last year, the skills that anyone can learn to thrive in the data-led world, how companies can be data-led with or without hiring a data scientist, what the best companies do with their data that others don’t, and so much more.
Select excerpts from the episode:
What are the biggest misconceptions around data?
[R]: There are definitely a few. The first is the idea that you want to be data-driven for everything. That is, you want to effectively reject all opinions if you can’t back them up with data. And that causes issues. We saw a lot of those issues during the pandemic, where a lot of companies did not have any data.
I work with the provincial government here in British Columbia, Canada, on the tourism side - one of the most heavily affected areas by the pandemic - and they had no answers. The numbers we were seeing were unprecedented. So they had no way of knowing: Is this good? Is this bad? How does this compare? What's going to happen? But they still had to make decisions without the data.
If you're stuck in this narrow, data-driven mindset, you're going to put yourself in a box the moment you don't have concrete data to back something up.
If you look at significantly experienced professionals who have been in an industry for 20-40 years, they probably have intuition about things that might work without any way of actually explaining why that might work. Are you going to just dismiss their intuition and experience?
[A]: You can't just follow what the data tells you blindly; you have to use your intuition and experience to make data-related decisions, so I really like what you said there.
[C]: In the data space, they always say “Don't go with your intuition, go with data.” But people started to take that too literally. Intuition can have a very big impact on a business, especially when it has some data history behind it.
The pandemic has really changed how people look at data. What shifts have you seen in the data space in the last year, specifically in regard to the human factor?
[R]: A major data-related change, as a result of the pandemic, is the shift to digital. This shift has been happening for years, but now it has been accelerated. So all of a sudden, companies can easily track a lot of things. They might have more visibility of what's going on. And they’ve had to learn new systems - so they had to learn how to track eCommerce purchases versus offline, how to measure digital spend, how to work in costs, how to create a digital presence, etc.
At the same time, the year was interesting because we are also seeing another growing trend in terms of privacy, and people wanting privacy over their data.
In the pandemic, for example, there's a big debate over vaccine passports right now. Some countries say yes, some say no, but it boils down to data privacy. Who's going to own this data? The government? A company? And Apple has now released their update that makes privacy a central focus.
So there are all of these different, competing forces. Businesses need to track more digital data now, but at the same time, the walls are closing in because of privacy. And that’s forcing companies to adjust how they collect data, how they store it, how they deal with limited data from big companies like Apple and in the future, likely Google.
With many companies going digital and increasing data privacy, executives need to work with data now more than ever. What are the skills they need to gain to enable their organizations to live in this data-led world?
[R]: Some executives want to be in the data, and some don't. But there is a set of skills that apply to almost everyone.
Basic statistics. Being able to look at a number and knowing: Is this the right sample size? Do we have enough data to make a decision here? Are we looking at a very extreme or obscure segment or portion of the data that looks really good when you put it under the right light and angle?
Basic probabilities. Being able to look at two possible ideas and saying: What's the probability of success? What's the expected value? If it becomes successful, how much can we expect? How do we weigh those things?
A basic understanding of technology. You don’t have to be an expert, but you want to understand what ETL is, what a data warehouse is, how things are flowing, roughly, from point A to point B, and some of the major challenges to be aware of.
Data proficiency general skills. If you or your teams get overwhelmed by data, what do you do? If you come across a number that you don’t trust, who do you go to, and how do you deal with it? So all of these basic housekeeping things that you're bound to run into when you have data, and knowing how to tackle them. Those four skills apply to anyone. And then some, like an executive, might want to pick up SQL, Tableau, or Power BI, etc. to build their own reports. But that’s optional.
[A]: It’s really important for people to be able to question the accuracy of data. And to do that, they need to be somewhat data literate. Otherwise, they’ll blindly follow the data that they see, and then they're likely to make mistakes.
And no company, no matter how good the data is, has 100% accuracy. Data quality is an issue for every company. In fact, the bigger you are and the more data you gather , the more data quality issues you have.
Can a company be data-led without hiring data scientists?
[R]: I think it'd be helpful to have a model here to understand where people fall into. In previous blog posts and interviews I’ve described this time source of data: we have the past, present, and future.
In the past, you’re trying to understand what happened, it’s very data analysis heavy. In the present, you’re trying to understand what's going on, that's really the world of BI tools - what's going on in the latest campaign or product launch. And in the future, you’re trying to predict what might happen - that’s the world of data science.
So if you take a question about the role of data science specifically, we can see it's mostly just for the future. So there's still an entire segment on the past and the present that's really valuable. Because if you understand what happened and what's going on, you can make better decisions, you can launch new things; there's a lot of things you can do without ever touching data science.
So the first answer is yes - of course you can be data-led without bringing machine learning experts and AI experts. And if you do bring them on, you want to put them in the right context. But there's definitely lots of things that a company could do, without ever having to talk about prediction models.
[C]: Data science for the future is a great way of putting it. Indeed, data scientists still work with the past data, trying to predict the future. And pretty much every analysis out there contains data science. Even when you look at a report, you look at it and say, “How can this help me tomorrow?”
So you actually do data science in your mind, but at a much smaller level than using a data scientist or statistics. So like you said, it's very important to have a basic understanding of statistics because it will change a lot in terms of how you use the data moving forward. Because you will actually apply that data science in your mind. Many people don't maybe understand that data science is statistics, just at a much higher level. But it’s all statistics in the end.
Let’s focus more on this company that should be data-led, even without a data scientist.
What do you think is the most important skill for that company or the people in that company to learn in order to be data-led?
[R]: Making the data easily available. We have a term for this: democratized data, and it means getting people access to data and getting them to use the data. I think companies really want to make it so that people metaphorically “trip” on the data.
So you start to see a number of different ways of accessing data, and making the data available. That is - you almost have to go out of your way to not see the data. That's what you really want.
Today’s companies want to have multiple dashboards; every individual could have their own dashboard with up to four or five KPIs that they care about, every team could have their own dashboard, the company might have multiple dashboards, and the dashboards are available on desktop or mobile.
There's also, for example, email digests that summarize the most important KPIs - maybe individually, team-wide, or company-wide. A lot of teams are on Slack or Microsoft Teams, so you want to find ways to pipe data into those places in little widgets. The data is now available, so you want to make it downloadable to a CSV or Excel. And of course, you may have the ability for people to run queries against it through SQL.
Once you have data available, it starts to seep in. You’ll start to say: “Okay, let me go check this number. I ran this campaign. Let me go check that.” The consumption of data should be more like a snack - you want to have a little bit at a time, but consistently over time. That's much more powerful than locking everyone in a room and telling them to look at one single dashboard and really memorize it.
[A]: The democratization of data is really important to make data accessible. And not just in the form of dashboards, but also, in the tools people use every day. So you mentioned Slack, which is super relevant. And it almost forces people to stay up to date with what's going on. But it's also going to make data available in the tools that different teams use, whether that’s the sales, marketing, support, or advertising teams. By making the right customer data available in these tools, you enable teams to actually act upon data, and derive insights from it which is the next natural step.
[C]: When data is made available to everyone in the company, easily and a lot, they start to ask questions. And when it's not, nobody asks any questions. And that makes a huge difference. Because what you want to do in a data-led company is have a lot of people asking questions and being curious, because that curiosity is the main driver towards the data-led culture.
Let's talk about what gets in the way of becoming data-led.
[R]: A few things, in no particular order:
Technology. Technology is key to this; you're going to need technology, there's no denying that. But over-emphasis of technology would be a killer. A lot of companies bring me in and say, “We want a Customer Data Platform (CDP). We want this, we want that,” and it’s all very technology-focused. And the question becomes: Do you really need that or can we do without? Can we consolidate?
So what do we really care about? We care about those insights, which means we need dashboards and data. And from there, we have a logical sequence to proceed. We’ll find the right technology for the steps. But the goal is not to have this amazingly set up CDP. The goal is to have insights. So if it doesn't have a CDP, so be it.
A lack of skills. Let's say you make data easily available, you follow this idea, and you have data everywhere - you found ten different ways to make it available. But people are still not using it. Now, this might be a question of training. You might talk to your team or company and realize that a lot of people don’t feel comfortable with basic statistics, basic math, numbers, and they don’t know how to read it, they're not sure how to interpret it, so they're just ignoring it altogether. So now we can offer coaching on the skills to work through what's needed. So this is solvable.
A complete disregard of data. So they’ve swung to the other extreme. You have the data-driven extreme, where you reject all opinions, and then you have the opinion-driven extreme, where they have no data and don't trust any numbers; it’s all opinion-driven. A lot of small companies operate like this, they don't have that much data. So they're driven by anecdotes and opinions. So that gets in the way of actually using data and making the most of it.
What leads some organizations to having data silos?
[R]: I've actually put data-overwhelm under the people issues, in that there's too much of it, there's too many reports, too many dashboards, you’re not quite sure what's true, what's not true, what’s the focus. Again, that’s a skill that can be coached or trained. And somebody can build a team relatively quickly, in terms of cutting down the amount of data that you're consuming.
Silos contain data from other teams. So the finance team has some data, and the marketing team has some data, but they're not talking to each other. They're living in their own little world. Silos are not a technology problem, which is what some people might think. It's really a people challenge, a political problem.
I once worked with a company in the finance space, and our entire project was vetoed because of compliance issues.
So maybe legal compliance shoots something down, or you have a team that's not willing to give other teams data for political reasons. And you have to work through those in a more sophisticated manner. It's not just a technology solution.
[A]: The people aspect is so true. The technologies are there, the dashboards are there, there's really no shortage of tools. But the people aspect is a big challenge.
What do you think the best companies do with their data that others don’t?
[R]: My go-to example here has always been Spotify. Not only is it a great product, but it's been really interesting to see how they use data. And I think that there are lessons to be learned from their product itself. This is technology, of course; this is a software company, but the ideas can apply to any industry.
First, Spotify has developed their own frameworks. Their main framework is called DIBB (Data Insights Beliefs Bets). It's their framework for how they take potential ideas within the company. They measure the potential impact, and then test them. It’s fantastic. So they’ve created a way to take all of the insights from the data and run them through some kind of objective criteria, but they’ve made it their own. Which is what you want - you want to make it your own. You want to use your own language, your own steps, etc.
Another great lesson to take away from Spotify is their use of machine learning and data science. We talked about the future, which is a hot topic - a lot of companies are thinking about how they actually want to use data science and AI. There are a few tangible use cases, and Spotify found one in particular: they build playlists for you effectively. When you open the app, the biggest challenge of a music app is that you have to go find your music. If you’re old enough to remember choosing 20 songs and burning them onto a CD or cassette, Spotify eliminates that process. They build these playlists for you, and as you use the app, you'll notice that the playlists change. So I noticed that a playlist that might be popular has been heavily tweaked to my taste, and my favorite types of songs. And it's probably different if you were to look at it in your own account. This is a lot of machine learning, prediction, and recommendations.
One more thing I’ll mention about companies like Spotify is that they invest in their data. Not just in their technology, which is clear, but in the people. In order to train them on how to use the data, and to bring the right people on board. So they have data analysts, data engineers, data science people. They’ve made significant investments in how a company uses and thinks about data at a large scale.
[C]: When you have teams with good data mindsets, and strong data-led companies, one of the outcomes is the building of great products, because that's the purpose of data. When you actually use data, when all of the teams are aligned on what needs to be tracked and measured, you always end up with a great product. Not doing that might get you to a maybe good product. But not a great one. So I think that's a great outcome of being a data-led company and having a solid data culture.
Conclusions:
Data-related tools are important, but the people using them are more important. The right skillset and the right mindset need to be there.
Empower the people to be data-led, embrace a data-led culture, and your chances of delivering an amazing product will be that much higher Subscribe for more episodes on your favorite streaming platform.
Welcome to The Data-led Professional Podcast: A podcast dedicated to helping folks become data-led to build better products and services.
In this week’s episode, join hosts Claudiu Murariu, CEO and Co-Founder of InnerTrends, Arpit Choudhury, Founder of Data-led Academy, and special guest Ruben Ugarte, Author of The Data Mirage, as they discuss the biggest misconceptions surrounding data, the impact that the pandemic has had on the data space in the last year, the skills that anyone can learn to thrive in the data-led world, how companies can be data-led with or without hiring a data scientist, what the best companies do with their data that others don’t, and so much more.
Select excerpts from the episode:
What are the biggest misconceptions around data?
[R]: There are definitely a few. The first is the idea that you want to be data-driven for everything. That is, you want to effectively reject all opinions if you can’t back them up with data. And that causes issues. We saw a lot of those issues during the pandemic, where a lot of companies did not have any data.
I work with the provincial government here in British Columbia, Canada, on the tourism side - one of the most heavily affected areas by the pandemic - and they had no answers. The numbers we were seeing were unprecedented. So they had no way of knowing: Is this good? Is this bad? How does this compare? What's going to happen? But they still had to make decisions without the data.
If you're stuck in this narrow, data-driven mindset, you're going to put yourself in a box the moment you don't have concrete data to back something up.
If you look at significantly experienced professionals who have been in an industry for 20-40 years, they probably have intuition about things that might work without any way of actually explaining why that might work. Are you going to just dismiss their intuition and experience?
[A]: You can't just follow what the data tells you blindly; you have to use your intuition and experience to make data-related decisions, so I really like what you said there.
[C]: In the data space, they always say “Don't go with your intuition, go with data.” But people started to take that too literally. Intuition can have a very big impact on a business, especially when it has some data history behind it.
The pandemic has really changed how people look at data. What shifts have you seen in the data space in the last year, specifically in regard to the human factor?
[R]: A major data-related change, as a result of the pandemic, is the shift to digital. This shift has been happening for years, but now it has been accelerated. So all of a sudden, companies can easily track a lot of things. They might have more visibility of what's going on. And they’ve had to learn new systems - so they had to learn how to track eCommerce purchases versus offline, how to measure digital spend, how to work in costs, how to create a digital presence, etc.
At the same time, the year was interesting because we are also seeing another growing trend in terms of privacy, and people wanting privacy over their data.
In the pandemic, for example, there's a big debate over vaccine passports right now. Some countries say yes, some say no, but it boils down to data privacy. Who's going to own this data? The government? A company? And Apple has now released their update that makes privacy a central focus.
So there are all of these different, competing forces. Businesses need to track more digital data now, but at the same time, the walls are closing in because of privacy. And that’s forcing companies to adjust how they collect data, how they store it, how they deal with limited data from big companies like Apple and in the future, likely Google.
With many companies going digital and increasing data privacy, executives need to work with data now more than ever. What are the skills they need to gain to enable their organizations to live in this data-led world?
[R]: Some executives want to be in the data, and some don't. But there is a set of skills that apply to almost everyone.
Basic statistics. Being able to look at a number and knowing: Is this the right sample size? Do we have enough data to make a decision here? Are we looking at a very extreme or obscure segment or portion of the data that looks really good when you put it under the right light and angle?
Basic probabilities. Being able to look at two possible ideas and saying: What's the probability of success? What's the expected value? If it becomes successful, how much can we expect? How do we weigh those things?
A basic understanding of technology. You don’t have to be an expert, but you want to understand what ETL is, what a data warehouse is, how things are flowing, roughly, from point A to point B, and some of the major challenges to be aware of.
Data proficiency general skills. If you or your teams get overwhelmed by data, what do you do? If you come across a number that you don’t trust, who do you go to, and how do you deal with it? So all of these basic housekeeping things that you're bound to run into when you have data, and knowing how to tackle them. Those four skills apply to anyone. And then some, like an executive, might want to pick up SQL, Tableau, or Power BI, etc. to build their own reports. But that’s optional.
[A]: It’s really important for people to be able to question the accuracy of data. And to do that, they need to be somewhat data literate. Otherwise, they’ll blindly follow the data that they see, and then they're likely to make mistakes.
And no company, no matter how good the data is, has 100% accuracy. Data quality is an issue for every company. In fact, the bigger you are and the more data you gather , the more data quality issues you have.
Can a company be data-led without hiring data scientists?
[R]: I think it'd be helpful to have a model here to understand where people fall into. In previous blog posts and interviews I’ve described this time source of data: we have the past, present, and future.
In the past, you’re trying to understand what happened, it’s very data analysis heavy. In the present, you’re trying to understand what's going on, that's really the world of BI tools - what's going on in the latest campaign or product launch. And in the future, you’re trying to predict what might happen - that’s the world of data science.
So if you take a question about the role of data science specifically, we can see it's mostly just for the future. So there's still an entire segment on the past and the present that's really valuable. Because if you understand what happened and what's going on, you can make better decisions, you can launch new things; there's a lot of things you can do without ever touching data science.
So the first answer is yes - of course you can be data-led without bringing machine learning experts and AI experts. And if you do bring them on, you want to put them in the right context. But there's definitely lots of things that a company could do, without ever having to talk about prediction models.
[C]: Data science for the future is a great way of putting it. Indeed, data scientists still work with the past data, trying to predict the future. And pretty much every analysis out there contains data science. Even when you look at a report, you look at it and say, “How can this help me tomorrow?”
So you actually do data science in your mind, but at a much smaller level than using a data scientist or statistics. So like you said, it's very important to have a basic understanding of statistics because it will change a lot in terms of how you use the data moving forward. Because you will actually apply that data science in your mind. Many people don't maybe understand that data science is statistics, just at a much higher level. But it’s all statistics in the end.
Let’s focus more on this company that should be data-led, even without a data scientist.
What do you think is the most important skill for that company or the people in that company to learn in order to be data-led?
[R]: Making the data easily available. We have a term for this: democratized data, and it means getting people access to data and getting them to use the data. I think companies really want to make it so that people metaphorically “trip” on the data.
So you start to see a number of different ways of accessing data, and making the data available. That is - you almost have to go out of your way to not see the data. That's what you really want.
Today’s companies want to have multiple dashboards; every individual could have their own dashboard with up to four or five KPIs that they care about, every team could have their own dashboard, the company might have multiple dashboards, and the dashboards are available on desktop or mobile.
There's also, for example, email digests that summarize the most important KPIs - maybe individually, team-wide, or company-wide. A lot of teams are on Slack or Microsoft Teams, so you want to find ways to pipe data into those places in little widgets. The data is now available, so you want to make it downloadable to a CSV or Excel. And of course, you may have the ability for people to run queries against it through SQL.
Once you have data available, it starts to seep in. You’ll start to say: “Okay, let me go check this number. I ran this campaign. Let me go check that.” The consumption of data should be more like a snack - you want to have a little bit at a time, but consistently over time. That's much more powerful than locking everyone in a room and telling them to look at one single dashboard and really memorize it.
[A]: The democratization of data is really important to make data accessible. And not just in the form of dashboards, but also, in the tools people use every day. So you mentioned Slack, which is super relevant. And it almost forces people to stay up to date with what's going on. But it's also going to make data available in the tools that different teams use, whether that’s the sales, marketing, support, or advertising teams. By making the right customer data available in these tools, you enable teams to actually act upon data, and derive insights from it which is the next natural step.
[C]: When data is made available to everyone in the company, easily and a lot, they start to ask questions. And when it's not, nobody asks any questions. And that makes a huge difference. Because what you want to do in a data-led company is have a lot of people asking questions and being curious, because that curiosity is the main driver towards the data-led culture.
Let's talk about what gets in the way of becoming data-led.
[R]: A few things, in no particular order:
Technology. Technology is key to this; you're going to need technology, there's no denying that. But over-emphasis of technology would be a killer. A lot of companies bring me in and say, “We want a Customer Data Platform (CDP). We want this, we want that,” and it’s all very technology-focused. And the question becomes: Do you really need that or can we do without? Can we consolidate?
So what do we really care about? We care about those insights, which means we need dashboards and data. And from there, we have a logical sequence to proceed. We’ll find the right technology for the steps. But the goal is not to have this amazingly set up CDP. The goal is to have insights. So if it doesn't have a CDP, so be it.
A lack of skills. Let's say you make data easily available, you follow this idea, and you have data everywhere - you found ten different ways to make it available. But people are still not using it. Now, this might be a question of training. You might talk to your team or company and realize that a lot of people don’t feel comfortable with basic statistics, basic math, numbers, and they don’t know how to read it, they're not sure how to interpret it, so they're just ignoring it altogether. So now we can offer coaching on the skills to work through what's needed. So this is solvable.
A complete disregard of data. So they’ve swung to the other extreme. You have the data-driven extreme, where you reject all opinions, and then you have the opinion-driven extreme, where they have no data and don't trust any numbers; it’s all opinion-driven. A lot of small companies operate like this, they don't have that much data. So they're driven by anecdotes and opinions. So that gets in the way of actually using data and making the most of it.
What leads some organizations to having data silos?
[R]: I've actually put data-overwhelm under the people issues, in that there's too much of it, there's too many reports, too many dashboards, you’re not quite sure what's true, what's not true, what’s the focus. Again, that’s a skill that can be coached or trained. And somebody can build a team relatively quickly, in terms of cutting down the amount of data that you're consuming.
Silos contain data from other teams. So the finance team has some data, and the marketing team has some data, but they're not talking to each other. They're living in their own little world. Silos are not a technology problem, which is what some people might think. It's really a people challenge, a political problem.
I once worked with a company in the finance space, and our entire project was vetoed because of compliance issues.
So maybe legal compliance shoots something down, or you have a team that's not willing to give other teams data for political reasons. And you have to work through those in a more sophisticated manner. It's not just a technology solution.
[A]: The people aspect is so true. The technologies are there, the dashboards are there, there's really no shortage of tools. But the people aspect is a big challenge.
What do you think the best companies do with their data that others don’t?
[R]: My go-to example here has always been Spotify. Not only is it a great product, but it's been really interesting to see how they use data. And I think that there are lessons to be learned from their product itself. This is technology, of course; this is a software company, but the ideas can apply to any industry.
First, Spotify has developed their own frameworks. Their main framework is called DIBB (Data Insights Beliefs Bets). It's their framework for how they take potential ideas within the company. They measure the potential impact, and then test them. It’s fantastic. So they’ve created a way to take all of the insights from the data and run them through some kind of objective criteria, but they’ve made it their own. Which is what you want - you want to make it your own. You want to use your own language, your own steps, etc.
Another great lesson to take away from Spotify is their use of machine learning and data science. We talked about the future, which is a hot topic - a lot of companies are thinking about how they actually want to use data science and AI. There are a few tangible use cases, and Spotify found one in particular: they build playlists for you effectively. When you open the app, the biggest challenge of a music app is that you have to go find your music. If you’re old enough to remember choosing 20 songs and burning them onto a CD or cassette, Spotify eliminates that process. They build these playlists for you, and as you use the app, you'll notice that the playlists change. So I noticed that a playlist that might be popular has been heavily tweaked to my taste, and my favorite types of songs. And it's probably different if you were to look at it in your own account. This is a lot of machine learning, prediction, and recommendations.
One more thing I’ll mention about companies like Spotify is that they invest in their data. Not just in their technology, which is clear, but in the people. In order to train them on how to use the data, and to bring the right people on board. So they have data analysts, data engineers, data science people. They’ve made significant investments in how a company uses and thinks about data at a large scale.
[C]: When you have teams with good data mindsets, and strong data-led companies, one of the outcomes is the building of great products, because that's the purpose of data. When you actually use data, when all of the teams are aligned on what needs to be tracked and measured, you always end up with a great product. Not doing that might get you to a maybe good product. But not a great one. So I think that's a great outcome of being a data-led company and having a solid data culture.
Conclusions:
Data-related tools are important, but the people using them are more important. The right skillset and the right mindset need to be there.
Empower the people to be data-led, embrace a data-led culture, and your chances of delivering an amazing product will be that much higher Subscribe for more episodes on your favorite streaming platform.
