Naama: Hi everyone. Welcome to another episode of Dopamin Lab. I have a very, very special guest with me today, and we'll talk about one of my favorite things in the world and in gaming, which is data. And I think the previous episodes, were more softer and today we will actually dive deeper into data which is not soft and gives us some more information Okay, so today I'm talking about data with someone who knows this world extremely well. Maor and I worked together at Slotomania where he was a business analyst. And I've always loved working with you because I brought you a question that felt incredibly simple, and suddenly we discovered that the question itself was wrong, and that We probably need to ask it differently And I think you're a great storyteller. and you just look at data from very different angles and I love that about you. And
Maor Frid: Yeah.
Naama: now you go ahead and tell us who you are and tell us about your relationship with data.
Maor Frid: So thank you very much. So first of all thank you for the warm and kind words. Yeah, I loved working with you too. it was a while back now, but still remember it very very good. so my name is Maor Frid, I'll introduce myself shortly. So I've been working in gaming and data for the last let's say nine years-ish. worked with you in Slotomania, then moved to other roles including monetization product, going back into analytics in a broader sense in the tech division, and currently in a small startup still in the gaming industry, in a very exciting startup in a new domain making games in the telegram ecosystem which is a whole different story and very different audience and very different habits for those players unlike the standard US audience that we all know and love from From mobile
Naama: Hm.
Maor Frid: games.
Naama: Well, can I guess? Probably shorter sessions and probably way more casual.
Maor Frid: It is shorter sessions but much more sessions in a day.
Naama: Mm.
Maor Frid: just think how much time you spend in WhatsApp. so just
Naama: Yeah. Wow. Yeah.
Maor Frid: think that you have like a game within the WhatsApp platform. It just like it's a click away, everything is super fast, loads up instantly, you don't need to download additional apps or stuff everything is within the same and the same platform china has it for years now in the WeChat which is like one app to rule them all like only China knows
Naama: Mm-hmm.
Maor Frid: how to do Telegram has let's call it similar features, obviously different audience, but but similar features and trying to create like a more holistic ecosystem and not WhatsApp which is primarily and mostly only messaging.
Naama: Yeah. Wow, here in the other side of the world I don't think I've ever heard about WeChat. I don't know if Telegram is a popular thing here. I know in Israel it is. but wow, it's super interesting. Very cool.
Maor Frid: No, Americans all know the I message so that's that that's it.
Naama: Yeah.
Maor Frid: That that's why they know.
Naama: Yep, that's it. Okay. make sure to stay with us until the end because Maor has a pro tip for something I think almost everyone who works with data struggle with. So stay tuned. And let's start with something that sounds almost stupid. When we say the players, who are we actually talking about? Because there isn't a player. There isn't one player. There are people who play for five minutes, people who play for five hours, people who spend one dollar, people who spend hundred dollars, people who play because they're competitive, and some that play because they're bored, because their friends are there. you see where I'm going.
Maor Frid: Yeah, yeah for sure. For so for a long time now the buzzword is personalization. Someone somewhere said hyper personalization.
Naama: Ha ha.
Maor Frid: I have to say that I think that not many companies and products are there, mostly are still at the place of segmentation. In different variations of it, and let's call it in different levels. At the end, personalization, it's a segmentation with n size equal to one, like a single player within a segment. I have to say that in most cases that I saw and like got people talking about personalization, we always came to a conclusion of overfitting. Something from like mathematics or and statistics models, overfitting, like personalization, it's amazing, but at the end, real personalization is if I'm sending you a message, if I can include your name inside, that's personalization, that's great, that's personal. But when I'm talking about okay, I want to send an email to everyone who did da da da da da da so it's not a person it's a segment of players that fit a certain criteria or criterias that we combine them together and into a segment. The the real question and the actual difficulty in our current rule is that we are drowning data and we need to make sense of the data and to understand how it comes together into actual player experience. What is actually relevant, what is irrelevant or what is relevant in a specific point of time or in a specific context. Just top of my head a few examples of trying to to make sense of what I said. So you said telegram probably have shorter sessions and I said yeah but more sessions a day. So is a longer session better than a shorter session? what does it mean? maybe for Netflix as a company, longer sessions are great. binge watching Netflix is amazing. But wait, what what's binge watching? So is it too long? Did you fall asleep? We all woke up on the couch
Naama: Yeah.
Maor Frid: to the screen. Are you still there?
Naama: Yeah.
Maor Frid: From Netflix. that no, I'm not here, I fell asleep on the couch. So My session was very long but I wasn't actually active on it. and so coming back to to to our world to the gaming world, so what's better if I have a single session which is like I don't know twenty minutes, thirty minutes long. Is it actually a good session? Like did someone play my game for twenty consecutive minutes or did he just get lost in those twenty minutes somewhere along the way? so I think that when we have a lot of data we need to tie it to what we want to learn.
Naama: I just want to say about session length that I think it's a great example because it's a metric that is hiding more data in it, and it's not necessarily a metric that we should look at because like you said, long session time can mean confusion can mean scraping coins or tickets or whatever currency it is within other things that are not the core loop. so it's very interesting. I hundred percent agree with you about session length or session time that it's not necessarily a metric that we should look at. And this is part of the problem of having endless data that a lot of time is perceived as a good thing. yeah, we have so much data, we have endless data, but what do we do with it? How do we approach it? How do we ask the right questions?
Maor Frid: so I'll so I'm I think that if in the past the issue was we don't have enough data or we don't have the right data, so those days are gone. we have the data. Probably each company has every data they can dream of. And if they don't they can access it via other points But the real question remains in each business, what do I want to achieve and how can I get to it? So we have a lot of junk data that we collect. Just I don't know. Every mobile game can request access and probably most people just approve it. to your I don't know to your gyroscope in the in the in your phone the accelerator saying how how is the device moving so probably for most I don't know gaming companies it's not that relevant but because they don't have again not that I can think of they don't have a lot of things to do with that information just collect it hoard it But Uber for example it's critical for them. Or if I'll take a more Israeli proud approach, waze are you moving? Are you stopping? Are you accelerating, slowing down, what's going on with your device? is it up in the air? Is it down facing down, etc. etc. All those Data points become more interesting again because the business question changes. So it's not just collecting data, it's collecting the relevant data and asking the relevant questions to your business.
Naama: Yep. Asking the relevant question is crucial and maybe later we can get to that point deeper and you can help us and tell us how do we ask the right questions?
Maor Frid: j regarding the right question. So there's funny story that I tell every time that I get to that conversation and every time that I have intro meeting with employees, wherever I work, Monetization guys product, etc. So I always tell about a friend of ours, a mutual friend. We call him Dubi even though his name is
Naama: Yeah.
Maor Frid: Guy. But so a while back I was an analyst, he was a copywriter, he popped his head into the business analytics room and guys how many users do we have?
Naama: Wow.
Maor Frid: So that turned on every cynical fiber
Naama: haha
Maor Frid: in any analyst in the room and God knows we have a lot of them. We started bombarding him with every question Wha what do you mean how many? Ever recently active? What's active? Active spinning? Purchasing?
Naama: Mm-hmm.
Maor Frid: buyers, non-buyers. in a s specific region in the world? we asked him ten minutes worth of
Naama: Mm-hmm.
Maor Frid: questions he just turned on and walked away. So I I I G I got up, I followed him to the kitchen. I'm like, listen man, you asked the question. We obviously didn't give you answer. So first
Naama: Yeah.
Maor Frid: of all I apologize, but what what do you need? Like as a copywriter I assume that the question how many players do we have in your head sounds like a very simple one As you saw, it's not that simple. So tell me what you need and let's work together. And he told me we're working on a partnership with a different company and I'm working on the copy for the ad that we're gonna advertise on their platform. So I'm trying to think if I can use the sentence join. Millions of other players already playing or should I say hundreds of thousands it was a great example. This is why I I say it all the time. You have a question in your in your head and you have a goal that you need to reach. Like can I write join millions of players playing playing our game? So that's what you need to know. Okay, that's your end goal. If I'll take it to a deeper sense, are players engage with my feature? That's every product manager question. And usually it becomes a whole new 30 minute session of what's engagement. What what did you want them to do? Engaged as in they saw your event within the game? Yeah, everybody saw it. Did they actually do what you wanted them? So the the best way to ask the right data question is not to ask a data question. is to be focused on the business question. Like what's the habit that you wanted to form? What's the change in habit, change in behavior you wanted to see in the players, And from there we can build what's the relevant data, what's the question, how do we need to approach it, etc etc. I think that's the the golden tip the golden tip is Data is just another point of the business. So you have a business question, and in the end, as this whole podcast is about, our business is people and how they behave. So at the end, every question comes down to Habits and behavioral change within people.
Naama: Yeah, thank you for the great definition. This podcast is about people and how they behave within the dopamine lab, which is called a game because the game is is the biggest dopamine lab. So yeah, that that's great. And also I think you really, really distilled the essence of how do we ask the right questions? What do you want to know? What are you driving? What do you want to drive? Is if the feature is driving more spins or more or higher bets or even number of clicks or number of clicks on an external arena or whatever that you're trying to drive, this is your question, right?
Maor Frid: also even before the what, the who like who are the players that you're trying to to change their behaviour. Some features, some aspects of any game are what we know as FTUE first time user experience. They have nothing to do with our VIP audience. with our I don't know top spenders of the game players that downloaded the game two years ago and are still playing so what happens on like the first level of the game they probably don't remember even how the game started what what feature it has and etc etc so even before the what is the who Who are you targeting?
Naama: Yeah, great point. Okay, I have a question because I have something that is really bothering me and it's the fact that a lot of our data is just aggregated and divided data, which is the average. and I think that average has two main risks in it. imagine we have ten players, nine players spend one dollar, one player spent hundred dollars, and the average is ten point ninety. So our data tells us that the average revenue per paying user is ten point ninety. And here's the funny part, no one of those ten players spent 10.90 dollars. So we created a character who doesn't actually exist. So that's one. I I guess the the right thing to do here is look at the median, but you tell me later. Another risk is that the average doesn't move, but it's because one segment pulled it this way, and another segment pulled it in the other direction. So that's actually two segments that moved dramatically, but the average didn't move.
Maor Frid: so without going into the the nerdy version of me so
Naama: Mm-hmm.
Maor Frid: first of all the average has its mathematical d disadvantages of being highly affected by extreme values like in your case if you average out a non-payer with high payers so The average is let's call it a medium to high payer, which doesn't exist. So to avoid that mathematical skewness, probably what we usually do we look at more precise values if we're talking about like the median or we're trying to break it down into percentiles. Percentile 25. 50 which is the median 75, 90, or 95 if we're looking at extreme cases. So that's one way of looking at it. The other thing is to look at the distribution of values. So we're talking about the VIPs the most valuable players, how much they spend, what's a VIP. So we need to take into consideration what are the relevant options in the game? I usually try to look at the distribution? How many players do we have in each range or buckets if you want, of spending values? Again, doesn't have to be money spending. Obviously, we have habits and have ranges for everything, the infamous session length or amount of core loops the player did, the amount of stickers he collected within the collectible feature of the game. so yeah the average is probably something that we shouldn't be looking at unless and I'm Jumping back and forth from what we talked about, and unless we have a really good defined segment and sub-segment, and then you know, okay, you know what? So averaging those players together is okay. but
Naama: Okay.
Maor Frid: If you're not sure, probably average is not a number that you can count on and for sure not when you're trying to define player experience. Like the average player doesn't exist.
Naama: Nice, nice. Okay, perfect. So you're saying ways to avoid this risk is one, to look at percentiles, two, to look at distribution, and I don't want listeners to leave this episode thinking average is a bad thing. So you answered that thing too. What if you have a very, very defined segment and a sub segment, you can totally look at the average and it's even I guess more efficient sometimes. please correct me if I'm wrong.
Maor Frid: No no you you're totally
Naama: Okay.
Maor Frid: right. percentiles and their ratio with the average probably also can give you some sense of what's going on with your audience. Is the average and the median very far apart? So depends on the direction but Do you have extreme high values, extreme low values? So also these two numbers, the median and the average combined, can give you some sense of where do you need to go? what's more important, if we have extreme high values. let's try to sub-segment the higher values into buckets according to percentile 75, 90, 95, etc. etc.
Naama: perfect. We have a few minutes left, because you know it's a very, very short podcast, and I don't see how I can cut whatever you said. So give us a tip because most of my listeners are from the monetization and live ops world. And since we are looking at dashboards and data all the time, we refresh them every minute I wanna say. How do we differentiate between a signal that we're seeing that is about to turn into a trend and just a noise or like a singular fluctuation in data?
Maor Frid: So first of all, yeah, most of us all live in a world that dashboards are alive and refreshing every second minute and we keep track of them all the day. I think the best way to differentiate between noise and actual signal is to first of all, I know it sounds obvious, but understand what you're looking at. it's not that obvious and in rich data organization like you and I are both part of we have a lot of dashboard reports charts that cover a lot of different metrics so you keep track of everything so first of all know what you're looking at if your chart that you're looking at is the minutely revenue, true story, I do have it. So this will fluctuate quite a bit. So you need to understand what you're looking at. if you're looking at the return retention rates for the month, So it probably shouldn't fluctuate that much, and you have you can set yourself a control chart so minus X and plus Y, those are the guardrails, preferably are also embedded in the actual chart that you're looking at. that someone in the data site did his job and provided those guardrails, but okay so I know that under this beneath this threshold I should be panicked or otherwise I'm pretty safe. so first of all understand and know what you're looking at and I think second of all is have supporting reports charts to understand the impact. the easiest example the inflow in currency is very high. Yeah, because players are purchasing. It's not a bad thing. You just need to understand what you're looking at and What's the reason behind it? Again going back to what's the business question, not what's the data question, and going back to what's the behavioral change of the players? What's changed? Not only what the numbers say.
Naama: I'm smiling because I've been there so many times and I just can relate to what you said. so yeah. all of those clarifications what are we looking for? what are we driving? what has changed? and yeah, just dig more. so I I just wanna say we haven't covered even a tiny part of what we plan to talk about. So let's have another episode
Maor Frid: Thank you for inviting me. It was great seeing you again.
Naama: really wanna Yeah. and thanks for listening to Dopamine Lab. I wasn't here for a while because I was under the weather and then I was traveling but I will be here every Tuesday and hopefully with Maor again that's it. See you next time.
Maor Frid: Yeah.
Naama: now you go ahead and tell us who you are and tell us about your relationship with data.
Maor Frid: So thank you very much. So first of all thank you for the warm and kind words. Yeah, I loved working with you too. it was a while back now, but still remember it very very good. so my name is Maor Frid, I'll introduce myself shortly. So I've been working in gaming and data for the last let's say nine years-ish. worked with you in Slotomania, then moved to other roles including monetization product, going back into analytics in a broader sense in the tech division, and currently in a small startup still in the gaming industry, in a very exciting startup in a new domain making games in the telegram ecosystem which is a whole different story and very different audience and very different habits for those players unlike the standard US audience that we all know and love from From mobile
Naama: Hm.
Maor Frid: games.
Naama: Well, can I guess? Probably shorter sessions and probably way more casual.
Maor Frid: It is shorter sessions but much more sessions in a day.
Naama: Mm.
Maor Frid: just think how much time you spend in WhatsApp. so just
Naama: Yeah. Wow. Yeah.
Maor Frid: think that you have like a game within the WhatsApp platform. It just like it's a click away, everything is super fast, loads up instantly, you don't need to download additional apps or stuff everything is within the same and the same platform china has it for years now in the WeChat which is like one app to rule them all like only China knows
Naama: Mm-hmm.
Maor Frid: how to do Telegram has let's call it similar features, obviously different audience, but but similar features and trying to create like a more holistic ecosystem and not WhatsApp which is primarily and mostly only messaging.
Naama: Yeah. Wow, here in the other side of the world I don't think I've ever heard about WeChat. I don't know if Telegram is a popular thing here. I know in Israel it is. but wow, it's super interesting. Very cool.
Maor Frid: No, Americans all know the I message so that's that that's it.
Naama: Yeah.
Maor Frid: That that's why they know.
Naama: Yep, that's it. Okay. make sure to stay with us until the end because Maor has a pro tip for something I think almost everyone who works with data struggle with. So stay tuned. And let's start with something that sounds almost stupid. When we say the players, who are we actually talking about? Because there isn't a player. There isn't one player. There are people who play for five minutes, people who play for five hours, people who spend one dollar, people who spend hundred dollars, people who play because they're competitive, and some that play because they're bored, because their friends are there. you see where I'm going.
Maor Frid: Yeah, yeah for sure. For so for a long time now the buzzword is personalization. Someone somewhere said hyper personalization.
Naama: Ha ha.
Maor Frid: I have to say that I think that not many companies and products are there, mostly are still at the place of segmentation. In different variations of it, and let's call it in different levels. At the end, personalization, it's a segmentation with n size equal to one, like a single player within a segment. I have to say that in most cases that I saw and like got people talking about personalization, we always came to a conclusion of overfitting. Something from like mathematics or and statistics models, overfitting, like personalization, it's amazing, but at the end, real personalization is if I'm sending you a message, if I can include your name inside, that's personalization, that's great, that's personal. But when I'm talking about okay, I want to send an email to everyone who did da da da da da da so it's not a person it's a segment of players that fit a certain criteria or criterias that we combine them together and into a segment. The the real question and the actual difficulty in our current rule is that we are drowning data and we need to make sense of the data and to understand how it comes together into actual player experience. What is actually relevant, what is irrelevant or what is relevant in a specific point of time or in a specific context. Just top of my head a few examples of trying to to make sense of what I said. So you said telegram probably have shorter sessions and I said yeah but more sessions a day. So is a longer session better than a shorter session? what does it mean? maybe for Netflix as a company, longer sessions are great. binge watching Netflix is amazing. But wait, what what's binge watching? So is it too long? Did you fall asleep? We all woke up on the couch
Naama: Yeah.
Maor Frid: to the screen. Are you still there?
Naama: Yeah.
Maor Frid: From Netflix. that no, I'm not here, I fell asleep on the couch. So My session was very long but I wasn't actually active on it. and so coming back to to to our world to the gaming world, so what's better if I have a single session which is like I don't know twenty minutes, thirty minutes long. Is it actually a good session? Like did someone play my game for twenty consecutive minutes or did he just get lost in those twenty minutes somewhere along the way? so I think that when we have a lot of data we need to tie it to what we want to learn.
Naama: I just want to say about session length that I think it's a great example because it's a metric that is hiding more data in it, and it's not necessarily a metric that we should look at because like you said, long session time can mean confusion can mean scraping coins or tickets or whatever currency it is within other things that are not the core loop. so it's very interesting. I hundred percent agree with you about session length or session time that it's not necessarily a metric that we should look at. And this is part of the problem of having endless data that a lot of time is perceived as a good thing. yeah, we have so much data, we have endless data, but what do we do with it? How do we approach it? How do we ask the right questions?
Maor Frid: so I'll so I'm I think that if in the past the issue was we don't have enough data or we don't have the right data, so those days are gone. we have the data. Probably each company has every data they can dream of. And if they don't they can access it via other points But the real question remains in each business, what do I want to achieve and how can I get to it? So we have a lot of junk data that we collect. Just I don't know. Every mobile game can request access and probably most people just approve it. to your I don't know to your gyroscope in the in the in your phone the accelerator saying how how is the device moving so probably for most I don't know gaming companies it's not that relevant but because they don't have again not that I can think of they don't have a lot of things to do with that information just collect it hoard it But Uber for example it's critical for them. Or if I'll take a more Israeli proud approach, waze are you moving? Are you stopping? Are you accelerating, slowing down, what's going on with your device? is it up in the air? Is it down facing down, etc. etc. All those Data points become more interesting again because the business question changes. So it's not just collecting data, it's collecting the relevant data and asking the relevant questions to your business.
Naama: Yep. Asking the relevant question is crucial and maybe later we can get to that point deeper and you can help us and tell us how do we ask the right questions?
Maor Frid: j regarding the right question. So there's funny story that I tell every time that I get to that conversation and every time that I have intro meeting with employees, wherever I work, Monetization guys product, etc. So I always tell about a friend of ours, a mutual friend. We call him Dubi even though his name is
Naama: Yeah.
Maor Frid: Guy. But so a while back I was an analyst, he was a copywriter, he popped his head into the business analytics room and guys how many users do we have?
Naama: Wow.
Maor Frid: So that turned on every cynical fiber
Naama: haha
Maor Frid: in any analyst in the room and God knows we have a lot of them. We started bombarding him with every question Wha what do you mean how many? Ever recently active? What's active? Active spinning? Purchasing?
Naama: Mm-hmm.
Maor Frid: buyers, non-buyers. in a s specific region in the world? we asked him ten minutes worth of
Naama: Mm-hmm.
Maor Frid: questions he just turned on and walked away. So I I I G I got up, I followed him to the kitchen. I'm like, listen man, you asked the question. We obviously didn't give you answer. So first
Naama: Yeah.
Maor Frid: of all I apologize, but what what do you need? Like as a copywriter I assume that the question how many players do we have in your head sounds like a very simple one As you saw, it's not that simple. So tell me what you need and let's work together. And he told me we're working on a partnership with a different company and I'm working on the copy for the ad that we're gonna advertise on their platform. So I'm trying to think if I can use the sentence join. Millions of other players already playing or should I say hundreds of thousands it was a great example. This is why I I say it all the time. You have a question in your in your head and you have a goal that you need to reach. Like can I write join millions of players playing playing our game? So that's what you need to know. Okay, that's your end goal. If I'll take it to a deeper sense, are players engage with my feature? That's every product manager question. And usually it becomes a whole new 30 minute session of what's engagement. What what did you want them to do? Engaged as in they saw your event within the game? Yeah, everybody saw it. Did they actually do what you wanted them? So the the best way to ask the right data question is not to ask a data question. is to be focused on the business question. Like what's the habit that you wanted to form? What's the change in habit, change in behavior you wanted to see in the players, And from there we can build what's the relevant data, what's the question, how do we need to approach it, etc etc. I think that's the the golden tip the golden tip is Data is just another point of the business. So you have a business question, and in the end, as this whole podcast is about, our business is people and how they behave. So at the end, every question comes down to Habits and behavioral change within people.
Naama: Yeah, thank you for the great definition. This podcast is about people and how they behave within the dopamine lab, which is called a game because the game is is the biggest dopamine lab. So yeah, that that's great. And also I think you really, really distilled the essence of how do we ask the right questions? What do you want to know? What are you driving? What do you want to drive? Is if the feature is driving more spins or more or higher bets or even number of clicks or number of clicks on an external arena or whatever that you're trying to drive, this is your question, right?
Maor Frid: also even before the what, the who like who are the players that you're trying to to change their behaviour. Some features, some aspects of any game are what we know as FTUE first time user experience. They have nothing to do with our VIP audience. with our I don't know top spenders of the game players that downloaded the game two years ago and are still playing so what happens on like the first level of the game they probably don't remember even how the game started what what feature it has and etc etc so even before the what is the who Who are you targeting?
Naama: Yeah, great point. Okay, I have a question because I have something that is really bothering me and it's the fact that a lot of our data is just aggregated and divided data, which is the average. and I think that average has two main risks in it. imagine we have ten players, nine players spend one dollar, one player spent hundred dollars, and the average is ten point ninety. So our data tells us that the average revenue per paying user is ten point ninety. And here's the funny part, no one of those ten players spent 10.90 dollars. So we created a character who doesn't actually exist. So that's one. I I guess the the right thing to do here is look at the median, but you tell me later. Another risk is that the average doesn't move, but it's because one segment pulled it this way, and another segment pulled it in the other direction. So that's actually two segments that moved dramatically, but the average didn't move.
Maor Frid: so without going into the the nerdy version of me so
Naama: Mm-hmm.
Maor Frid: first of all the average has its mathematical d disadvantages of being highly affected by extreme values like in your case if you average out a non-payer with high payers so The average is let's call it a medium to high payer, which doesn't exist. So to avoid that mathematical skewness, probably what we usually do we look at more precise values if we're talking about like the median or we're trying to break it down into percentiles. Percentile 25. 50 which is the median 75, 90, or 95 if we're looking at extreme cases. So that's one way of looking at it. The other thing is to look at the distribution of values. So we're talking about the VIPs the most valuable players, how much they spend, what's a VIP. So we need to take into consideration what are the relevant options in the game? I usually try to look at the distribution? How many players do we have in each range or buckets if you want, of spending values? Again, doesn't have to be money spending. Obviously, we have habits and have ranges for everything, the infamous session length or amount of core loops the player did, the amount of stickers he collected within the collectible feature of the game. so yeah the average is probably something that we shouldn't be looking at unless and I'm Jumping back and forth from what we talked about, and unless we have a really good defined segment and sub-segment, and then you know, okay, you know what? So averaging those players together is okay. but
Naama: Okay.
Maor Frid: If you're not sure, probably average is not a number that you can count on and for sure not when you're trying to define player experience. Like the average player doesn't exist.
Naama: Nice, nice. Okay, perfect. So you're saying ways to avoid this risk is one, to look at percentiles, two, to look at distribution, and I don't want listeners to leave this episode thinking average is a bad thing. So you answered that thing too. What if you have a very, very defined segment and a sub segment, you can totally look at the average and it's even I guess more efficient sometimes. please correct me if I'm wrong.
Maor Frid: No no you you're totally
Naama: Okay.
Maor Frid: right. percentiles and their ratio with the average probably also can give you some sense of what's going on with your audience. Is the average and the median very far apart? So depends on the direction but Do you have extreme high values, extreme low values? So also these two numbers, the median and the average combined, can give you some sense of where do you need to go? what's more important, if we have extreme high values. let's try to sub-segment the higher values into buckets according to percentile 75, 90, 95, etc. etc.
Naama: perfect. We have a few minutes left, because you know it's a very, very short podcast, and I don't see how I can cut whatever you said. So give us a tip because most of my listeners are from the monetization and live ops world. And since we are looking at dashboards and data all the time, we refresh them every minute I wanna say. How do we differentiate between a signal that we're seeing that is about to turn into a trend and just a noise or like a singular fluctuation in data?
Maor Frid: So first of all, yeah, most of us all live in a world that dashboards are alive and refreshing every second minute and we keep track of them all the day. I think the best way to differentiate between noise and actual signal is to first of all, I know it sounds obvious, but understand what you're looking at. it's not that obvious and in rich data organization like you and I are both part of we have a lot of dashboard reports charts that cover a lot of different metrics so you keep track of everything so first of all know what you're looking at if your chart that you're looking at is the minutely revenue, true story, I do have it. So this will fluctuate quite a bit. So you need to understand what you're looking at. if you're looking at the return retention rates for the month, So it probably shouldn't fluctuate that much, and you have you can set yourself a control chart so minus X and plus Y, those are the guardrails, preferably are also embedded in the actual chart that you're looking at. that someone in the data site did his job and provided those guardrails, but okay so I know that under this beneath this threshold I should be panicked or otherwise I'm pretty safe. so first of all understand and know what you're looking at and I think second of all is have supporting reports charts to understand the impact. the easiest example the inflow in currency is very high. Yeah, because players are purchasing. It's not a bad thing. You just need to understand what you're looking at and What's the reason behind it? Again going back to what's the business question, not what's the data question, and going back to what's the behavioral change of the players? What's changed? Not only what the numbers say.
Naama: I'm smiling because I've been there so many times and I just can relate to what you said. so yeah. all of those clarifications what are we looking for? what are we driving? what has changed? and yeah, just dig more. so I I just wanna say we haven't covered even a tiny part of what we plan to talk about. So let's have another episode
Maor Frid: Thank you for inviting me. It was great seeing you again.
Naama: really wanna Yeah. and thanks for listening to Dopamine Lab. I wasn't here for a while because I was under the weather and then I was traveling but I will be here every Tuesday and hopefully with Maor again that's it. See you next time.
