TRANSCRIPTION 🔗
WEBVTT
00:00:06.879 --> 00:00:11.599
Welcome to Responsibly Human, a responsibly different podcast from Campfire Consulting.
00:00:11.839 --> 00:00:18.800
We live in a world getting remarkably good at finding patterns in human behavior, but understanding people is something different.
00:00:19.039 --> 00:00:30.399
On this show, we talk with researchers, technologists, marketers, creators, and leaders about what actually moves us, earns our trust, and makes us feel known instead of simply seen.
00:00:30.640 --> 00:00:34.960
And as more of our world becomes automated, one question keeps getting more important.
00:00:35.200 --> 00:00:37.600
What does technology still miss?
00:00:38.560 --> 00:00:43.280
Before we get into it, one small way you can support more thoughtful work is by hitting subscribe.
00:00:43.439 --> 00:00:52.560
That simple act helps this show reach more people who are trying to build businesses, communities, and a world that works a little better for all of us.
00:00:52.799 --> 00:00:59.920
If something you hear in these conversations helps someone else think differently or do their work differently, that's the whole point.
00:01:00.240 --> 00:01:05.920
I'm Chris Marine, founder and CEO of Campfire Consulting, and this is responsible to you.
00:01:49.760 --> 00:01:52.319
ROAS is up, conversions are down.
00:01:52.480 --> 00:01:54.560
The creative perform better than that one.
00:01:54.719 --> 00:02:01.040
We put all that into dashboards and models, and eventually, those outputs start to feel a lot like truth.
00:02:01.280 --> 00:02:05.280
But what if some of the most important signals are the ones we've been calling noise?
00:02:05.519 --> 00:02:07.920
That's where my conversation with Rom Sing went.
00:02:08.080 --> 00:02:12.479
Rom had spent his career building measurement systems and challenging the assumptions underneath them.
00:02:12.639 --> 00:02:17.680
And what I found most interesting wasn't a new model or another way to optimize media.
00:02:17.840 --> 00:02:24.000
It was his argument that better technology should actually make us more comfortable with uncertainty.
00:02:24.240 --> 00:02:25.280
Not less.
00:02:25.680 --> 00:02:28.879
Because people don't always behave the way the data says they should.
00:02:29.039 --> 00:02:32.960
Sometimes the campaign that looked successful was going to get the sale anyway.
00:02:33.199 --> 00:02:38.719
Sometimes the message that looks like it failed simply hasn't had enough time to become familiar.
00:02:38.879 --> 00:02:45.439
And sometimes the most important thing a model can do is tell us something we didn't want it to tell us.
00:02:45.919 --> 00:02:59.439
So we talked about what marketers are measuring because it's useful, what we're measuring simply because it's convenient, and why the ability to challenge the answer may be one of the most important things we still bring to the table.
00:02:59.840 --> 00:03:02.960
Here's my conversation with Rom Singh.
00:03:06.400 --> 00:03:26.719
So, Rom, what pulled you toward computer science, analytics, and eventually marketing to kind of just get everyone familiar with your background and how when we were talking a few minutes ago before hitting record, uh that idea that this was kind of a sounded like an a place you didn't know you were gonna end up in.
00:03:27.520 --> 00:03:28.080
Yes.
00:03:28.639 --> 00:03:31.520
Well, first off, thank you, Chris, for having me on the show.
00:03:31.599 --> 00:03:32.719
I appreciate this.
00:03:32.879 --> 00:03:41.039
Um, just like we were discussing, my journey has been more defined by the need and then jumping in to solve a problem.
00:03:41.199 --> 00:03:48.479
I I started uh with an undergrad in computer science, I started at IEBM as a software engineer.
00:03:48.719 --> 00:04:00.000
But very quickly, when I was in the field talking to clients, talking to partners, I started to enjoy learning about what their problems were and how to solve them.
00:04:00.080 --> 00:04:06.560
And that led me to a lot of management consulting realm, and I started using data a lot.
00:04:07.280 --> 00:04:10.159
I was a computer scientist by trade, not a data scientist.
00:04:10.240 --> 00:04:12.479
In fact, the word data science didn't exist then.
00:04:12.719 --> 00:04:20.800
And then in order to solve those questions, I often had to look at volumes and volumes of data and try to decipher what I'm going to be building.
00:04:20.959 --> 00:04:47.279
That is what started to lead me into data science, and that to specifically marketing decision sciences, eventually followed the road of becoming a marketer, became a CMO at a very large company, started to run marketing, and then standing up in front of a board of directors and trying to answer what in marketing is working, why something happens that you can't explain in data, that kept refining my purview.
00:04:47.360 --> 00:04:53.360
And eventually, then I spent the last decade and a half purely focused on advanced measurement.
00:04:53.519 --> 00:04:54.720
So it was a journey.
00:04:54.800 --> 00:04:56.240
Uh, it found me.
00:04:56.480 --> 00:05:05.680
I didn't uh I don't know if I will I will say I started uh by that thought in mind, but I certainly enjoyed uh doing that.
00:05:05.759 --> 00:05:06.720
It it found me.
00:05:07.120 --> 00:05:08.560
Those are the best kind of journeys, though.
00:05:08.639 --> 00:05:14.319
I mean, I think that's what makes like a well-rounded perspective, and that perspective is something that makes us all human.
00:05:14.399 --> 00:05:30.079
So was there a moment though, like through that journey that you ever think back and point to where it's like there was that moment of understanding there is a number or a story that people were trusting where you're like, that just doesn't seem right to me?
00:05:30.399 --> 00:05:30.720
Correct.
00:05:30.879 --> 00:05:31.279
Correct.
00:05:31.519 --> 00:05:33.839
Many, many, Chris, along the way.
00:05:34.079 --> 00:05:43.439
Load full transcript
Look, at the end of the day, one of the things you'll probably hear this from a lot of data professionals, and it is one of the things that I've sent a significant amount of time.
00:05:43.600 --> 00:05:50.720
When we use data to find a pattern or prove something that you will do, you will invariably find that data.
00:05:50.959 --> 00:06:02.720
When I first, my first foray into the marketing world where I was running marketing and I was answerable to things, I would often look at reports that would say, okay, this is an opportunity.
00:06:02.800 --> 00:06:14.720
I whether it's a campaign, whether it's an in-market spend, what have you, right off the bat, there would be so many instances that you make a decision based on a metric that is available, you direct spend, but then it doesn't work.
00:06:15.040 --> 00:06:17.680
So uh it doesn't produce the results that you were expecting.
00:06:17.839 --> 00:06:24.879
My point in sharing that is there were so many instances from early on where I saw you can apply this to bids.
00:06:25.120 --> 00:06:42.800
One of the things I did early on is build a bid engine based on a portfolio model because oftentimes when you were bidding on certain digital campaigns, whether it was ad placements, ranks, etc., you would find you do it with one purpose looking at data, but the outcome is different.
00:06:42.959 --> 00:06:52.879
So my point in sharing all of this is there were many instances where I saw what was reported out or what was in front of my my eyes didn't produce the results.
00:06:53.040 --> 00:06:58.399
So that is when I went on a quest of like, is there a better way to make decisions?
00:06:58.560 --> 00:07:05.920
Is there a better way to build a portfolio model instead of just looking at a report and doing a highest bid win kind of thing?
00:07:06.079 --> 00:07:17.600
Let me pause for a second and try to give you some examples of these are the things which really challenged my thought process, saying, am I looking at it correctly or is there an alternate way of looking at the problem?
00:07:18.160 --> 00:07:22.959
I mean, and that's always a challenging thing going from like what you know to then building something different.
00:07:23.600 --> 00:07:27.040
Is there in your journey?
00:07:27.279 --> 00:07:36.879
I know in my personal journey, like there's all sorts of things that I had to unlearn going from like the working within media companies and like the ad sales side to the buying side.
00:07:37.040 --> 00:07:45.360
Are there things in your journey that you had to unlearn to get you to the point where you could build something different?
00:07:45.759 --> 00:07:50.160
Uh I I I yes, and I think the answer is in your question.
00:07:50.319 --> 00:07:56.000
Um, you know, in my case, I've I've been a I've been a builder for a long time, uh very long time.
00:07:56.160 --> 00:07:59.920
Uh almost the entirety of my work has been building out solutions, right?
00:08:00.160 --> 00:08:06.800
One thing I learned early on is in order to improve your solution, uh, and this is just my point of view.
00:08:06.879 --> 00:08:09.680
So I just want to make sure that I'm the best point of view.
00:08:10.160 --> 00:08:13.040
Yeah, I I there could be differences of opinion.
00:08:13.279 --> 00:08:19.279
In my case, the learning was if any solution that I build is not my life's work.
00:08:19.519 --> 00:08:25.680
Because if I attach myself so hard to it, I start to see what I could do to improve it.
00:08:25.839 --> 00:08:28.959
And I have built a lot of blockbuster solutions.
00:08:29.199 --> 00:08:37.759
And as builders, we have a natural affinity to stand very, very, very close to what we build because we're passionate about this, right?
00:08:37.919 --> 00:08:38.799
I build a new model.
00:08:38.879 --> 00:08:40.639
I just told you I build a portfolio model.
00:08:40.720 --> 00:08:44.000
At that point, second price auction was a new concept.
00:08:44.159 --> 00:08:49.919
So, me, even building a bid engine using portfolio theory from finance was a new thing.
00:08:50.080 --> 00:08:51.440
I was so proud of it.
00:08:51.679 --> 00:08:58.960
But in doing so, you attach yourself to that position, even if you don't know it, your ego gets attached to what you build.
00:08:59.039 --> 00:09:10.320
And one of the things I had to unlearn over the years is if I have to go back and make changes and improve my product, I have to be very conscious about the distance between my product and my ego.
00:09:10.480 --> 00:09:17.840
So one of the things I consciously try to do is if I build something, I'm also the one occasionally challenging my own thinking.
00:09:18.000 --> 00:09:20.720
That doesn't come naturally or that didn't come naturally to me.
00:09:20.879 --> 00:09:29.440
I had to learn it because if I don't do that, I'll be stuck with this one thing that I built while the rest of the world moves on, and then I'm playing catch up.
00:09:29.919 --> 00:09:30.639
That's interesting.
00:09:30.879 --> 00:09:32.559
I love that framework and thing.
00:09:32.879 --> 00:09:38.559
That's just uh self-awareness, I feel like is everything, and that's a huge indicator of it.
00:09:38.639 --> 00:09:40.080
Are there people in your circle though?
00:09:40.240 --> 00:09:43.840
Because I know when trying to like question yourself, it can be challenging.
00:09:44.000 --> 00:09:52.879
Are there, do you have do you build like a close trust circle of humans where they are you go to and you're like, I'm I'm coming to you because I want you to push back?
00:09:52.960 --> 00:09:53.279
Yes.
00:09:53.600 --> 00:09:55.919
Or how do you kind of frame that up?
00:09:56.000 --> 00:09:58.480
Or is it much more expansive than a small circle of people?
00:09:58.960 --> 00:10:02.960
No, you're your your very astute, your question hit the nail on the head.
00:10:03.120 --> 00:10:12.480
When I first came to the agency side 11 years ago, I started building sandboxes, whether it was for Publicis or any
company that I work for.
00:10:12.720 --> 00:10:20.720
In fact, my current company, Etacore, is founded off of a work that we were doing in a sandbox for multiple years.
00:10:20.960 --> 00:10:30.159
The idea of a sandbox is to get thinkers in a room and have a productive point-counter point kind of discussion.
00:10:30.320 --> 00:10:35.759
When we are in the world of measurement and analytics, things can get very passionate.
00:10:36.000 --> 00:10:49.759
You are building a new algorithm, you're building a new solution, and oftentimes it is more usual than an exception, more of a known than an exception that you'll find people put something out and not vigorously stand behind that.
00:10:49.919 --> 00:10:52.960
So I have been using sandboxes for the last 11 years.
00:10:53.039 --> 00:10:56.399
And the purpose of a sandbox is like an incubator.
00:10:56.559 --> 00:11:02.799
That is where you work on a solution and you give others the opportunity to challenge, or you can challenge your own thing.
00:11:02.879 --> 00:11:08.639
One of the things I've always talked to my team members building product is identify the assumptions.
00:11:08.879 --> 00:11:16.960
Once you take the effort of listing the assumption, you're telling your brain what are the things I may have missed to look in building the product.
00:11:17.039 --> 00:11:28.559
So you're not only becoming a little self-aware, list that out when you go into a sandbox, a lot of then discussion will, you'll be surprised how much discuss the discussion can go in terms of improving the outcome.
00:11:28.720 --> 00:11:33.200
You sowed a seed, but the nurturing can be done by multiple different people.
00:11:33.360 --> 00:11:40.639
And that has always worked for me, Chris, for 11 years, because in product building, you need to be aware of what are your oversights.
00:11:40.799 --> 00:11:47.360
Einstein said you cannot solve a problem by looking at it from the same angle when you discovered the problem.
00:11:47.519 --> 00:11:49.279
Otherwise, you're going to keep seeing the problem.
00:11:49.519 --> 00:11:50.879
You have to change your point of view.
00:11:51.360 --> 00:12:04.480
The best way to change your point of view is do a little self-pretty and do a point-counter point discussion with other experts who can point out potential issues that you may have missed to look.
00:12:05.200 --> 00:12:11.679
And you've expressed that distinction before between systems that confirm what happened and learning models that tell us what should happen next.
00:12:12.000 --> 00:12:24.159
I mean, to everything you just said, you I think you partially answered it, but but why why is that distinction so hard for organizations to accept and actually move forward through?
00:12:24.399 --> 00:12:27.039
Is it that attachment to ego or is it something?
00:12:27.840 --> 00:12:30.399
I I think there is a mindset in play here.
00:12:30.559 --> 00:12:41.919
I think when you look at data or a measurement solution to prove or justify what was done, the I'm not a neuroscientist.
00:12:42.000 --> 00:12:45.600
I'm just putting it in the ways I understood it myself.
00:12:45.919 --> 00:12:58.320
When you use data analytics to prove what you are doing, you're basically sending a signal to your brain saying, I did something right and I need to find evidence on why that was right.
00:12:58.639 --> 00:13:04.879
If you take the other approach, you've just built something, and you're like, I might be surprised by the finding.
00:13:05.039 --> 00:13:13.759
I build a data and analytic solution, I build an NL model, I build an AI native solution that might present to me an outcome I was not expecting.
00:13:14.000 --> 00:13:23.039
It asks me to cut a certain campaign because it's detecting cannibalization that otherwise would not be detected in your standard way of just looking at tech logs.
00:13:23.519 --> 00:13:31.200
It says, reduce the spend on this, but you're like, no, that's my hero campaign, that's my main brand campaign, or that's my main lead gen campaign.
00:13:31.279 --> 00:13:33.360
I need to keep it ever green.
00:13:33.519 --> 00:13:38.320
My point in giving that example is you were presented an outcome that you were not expecting.
00:13:38.559 --> 00:13:40.159
That's a different signal to the brain.
00:13:40.320 --> 00:13:44.320
You're essentially going back and saying, okay, what could I have missed?
00:13:44.559 --> 00:13:54.879
So when you find data to prove what you're doing, versus when you try to look at outputs that present to you something that you were not expecting, those are two different patterns of working.
00:13:55.039 --> 00:13:57.120
That's not just a difference in observation.
00:13:57.279 --> 00:14:03.360
It's not report is wrong or outcome is wrong or they are not aligning.
00:14:03.679 --> 00:14:06.799
You're sending a different signal to yourself, to the team members.
00:14:06.879 --> 00:14:10.960
In one case, you're gonna go put a lot of effort in trying to justify what was done.
00:14:11.120 --> 00:14:17.120
In the other one, you're going to be more forward-looking and saying, maybe there's a different way of addressing the problem.
00:14:17.279 --> 00:14:32.080
So I that's why I feel those things are not only important from a measurement perspective, but they're also uh fundamental to defining how measurement works for you, your team members, your clients, your audiences, etc.
00:14:32.399 --> 00:14:34.720
Distinctly different patterns of working.
00:14:35.440 --> 00:14:44.559
Which goes to one of your core arguments, which is around that there are a lot of organizations still running AI area analytics through operating models built for retrospective BI.
00:14:44.799 --> 00:14:45.039
Correct.
00:14:45.200 --> 00:14:45.440
Correct.
00:14:45.600 --> 00:14:45.840
Correct.
00:14:46.000 --> 00:14:46.159
Correct.
00:14:47.279 --> 00:14:56.480
Is there one metric that you find uh that marketers lean on uh most when it's not entirely useful?
00:14:56.799 --> 00:15:01.919
Especially if they're using the models in the wrong in the wrong uh lens?
00:15:03.200 --> 00:15:05.519
I'm sure you've heard this from others as well.
00:15:05.759 --> 00:15:11.039
ROAS tends to be used as a as a gold standard to performance.
00:15:11.279 --> 00:15:22.879
Um it used to be ROAS, then we pivoted to incrementality-oriented ROAS, but the whole idea of using a metric, and you can you can take your pick.
00:15:22.960 --> 00:15:24.399
There are others like that too, right?
00:15:24.639 --> 00:15:25.919
Conversions could be one of those.
00:15:27.120 --> 00:15:27.600
Right.
00:15:28.159 --> 00:15:43.759
Because the thing is, the implied assumption there is if you think, if you break down return on ad spend and how it not only how is it calculated, but how people approach it, it's essentially trying to say, I want to quantify a benefit per unit spend.
00:15:44.080 --> 00:15:44.799
That is what it is.
00:15:45.120 --> 00:15:45.840
What is the benefit?
00:15:46.000 --> 00:15:48.720
That benefit could be revenue, that could be latent revenue.
00:15:48.879 --> 00:15:52.159
You we're ex- The good news is we're expanding our consideration.
00:15:52.320 --> 00:15:58.720
But in that metric is included a basic premise, a quantification of output.
00:15:59.279 --> 00:16:00.240
What does that mean?
00:16:00.720 --> 00:16:02.559
Spend is not debatable.
00:16:02.720 --> 00:16:05.120
That that you can you you you know how much money you spend.
00:16:05.279 --> 00:16:08.320
So there's no there's no uh gray area in there.
00:16:08.559 --> 00:16:13.440
But the quantification of benefit itself is a very undefined space.
00:16:13.600 --> 00:16:29.039
You can quantify the benefit as uh revenue attained in the next 30 days, you can uh quantify revenue, uh uh benefit as revenue now plus future revenue, you can uh keep thinking of engagement metrics, quality metric.
00:16:29.279 --> 00:16:33.679
My point here is that quantification becomes subjective.
00:16:33.840 --> 00:16:39.440
And if you don't have a good measurement basis to define that, your ROAS number will be misleading.
00:16:39.679 --> 00:16:45.120
Um I I I'll I'll pause for a second to see if you have uh any question uh before I build up on it.
00:16:45.360 --> 00:16:49.679
I mean, I think it uh I'm and maybe you're going here with this, but I mean, we see that quite a bit.
00:16:49.840 --> 00:16:53.840
Media sometimes gets credit it deserves and sometimes a credit that it doesn't deserve.
00:16:54.080 --> 00:17:06.079
So if ROAS looks great, but the casual model says the spend got credit for demand, that would have happened anyway, which we have seen where literally media hasn't even been running, but they're still trying to attach a ROAS.
00:17:06.160 --> 00:17:08.319
We're like, media wasn't even active when that was happening.
00:17:08.480 --> 00:17:08.640
Correct.
00:17:08.799 --> 00:17:12.079
So I mean, how should marketers respond in those moments?
00:17:12.240 --> 00:17:23.279
Um, if if that one metric ROAS, which is kind of this now umbrella term to all how your brand moves through the world, how how do you respond to that, to those metrics?
00:17:23.599 --> 00:17:33.519
See, one of the things I'm trying to build, Chris, is a balance between what gets immediately measured and what are some of the latent benefits of measuring.
00:17:33.680 --> 00:17:38.240
So one of the examples I'll use is uh and I'll bring this back to marketing in a second.
00:17:38.400 --> 00:17:41.759
Think, for example, how stock valuation metrics work, right?
00:17:42.000 --> 00:17:51.920
If you're buying a stock and you're looking at a PE multiple, a PE multiple of 30 for a tech start might be considered okay or low.
00:17:52.079 --> 00:17:58.960
A PE multiple of 120 for a start like Palantir is still like, okay, still buy, even though it is super expensive.
00:17:59.200 --> 00:18:12.480
If you think of ROAS in that kind of a consideration, until unless you have some form of a valuation metric in marketing, it's very difficult to assign interpretation to a value of ROAS.
00:18:12.640 --> 00:18:16.400
So where I'm going with this thing is any of these, that was my premise.
00:18:17.279 --> 00:18:26.640
The computation of a return or a return on benefit is reliant on what are all the potential ways in which marketing could spend could help.
00:18:26.799 --> 00:18:30.720
It can increase revenue, it can increase conversion, those are the direct measures.
00:18:30.960 --> 00:18:35.519
But are we measuring any latent lift in brand value?
00:18:35.680 --> 00:18:45.759
Are we measuring any latent lift in uh you are in a very competitive segment and it will require a minimum set of exposures before any conquesting can happen?
00:18:45.920 --> 00:18:51.440
It will require a minimum amount of investment before your saturation in a given market moves.
00:18:51.680 --> 00:18:58.160
Are we measuring or have a proxy for any of those benefits that reflect in ROAS calculation?
00:18:58.319 --> 00:19:00.400
Because that contextualizes ROAS.
00:19:00.640 --> 00:19:04.960
And without that, you may find yourself in a trap.
00:19:05.200 --> 00:19:06.880
Oh, my number looks weak.
00:19:07.119 --> 00:19:16.480
My number looks great, either pull back or put more money into it, only to then realize, oh, money was already spent or wasted, and you didn't see the return.
00:19:17.119 --> 00:19:23.200
And I think that's important too, like in regards to what's impacting that.
00:19:23.279 --> 00:19:31.759
And one of the things that I've I've read from some of your posts is arguing that creative itself may be one of the marketing's biggest unexplained variables.
00:19:32.000 --> 00:19:42.079
So I mean, and I mean, we feel this we solely focus on the paid media strategy and the planning side of everything, which influences brand.
00:19:42.160 --> 00:20:03.680
But when we're given creative, there's still there are some reports that you can do in regards to
creative, but it seems like the industry for so long has been so fixated on ad placement and what that unit return is versus trying to understand how that message is actually trying to make not even trying, making people feel.
00:20:04.079 --> 00:20:16.319
Um why do you think for someone that's been in this, uh, why do you think uh got so good on veered one way and neglecting that other really important pillar of advertising and marketing?
00:20:16.559 --> 00:20:21.200
I I I I I I might underwhelm you with this answer.
00:20:21.440 --> 00:20:27.759
I think it's measuring what is convenient versus measuring what should be measured.
00:20:28.000 --> 00:20:34.319
And the reason I bring that up is oftentimes we grew up looking at at tech lots.
00:20:34.480 --> 00:20:40.319
We grew up at analyzing clicks, costs, saturation, frequency, reach, conversion, etc.
00:20:40.799 --> 00:20:42.079
Nothing wrong with that.
00:20:42.240 --> 00:20:48.400
Look, I think direct values and building analytics around them is absolutely the basic thing that we can do.
00:20:48.559 --> 00:20:54.880
We, for example, now also look at interpretation variables between combination of factors, derived variables.
00:20:54.960 --> 00:21:05.839
But before I go out, go off too far of the deep end, my key premise here is metrics that you find in any ad tech log or that can be captured, they are convenience metrics.
00:21:06.000 --> 00:21:08.319
They are good metrics, but they are convenience metrics.
00:21:08.559 --> 00:21:33.119
What we're trying about measuring is harder to measure because there is no ad tech log that says when I expose somebody to a message eight times and four of those happen to be about a feature or a benefit, and then follow that uh four exposures about a feature or benefit, then expose them to a brand message so that the brand becomes the answer to the problem.
00:21:33.279 --> 00:21:50.559
There is no metric that will define that this needs to be measured for you to then go back and correlate to say, okay, when you do it in this fashion, in this creative, in this narrative order, using these tenth pole classification, that increases my foot traffic.
00:21:50.720 --> 00:21:51.759
That increases the bike.
00:21:52.079 --> 00:21:53.279
It's hard to measure.
00:21:53.519 --> 00:21:56.960
One of the things we have been very passionate about is weak signal detection.
00:21:57.119 --> 00:21:59.759
The whole idea of weak signal detection is.
00:22:00.319 --> 00:22:07.519
What we have thrown out as noise is was because it was hard to measure.
00:22:07.680 --> 00:22:15.279
Anything that you couldn't explain immediately, the outside temperature is low, so my thermostat must be set to a higher temperature.
00:22:15.440 --> 00:22:18.480
Anything outside of that is noise.
00:22:18.720 --> 00:22:29.279
I am challenging that basic premise because the thing that we are throwing out, calling noise, might be the most important factor for optimizing, she uh optimizing that is left in our hands.
00:22:29.440 --> 00:22:35.680
So to answer your question, that is one of the reasons why you haven't necessarily seen these areas picked up as more, because they tend to be a lot more difficult.
00:22:35.839 --> 00:22:38.240
They're not based off of convenience KPIs.
00:22:38.400 --> 00:22:44.160
They actually do require dis different observations, different derivations, different interpretations.
00:22:44.240 --> 00:22:47.759
It requires a lot of work to be able to say what I'm discarding as noise.
00:22:47.920 --> 00:22:53.200
There are nuggets in that noise that I can correlate better to outcome and then push on those things.
00:22:53.359 --> 00:22:55.440
So it is a little bit harder to measure.
00:22:55.839 --> 00:23:03.680
Are there some of those nuggets that you that that you look for or one that stands out to you as being most important?
00:23:03.920 --> 00:23:22.000
I know it's a collective set of signals that you're looking at across to get enough nuggets that make a meal for a really bad analogy, but like what are some of those nuggets for people to look out for that are often probably missed because they're looking at those louder ones like ROAS or those traditional media metrics.
00:23:22.319 --> 00:23:22.799
Right, correct.
00:23:23.119 --> 00:23:26.720
We find a lot of those in creative and narrative analysis.
00:23:26.880 --> 00:23:28.640
That's the place where we're finding most of those.
00:23:28.720 --> 00:23:32.319
And some of those examples could be placement of a class.
00:23:32.559 --> 00:23:35.359
A class is let's say defined as we break the creative down.
00:23:35.519 --> 00:23:50.400
Let's say classes are defined as feature-oriented messaging, benefit-oriented messaging, presence of a brand, uh uh a promotion, uh, what kind of uh image invokes what kind of uh recollection in your mind?
00:23:50.480 --> 00:23:52.799
We we build out classes in creative, right?
00:23:52.960 --> 00:24:00.240
And so oftentimes we'll we're finding non-zero weak correlation between these classes and outcomes.
00:24:00.400 --> 00:24:05.839
So we will go back and say these classes tend to maximize click-throughs.
00:24:06.240 --> 00:24:18.400
You on the ad text log side, I don't I mean rhetorical view, people on the ad tech log side will see, oh, this is why I have a creative with a very high hook but zero sustain.
00:24:18.720 --> 00:24:23.920
I only see the first 10% of a creative view through, and after that, people click out.
00:24:24.079 --> 00:24:25.839
That doesn't mean that the creative is bad.
00:24:25.920 --> 00:24:33.279
That simply means that the sequencing of the signals that you're sending in that creative is designed for a high impulse.
00:24:33.440 --> 00:24:36.799
And after that, there is not much left in that creative to tell.
00:24:37.039 --> 00:24:47.519
So you shouldn't be throwing that creative out the uh out the window saying, I put creative uh that only works seven seconds out of a 30-second ad, and so
toss it out.
00:24:47.680 --> 00:25:02.240
No, that creative has, because we've done a breakdown, it could be the single biggest lever that you can couple with, let's say, say skippable buys and increase your click-throughs or view-throughs, then sequence it out with something that has a higher sustain.
00:25:02.319 --> 00:25:03.920
Now all of a sudden you're pull through.
00:25:04.079 --> 00:25:08.640
My point in sharing a little bit and going a little off of the deep end here is this is how we think.
00:25:08.799 --> 00:25:14.480
These are some of the signals that we are detecting, which otherwise there's no place to go to.
00:25:14.640 --> 00:25:19.200
When I present this outcome to you, you don't have to change your experience, you don't have to change your story.
00:25:19.440 --> 00:25:23.359
Just knowing that this is what causes a outcome.
00:25:23.519 --> 00:25:29.359
That outcome may not be a measurable increase in revenue or a conversion just yet, but it is a outcome.
00:25:29.519 --> 00:25:34.799
Because when you drive this outcome, the next step is it'll show up in revenue or conversion.
00:25:34.960 --> 00:25:41.599
So that's how we're thinking about finding these signals that otherwise you simply cannot find looking at an ad tech log.
00:25:41.839 --> 00:25:42.960
Am I answering your quiz correctly?
00:25:43.279 --> 00:25:43.920
You are, yeah.
00:25:44.079 --> 00:25:56.000
I mean, in and it's kind of a natural segue into the tension of all of this and where technology and and where people that study human behavior are coming into I don't know, some friction.
00:25:56.079 --> 00:26:08.960
But I mean, are there models that you've been able to either create or see that tell the difference between content or creative that predicts a response and content that creates real recognition?
00:26:09.200 --> 00:26:12.160
Like I'm deep into wanting to understand.
00:26:12.319 --> 00:26:14.559
I think trust has been looked at for a while.
00:26:14.960 --> 00:26:18.880
I'm in what we're building is really around this idea of familiarity.
00:26:18.960 --> 00:26:28.799
And what does familiarity mean to people and to how people consume media and how people think of their response to brands and the messaging that gets served to them?
00:26:28.960 --> 00:26:34.000
But are there models that you that you see that can help predict a response?
00:26:35.200 --> 00:26:39.119
There are there are quite a few models beyond your standard model.
00:26:39.359 --> 00:26:42.640
So almost every I say this with a lot of empathy.
00:26:42.799 --> 00:26:51.839
A lot of times the answer to these hard questions are let's just use a Bayesian MMM or a Markov chain and study attribution.
00:26:52.160 --> 00:26:53.519
Nothing against these models.
00:26:53.680 --> 00:26:56.400
I teach them in classrooms, we've used them for a while.
00:26:56.640 --> 00:27:00.640
So to answer your question, this goes back to also one of the earlier discussions.
00:27:00.720 --> 00:27:02.880
You
need to think outside the box.
00:27:03.200 --> 00:27:09.200
These models were built for a certain need at a given point in time.
00:27:09.359 --> 00:27:10.480
Those needs have evolved.
00:27:10.559 --> 00:27:22.960
So to answer your question, there are many more models that we are trying to expose our uh industry to that are less known but can do a better job of prediction.
00:27:23.519 --> 00:27:26.480
Those are ensemble models, those are different.
00:27:26.799 --> 00:27:27.359
Go ahead.
00:27:27.759 --> 00:27:29.039
No, I you keep going.
00:27:29.119 --> 00:27:32.799
I mean, that's I I you're you're making sparks go off in my brain.
00:27:32.960 --> 00:27:46.799
So no, I I uh the the reason I the reason I pause is because uh just like you were saying, look, we use allow me, I'll be very brief, allow me to give you a little bit more specifics on this thing.
00:27:46.880 --> 00:27:55.759
I I will I will not go off of the the deep end here, but we started to bring to the industry what we call USEM, uncertainty aware causal models.
00:27:55.920 --> 00:27:57.359
A lot of people haven't heard about that.
00:27:57.519 --> 00:28:01.759
The reason they haven't is because those models originated in the world of healthcare.
00:28:02.000 --> 00:28:05.680
A lot of them were used for cancer cell propagation detection.
00:28:05.920 --> 00:28:18.400
Again, in the sandbox, and now my co-founder, we've worked across industries, so we are aware of models that are aware outside of the world of marketing that marketing veterans don't have access to.
00:28:18.559 --> 00:28:40.400
So when we bring USEM models to bear, we're not trying to predict cancer cell propagation, but we are trying to say that in marketing data, because there's so much noise, there is a better way to pick up a signal that than the standard Bayesian regressions that you've been using or uh Markov chain to try to uh uh assign attribution.
00:28:40.559 --> 00:28:53.599
Similarly, when it comes to creative analysis, we brought to bear a lot of theories, quantitative theories from they're called NPT, narrative performance theory, from the world of anthropology.
00:28:53.759 --> 00:29:02.720
So my point here is we go out and look at models that are outside, that are not as you as frequently used in marketing.
00:29:02.799 --> 00:29:05.359
So we're trying to bring them here for better purposes.
00:29:05.519 --> 00:29:07.920
That's part one of the answer of your question.
00:29:08.079 --> 00:29:09.599
Yes, there are better models.
00:29:09.920 --> 00:29:21.440
Part two of the answer is I think what I'm trying to affect a little bit uh uh uh uh uh uh by raising awareness is not everything has to be predicting a response.
00:29:21.680 --> 00:29:30.079
When you use better models, it also gives you a little bit of a better power to do more informed experimentation and prediction.
00:29:30.240 --> 00:29:43.759
In other words, if you're in the in the business of predicting a point, a stock price that will
be Nvidia will be at X price on day 60 from today, it's almost a guaranteed way of failure.
00:29:43.839 --> 00:29:49.359
And so if you think of everything I can predict as a point estimate, you're gonna be disappointed.
00:29:49.519 --> 00:30:03.279
But if you think of using these prediction models to produce an outcome, whether that is a range of prediction or something you must be doing as a marketer to try out uh a new way of experimentation, that's the right way of using it.
00:30:03.440 --> 00:30:08.960
So short answer to your question: yes, there are models uh that are better than the ones that we use.
00:30:09.119 --> 00:30:18.240
Uh also the caveat I would say is how we use those models is also something that we should drive towards a little bit more maturity in our industry.
00:30:18.640 --> 00:30:19.920
I like the stock analogy.
00:30:20.000 --> 00:30:27.359
That's something that uh we've always tried to correlate and how people think about media as investing in in stocks in that capacity.
00:30:27.519 --> 00:30:31.680
It's like, you know, there's there's a lot to be learned from that correlation.
00:30:31.839 --> 00:30:58.559
But I mean, is there something one of my fears, just to put my cards on the table, is that sometimes through all of this, I think about are we engineering something that humans experience as authentic when sometimes, especially with advertising, I mean, there's advertising and content, and we can have a whole other conversation about the differences or what some people think of similarities in that.
00:30:58.640 --> 00:31:10.240
But it's like sometimes we reach people and there's just casual moments, and it's not this thing that can be even have some sort of causal effect on or model out.
00:31:10.400 --> 00:31:12.240
So I'm like, Correct.
00:31:12.640 --> 00:31:16.240
Are we engineering something that's not there that's just like intrinsic to being human?
00:31:16.480 --> 00:31:18.559
I I appreciate the question.
00:31:18.720 --> 00:31:41.279
Uh the way the the way I look at it is we shouldn't try to engineer the expect uh uh experience because there will be moments that you will have a certain set of campaigns that will reach people, like you said, they reach them in a certain they reach them at a certain point in time that there's no response, or they reach them at a point where they have already been served a lot of ads.
00:31:41.519 --> 00:31:57.680
So there's a relative desensitization built into that, or they could reach them in a point where it is an absolute opposite of the thought process they are in, and you could see a spike in recognition by saying, Oh, I just saw a message that just piqued me or challenged me in my thinking.
00:31:57.759 --> 00:32:01.119
My point here is we shouldn't try to engineer a response.
00:32:01.279 --> 00:32:12.160
One of the things that we're using analytics for is a simple way of saying is analytics should be used to try to understand, try to meet the customer on their terms.
00:32:12.400 --> 00:32:20.640
If I have somebody who has a certain need state, and that need state, I cannot predict at what point in time that need state will be.
00:32:20.799 --> 00:32:34.240
I have to have analytics that'll be say, that'll be able to say at what distribution should I be trying to reach them at a need state so that when they are in that need state, the response feels more natural.
00:32:34.400 --> 00:32:42.000
Oh, I was looking for this, I was searching for this experience, I wanted to buy a certain product for a need state that's coming up in the future.
00:32:42.160 --> 00:32:52.960
But if I try to if I try to zero in at that point in time, at that second, all we're doing is responding to a signal that may or may not be true.
00:32:53.200 --> 00:33:08.880
Just because there's an opportunity to serve an ad on an inventory that is assigned with a certain characteristic does not mean that the user, the human sitting behind that inventory, is in that exact need state and can will respond to your ad in exactly the same way that you're trying to do.
00:33:09.359 --> 00:33:20.960
I think that need state is really important framework to keep in mind because every platform is coming out with all these AI tools that are doing exactly to what you're speaking of.
00:33:21.039 --> 00:33:26.000
It's like they're taking that immediacy of a data point and then trying to give you something to react to.
00:33:26.480 --> 00:33:31.119
And it's like there's a difference between reacting and then responding to enough to be able to give you a more clear picture.
00:33:31.519 --> 00:33:51.200
So, I mean, with that, is there any decisions that you see from your work or marketers in the field that you're just like, I would never hand that to an algorithm to be able to dictate how we're responding, not reacting to something, even if the model's statistically more accurate than the people in the room.
00:33:55.519 --> 00:33:56.480
I'll just say my answer.
00:33:56.640 --> 00:33:59.920
For I'm a data guy, it might sound a little strange coming in.
00:34:00.000 --> 00:34:02.079
I'm talking about the empathy of being a human.
00:34:02.160 --> 00:34:07.680
So the short answer is by nature, and you see this across measurement people, you see this across everything.
00:34:07.759 --> 00:34:11.440
We tend to be, our minds tend to be critically oriented, right?
00:34:11.519 --> 00:34:18.480
You I'll present to you a solution, you will like that solution, but naturally there'll be some questions you're gonna want to ask of that solution.
00:34:18.639 --> 00:34:21.760
Anything that I present to you, like, what about this?
00:34:21.920 --> 00:34:28.800
What your brain is essentially doing is it is taking a response and it is critically challenging the output.
00:34:29.039 --> 00:34:32.000
That challenging part should always stay human.
00:34:32.079 --> 00:34:35.119
Whether it's a measurement solution, whether it's a campaign or what have you.
00:34:35.280
--> 00:34:37.360
In the world of measurement, we call hypotheses.
00:34:37.440 --> 00:34:39.679
In the world of measurement, we call that context setting.
00:34:39.760 --> 00:34:45.920
But the whole idea is naturally, as humans, you don't just give something to somebody and they take it and run with it.
00:34:46.079 --> 00:34:54.639
There is a natural human trait of understanding or challenging the output for either better understanding or pointing out what could be done better.
00:34:54.800 --> 00:35:10.800
That I think should stay squarely and will stay squarely human because AI models run off of prediction, run off of correlation, run off of vectors that will predict based on a certain vector what the next vector should be.
00:35:11.039 --> 00:35:21.199
They're not at a point, or I don't think uh anywhere in the near future, they'll be at a point where they will do independent critique of all of these things.
00:35:21.280 --> 00:35:22.159
They need an input.
00:35:22.239 --> 00:35:30.079
They need you to present a hypothesis, they need you to go back and say, hey, um, audience can be a confounder.
00:35:30.239 --> 00:35:36.480
Audience can affect spend as an input, and it can affect revenue or take rate as an output.
00:35:36.639 --> 00:35:43.920
When an input affects both input and when a factor affects both input and output, it will throw your models off.
00:35:44.079 --> 00:35:57.039
So until unless you as a human can point that out saying, this is a confounder, this is my critical thinking going to work, setting the right context, AI will not have the right context to keep automatically running models.
00:35:57.280 --> 00:36:09.199
So this context of challenging critical analysis, hypothesis, confounder identification, no matter what you want to call it, that I squarely see sitting in the human domain.
00:36:09.519 --> 00:36:10.719
Oh, that gives me hope.
00:36:10.960 --> 00:36:14.719
Sometimes I wake up in a cold sweat in the middle of the night being like, they're gonna take over.
00:36:14.960 --> 00:36:18.320
And then sometimes I'm, you know, have all the hope in the world.
00:36:18.480 --> 00:36:20.960
It's really a roller coaster of emotions with all of this.
00:36:21.360 --> 00:36:25.679
But I mean, on that notion, I am always trying to challenge myself, and I'd be interested to hear from you.
00:36:25.840 --> 00:36:36.400
And the idea is like, is there are there things that we're underestimating that these machines can do that it is getting right about um kind of pinpointing and understanding human behavior?
00:36:36.800 --> 00:36:43.360
See, one of the things that and I see this in data all the time, you you you you you've seen this forever.
00:36:45.760 --> 00:36:57.199
The difference between when you pull somebody and say, take a survey and say, what are they going to do, and then the reality of what actually happens, we've known of the disconnect, implicit versus explicit for a long time.
00:36:57.360 --> 00:37:05.199
I'm using
that as an example to be able to go back and say that is one of the things that will always get underestimated.
00:37:05.440 --> 00:37:21.280
If I provide a data to any model, any AI native model, ML model, it will do the best to decipher that data and anything that it finds in that data, it takes that as an instance of a fact.
00:37:22.239 --> 00:37:35.360
But my response as a human being to an ad being served to me or to a qualification of my traits as an audience member of uh X, Y, Z audience group.
00:37:35.440 --> 00:37:42.559
My point here is those qualification factors, the data points that define me, take those definitions as an absolute truth.
00:37:42.800 --> 00:37:50.000
But the behavior will always vary from it because there is a big gap between what we say and what we do.
00:37:50.239 --> 00:37:57.599
And I think if you think of that realm, there is no basis for any AI engine or any machine to know that.
00:37:57.760 --> 00:38:00.639
Whatever they see as data, they will treat it as a fact.
00:38:00.960 --> 00:38:12.800
When you do something different, you were served an ad, every prediction said, you are 78% likely to either be grouped into a click-through or a view-through based off of that ad being served to you seven times.
00:38:13.039 --> 00:38:17.119
There is an absolute level of precision attached to that, but you didn't do that.
00:38:17.440 --> 00:38:22.320
The moment the machine detects that difference, that call it calls it as a noise.
00:38:22.559 --> 00:38:24.320
The reality is you are a human.
00:38:24.480 --> 00:38:31.840
There is a non-zero chance that you will not take action because you may do different than what your data describes to you.
00:38:32.000 --> 00:38:38.079
So, long way of saying that is the difference between stated behavior and predicted behavior, that cannot be model.
00:38:38.480 --> 00:38:41.519
And a machine will always throw that out as a noise.
00:38:41.679 --> 00:38:46.239
A human being will always be keep, will always keep that inherent to their behavior.
00:38:46.400 --> 00:38:51.440
That's the one disconnect that I don't think machines are anywhere close to one.
00:38:52.079 --> 00:38:53.760
How do you balance that disconnect?
00:38:54.639 --> 00:39:03.920
Especially, I mean, as you're talking with people who aren't in the weeds like you, I mean, that is a juggernaut to kind of try to wrap your mind around if you're not used to living in this world all the time.
00:39:04.000 --> 00:39:06.079
And most CMOs aren't.
00:39:06.239 --> 00:39:08.400
They have a million other things that are on their plate.
00:39:08.480 --> 00:39:13.679
So it's like when you're showing them what these models are putting out.
00:39:14.400 --> 00:39:21.920
Are you training people like how to balance the intake of this information with what is real human behavior?
00:39:22.239 --> 00:39:22.480
Yes.
00:39:22.559 --> 00:39:31.519
So so we we we certainly don't always go if we're if we're in the room with um uh we we sit
in the room with a lot of global analytics team across the world.
00:39:31.599 --> 00:39:35.599
So if we're in an analytics discussion, uh this discussion can go really into the depth.
00:39:35.760 --> 00:39:42.239
If we're sitting with our um key partners in marketing and finance, it doesn't usually go into the depth of it.
00:39:42.320 --> 00:39:50.320
But the essence of your question is um we try to, in our measurement solution, account for uncertainty.
00:39:50.480 --> 00:39:54.880
That is why I had mentioned something called uncertainty aware causal models, USCM.
00:39:55.039 --> 00:40:03.199
Uncertainty aware models, the solutions we build assign a certain level of inherent risk to an outcome.
00:40:03.519 --> 00:40:10.480
We I don't think I or anybody here at this point in time can say we've got that cover.
00:40:10.880 --> 00:40:23.199
The difference between predicted behavior and actual behavior, if anybody says, I can totally capture that and account for that, I would call them on it.
00:40:23.360 --> 00:40:25.280
Because that is just an area.
00:40:25.440 --> 00:40:26.880
There's no specifics in it.
00:40:26.960 --> 00:40:33.840
I can say, as a block of things, a block of factors, 40% of the decision is uncertain.
00:40:34.159 --> 00:40:41.519
Like so, my point here is what we try to do, Chris, is make people aware that there is a range within which you have to operate.
00:40:41.679 --> 00:40:44.159
There is a portfolio within which you have to operate.
00:40:44.400 --> 00:41:10.559
We can define with reasonable confidence what the bounds of that, that uh those walls are, but we try not to make it into a point prediction because there is, it's it's impossible to, you as a human being, today, you're right now you're talking to me and you feel that you are interested in a subject X, two hours later, you may meet somebody else, you may talk to another human being, and you may say, I'm interested in subject Y.
00:41:10.960 --> 00:41:21.039
There was a difference between X and Y, and it would be amiss for me to go back and say, I know Chris so well through my models that I could predict that he was going to change from X to Y.
00:41:21.199 --> 00:41:21.920
That's a fallacy.
00:41:22.239 --> 00:41:33.280
But I could say, after analyzing Chris's behavior, I can say that the chance of Chris moving from X to Y or not exactly falling on point is a plus minus 5%.
00:41:33.760 --> 00:41:35.760
That I can say with a reasonable confidence.
00:41:36.000 --> 00:41:38.400
So that's how we approach uncertainty.
00:41:38.559 --> 00:41:43.280
We model a certain level of risk with every solution that we're bringing forward.
00:41:44.239 --> 00:41:47.679
I don't want to lead the question, but I think you just answered one of mine.
00:41:47.760 --> 00:41:53.440
But I'm interested in what you would tell someone to stop measuring and start questioning.
00:41:53.840 --> 00:41:56.559
I I I uh I appreciate that.
00:41:56.800 --> 00:42:02.159
I I I since you like
stock uh uh analogies, I'll give you the valuation metrics.
00:42:02.239 --> 00:42:05.360
There is no valuation equivalent of metrics in marketing.
00:42:05.599 --> 00:42:11.519
There is no PE ratio, there is no PE floor, there is no price to book, there is no price to sales.
00:42:11.760 --> 00:42:19.519
Because one of the things that finance has it right, that marketing needs to grow up to is understanding forward-looking metrics.
00:42:19.760 --> 00:42:31.679
Everything, going back to the ROAS kind of conversation, revenue calculation, everything has to be immediate, everything has to be measurable, everything has to be precise for that ROAS to work.
00:42:31.840 --> 00:42:35.360
There are very few, if at all, forward-looking valuation metrics.
00:42:35.440 --> 00:42:45.440
So there is no equivalent of am I tracking movement in plus minus movement in any level of awareness of a product or brand?
00:42:45.760 --> 00:42:47.920
I'm giving you simple examples, those could be measured.
00:42:48.000 --> 00:42:49.599
There are some complex ones too.
00:42:49.840 --> 00:42:54.559
Um, the uh level of saturation and its effect on conversion.
00:42:54.719 --> 00:42:56.559
My point here is there's no forward-looking metrics.
00:42:56.639 --> 00:42:59.440
So I would encourage CMOs or anybody to start.
00:42:59.519 --> 00:43:00.719
Don't give up what you have.
00:43:00.880 --> 00:43:05.519
You have to stand in front of board of directors, your CEO to be able to justify what you're doing.
00:43:05.599 --> 00:43:06.400
That doesn't change.
00:43:06.480 --> 00:43:08.960
So you need the measurement that exists today.
00:43:09.199 --> 00:43:19.039
But actively start thinking about forward-looking metrics that you can include in your consideration, because a lot of what we do is a pay forward model.
00:43:19.199 --> 00:43:22.559
If you are a huge brand, your incrementality is going to be low.
00:43:22.719 --> 00:43:24.639
There's no two ways of looking at it.
00:43:24.800 --> 00:43:33.599
If somebody just keeps coming to you and says, I've got an incrementality solution that can give you a 20% incrementality, my first question to those people is There are so many of those, by the way.
00:43:34.239 --> 00:43:34.480
Right.
00:43:34.719 --> 00:43:35.119
Right?
00:43:35.360 --> 00:43:38.400
So that but but my point, it that is exactly right, Chris.
00:43:38.480 --> 00:43:43.440
But that conversation, the first question in that conversation is what's the level of your brand presence?
00:43:43.519 --> 00:43:46.400
If your level of your brand presence is low, incrementality will be high.
00:43:46.480 --> 00:43:56.159
But if you're, let's say, a recognized brand, if you're an Apple or a Home Depot or any of no matter what model you run, your incrementality dimension will be low.
00:43:56.320 --> 00:43:58.000
Now think about forward looking metrics.
00:43:58.159 --> 00:44:01.920
If I said the only range of Incrementality you can affect is 5%.
00:44:02.320 --> 00:44:05.519
But over a period of time, that five can go to five and a half, six percent.
00:44:05.679 --> 00:44:07.599
I have a forward-looking metric.
00:44:07.679 --> 00:44:17.199
And it gives people the ability to have a dialogue on what should you do now that will impact your current results, but also not give away future benefits.
00:44:17.360 --> 00:44:23.360
So I I will wrap up by saying is I don't know if marketing currently has very many forward-looking metrics.
00:44:25.039 --> 00:44:33.760
A lot of what we've been kind of talking about is marketing in the sense of what most marketers are looking for.
00:44:33.840 --> 00:44:38.159
They're like, grow my business, I'm looking for that sales, that foot traffic lift, whatever.
00:44:38.320 --> 00:44:41.519
I that clicks to convert to econ.
00:44:42.400 --> 00:44:53.920
But I mean, at the core of that, something I believe deeply in and our and our team at Campfires constantly always trying to like wrap our arms around is like really at the core of this, though, is like we're really trying to connect with humans.
00:44:54.159 --> 00:44:54.400
Yes.
00:44:54.639 --> 00:45:09.920
And so it's like if the goal of marketing isn't to merely just generate a response, but create that connection and like real connection, something that mobilizes people around something more than just a product or service, but a way of thinking and doing and moving through the world.
00:45:10.559 --> 00:45:19.039
What does measurement need to become better at seeing in order to grasp that true tie?
00:45:20.800 --> 00:45:22.480
I I think you hit the nail on the head.
00:45:22.639 --> 00:45:26.159
I think measurement is a vehicle, not an end.
00:45:26.320 --> 00:45:27.920
It's a means to a delivery.
00:45:28.079 --> 00:45:37.920
Measurement needs to try to understand how to bring what you're doing closer to the consumer over a certain period of time.
00:45:38.320 --> 00:45:45.920
One of the things I have not seen very many people do is buy their brand or even within a brand, buy their product.
00:45:46.400 --> 00:45:50.400
What is the momentum of your message?
00:45:50.559 --> 00:45:54.480
And when does it start to even show up in the mind of a customer?
00:45:54.639 --> 00:45:56.559
That journey could be three weeks.
00:45:56.800 --> 00:45:59.679
That journey could be three months, it could be a year long.
00:45:59.760 --> 00:46:04.960
But if you miss to account for something as simple as that, your campaigns will become mechanical.
00:46:05.119 --> 00:46:10.559
Your campaigns will be in market, you'll run it for three weeks, four weeks, eight weeks, and say it's not performing, pull it away.
00:46:10.719 --> 00:46:18.000
What why I bring that up is that is a genuine missed uh opportunity to make the connection with the customer.
00:46:18.079 --> 00:46:26.559
Because it's not, you you just threw a threw something out that could actually make that connection because you didn't understand what was the time to make that connection with the customer.
00:46:26.719 --> 00:46:32.719
You didn't account for the fact that you, I, as busy human beings, are exposed to a few thousand ads in a given day.
00:46:32.880 --> 00:46:35.599
I might not be in the neat state when that ad was run.
00:46:35.920 --> 00:46:38.880
By the way, I'm not advocating that everybody run unlimited budget.
00:46:39.039 --> 00:46:51.840
This is also one of those things that we do in measurement to be able to say how do you maximize the stretch of your dollar so that you're not like my my answer isn't uh uh isn't uh advocating for the fact that you need to throw a lot of money.
00:46:52.000 --> 00:47:03.840
My point here is an example like this is, in my opinion, a trap that people fall into because you were genuinely trying to build a connection, but you cut it off before the tipping point happened.
00:47:04.639 --> 00:47:06.800
The converse also happens oftentimes.
00:47:07.039 --> 00:47:26.000
You sink a lot of money right off the bat, hoping for an immediate effect, but that immediate effect was never gonna be the case because you're a new product reaching out to a new audience or saturating the market with a new message, or people who you've not connected with and you're conquesting and you're trying to get an immediate result that will not happen.
00:47:26.079 --> 00:47:46.239
So, my point here is we should at least have some baselines based off of what measurement should be run so that you have an understanding of what does it take to resonate with the customer instead of putting absolute values on these metrics and then cutting off things that could actually work for you or over-indexing on things that are going to be a waste.
00:47:46.880 --> 00:47:50.960
And is it fair those baselines take time to generate a baseline?
00:47:52.159 --> 00:47:52.719
You are right.
00:47:52.880 --> 00:48:00.960
I mean, baseline generation does require bad data, but because we are living in a high compute world today, you don't always need a massive amount of data.
00:48:01.360 --> 00:48:07.440
The reason I bring high compute is let's say you're a new product and launching a new product and a new market, you will not have any data.
00:48:07.599 --> 00:48:11.440
You will not have any baseline because it's a new product and a new market, it's a new message.
00:48:11.599 --> 00:48:14.960
But here's the beauty of measurement, and this is why I get excited.
00:48:15.280 --> 00:48:19.760
We use a lot of models that are backpropagating models.
00:48:20.079 --> 00:48:21.679
It's like an SAT.
00:48:22.159 --> 00:48:29.440
You, a good score in an SAT or ACT is a function of getting multiple questions right, not one question right.
00:48:29.599 --> 00:48:44.639
So if you're in a new product in a new category in a new market, your first output, your first action, whether it's spend assigned to that campaign, whether it's audience assigned to that, whether it's creative assigned to that, your first one has a very high amount of
risk.
00:48:44.880 --> 00:48:51.599
But because we have high compute models available, I can literally take that data, feed it back to the model and say, what do I need to correct?
00:48:51.679 --> 00:49:01.119
So in this case, I don't have a baseline, but I'm self-correcting the output at each run and kind of using a way of zeroing down to a natural state.
00:49:01.199 --> 00:49:05.679
So to answer your question, baseline generation in a standard way does require data.
00:49:05.760 --> 00:49:13.840
But if you are not there, if you don't have that much data to generate a baseline, rely on the power of models now available to you.
00:49:14.079 --> 00:49:16.559
You don't need to be a data scientist.
00:49:16.719 --> 00:49:20.400
Now you have libraries available, you can ask Claude to code it for them.
00:49:20.639 --> 00:49:29.280
There are models that are available that can zero in on an effect, and now all of a sudden you have just filled the gap for a missing baseline.
00:49:29.519 --> 00:49:31.360
Now you have better data to work with.
00:49:31.679 --> 00:49:44.159
One of the traps that I've seen when it comes to measurement, and we'll kind of start start rounding out, but is that measurement seems to be, again, one of those things that's reactive to being like, we need to hit a number.
00:49:44.320 --> 00:49:48.960
And it comes much later in the in the strategic decision making process.
00:49:49.119 --> 00:49:56.639
Like it seems like it's almost like how people used to think of the paid media where it's like one of those last boxes to check.
00:49:56.719 --> 00:50:01.039
It's like, cool, we've done all this creative brand, all of these things have been checked out.
00:50:01.199 --> 00:50:03.840
Now we get to media and measurement as those last stops.
00:50:04.000 --> 00:50:04.159
Yes.
00:50:04.320 --> 00:50:05.840
Is that a fair assessment?
00:50:06.480 --> 00:50:12.559
That comes with complete bias as someone that's worked in media for close to two decades, or it's like, no, that's how I've seen it.
00:50:12.960 --> 00:50:18.320
It it is definitely a fair, it is a very fair assessment, and it is definitely something that we run into all the time.
00:50:18.559 --> 00:50:32.320
Even in the example I was providing a new product being launched and a new market or whatever, they always invariably come with uh, I expect this to be hitting a hundred dollar cost per acquisition, and I need this to be uh 18% growth in the next six months since launch.
00:50:32.480 --> 00:50:40.960
So there's already a lot of targets that are associated with this, and usually that's when measurement gets involved to be able to say, how do I hit those?
00:50:41.199 --> 00:50:42.800
It's not upfront.
00:50:42.880 --> 00:51:01.199
Uh there is a missed opportunity of including it more in the upfront process because the first part of this process should be about what your potential is, and that you can unlock through measurement and then work your way downward from there to hit the target, not the other way around.
00:51:01.440 --> 00:51:16.480
Long way of saying is and I've also seen what you're saying, that's where we get pulled in oftentimes once the decisions have been made and the targets have been set, and then measurement is brought in to bridge the gap.
00:51:16.719 --> 00:51:18.719
It could benefit by being up front.
00:51:18.880 --> 00:51:30.400
It could also give you insights on product launch, product changes from the upfront itself that might put you in a better place to deliver on those goals if brought up front.
00:51:30.639 --> 00:51:30.960
Yeah.
00:51:31.119 --> 00:51:32.159
I look at it as a thing.
00:51:32.239 --> 00:51:34.239
It doesn't need to be a thing that's bridging a gap.
00:51:34.400 --> 00:51:38.960
It can be the road that leads to the bridge that leads to the opportunity on the other side.
00:51:39.119 --> 00:51:52.559
It's like if we're talking about really trying to grow our businesses, we know we have to look outside of those traditional KPIs of business and marketing and look towards the impact on humans.
00:51:52.719 --> 00:51:58.480
Well, the only way to do that is if you do bring that measurement strategy up to the forefront of the conversation.
00:51:58.639 --> 00:51:58.719
Yes.
00:51:58.960 --> 00:52:10.079
And you can start ideating around what is actually what's our path to measurement once we hit those standard namesake KPIs that everyone's aware that they need to report to their board.
00:52:10.400 --> 00:52:12.719
I I it it's so true.
00:52:12.960 --> 00:52:32.079
And if I were to show you empirical evidence on that, the amount of and I say this with a lot of empathy, no critique, the amount of um uh wasted spend or underperforming spend that typically happens at the beginning of any cycle tends to be the highest.
00:52:32.400 --> 00:52:40.159
And one of the things that because if you think if you think of measurement as just any other business factor, right?
00:52:40.400 --> 00:52:51.360
Yes, we do we do oftentimes get into the weeds of things, but if you think of measurement as an enabler, just like any other business factor, bringing it up front also saves you.
00:52:51.440 --> 00:52:54.800
Because I I I have crunched so much data, I'm sure you have seen this too.
00:52:55.039 --> 00:53:03.440
The level of waste that happens at the beginning of a cycle without measurement already puts you in a hole because that you're then trying to claw out of.
00:53:03.599 --> 00:53:06.719
If brought up front, you could have avoided that.
00:53:06.880 --> 00:53:09.760
So yes, uh, that's a big flag.
00:53:10.079 --> 00:53:10.400
Yeah.
00:53:11.440 --> 00:53:18.639
So, from how we end every episode here is with this uh big lofty question.
00:53:19.280 --> 00:53:20.639
Would love your take on it.
00:53:20.960 --> 00:53:27.039
What is one thing about human behavior that no algorithm will ever be able to replicate?
00:53:27.440 --> 00:53:30.480
I I think I might have touched upon this.
00:53:30.639 --> 00:53:33.679
It's the ability to critique, it's the ability to challenge.
00:53:33.840 --> 00:53:59.679
I I do not think, and I'm saying this not just out of the empathy of my heart, but also because of an understanding of how AI, ML models, GPT transforms were the ability to critically challenge an output and introduce a new dimension for consideration, um, I don't think there is a replacement for that right now.
00:53:59.920 --> 00:54:04.400
Um it it is don't get me wrong, it is not an easy task.
00:54:04.559 --> 00:54:11.760
A lot of times that doesn't happen because maybe you're you're gonna find critiquing your own solution that I talked about.
00:54:11.920 --> 00:54:13.679
Maybe there's many factors in there.
00:54:13.920 --> 00:54:26.960
But if I were to answer that question in one way, the ability to critique and the ability to poke holes in a solution squarely sits human today, in my mind.
00:54:27.599 --> 00:54:38.800
I think there's a great theme, one that I'm picking up from our whole conversation, which I'm super grateful for and your time and sharing all your all your insight, is like you've touched on it numerous times around empathy.
00:54:38.880 --> 00:54:52.400
And you can tell that's like a core part of what brings you to this work, but also how we can remain that human in the loop and make sure that humans are making sure that we're responsible for each other.
00:54:52.639 --> 00:54:54.639
So I mean, you're touching on empathy.
00:54:54.800 --> 00:54:59.119
Our previous episode conversation ended with this idea of wonder.
00:54:59.280 --> 00:55:00.800
We have wonder and empathy.
00:55:00.960 --> 00:55:05.599
I can't think of like better, more human in instincts to to end on.
00:55:05.760 --> 00:55:08.880
So thank you for reminding us of empathy today.
00:55:08.960 --> 00:55:09.840
That's great.
00:55:10.159 --> 00:55:11.679
I I I appreciate that.
00:55:11.920 --> 00:55:14.239
I appreciate you sharing me about wonder.
00:55:14.320 --> 00:55:16.719
That is a really good one as well.
00:55:17.119 --> 00:55:18.559
Empathy is up there too.
00:55:18.639 --> 00:55:19.360
I appreciate it.
00:55:19.519 --> 00:55:22.480
Well, thank you so much, and uh appreciate your time.
00:55:22.880 --> 00:55:23.679
Thank you.
00:55:25.039 --> 00:55:27.519
One thing from this conversation keeps sticking with me.
00:55:27.760 --> 00:55:31.840
ROMSAD measurement should be a vehicle, not the destination.
00:55:32.159 --> 00:55:34.320
That feels particularly important right now.
00:55:34.480 --> 00:55:40.000
We have more ways than ever to predict behavior, optimize toward outcomes, and identify patterns.
00:55:40.159 --> 00:55:46.239
But there's a difference between predicting what someone might do and understanding why they do it.
00:55:46.480 --> 00:55:50.960
There's also a difference between optimizing a response and building a connection.
00:55:51.199 --> 00:55:58.400
And maybe a part of being responsible with all of this technology is being willing to leave room for the things it can't explain.
00:55:58.639 --> 00:56:07.440
The person who doesn't click when the model says they should, the campaign that needs more time, the signal buried inside what we've dismissed as noise.
00:56:07.760 --> 00:56:11.599
Rom's answer to our closing question was critique.
00:56:11.840 --> 00:56:19.280
The ability to poke holes in the answer, introduce another possibility, and ask what we might have missed.
00:56:19.760 --> 00:56:23.119
The last conversation on this show left me thinking about wonder.
00:56:23.360 --> 00:56:27.199
This one left me thinking about empathy and the ability to critique.
00:56:27.440 --> 00:56:32.000
And I'm starting to think those ideas are more connected than they first appear.
00:56:32.639 --> 00:56:36.320
The machines are going to keep getting better at giving us answers.
00:56:36.559 --> 00:56:37.840
Smart ones at that.
00:56:38.079 --> 00:56:43.360
Maybe our job is to keep getting better at asking whether they're the right ones.
00:56:47.840 --> 00:56:52.480
This has been Responsibly Human, a responsibly different podcast from Campfire Consulting.
00:56:52.719 --> 00:56:55.840
At Campfire, we believe data can tell us a lot about people.
00:56:56.000 --> 00:57:04.800
But building brands people actually connect with still takes curiosity, judgment, empathy, and a willingness to see the person behind the pattern.
00:57:05.039 --> 00:57:10.800
If this conversation gave you something worth thinking about, subscribe and share it with someone else doing the work that could benefit from it.
00:57:11.039 --> 00:57:19.360
Every listen, follow, and share helps these ideas reach more people trying to make business, media, and culture a little more human.
00:57:19.840 --> 00:57:26.719
Responsibly Human is hosted by me, Chris Maureen, with Pip Colmar and Kaylee Elwell helping shape and produce the series.
00:57:26.880 --> 00:57:32.719
And yes, our cover art was made by actual humans, illustrated by our friends at Narrative.
00:57:33.119 --> 00:57:37.840
Until next time, keep making room for what can't be measured.