WEBVTT
00:00:00.000 --> 00:00:06.644
I realized that I was a very good strategy consultant, but I had no idea how to implement my strategies.
00:00:06.644 --> 00:00:12.404
Just make reasonable assumptions, be transparent about your assumptions, and just get to a directionally correct answer.
00:00:12.974 --> 00:00:17.324
And directionally correct is, is much, much better than a blank page.
00:00:17.375 --> 00:00:22.214
So the fastest path forward is just find people who do.
00:00:22.695 --> 00:00:25.574
And the most important thing is kind of like.
00:00:25.995 --> 00:00:27.975
Just check your ego, right?
00:00:28.245 --> 00:00:31.155
And then once you're done checking your ego, check it again.
00:00:31.625 --> 00:00:38.164
You can be absolutely brilliant, but to do big things, you need a lot of good people.
00:00:38.755 --> 00:00:42.070
If you're always the smartest guy or the smartest gal in the room.
00:00:42.850 --> 00:00:44.079
You're in the wrong room.
00:00:44.750 --> 00:00:47.570
So just deliver, deliver, deliver, deliver.
00:00:47.570 --> 00:00:48.619
Because I think.
00:00:48.634 --> 00:01:01.770
If you, if you, if you get, if you gain a reputation of someone who can take on difficult challenges and just get things done influence will come, like, and, and progression will come as well.
00:01:14.638 --> 00:01:22.123
Welcome to Data Analytics Chat, the podcast where we discuss the world of data AI and the careers shaping it.
00:01:22.843 --> 00:01:30.584
My guest today is Ilya Ma, who is the SVP head of AI Solutions at Dun and Bradstreet.
00:01:31.123 --> 00:01:39.194
In this episode, we will explore his journey, the challenges and insights he has learned during his career and the data topic.
00:01:39.194 --> 00:01:42.433
We'll look at the rise on AI agents.
00:01:42.823 --> 00:01:44.864
Ilia, welcome to the podcast.
00:01:45.638 --> 00:01:46.728
Thank you very much, Ben.
00:01:46.728 --> 00:01:48.108
Thank you for having me on your show.
00:01:48.733 --> 00:01:49.573
No, it's a problem.
00:01:49.933 --> 00:01:54.013
Obviously the topic is a popular one and everyone's, yeah.
00:01:54.013 --> 00:01:55.123
Getting involved with this.
00:01:55.603 --> 00:01:58.453
Simon, do you wanna start off with Yeah.
00:01:58.483 --> 00:02:00.554
Introducing yourself and sharing your career journey?
00:02:00.554 --> 00:02:00.584
I.
00:02:01.483 --> 00:02:02.093
Yeah, sure.
00:02:02.269 --> 00:02:06.019
So like you said I'm the head of AI Solutions at Dun and Bradstreet.
00:02:06.713 --> 00:02:13.913
Obviously we are doing a lot with AI and agents and all things LLM.
00:02:14.574 --> 00:02:20.033
My career path has been anything but linear.
00:02:20.063 --> 00:02:23.723
So I started out in management consulting, then moved into strategy.
00:02:24.193 --> 00:02:27.074
Worked with private equity shops a lot, large corporates.
00:02:27.764 --> 00:02:35.193
Kind of, like never specialized in any industry vertical and did everything under the sun from automotive to.
00:02:35.824 --> 00:02:40.229
Aviation, metals, retail, consumer products which is it was great for me.
00:02:40.234 --> 00:02:41.804
It suits my personality.
00:02:41.804 --> 00:02:45.283
I really like learning and actually I get bored pretty quickly.
00:02:45.813 --> 00:02:53.044
So then I, at some point I realized that I was a good strategy professional, but I didn't know anything about what it.
00:02:53.359 --> 00:02:55.878
Takes to actually implement strategies.
00:02:56.299 --> 00:03:03.019
So I started looking to transition more into implementation roles, so to speak.
00:03:03.829 --> 00:03:05.479
Then came to Dun Bradstreet.
00:03:05.973 --> 00:03:29.383
About 12 years ago, started in strategic business planning and then did a lot of really interesting things across the company from technology finance, and eventually found my home and data science and ai, which again, just like it's a perfect field for someone who really likes variety and gets bored pretty quickly because ai, it's not really a discipline, it's just a collection of.
00:03:29.938 --> 00:03:35.149
Tools and solutions that can be applied to an infinite range of problems really.
00:03:35.209 --> 00:03:42.359
So I keep learning every day and, we're incredibly busy with all the cool AI and agent stuff, so it's been really great.
00:03:42.959 --> 00:03:43.079
I,
00:03:43.364 --> 00:03:58.419
Yeah, I think it's definitely one you gotta keep on your toes and keep learning this at the moment, with yourself, you, me, you mentioned you are one that's sort open to working in many different industries.
00:03:59.079 --> 00:04:10.868
Do you think that's just more down to your,'cause obviously a lot of people, businesses want people that have got the domain expertise and I guess there's always that element of can someone learn a new industry?
00:04:11.288 --> 00:04:15.419
Was that just something you picked up as a sort of in your sort of consulting days?
00:04:16.019 --> 00:04:18.298
Yeah, I think it was just a combination of things.
00:04:18.329 --> 00:04:18.928
Um.
00:04:19.528 --> 00:04:28.139
The group that, the consulting group that I was working for was relatively small, so everybody kind of expected to be able to do everything.
00:04:28.619 --> 00:04:34.738
And then once I got into the mode of doing that, I discovered that I really enjoy it.
00:04:35.129 --> 00:04:42.778
'cause you know, like it can be very stressful'cause you're constantly learning new things and you're basically, you're never doing anything twice.
00:04:43.259 --> 00:04:47.369
So there is a little bit of stress and the learning curve is always very steep.
00:04:47.908 --> 00:04:50.189
But it's just like, uh, it's, it's a lot of fun.
00:04:50.218 --> 00:04:56.699
'cause like I said, you keep learning and you're developing a very broad perspective on things.
00:04:56.699 --> 00:05:11.038
And then you can start, at some point you realize that you can draw some higher level inferences about, you know, like all these seemingly disparate industries, but you kind of like, you start seeing patterns.
00:05:11.509 --> 00:05:12.738
And that really helps.
00:05:12.809 --> 00:05:15.298
You know, and I've kind of, you know, once I discovered that.
00:05:15.718 --> 00:05:25.468
I really enjoy that and I'm okay, like, putting up with a little bit of stress that comes with, uh, you know, like I have to get up to speed on this thing in, in two, three weeks.
00:05:26.069 --> 00:05:33.028
I, I realized that I really, really like it and I've always tried to, to sort of like look for roles that allow me to do that.
00:05:33.629 --> 00:05:34.079
Okay.
00:05:34.079 --> 00:05:34.499
Brilliant.
00:05:35.038 --> 00:05:41.449
And then so if you had to pinpoint one or two defining moments that's changed your career, what would they be?
00:05:42.048 --> 00:05:45.949
So, you know, the pivot from, from strategy to implementation.
00:05:45.949 --> 00:05:54.249
So, you know, I mentioned, I, I realized that I was a very good strategy consultant, but I had no idea how to implement my strategies.
00:05:54.249 --> 00:05:57.819
And I actually, you know, I got that feedback from a very senior.
00:05:58.494 --> 00:06:11.934
Person in the firm I was working for and you know, in the moment, you know, I was taken aback, like walk out of the meeting and you mumble all kinds of words under your breath as like, what does, you know, of course I know how to implement.
00:06:12.384 --> 00:06:17.064
But then, you know, like I took a hard look at myself.
00:06:17.064 --> 00:06:21.713
I was like, well, yeah, like I've always been a, a consultant, so what do I know?
00:06:21.713 --> 00:06:24.144
And that's, that, that was really the point when.
00:06:24.774 --> 00:06:34.413
I started looking for roles outside of consulting and really like looking for roles that were not even on the strategy side, that were much more on implementation side.
00:06:34.963 --> 00:06:45.944
So that was a very, and, and, you know, you know, with 12 or or so years later or 13 years later I'm really happy that I did this.
00:06:45.944 --> 00:06:46.483
Um.
00:06:47.473 --> 00:06:48.014
Pivot.
00:06:48.163 --> 00:06:56.723
Like obviously, you know, like the, the, what I discovered is the world is infinitely messier and complicated, more complicated.
00:06:56.723 --> 00:07:02.363
And you know, like things that look great on a piece of paper, they just don't work, right.
00:07:02.363 --> 00:07:10.163
So it's, it's kind of like, you know, I, before I switched to industry as consultants say, I didn't truly realize that.
00:07:10.764 --> 00:07:13.134
I didn't see the, I didn't have the full picture of the world.
00:07:13.213 --> 00:07:15.043
So that was one very important moment.
00:07:15.384 --> 00:07:23.434
Another one I would say like learning to to be comfortable with ambiguity and uncertainty.
00:07:23.853 --> 00:07:29.374
And there's one very clear moment that like I always go back to in my mind.
00:07:29.754 --> 00:07:34.403
Back in my consulting days, I was on, on a project where I had to build.
00:07:35.103 --> 00:07:37.894
What consultants call a supply curve.
00:07:37.923 --> 00:07:46.024
Basically, it's a, it's an analysis and, uh, a framework of which producers in a specific industry can profit.
00:07:46.579 --> 00:07:51.348
At different price levels and who's gonna be essentially like, priced out of production.
00:07:51.649 --> 00:07:53.149
And I had to do that for ethanol.
00:07:53.519 --> 00:07:56.459
Which is, you know, like ethanol is a really complex industry.
00:07:56.459 --> 00:07:59.788
Like you can make ethanol from pretty much anything, right?
00:07:59.788 --> 00:08:04.019
Like you could, you could make ethanol from like stale beer.
00:08:04.379 --> 00:08:10.619
And then, you know, like the, the ethanol prices are linked to multiple commodities, oil and corn and sugar, and.
00:08:11.038 --> 00:08:14.249
There's regulations and subsidies all over the place.
00:08:14.399 --> 00:08:27.959
Capacity was surging so you know, it, it was an impossible thing to seemingly to analyze and I complained to my mentor, um, mark Chenko, who is now a senior partner at KPMG.
00:08:28.379 --> 00:08:32.458
That it was an impossible task, and basically like nobody on Earth knew the answer, right?
00:08:32.908 --> 00:08:35.609
And Mark's response was, that's right.
00:08:35.609 --> 00:08:37.558
That's exactly why the client is paying us.
00:08:38.068 --> 00:08:43.894
And then he just he kind of explained to me that everybody knows that you don't have a magic wand.
00:08:44.703 --> 00:08:47.673
Like nobody expects you to give a perfect answer.
00:08:48.604 --> 00:08:54.364
Just make reasonable assumptions, be transparent about your assumptions, and just get to a directionally correct answer.
00:08:54.933 --> 00:08:59.283
And directionally correct is, is much, much better than a blank page.
00:08:59.384 --> 00:09:07.943
And so that, that advice that I got from him just completely reframed how I think about really difficult problems.
00:09:07.943 --> 00:09:12.964
And, it probably has helped me more in my career than just about anything else, right?
00:09:13.384 --> 00:09:19.653
So I kind of like, I've always had to operate in environments where, like I said, it's the first time I'm doing something.
00:09:19.653 --> 00:09:21.333
Things are very much in flux.
00:09:21.948 --> 00:09:24.678
There's never enough information to make decisions.
00:09:25.308 --> 00:09:29.658
And actually most of the time I don't even know if I have enough information to make a decision, right?
00:09:29.989 --> 00:09:43.769
So that ability to just like separate things into what's knowable, what's not knowable, understand the risks of not knowing stuff, it's just finding a way to move forward and, and starting to do things.
00:09:44.269 --> 00:09:48.158
Like without that masterclass from Mark, like honestly I don't think.
00:09:49.048 --> 00:09:51.749
I would've been nearly as successful in my career.
00:09:52.349 --> 00:09:52.828
Amazing.
00:09:52.828 --> 00:09:57.239
It's, I think, yeah, like you said, it's something getting right, starting something.
00:09:57.239 --> 00:09:58.708
It gets your brain going, doesn't it?
00:09:58.708 --> 00:10:01.344
The creativity sparks and I just.
00:10:01.943 --> 00:10:08.933
Things just start, you think your brain starts getting creative and then new ideas pop up and it just, things happen, doesn't it?
00:10:09.068 --> 00:10:09.698
Exactly.
00:10:09.698 --> 00:10:09.879
Yeah.
00:10:09.879 --> 00:10:12.399
And then like you get feedback from your environments.
00:10:12.399 --> 00:10:15.399
Like you try things, some things work, some things don't work.
00:10:15.399 --> 00:10:18.058
Like the most important thing is just to get going.
00:10:18.058 --> 00:10:18.448
So
00:10:19.048 --> 00:10:19.198
And
00:10:19.274 --> 00:10:21.418
avoid the, like another, yeah.
00:10:21.989 --> 00:10:25.889
Another favorite consulting term, like just avoid analysis paralysis.
00:10:25.889 --> 00:10:26.158
Right.
00:10:26.759 --> 00:10:27.058
Yeah.
00:10:27.119 --> 00:10:30.509
And that I said that sometimes starting you get so worked up, don't you?
00:10:30.509 --> 00:10:31.109
And then you end
00:10:31.349 --> 00:10:31.979
Yes.
00:10:31.979 --> 00:10:32.129
yeah.
00:10:32.129 --> 00:10:33.719
You just need to get something written out.
00:10:34.479 --> 00:10:34.839
Brilliant.
00:10:35.139 --> 00:10:40.448
So what's been the tough challenges you faced along the way, and I guess how have you overcome them?
00:10:40.798 --> 00:10:43.499
So I mean, you know, it's, it's like I mentioned, right?
00:10:43.499 --> 00:10:48.208
Like, I make it sound like a lot of fun, like constantly learning and doing new things.
00:10:48.208 --> 00:10:49.558
But there is that element of.
00:10:50.158 --> 00:10:56.038
Of stress and sort of like constant discomfort because you're always doing things that you've never done before.
00:10:56.099 --> 00:11:02.249
And you know, like sometimes you have weeks or even days to just learn something completely new.
00:11:02.278 --> 00:11:09.813
Like back in consulting was a new industry or like in ai, like the, the pace of change is so quick that Yeah.
00:11:10.264 --> 00:11:14.673
Something that was state of the art a few months ago is outdated.
00:11:15.043 --> 00:11:15.464
Today.
00:11:15.783 --> 00:11:24.124
And, uh, you know, I think, I, I think the way I've always gone about it is just like, figure out who the real experts are, right?
00:11:24.333 --> 00:11:28.293
So when you're tackling something new, you don't even know what you don't know.
00:11:28.774 --> 00:11:33.614
So the fastest path forward is just find people who do.
00:11:34.094 --> 00:11:36.974
And the most important thing is kind of like.
00:11:37.394 --> 00:11:39.374
Just check your ego, right?
00:11:39.644 --> 00:11:42.553
And then once you're done checking your ego, check it again.
00:11:42.974 --> 00:11:51.274
So I literally call people every day and say, forgive me, but I'm about to ask you a lot of really dumb questions.
00:11:51.274 --> 00:11:52.504
Please bear with me.
00:11:52.984 --> 00:11:58.114
Um, and that, that, that mindset has served me incredibly well.
00:11:58.114 --> 00:11:58.384
Right?
00:11:58.384 --> 00:12:01.714
So you have three, four conversations with the right people.
00:12:02.359 --> 00:12:06.859
And you sort of like start seeing the contours of the problem space, right?
00:12:07.219 --> 00:12:13.313
So suddenly you understand what there is to know what you need to know and what you still don't know.
00:12:13.553 --> 00:12:15.264
And that's a huge win, right?
00:12:15.264 --> 00:12:17.514
When you can finally map out the landscape.
00:12:18.004 --> 00:12:18.964
That's a huge win.
00:12:18.964 --> 00:12:19.509
And you can.
00:12:20.149 --> 00:12:24.168
Go deeper, ask better questions, start connecting the dots, et cetera.
00:12:24.168 --> 00:12:24.469
Right?
00:12:24.889 --> 00:12:43.958
So I think, you know, like that's ultimately, this is how I've, I've handled, um, all these big challenges, just like staying curious, being patient and surrounding myself with people who know much more than I do and just being honest with myself and with the environment.
00:12:43.958 --> 00:12:46.089
It's like, this is something that I don't know anything about.
00:12:46.538 --> 00:12:48.068
Help me get up to speed.
00:12:48.469 --> 00:12:49.019
Okay good.
00:12:49.229 --> 00:12:52.889
And then, so obviously you mentioned learning's been quite key for you,
00:12:53.683 --> 00:12:53.974
Yeah.
00:12:54.188 --> 00:13:03.484
Obviously you've been successful to get to leadership and there's many people out there that wanna do it, but maybe uncertain or feel they've lack of skills.
00:13:03.484 --> 00:13:07.573
How have you, how has your career progressed into leadership?
00:13:07.573 --> 00:13:11.624
Was it, has there been any particular, have you had mentors, training?
00:13:11.714 --> 00:13:12.884
Has it been self-study?
00:13:12.884 --> 00:13:13.724
How have you gone about it?
00:13:14.053 --> 00:13:28.823
Yeah I've always had an incredible fortune of working for very smart, very successful people who I learned I've learned from and tremendous amount.
00:13:29.649 --> 00:13:37.178
And but also I've always had the great fortune of being surrounded by incredible team members.
00:13:37.589 --> 00:13:51.193
'Cause at the end of the day, I, I don't, like I, if someone junior is, or, like aspiring leaders are listening to this the most important thing to understand that it's all about people, right?
00:13:51.193 --> 00:13:55.124
Sometimes we get so wrapped up and we're solving this analytic problem, or.
00:13:55.734 --> 00:13:57.264
This is the thing that we're building.
00:13:57.683 --> 00:13:59.063
It's all about people.
00:13:59.094 --> 00:14:11.943
And as you progress, your success singularly depends on who you work with and your team members and the collaboration that you build with them and, and the relationships.
00:14:12.423 --> 00:14:18.303
So it's, it's, you know, more than anything, it's like it, people are people, right?
00:14:18.303 --> 00:14:27.244
Like people give their best when they feel understood and they feel appreciated and see how their work is creating genuine impact.
00:14:27.663 --> 00:14:38.573
So I think it's, it's always it's critical to remember that, like you can be absolutely brilliant, but to do big things, you need a lot of good people.
00:14:39.094 --> 00:14:43.533
To support you and, and drive that thing alongside with you.
00:14:43.994 --> 00:14:45.524
And I guess, you know, so that's one thing.
00:14:45.524 --> 00:14:53.594
And another thing would be what has worked really well for me is the ability to play in multiple worlds, right?
00:14:53.953 --> 00:15:00.254
My, my degree actually is in philosophy, but I have enough physics and math background to.
00:15:00.943 --> 00:15:04.423
To, uh, hold my own with very quantitative PhDs.
00:15:05.053 --> 00:15:07.063
And then I have my strategy background.
00:15:07.234 --> 00:15:13.443
You know, like, so at my core, I am quote unquote a business person, right?
00:15:13.744 --> 00:15:24.504
But obviously I can operate, I'm very comfortable operating in the, you know, AI data science world and leading these incredibly capable.
00:15:24.969 --> 00:15:27.009
More technical resources.
00:15:27.428 --> 00:15:38.818
So that ability kind of like to be on the intersection of the business and the AI has been incredibly helpful in my career.
00:15:38.818 --> 00:15:49.078
Just like, like blending technical insights, analytical insights, strategic insights, and just like going in and out of both those worlds, right?
00:15:49.438 --> 00:15:58.528
And they're like, there's so many, like AI is creating so many really interesting opportunities for, for people to do that, to play in both worlds.
00:15:58.528 --> 00:16:17.249
Like one example that like I kind of like talking about, so without getting too technical, that you know, you can take the way large language models, one of the things that they do, you can take a lot of unstructured text and you can turn that.
00:16:17.594 --> 00:16:21.524
Unstructured text into what they call like vector embeddings.
00:16:21.524 --> 00:16:21.764
Right?
00:16:21.764 --> 00:16:27.134
Which is basically like numeric representation of that text.
00:16:27.134 --> 00:16:29.114
A lot of ones and zeros, right?
00:16:29.563 --> 00:16:31.994
So there's this interesting question in my head.
00:16:31.994 --> 00:16:44.803
So when somebody does that, when you take, let's say a document and then you create a vector embedded representation of that document, is that the same data but in a different format?
00:16:45.359 --> 00:16:49.168
Or is that a completely new artifact?
00:16:49.528 --> 00:16:59.849
And the reason it's, it's important is because like, you know, like when you're talking about the legal aspect of ai, like contractual aspect of ai, so that's a really important question.
00:16:59.908 --> 00:17:00.239
Right?
00:17:00.649 --> 00:17:10.068
When, let's say your customer takes your data and creates some beddings out of it, is that a derivative output or is it basically the same data just in a different format?
00:17:10.519 --> 00:17:13.368
So I think Eli, so that's an example like lawyers.
00:17:13.743 --> 00:17:18.364
Who will be able to engage with these issues that didn't exist.
00:17:18.364 --> 00:17:20.344
Like even, you know, like two, three years ago.
00:17:21.064 --> 00:17:26.213
They will be, they'll have a lot of work and like, I think it's gonna be really interesting work.
00:17:26.463 --> 00:17:30.183
I think, like I said, like AI is creating a lot of these opportunities to be.
00:17:31.054 --> 00:17:33.713
Play in in, in multiple domains.
00:17:33.713 --> 00:17:38.213
And I think it's, you know, I think it's fun and, uh, there's not that many people who can do that.
00:17:38.213 --> 00:17:44.773
So, you know, that that's a pretty good way to, to get going with or accelerate your career rather.
00:17:45.689 --> 00:17:49.663
Yeah, I definitely think now the, the skill set is.
00:17:50.263 --> 00:17:51.253
Changing, isn't it?
00:17:51.294 --> 00:17:52.673
You need the blender skills now.
00:17:52.673 --> 00:17:55.078
I think you need obviously the technical skills.
00:17:55.679 --> 00:18:01.989
If you can talk, obviously there's so many different roles as well but you need, technical skills are getting less technical now'cause you've got the heavy lifting being done.
00:18:01.989 --> 00:18:07.269
So you need to now have that business knowledge strategy.
00:18:07.778 --> 00:18:10.868
It's some, so it's just evolving, isn't it?
00:18:10.868 --> 00:18:17.528
Now I think even if you're a data scientist now you and you're pure techie, I think you need to start learning them business skills.
00:18:17.558 --> 00:18:22.804
'cause you're gonna need to be able to communicate with the business and add value or you skill you.
00:18:23.628 --> 00:18:25.023
Roll's just gonna get void.
00:18:25.624 --> 00:18:26.523
Yeah, exactly.
00:18:26.554 --> 00:18:27.003
Exactly.
00:18:27.003 --> 00:18:41.314
And like, you know, the AI is blending so much, you know, all this age agentic all these age agentic capabilities, and, and in a few years, if you're a business leader who, who doesn't really understand these tools or these technologies.
00:18:41.983 --> 00:18:52.663
I think you're really, um, locking yourself out of a huge part of, like the problems that you can credibly try to solve.
00:18:53.003 --> 00:18:54.354
Yeah, I completely agree with you.
00:18:54.953 --> 00:18:57.689
And you men, oh, you mentioned you've got, you've got a blend of good skills.
00:18:57.689 --> 00:19:01.294
So is there, is there anything else that sort of helped you progress into leadership?
00:19:01.644 --> 00:19:05.419
I mean, like I said, like I, you know, I, I've been very, very lucky.
00:19:05.419 --> 00:19:18.963
I've, I've always worked for and with incredibly good people and the way I sort of like think about it if you're always the smartest guy or the smartest gal in the room.
00:19:19.743 --> 00:19:20.973
You're in the wrong room.
00:19:21.693 --> 00:19:32.644
You know, I, I, I think every person on teams that I've been, that I've had the privilege to lead is much better than me at many things.
00:19:33.153 --> 00:19:34.054
And that's great.
00:19:34.054 --> 00:19:39.243
That's how it should be if you ask me, like, so I don't, you know, I don't try to.
00:19:40.219 --> 00:19:43.249
To kind of like pretend that this is not the case.
00:19:43.519 --> 00:19:44.808
I lean into it.
00:19:44.858 --> 00:19:51.419
I want people to be better than me and learn thing, I mean, know things that I don't know and I can learn from them.
00:19:52.239 --> 00:19:55.088
I think that's been really critical for me.
00:19:55.388 --> 00:20:00.769
Then so the, the, the another thing is I'm pretty, uh.
00:20:01.068 --> 00:20:03.979
Pretty crazy about protecting my focus, right?
00:20:04.219 --> 00:20:09.858
There's just like so much stuff going on and, and the pace is just unrelenting, right?
00:20:09.858 --> 00:20:20.538
So even if days were 72 hour long, not 24 hours, if I engaged with everything that I can engage, I wouldn't get anything done.
00:20:21.138 --> 00:20:23.989
And that's one of the skills that I learned in consulting.
00:20:24.019 --> 00:20:25.114
'cause sometimes we would do.
00:20:25.868 --> 00:20:32.949
Projects where we had to help a private equity shop make a decision about buying a company and not buying a company.
00:20:32.949 --> 00:20:41.148
And we had like three weeks to to, learn about a target company and completely new space.
00:20:41.479 --> 00:20:50.723
So if that's, if that's the timelines that you're operating with, then you learn very quickly to separate things into those that are critical to understand in this context.
00:20:51.378 --> 00:20:54.469
And things that are nice to have to understand, right?
00:20:54.888 --> 00:21:00.638
So it's sort of like I've been applying the same mindset also to problems, right?
00:21:00.638 --> 00:21:06.038
Like, so I'm trying to do whatever I'm trying to do what is critical, what is not critical.
00:21:06.038 --> 00:21:12.479
And it's just like, I don't I don't, I don't have the, the luxury of engaging things that are not critical.
00:21:12.479 --> 00:21:14.068
And that's actually really helpful, right?
00:21:14.068 --> 00:21:17.848
So you end up, you end up doing more and you end up doing.
00:21:18.223 --> 00:21:22.183
Bigger things that drive bigger value for your company or your organization?
00:21:22.784 --> 00:21:38.713
Yeah, I think, I think the, you know, we talked about the ability to play in, in multiple worlds, and I guess, you know, like the last, but very much, not least, maybe it is the most important thing is build credibility before, before you seek influence, right?
00:21:39.044 --> 00:21:41.864
So just deliver, deliver, deliver, deliver.
00:21:41.864 --> 00:21:42.913
Because I think.
00:21:42.929 --> 00:21:56.064
If you, if you, if you get, if you gain a reputation of someone who can take on difficult challenges and just get things done influence will come, like, and, and progression will come as well.
00:21:56.364 --> 00:21:58.844
So that's probably the main thing.
00:21:59.443 --> 00:22:01.784
Yeah, I like comment about the focus.
00:22:01.814 --> 00:22:04.153
'cause I think that is key, isn't it?
00:22:04.153 --> 00:22:10.683
I mean, you need to, there's so much going on and you need to, what's gonna move the needle basically isn't what's gonna, what's the big thing.
00:22:11.013 --> 00:22:12.153
And again, that's the same with business.
00:22:12.153 --> 00:22:15.213
I think everyone's got bit loss, regenerative ai, haven't they?
00:22:15.213 --> 00:22:20.463
Again, tr there's so many projects you wanna do, you need to, what's gonna impact the business right now?
00:22:20.773 --> 00:22:23.114
'Cause there'll be so much you wanna do in the business now.
00:22:23.114 --> 00:22:24.673
There's so much opportunity now, but.
00:22:25.273 --> 00:22:27.433
What's the current challenge, isn't it really?
00:22:27.433 --> 00:22:30.884
You need to focus on what's gonna be the big impact for you right now.
00:22:31.723 --> 00:22:32.263
That's right.
00:22:32.324 --> 00:22:32.834
That's right.
00:22:33.433 --> 00:22:33.794
Okay.
00:22:34.173 --> 00:22:40.203
So if say, I wanted, I was an aspiring leader, what advice would you give myself?
00:22:40.354 --> 00:22:40.443
Give.
00:22:41.044 --> 00:22:42.604
Advice to give yourself?
00:22:43.144 --> 00:22:45.993
Um, that's a great question.
00:22:46.594 --> 00:22:58.144
I think, you know, ev everything, so you, you get, you're in a really interesting position because you get to talk to people from many different walks of life and different roles.
00:22:58.743 --> 00:23:00.844
And many different industries.
00:23:01.114 --> 00:23:05.733
So you kind of, you know, like to do successfully what you do on this podcast.
00:23:06.304 --> 00:23:13.443
So you must be curious and you must be willing to engage with things that you maybe don't know much about.
00:23:13.443 --> 00:23:15.624
And, sounds like you're doing really, really well with that.
00:23:16.223 --> 00:23:16.703
I hope so.
00:23:17.304 --> 00:23:17.963
That's fine.
00:23:18.304 --> 00:23:22.673
Okay, so let's tap into the question then on AI agents.
00:23:23.023 --> 00:23:38.943
Obviously this is, yeah, massive next evolution, obviously brings many opportunities and I guess there's obviously challenges and risks in the business environment, so they're obviously pushed as the next layer on top of LLMs.
00:23:39.574 --> 00:23:46.864
In your view, what capabilities do they unlock that base models cannot achieve on their own right now?
00:23:47.638 --> 00:23:48.959
This is an awesome question.
00:23:48.959 --> 00:23:52.019
This is really like truly foundational to understand.
00:23:52.229 --> 00:23:53.489
So there's a couple of things, right?
00:23:53.489 --> 00:24:01.769
So first of all, LLMs understand and agents do, right?
00:24:01.769 --> 00:24:12.028
So there, there's there's a huge progression from LLMs that are essentially like passive interlocutors or assistance.
00:24:12.588 --> 00:24:22.199
Where whereas agents are active problem solvers, you know, like a digital coworker that can reason, plan and execute things.
00:24:22.528 --> 00:24:29.398
So just to get like a tad more technical agents add state.
00:24:30.324 --> 00:24:33.894
Planning memory and tool usage, right?
00:24:33.894 --> 00:24:36.773
So like I planning memory and tool usage.
00:24:36.773 --> 00:24:38.304
I think that's pretty self-explanatory.
00:24:38.304 --> 00:24:40.044
Let me talk a little bit about state.
00:24:40.493 --> 00:24:49.943
So LLMs are, are awesome at understanding and, and generating language, but they're not inherently stateful, right?
00:24:50.243 --> 00:24:58.403
They don't retain goals, they don't retain memory or, or continuity beyond.
00:24:59.334 --> 00:25:01.104
The current context, right?
00:25:01.433 --> 00:25:07.794
And agents add that missing layer of planning and, and memory and action.
00:25:08.064 --> 00:25:10.223
They can decide what to do next.
00:25:10.223 --> 00:25:12.114
They can call external tools.
00:25:12.403 --> 00:25:16.114
They can coordinate steps over time.
00:25:16.473 --> 00:25:17.163
So.
00:25:17.763 --> 00:25:23.778
And, and, and all those things is just again, like, just to bring it back like LLMs talk.
00:25:23.808 --> 00:25:32.479
LLMs can converse with you, but agents can actually do things and like pretty, pretty complex things at that.
00:25:32.578 --> 00:25:38.969
So, and speaking of complexity, so the other thing is agents, I apologize, lost my voice.
00:25:39.568 --> 00:25:52.138
And so sorry agents can decompose large, large, complex problems into smaller tasks and orchestrate how those tasks are solved.
00:25:52.739 --> 00:25:57.659
Something that even smartest models, smartest LMS really struggle with, right?
00:25:57.898 --> 00:26:02.493
So you can, like, if you throw, if you ask even the smartest LLM.
00:26:03.094 --> 00:26:13.263
In the world to interact with petabytes of diverse data and ask it to, to derive conclusions from that data.
00:26:14.213 --> 00:26:15.294
It's too much, right?
00:26:15.294 --> 00:26:18.983
Like it's just too much data and there's too much variety and there's too much complexity.
00:26:18.983 --> 00:26:22.733
So LLMs in these situations, hallucinate like crazy.
00:26:23.064 --> 00:26:25.074
They make, they just make things up.
00:26:25.074 --> 00:26:28.463
Like their answers sound very confident, but factually.
00:26:29.439 --> 00:26:30.729
There are a lot of inaccuracies.
00:26:31.028 --> 00:26:34.538
So agents on the other hand work very differently.
00:26:34.538 --> 00:26:37.719
They kind of, they embrace tool use.
00:26:37.749 --> 00:26:45.249
They can call APIs, they can run database queries, they can delegate to specialists, sub-agents.
00:26:45.608 --> 00:26:50.979
And the beauty is, so those tools or, or subagents or, or APIs.
00:26:51.189 --> 00:26:56.169
You, they can be specialists like in just one thing, right?
00:26:56.588 --> 00:27:07.479
So you can have an agent tool that is only good at retrieving, like in, in our world, for example, data about ESG, right?
00:27:07.719 --> 00:27:12.909
And then you'll have ESG data, and then you'll have another tool that is exceptionally good at.
00:27:13.433 --> 00:27:19.324
Retrieving data about I don't know, like news about a company, right?
00:27:19.773 --> 00:27:23.763
And all those specialist tools, they'll be extremely good.
00:27:24.153 --> 00:27:29.074
They will use deterministic mechanisms of finding that information.
00:27:29.344 --> 00:27:39.013
Meaning, you know, like they, they will convert fluid conversational language from the user into whatever, like SQL calls or, or whatever, right?
00:27:39.723 --> 00:27:51.784
And by breaking the problem down into a bunch of smaller pieces for each piece, you have a specialist that is really, really good at doing that particular thing.
00:27:52.203 --> 00:28:02.223
So suddenly you can handle like enormous amounts of data and very complex, multifaceted user queries.
00:28:02.463 --> 00:28:03.364
And you can do that.
00:28:03.963 --> 00:28:05.493
Very successfully.
00:28:05.614 --> 00:28:05.943
Right.
00:28:06.223 --> 00:28:07.784
And, and that's just huge, right?
00:28:07.784 --> 00:28:15.253
Like you it's, even today, even like the lo the, the biggest, smartest lms, they hallucinate quite a bit.
00:28:15.614 --> 00:28:19.574
So it's like I said, like you can't just unleash them directly in the business context.
00:28:19.574 --> 00:28:21.864
Like, like you can't afford.
00:28:22.463 --> 00:28:25.854
You know, your tolerance for hallucination is extremely small.
00:28:25.854 --> 00:28:29.003
Like you can't even afford, like 5% of hallucinations.
00:28:29.374 --> 00:28:40.558
So that in those environments, like you really need to bring in agents that are exceptionally good at breaking those enormous problems down and just solving them one step at a time.
00:28:41.159 --> 00:28:45.314
And so what would be like a telltale sign of a high value use case for agents then?
00:28:45.663 --> 00:28:47.403
So I think there's a couple of things, right?
00:28:47.403 --> 00:28:54.463
So obviously, you know, everything, everything that you do, like it, it needs to have, clear value, right?
00:28:54.463 --> 00:28:59.364
Like, like if you're deploying an agent, you need to understand why you're doing it.
00:28:59.364 --> 00:29:02.449
What's the value that, that you want to drive, be it.
00:29:03.263 --> 00:29:13.173
Time savings or, or, or new revenue or efficiencies, or you can just, in some cases you can do something they were not able to do previously.
00:29:13.173 --> 00:29:16.534
So the first thing is like, what's the actual tangible impact, right?
00:29:17.433 --> 00:29:18.874
So that's sort of a given, right?
00:29:19.263 --> 00:29:24.243
That specifically for agents, I think repeatability is a big thing.
00:29:24.624 --> 00:29:28.824
If there is a workflow or a task that happens.
00:29:29.528 --> 00:29:35.409
Often enough, and it's sort of like more or less the same thing that is being done repeatedly over time.
00:29:35.798 --> 00:29:41.858
That's a really good signal that this is something where an agent can be quite impactful.
00:29:42.398 --> 00:29:51.699
Also complexity is actually like, I would say that complexity is, and by complexity I mean, a task may require.
00:29:52.104 --> 00:29:54.683
Reasoning and context, right?
00:29:54.683 --> 00:30:09.403
So you're not just like, not a single deterministic answer because like we said a little bit earlier, like, agents can actually break down problems, figure out what path to choose, which tools to use and which sequence.
00:30:09.763 --> 00:30:13.604
So if there's a little bit of complexity that's, that's also really good.
00:30:14.203 --> 00:30:18.763
And I guess, you know, like there's, there's also a dose of realism.
00:30:19.598 --> 00:30:24.068
Agents are very, very powerful tools, but it's not like it's a magic wand.
00:30:24.068 --> 00:30:38.544
So the sort of like the way to think about it is if you drop the brilliant person into the same task and ask them to do it, would they say, yep, I've got everything that I need to do this task.
00:30:38.933 --> 00:30:43.943
Or would they say, you didn't give me the data or the right tools to execute this task.
00:30:44.044 --> 00:30:54.453
If it's the latter, then agents are not going to be successful to execute again, like super, super powerful, but they can't create something out of nothing.
00:30:54.693 --> 00:31:01.374
It means that if that's the answer, then it means that you have to do some pre-work before deploying agents into the space.
00:31:01.673 --> 00:31:02.919
And of course, you know, in, in.
00:31:03.519 --> 00:31:13.038
In enterprise context, of course you have to be very mindful about regulatory compliance and sort of like government context.
00:31:13.608 --> 00:31:26.709
There are some use cases that inherently riskier than other use cases, and you just need to be, you need to understand very well the risk profiles of different use cases and judge accordingly.
00:31:26.709 --> 00:31:27.818
If this is something that.
00:31:28.419 --> 00:31:35.949
Um, you're very comfortable deploying agents into, or something that maybe you want to wait for a few months to get more clarity.
00:31:36.818 --> 00:31:50.769
So I would say just see, just to kind of like recap, impact repeatability, um, some, you know, like some complexity, some, like just realistic realism, right?
00:31:50.769 --> 00:31:55.429
And just remembering to think of regulatory and compliance stuff.
00:31:56.028 --> 00:32:01.878
So does that mean, oh, so this is a new, new thing for businesses now?
00:32:01.878 --> 00:32:04.858
So is this mean you need to get, when you're getting involved with this?
00:32:04.878 --> 00:32:10.159
This has gotta be bringing in multiple different departments within the business to cover all bases.
00:32:10.429 --> 00:32:12.709
So it's like not more team effort now.
00:32:13.683 --> 00:32:14.854
yeah, exactly right.
00:32:14.854 --> 00:32:19.709
So I I always say that Gen AI or agents, it's a team sport it like it truly is right.
00:32:20.509 --> 00:32:29.398
If you have a bunch of super smart people doing something in the basement I don't think it's gonna be I don't think it's going to scale.
00:32:29.398 --> 00:32:30.838
I don't think it's going to succeed.
00:32:30.919 --> 00:32:35.898
So you really need to have, you know, like different people call it differently.
00:32:35.989 --> 00:32:42.943
Like I've heard a name, gen AI pod like you, you have to have your product people, you have to have your technology people.
00:32:43.703 --> 00:32:50.624
You need to have legal, cybersecurity, um, you know, like all, all the, the folks who commercialize things.
00:32:50.624 --> 00:32:56.923
So all, all the sort of like, how do we price it, how do we bill it, how do we manage, uh, usage?
00:32:57.284 --> 00:33:01.993
So you gotta have, and I'm probably forgetting a bunch of important functions, right?
00:33:02.324 --> 00:33:05.384
So if you don't have all that.
00:33:05.693 --> 00:33:13.374
As a cohesive group or cohesive team, it's gonna be pretty hard to to, to deploy at scale.
00:33:13.973 --> 00:33:14.423
Okay.
00:33:14.844 --> 00:33:19.074
And then obviously many pilots fail to scale.
00:33:19.503 --> 00:33:26.433
So what are the hidden blockers that organizations overlook when deploying agents?
00:33:27.034 --> 00:33:28.459
So, you know, it's interesting.
00:33:28.808 --> 00:33:31.778
Complexity doesn't scale linearly.
00:33:32.439 --> 00:33:34.659
It kind of scales exponentially.
00:33:35.259 --> 00:33:38.318
I'm so sorry, I'm still having my voice issues.
00:33:38.769 --> 00:33:52.358
So, if you prototype an agent that works on a small, clean sliver of data for a pretty narrow use case, you can't assume that it will behave the same way.
00:33:52.959 --> 00:33:57.729
When it's exposed to the full scope of, of your data or workflows, right?
00:33:58.328 --> 00:34:18.789
So you, you, you may, something that works beautifully at small scale, you may discover that when you scaled up, you're running into memory limits context, window constraints, retrieval, quality orchestration load, and it's like all the things that don't show up in, in your demos.
00:34:19.614 --> 00:34:21.744
But they will show up in production.
00:34:22.103 --> 00:34:22.914
So that's one.
00:34:23.514 --> 00:34:26.844
Second, you gotta have agent ready data, right?
00:34:26.844 --> 00:34:32.003
Like agents can only reason as well as the data that they can see and interact with.
00:34:32.423 --> 00:34:36.534
So you need to make sure that you have the right data for the agents to work with.
00:34:37.114 --> 00:34:40.503
Meaning your data has to be trusted, timely, contextualized.
00:34:40.853 --> 00:34:54.184
If it's sort of like locked in silos or poorly mapped where you can just tr you can't trust your data, um, that that's, it's gonna be pretty hard to, to do anything at scale.
00:34:54.963 --> 00:35:01.143
And then there's this, this interesting concept of like signal to noise ratio, right?
00:35:01.623 --> 00:35:04.233
So let's say you do have agent ready data.
00:35:04.653 --> 00:35:07.893
So what's really important is to not overwhelm the agent.
00:35:08.688 --> 00:35:14.449
With data that it doesn't really need to solve the problems that you want it to solve.
00:35:14.958 --> 00:35:23.478
So it's sort of like, you know, if I go back to my, you know, like if you drop the really smart person into the problem, would they be able to do it?
00:35:24.079 --> 00:35:33.699
You know how like they say that most people can remember and repeat back only about seven to 10 digits if you tried to make people memorize.
00:35:34.298 --> 00:35:38.588
20 digits or 30 digits, almost everyone would fail, right?
00:35:38.588 --> 00:35:43.958
And there comes a point, you know, like you get to 50 digits, like 99.9% of people would fail.
00:35:44.438 --> 00:35:46.208
And it's the same thing with agents, right?
00:35:46.208 --> 00:35:53.378
Like the more data you throw at them, the higher the probability that they will get confused and will start hallucinating.
00:35:53.914 --> 00:35:58.418
So and so, the challenge is not to stuff more data in the context window.
00:35:58.778 --> 00:36:00.188
Like you really gotta know.
00:36:00.954 --> 00:36:07.164
What is the right data and the most impactful data that you want to bring into the problem.
00:36:07.583 --> 00:36:10.824
So this is not, you know, like the more, the better.
00:36:11.034 --> 00:36:17.778
Like you need to have, like this is the most impactful data that I can bring into this problem.
00:36:17.858 --> 00:36:23.648
So, and I think it's interesting that this concept of context engineering, it's really starting to evolve.
00:36:24.318 --> 00:36:28.068
It's, so, it's definitely an area to watch as you're scaling.
00:36:28.338 --> 00:36:32.568
And of course, you know, like I, it goes without saying, you gotta have the right infrastructure.
00:36:32.918 --> 00:36:43.778
Because I, you know, like some pilots often run on sort of like highly bes, bespoke setups, which don't translate into production, right?
00:36:43.778 --> 00:36:48.278
Because, you know, like, just because something runs.
00:36:48.878 --> 00:36:50.063
I, you know what?
00:36:50.094 --> 00:36:59.454
Well enough for, for your pilot, you may very quickly discover that that setup that you have to run the pilot is way too expensive to run in production, right?
00:36:59.844 --> 00:37:02.664
And like I said, you know, like some things just don't scale.
00:37:03.253 --> 00:37:09.284
So the question is not when you do your, your pilot, the question is not like, Hey, can I get it to work?
00:37:09.583 --> 00:37:15.974
The question really should be, can I get it to work 10,000 times a day reliably?
00:37:16.693 --> 00:37:19.034
At enterprise level.
00:37:19.123 --> 00:37:20.833
So I think these are the big things.
00:37:21.434 --> 00:37:27.018
I definitely hearing a lot around data quality is getting so key now, isn't it?
00:37:27.349 --> 00:37:27.588
Yeah.
00:37:27.923 --> 00:37:28.164
Absolutely.
00:37:28.548 --> 00:37:29.268
Yeah, exactly.
00:37:29.268 --> 00:37:39.759
It's sometimes what's really interesting, LLMs can do amazing things and then you throw agents into the mix and it's I look at what my team creates and I'm like.
00:37:40.344 --> 00:37:42.353
Wow, this is science fiction, right?
00:37:42.724 --> 00:37:46.773
I'm in it 24 7, but I still have those moments.
00:37:46.773 --> 00:37:49.623
Like I look at the stuff that they're building, I'm like, wow.
00:37:50.074 --> 00:37:52.954
Like I, I literally can't believe what I'm seeing.
00:37:53.463 --> 00:38:00.684
But so sometimes like we get really anchored on that, that we forget, like agents are still software, right?
00:38:01.023 --> 00:38:04.123
Like the old, garbage in, garbage out.
00:38:04.393 --> 00:38:09.884
As smart and capable as agents are like, you put bad data in, you're gonna get that stuff out.
00:38:10.454 --> 00:38:17.864
So yeah, having the right data, trusted, timely, contextualized well, accurate obviously.
00:38:18.304 --> 00:38:22.293
That is, you're going to fail if you don't have all that.
00:38:22.594 --> 00:38:32.884
A lot of people have said especially with lms, like building LM is easy, the easy part, easier part, the back, like the getting data is the hardest bit.
00:38:32.884 --> 00:38:36.873
Is that the same with building agents?
00:38:37.224 --> 00:38:39.764
So it, it's an interesting question, right?
00:38:40.608 --> 00:38:44.478
Um, building good agents is.
00:38:45.079 --> 00:38:46.639
Pretty hard, right?
00:38:46.639 --> 00:38:52.619
It's, it takes a lot of very specialized skills ability to innovate very, very quickly.
00:38:52.949 --> 00:38:54.688
So I think, I would say it's both right?
00:38:54.688 --> 00:39:03.398
Like you do need excellent teams and excellent infrastructure and like, just excellent sort of like, innovation process.
00:39:04.253 --> 00:39:13.193
But yeah it's, you can't do, even if you, you're exceptionally good at building stuff, you're not gonna be successful if you don't have great data.
00:39:13.193 --> 00:39:14.518
So it's, you really need both.
00:39:15.119 --> 00:39:15.989
Okay, perfect.
00:39:16.719 --> 00:39:22.059
And how do you separate orchestration complexity from genuine intelligence?
00:39:22.659 --> 00:39:26.559
See that, that's, that's an interesting, that's a really interesting question.
00:39:26.559 --> 00:39:27.548
Like intelligence is.
00:39:28.148 --> 00:39:30.278
Intelligence is a very loaded term.
00:39:30.818 --> 00:39:33.903
And, and I have, I have a degree in philosophy.
00:39:33.903 --> 00:39:42.403
Um, during my time at Yale, actually, I took a seminar with Nick Bostrom, who is one of the leading thinkers in, philosophy of ai.
00:39:42.943 --> 00:39:49.373
So I can debate intelligence and what that means and whether we have it or not until the cows come home.
00:39:49.974 --> 00:40:00.804
I think honestly, like I, I will punt a little bit on the intelligence thing, and I instead, I will say that in practice, the real metric is not intelligence, but value, right?
00:40:01.313 --> 00:40:12.653
So to me, the real question is, are agents creating value beyond, you know, like your traditional workflows or, you know, what, what automation could deliver?
00:40:12.653 --> 00:40:16.583
And to me the answer is an unequivocal yes.
00:40:16.684 --> 00:40:18.333
I think, you know, I, I think.
00:40:19.159 --> 00:40:31.048
So it's interesting, like sometimes when people say agents, they think of like early day agents that didn't really have agency or autonomy.
00:40:31.048 --> 00:40:41.318
They were really just like very sophisticated workflows wrapped around an LLM like executing static plans with, with style and flair, right?
00:40:41.378 --> 00:40:43.688
And that's sort of like that, that linear.
00:40:44.469 --> 00:40:48.398
Chains of things that agents were asked to ex execute.
00:40:48.608 --> 00:40:51.659
That's where, like the term chain of thought came from.
00:40:51.898 --> 00:40:56.159
There's just linearly, linear chains of, of execution.
00:40:56.489 --> 00:41:10.068
But now the agents that, that we have today with memory, like really good memory and planning and tool usage, they truly can, can act autonomously and decide how to approach a problem.
00:41:10.668 --> 00:41:12.528
Instead of following a fixed script.
00:41:12.528 --> 00:41:12.768
Right.
00:41:12.768 --> 00:41:15.139
So, and I mentioned agents are stateful.
00:41:15.768 --> 00:41:19.588
Um, they don't start from scratch with every request.
00:41:19.588 --> 00:41:23.789
Like they remember prior steps and they can track goals.
00:41:23.789 --> 00:41:31.349
They can track their execution against the goals, and they can adjust their next move based on.
00:41:32.009 --> 00:41:33.298
On outcomes, right?
00:41:33.568 --> 00:41:38.219
So they don't have to move linearly through predefined paths anymore, right?
00:41:38.548 --> 00:41:45.748
They can navigate dynamically through almost like a graph of actions that they can take.
00:41:46.228 --> 00:41:49.798
You know, like and then they can choose the most relevant path in real time.
00:41:50.099 --> 00:41:55.858
They have the autonomy to loop back, skip steps, like change course when new information comes in.
00:41:56.713 --> 00:42:01.333
It's, it's really like pretty similar to how humans solve problems, right?
00:42:01.704 --> 00:42:13.744
So I think, you know, like the agents that we have today absolutely add tremendous value on top of what those early day agents we're bringing to the table.
00:42:13.983 --> 00:42:15.514
And it's important to remember, like.
00:42:16.278 --> 00:42:22.458
It feels like there's so much has happened and the cha the pace of innovation so quick that it feels like these things have been around forever.
00:42:22.849 --> 00:42:27.389
We're still very early days of these, of this agentic revolution.
00:42:27.929 --> 00:42:32.128
We'll get to a place where these systems can learn and they can adapt.
00:42:32.719 --> 00:42:35.539
They can continuously improve outcomes.
00:42:35.969 --> 00:42:40.798
You know, I, I, I know that I totally punted on, on, uh, intelligence, but.
00:42:41.099 --> 00:42:41.418
Yeah.
00:42:41.429 --> 00:42:45.869
To me the value is unquestionably there.
00:42:45.869 --> 00:42:48.268
I see it every day many times.
00:42:48.699 --> 00:42:51.268
And the beauty is we barely scratch the surface.
00:42:52.188 --> 00:42:52.478
Yeah.
00:42:52.478 --> 00:42:53.173
Yeah, true.
00:42:53.173 --> 00:42:55.393
Like it's still early days, isn't it?
00:42:55.393 --> 00:42:55.844
Like it's.
00:42:56.248 --> 00:42:56.699
Yeah
00:42:56.963 --> 00:43:05.034
again, this is where I guess obvious you've got the technical knowledge, but also if you've got the business domain expertise here, like how can you actually, again, add value to the business?
00:43:05.034 --> 00:43:10.164
It's gonna, you're gonna add some small power to what you are trying to achieve.
00:43:11.043 --> 00:43:11.889
Yeah, exactly.
00:43:12.728 --> 00:43:13.268
Exactly.
00:43:13.869 --> 00:43:17.813
So you see like a big risk in over reliance agents for businesses.
00:43:18.414 --> 00:43:29.983
Um, I think you know it's a little bit of what I said earlier, unbelievably powerful tools, frameworks, um, when well designed, they can do amazing things.
00:43:30.583 --> 00:43:40.574
But agents are software, albeit software with autonomy to make very complex decisions and based on complex inputs, all that stuff, right?
00:43:41.114 --> 00:43:55.304
But you know, like it's really important not to get complacent and just like stay on top of how those environments evolve and how inputs shift over time.
00:43:55.414 --> 00:43:56.074
Um.
00:43:56.673 --> 00:44:03.514
So I would say, you know, like today, agents should augment human judgment.
00:44:04.023 --> 00:44:10.983
But you shouldn't just say, all right, like, we have an agent running, like we can go fishing for the rest of the month, right?
00:44:10.983 --> 00:44:15.378
Like, maybe we'll happen very, very quickly as models get better and these frameworks get better.
00:44:15.483 --> 00:44:19.923
Like maybe we'll be able to, to rely on these things more.
00:44:19.983 --> 00:44:22.173
But again, like unbelievably powerful tools.
00:44:22.173 --> 00:44:24.219
But it's important to remember that today they're tools.
00:44:24.568 --> 00:44:36.599
So yeah, I, I think it's really important to stay on top of, like to monitor what goes in monitor, what goes out track performance really, really closely.
00:44:36.599 --> 00:44:41.188
And there's some, you know, like really powerful tools that allow you to do that today, right?
00:44:41.188 --> 00:44:45.498
Like that, that can see all the like prompts, for example.
00:44:46.114 --> 00:44:58.143
Into an agent system and see all the outputs and do some like, extremely intelligent things like analyzing analyzing outputs from the quality perspective and like even governance, right?
00:44:58.143 --> 00:45:09.204
Like so I think, it, the, the risk is forgetting that we still need people very closely involved with these powerful tools.
00:45:09.804 --> 00:45:10.023
It.
00:45:10.623 --> 00:45:11.628
Yeah, definitely.
00:45:12.228 --> 00:45:12.619
Cool.
00:45:12.679 --> 00:45:24.469
So then if we look, say two to three years ahead, do you see agents becoming embedded into the enterprise architecture or is it gonna be more of a like a niche tactical tool for businesses?
00:45:24.498 --> 00:45:24.528
I.
00:45:25.128 --> 00:45:30.289
I think, um, I think it's definitely in the former, I think agents will become very embedded.
00:45:30.873 --> 00:45:35.974
And you know, like in a way they'll simply be how enterprise systems work, right?
00:45:36.514 --> 00:45:50.773
So you'll see agent capabilities built directly into CRMs, ERPs master data management platforms, often behind the scenes, sort of like similar to how APIs operates today.
00:45:51.224 --> 00:45:55.184
And I think, you know, the key enabler of that, of that embeddedness.
00:45:55.829 --> 00:45:57.599
It's going to be standardization, right?
00:45:57.599 --> 00:46:05.338
So if you take things like model context, protocol, MCP they will make a lot of data.
00:46:05.338 --> 00:46:08.938
Like MCP will make a lot of really, really good data sets.
00:46:09.358 --> 00:46:11.728
Enterprise, sorry.
00:46:11.728 --> 00:46:14.398
It will make really good data sets like agent ready.
00:46:14.728 --> 00:46:23.458
Um, they will make real good data sets more accessible, uh, well described, easier for agents to consume and understand.
00:46:23.938 --> 00:46:32.318
Um, like, so MCP will give agents a common language to, to use for communications across systems, right?
00:46:32.528 --> 00:46:36.128
So that's gonna be a very, very important driver of that embeddedness.
00:46:36.159 --> 00:46:41.798
The other one is going to be a to a, or, you know, like the protocol for agent to agent collaboration.
00:46:42.559 --> 00:46:59.378
I think, you know, like we're, I, maybe it hasn't happened at scale yet, but I think very quickly we're gonna get to a point where agents communicate with other agents across different enterprises, across different systems.
00:46:59.949 --> 00:47:06.219
So you kind of, you know, that, that will allow enterprises to move from having isolated.
00:47:06.818 --> 00:47:16.688
AI tools or agents to really this, these like connected collaborative ecosystems that reason and act together.
00:47:17.048 --> 00:47:24.268
So I I, I, I definitely think that the, the some of this embeddedness is happening, is happening already, right?
00:47:24.268 --> 00:47:34.849
So for example, like DDMV has pretty incredible MCP, servers that we already exposed to customers, we have a, to a capabilities.
00:47:34.849 --> 00:47:42.599
So we're starting to see that embeddedness already, and I think it's, it's only going to get much more pronounced very quickly.
00:47:42.838 --> 00:48:15.228
So I, I think like the vision is sort of like a lot of these interconnected agents powering workflows, powering important decisions, powering analytics behind the scenes, and sort of like seamlessly integrated through protocols like MCP and coordination layers, like eight to a again, I think this is still pretty early stages of that transformation, but I think we're starting to see the curve starting to bend up pretty quickly.
00:48:15.829 --> 00:48:16.489
Brilliant.
00:48:16.699 --> 00:48:16.969
Cool.
00:48:17.248 --> 00:48:17.579
Here we go.
00:48:17.579 --> 00:48:22.778
It's been a pleasure having you on shared some great insights and yeah, so obviously a hot topic.
00:48:23.148 --> 00:48:24.199
So thanks for joining us.
00:48:25.034 --> 00:48:25.963
Thank you very much.
00:48:25.963 --> 00:48:28.094
Thank you again for having me on your show.
00:48:28.094 --> 00:48:29.983
Like really interesting questions.
00:48:30.034 --> 00:48:38.364
There's a lot going on in the space and things are evolving very quickly, so it's definitely an area to watch and to learn about and to be excited about.
00:48:38.963 --> 00:48:39.264
Yep.
00:48:39.623 --> 00:48:39.804
I agree.
00:48:39.954 --> 00:48:43.523
And to our listeners, if you enjoyed today's episode Yep.
00:48:43.583 --> 00:48:47.934
If you could give us a follow, leave, a quick review, just really helps others discover the show.
00:48:48.233 --> 00:48:48.474
Thank you.
00:00:00.000 --> 00:00:06.644
I realized that I was a very good strategy consultant, but I had no idea how to implement my strategies.
00:00:06.644 --> 00:00:12.404
Just make reasonable assumptions, be transparent about your assumptions, and just get to a directionally correct answer.
00:00:12.974 --> 00:00:17.324
And directionally correct is, is much, much better than a blank page.
00:00:17.375 --> 00:00:22.214
So the fastest path forward is just find people who do.
00:00:22.695 --> 00:00:25.574
And the most important thing is kind of like.
00:00:25.995 --> 00:00:27.975
Just check your ego, right?
00:00:28.245 --> 00:00:31.155
And then once you're done checking your ego, check it again.
00:00:31.625 --> 00:00:38.164
You can be absolutely brilliant, but to do big things, you need a lot of good people.
00:00:38.755 --> 00:00:42.070
If you're always the smartest guy or the smartest gal in the room.
00:00:42.850 --> 00:00:44.079
You're in the wrong room.
00:00:44.750 --> 00:00:47.570
So just deliver, deliver, deliver, deliver.
00:00:47.570 --> 00:00:48.619
Because I think.
00:00:48.634 --> 00:01:01.770
If you, if you, if you get, if you gain a reputation of someone who can take on difficult challenges and just get things done influence will come, like, and, and progression will come as well.
00:01:14.638 --> 00:01:22.123
Welcome to Data Analytics Chat, the podcast where we discuss the world of data AI and the careers shaping it.
00:01:22.843 --> 00:01:30.584
My guest today is Ilya Ma, who is the SVP head of AI Solutions at Dun and Bradstreet.
00:01:31.123 --> 00:01:39.194
In this episode, we will explore his journey, the challenges and insights he has learned during his career and the data topic.
00:01:39.194 --> 00:01:42.433
We'll look at the rise on AI agents.
00:01:42.823 --> 00:01:44.864
Ilia, welcome to the podcast.
00:01:45.638 --> 00:01:46.728
Thank you very much, Ben.
00:01:46.728 --> 00:01:48.108
Thank you for having me on your show.
00:01:48.733 --> 00:01:49.573
No, it's a problem.
00:01:49.933 --> 00:01:54.013
Obviously the topic is a popular one and everyone's, yeah.
00:01:54.013 --> 00:01:55.123
Getting involved with this.
00:01:55.603 --> 00:01:58.453
Simon, do you wanna start off with Yeah.
00:01:58.483 --> 00:02:00.554
Introducing yourself and sharing your career journey?
00:02:00.554 --> 00:02:00.584
I.
00:02:01.483 --> 00:02:02.093
Yeah, sure.
00:02:02.269 --> 00:02:06.019
So like you said I'm the head of AI Solutions at Dun and Bradstreet.
00:02:06.713 --> 00:02:13.913
Obviously we are doing a lot with AI and agents and all things LLM.
00:02:14.574 --> 00:02:20.033
My career path has been anything but linear.
00:02:20.063 --> 00:02:23.723
So I started out in management consulting, then moved into strategy.
00:02:24.193 --> 00:02:27.074
Worked with private equity shops a lot, large corporates.
00:02:27.764 --> 00:02:35.193
Kind of, like never specialized in any industry vertical and did everything under the sun from automotive to.
00:02:35.824 --> 00:02:40.229
Aviation, metals, retail, consumer products which is it was great for me.
00:02:40.234 --> 00:02:41.804
It suits my personality.
00:02:41.804 --> 00:02:45.283
I really like learning and actually I get bored pretty quickly.
00:02:45.813 --> 00:02:53.044
So then I, at some point I realized that I was a good strategy professional, but I didn't know anything about what it.
00:02:53.359 --> 00:02:55.878
Takes to actually implement strategies.
00:02:56.299 --> 00:03:03.019
So I started looking to transition more into implementation roles, so to speak.
00:03:03.829 --> 00:03:05.479
Then came to Dun Bradstreet.
00:03:05.973 --> 00:03:29.383
About 12 years ago, started in strategic business planning and then did a lot of really interesting things across the company from technology finance, and eventually found my home and data science and ai, which again, just like it's a perfect field for someone who really likes variety and gets bored pretty quickly because ai, it's not really a discipline, it's just a collection of.
00:03:29.938 --> 00:03:35.149
Tools and solutions that can be applied to an infinite range of problems really.
00:03:35.209 --> 00:03:42.359
So I keep learning every day and, we're incredibly busy with all the cool AI and agent stuff, so it's been really great.
00:03:42.959 --> 00:03:43.079
I,
00:03:43.364 --> 00:03:58.419
Yeah, I think it's definitely one you gotta keep on your toes and keep learning this at the moment, with yourself, you, me, you mentioned you are one that's sort open to working in many different industries.
00:03:59.079 --> 00:04:10.868
Do you think that's just more down to your,'cause obviously a lot of people, businesses want people that have got the domain expertise and I guess there's always that element of can someone learn a new industry?
00:04:11.288 --> 00:04:15.419
Was that just something you picked up as a sort of in your sort of consulting days?
00:04:16.019 --> 00:04:18.298
Yeah, I think it was just a combination of things.
00:04:18.329 --> 00:04:18.928
Um.
00:04:19.528 --> 00:04:28.139
The group that, the consulting group that I was working for was relatively small, so everybody kind of expected to be able to do everything.
00:04:28.619 --> 00:04:34.738
And then once I got into the mode of doing that, I discovered that I really enjoy it.
00:04:35.129 --> 00:04:42.778
'cause you know, like it can be very stressful'cause you're constantly learning new things and you're basically, you're never doing anything twice.
00:04:43.259 --> 00:04:47.369
So there is a little bit of stress and the learning curve is always very steep.
00:04:47.908 --> 00:04:50.189
But it's just like, uh, it's, it's a lot of fun.
00:04:50.218 --> 00:04:56.699
'cause like I said, you keep learning and you're developing a very broad perspective on things.
00:04:56.699 --> 00:05:11.038
And then you can start, at some point you realize that you can draw some higher level inferences about, you know, like all these seemingly disparate industries, but you kind of like, you start seeing patterns.
00:05:11.509 --> 00:05:12.738
And that really helps.
00:05:12.809 --> 00:05:15.298
You know, and I've kind of, you know, once I discovered that.
00:05:15.718 --> 00:05:25.468
I really enjoy that and I'm okay, like, putting up with a little bit of stress that comes with, uh, you know, like I have to get up to speed on this thing in, in two, three weeks.
00:05:26.069 --> 00:05:33.028
I, I realized that I really, really like it and I've always tried to, to sort of like look for roles that allow me to do that.
00:05:33.629 --> 00:05:34.079
Okay.
00:05:34.079 --> 00:05:34.499
Brilliant.
00:05:35.038 --> 00:05:41.449
And then so if you had to pinpoint one or two defining moments that's changed your career, what would they be?
00:05:42.048 --> 00:05:45.949
So, you know, the pivot from, from strategy to implementation.
00:05:45.949 --> 00:05:54.249
So, you know, I mentioned, I, I realized that I was a very good strategy consultant, but I had no idea how to implement my strategies.
00:05:54.249 --> 00:05:57.819
And I actually, you know, I got that feedback from a very senior.
00:05:58.494 --> 00:06:11.934
Person in the firm I was working for and you know, in the moment, you know, I was taken aback, like walk out of the meeting and you mumble all kinds of words under your breath as like, what does, you know, of course I know how to implement.
00:06:12.384 --> 00:06:17.064
But then, you know, like I took a hard look at myself.
00:06:17.064 --> 00:06:21.713
I was like, well, yeah, like I've always been a, a consultant, so what do I know?
00:06:21.713 --> 00:06:24.144
And that's, that, that was really the point when.
00:06:24.774 --> 00:06:34.413
I started looking for roles outside of consulting and really like looking for roles that were not even on the strategy side, that were much more on implementation side.
00:06:34.963 --> 00:06:45.944
So that was a very, and, and, you know, you know, with 12 or or so years later or 13 years later I'm really happy that I did this.
00:06:45.944 --> 00:06:46.483
Um.
00:06:47.473 --> 00:06:48.014
Pivot.
00:06:48.163 --> 00:06:56.723
Like obviously, you know, like the, the, what I discovered is the world is infinitely messier and complicated, more complicated.
00:06:56.723 --> 00:07:02.363
And you know, like things that look great on a piece of paper, they just don't work, right.
00:07:02.363 --> 00:07:10.163
So it's, it's kind of like, you know, I, before I switched to industry as consultants say, I didn't truly realize that.
00:07:10.764 --> 00:07:13.134
I didn't see the, I didn't have the full picture of the world.
00:07:13.213 --> 00:07:15.043
So that was one very important moment.
00:07:15.384 --> 00:07:23.434
Another one I would say like learning to to be comfortable with ambiguity and uncertainty.
00:07:23.853 --> 00:07:29.374
And there's one very clear moment that like I always go back to in my mind.
00:07:29.754 --> 00:07:34.403
Back in my consulting days, I was on, on a project where I had to build.
00:07:35.103 --> 00:07:37.894
What consultants call a supply curve.
00:07:37.923 --> 00:07:46.024
Basically, it's a, it's an analysis and, uh, a framework of which producers in a specific industry can profit.
00:07:46.579 --> 00:07:51.348
At different price levels and who's gonna be essentially like, priced out of production.
00:07:51.649 --> 00:07:53.149
And I had to do that for ethanol.
00:07:53.519 --> 00:07:56.459
Which is, you know, like ethanol is a really complex industry.
00:07:56.459 --> 00:07:59.788
Like you can make ethanol from pretty much anything, right?
00:07:59.788 --> 00:08:04.019
Like you could, you could make ethanol from like stale beer.
00:08:04.379 --> 00:08:10.619
And then, you know, like the, the ethanol prices are linked to multiple commodities, oil and corn and sugar, and.
00:08:11.038 --> 00:08:14.249
There's regulations and subsidies all over the place.
00:08:14.399 --> 00:08:27.959
Capacity was surging so you know, it, it was an impossible thing to seemingly to analyze and I complained to my mentor, um, mark Chenko, who is now a senior partner at KPMG.
00:08:28.379 --> 00:08:32.458
That it was an impossible task, and basically like nobody on Earth knew the answer, right?
00:08:32.908 --> 00:08:35.609
And Mark's response was, that's right.
00:08:35.609 --> 00:08:37.558
That's exactly why the client is paying us.
00:08:38.068 --> 00:08:43.894
And then he just he kind of explained to me that everybody knows that you don't have a magic wand.
00:08:44.703 --> 00:08:47.673
Like nobody expects you to give a perfect answer.
00:08:48.604 --> 00:08:54.364
Just make reasonable assumptions, be transparent about your assumptions, and just get to a directionally correct answer.
00:08:54.933 --> 00:08:59.283
And directionally correct is, is much, much better than a blank page.
00:08:59.384 --> 00:09:07.943
And so that, that advice that I got from him just completely reframed how I think about really difficult problems.
00:09:07.943 --> 00:09:12.964
And, it probably has helped me more in my career than just about anything else, right?
00:09:13.384 --> 00:09:19.653
So I kind of like, I've always had to operate in environments where, like I said, it's the first time I'm doing something.
00:09:19.653 --> 00:09:21.333
Things are very much in flux.
00:09:21.948 --> 00:09:24.678
There's never enough information to make decisions.
00:09:25.308 --> 00:09:29.658
And actually most of the time I don't even know if I have enough information to make a decision, right?
00:09:29.989 --> 00:09:43.769
So that ability to just like separate things into what's knowable, what's not knowable, understand the risks of not knowing stuff, it's just finding a way to move forward and, and starting to do things.
00:09:44.269 --> 00:09:48.158
Like without that masterclass from Mark, like honestly I don't think.
00:09:49.048 --> 00:09:51.749
I would've been nearly as successful in my career.
00:09:52.349 --> 00:09:52.828
Amazing.
00:09:52.828 --> 00:09:57.239
It's, I think, yeah, like you said, it's something getting right, starting something.
00:09:57.239 --> 00:09:58.708
It gets your brain going, doesn't it?
00:09:58.708 --> 00:10:01.344
The creativity sparks and I just.
00:10:01.943 --> 00:10:08.933
Things just start, you think your brain starts getting creative and then new ideas pop up and it just, things happen, doesn't it?
00:10:09.068 --> 00:10:09.698
Exactly.
00:10:09.698 --> 00:10:09.879
Yeah.
00:10:09.879 --> 00:10:12.399
And then like you get feedback from your environments.
00:10:12.399 --> 00:10:15.399
Like you try things, some things work, some things don't work.
00:10:15.399 --> 00:10:18.058
Like the most important thing is just to get going.
00:10:18.058 --> 00:10:18.448
So
00:10:19.048 --> 00:10:19.198
And
00:10:19.274 --> 00:10:21.418
avoid the, like another, yeah.
00:10:21.989 --> 00:10:25.889
Another favorite consulting term, like just avoid analysis paralysis.
00:10:25.889 --> 00:10:26.158
Right.
00:10:26.759 --> 00:10:27.058
Yeah.
00:10:27.119 --> 00:10:30.509
And that I said that sometimes starting you get so worked up, don't you?
00:10:30.509 --> 00:10:31.109
And then you end
00:10:31.349 --> 00:10:31.979
Yes.
00:10:31.979 --> 00:10:32.129
yeah.
00:10:32.129 --> 00:10:33.719
You just need to get something written out.
00:10:34.479 --> 00:10:34.839
Brilliant.
00:10:35.139 --> 00:10:40.448
So what's been the tough challenges you faced along the way, and I guess how have you overcome them?
00:10:40.798 --> 00:10:43.499
So I mean, you know, it's, it's like I mentioned, right?
00:10:43.499 --> 00:10:48.208
Like, I make it sound like a lot of fun, like constantly learning and doing new things.
00:10:48.208 --> 00:10:49.558
But there is that element of.
00:10:50.158 --> 00:10:56.038
Of stress and sort of like constant discomfort because you're always doing things that you've never done before.
00:10:56.099 --> 00:11:02.249
And you know, like sometimes you have weeks or even days to just learn something completely new.
00:11:02.278 --> 00:11:09.813
Like back in consulting was a new industry or like in ai, like the, the pace of change is so quick that Yeah.
00:11:10.264 --> 00:11:14.673
Something that was state of the art a few months ago is outdated.
00:11:15.043 --> 00:11:15.464
Today.
00:11:15.783 --> 00:11:24.124
And, uh, you know, I think, I, I think the way I've always gone about it is just like, figure out who the real experts are, right?
00:11:24.333 --> 00:11:28.293
So when you're tackling something new, you don't even know what you don't know.
00:11:28.774 --> 00:11:33.614
So the fastest path forward is just find people who do.
00:11:34.094 --> 00:11:36.974
And the most important thing is kind of like.
00:11:37.394 --> 00:11:39.374
Just check your ego, right?
00:11:39.644 --> 00:11:42.553
And then once you're done checking your ego, check it again.
00:11:42.974 --> 00:11:51.274
So I literally call people every day and say, forgive me, but I'm about to ask you a lot of really dumb questions.
00:11:51.274 --> 00:11:52.504
Please bear with me.
00:11:52.984 --> 00:11:58.114
Um, and that, that, that mindset has served me incredibly well.
00:11:58.114 --> 00:11:58.384
Right?
00:11:58.384 --> 00:12:01.714
So you have three, four conversations with the right people.
00:12:02.359 --> 00:12:06.859
And you sort of like start seeing the contours of the problem space, right?
00:12:07.219 --> 00:12:13.313
So suddenly you understand what there is to know what you need to know and what you still don't know.
00:12:13.553 --> 00:12:15.264
And that's a huge win, right?
00:12:15.264 --> 00:12:17.514
When you can finally map out the landscape.
00:12:18.004 --> 00:12:18.964
That's a huge win.
00:12:18.964 --> 00:12:19.509
And you can.
00:12:20.149 --> 00:12:24.168
Go deeper, ask better questions, start connecting the dots, et cetera.
00:12:24.168 --> 00:12:24.469
Right?
00:12:24.889 --> 00:12:43.958
So I think, you know, like that's ultimately, this is how I've, I've handled, um, all these big challenges, just like staying curious, being patient and surrounding myself with people who know much more than I do and just being honest with myself and with the environment.
00:12:43.958 --> 00:12:46.089
It's like, this is something that I don't know anything about.
00:12:46.538 --> 00:12:48.068
Help me get up to speed.
00:12:48.469 --> 00:12:49.019
Okay good.
00:12:49.229 --> 00:12:52.889
And then, so obviously you mentioned learning's been quite key for you,
00:12:53.683 --> 00:12:53.974
Yeah.
00:12:54.188 --> 00:13:03.484
Obviously you've been successful to get to leadership and there's many people out there that wanna do it, but maybe uncertain or feel they've lack of skills.
00:13:03.484 --> 00:13:07.573
How have you, how has your career progressed into leadership?
00:13:07.573 --> 00:13:11.624
Was it, has there been any particular, have you had mentors, training?
00:13:11.714 --> 00:13:12.884
Has it been self-study?
00:13:12.884 --> 00:13:13.724
How have you gone about it?
00:13:14.053 --> 00:13:28.823
Yeah I've always had an incredible fortune of working for very smart, very successful people who I learned I've learned from and tremendous amount.
00:13:29.649 --> 00:13:37.178
And but also I've always had the great fortune of being surrounded by incredible team members.
00:13:37.589 --> 00:13:51.193
'Cause at the end of the day, I, I don't, like I, if someone junior is, or, like aspiring leaders are listening to this the most important thing to understand that it's all about people, right?
00:13:51.193 --> 00:13:55.124
Sometimes we get so wrapped up and we're solving this analytic problem, or.
00:13:55.734 --> 00:13:57.264
This is the thing that we're building.
00:13:57.683 --> 00:13:59.063
It's all about people.
00:13:59.094 --> 00:14:11.943
And as you progress, your success singularly depends on who you work with and your team members and the collaboration that you build with them and, and the relationships.
00:14:12.423 --> 00:14:18.303
So it's, it's, you know, more than anything, it's like it, people are people, right?
00:14:18.303 --> 00:14:27.244
Like people give their best when they feel understood and they feel appreciated and see how their work is creating genuine impact.
00:14:27.663 --> 00:14:38.573
So I think it's, it's always it's critical to remember that, like you can be absolutely brilliant, but to do big things, you need a lot of good people.
00:14:39.094 --> 00:14:43.533
To support you and, and drive that thing alongside with you.
00:14:43.994 --> 00:14:45.524
And I guess, you know, so that's one thing.
00:14:45.524 --> 00:14:53.594
And another thing would be what has worked really well for me is the ability to play in multiple worlds, right?
00:14:53.953 --> 00:15:00.254
My, my degree actually is in philosophy, but I have enough physics and math background to.
00:15:00.943 --> 00:15:04.423
To, uh, hold my own with very quantitative PhDs.
00:15:05.053 --> 00:15:07.063
And then I have my strategy background.
00:15:07.234 --> 00:15:13.443
You know, like, so at my core, I am quote unquote a business person, right?
00:15:13.744 --> 00:15:24.504
But obviously I can operate, I'm very comfortable operating in the, you know, AI data science world and leading these incredibly capable.
00:15:24.969 --> 00:15:27.009
More technical resources.
00:15:27.428 --> 00:15:38.818
So that ability kind of like to be on the intersection of the business and the AI has been incredibly helpful in my career.
00:15:38.818 --> 00:15:49.078
Just like, like blending technical insights, analytical insights, strategic insights, and just like going in and out of both those worlds, right?
00:15:49.438 --> 00:15:58.528
And they're like, there's so many, like AI is creating so many really interesting opportunities for, for people to do that, to play in both worlds.
00:15:58.528 --> 00:16:17.249
Like one example that like I kind of like talking about, so without getting too technical, that you know, you can take the way large language models, one of the things that they do, you can take a lot of unstructured text and you can turn that.
00:16:17.594 --> 00:16:21.524
Unstructured text into what they call like vector embeddings.
00:16:21.524 --> 00:16:21.764
Right?
00:16:21.764 --> 00:16:27.134
Which is basically like numeric representation of that text.
00:16:27.134 --> 00:16:29.114
A lot of ones and zeros, right?
00:16:29.563 --> 00:16:31.994
So there's this interesting question in my head.
00:16:31.994 --> 00:16:44.803
So when somebody does that, when you take, let's say a document and then you create a vector embedded representation of that document, is that the same data but in a different format?
00:16:45.359 --> 00:16:49.168
Or is that a completely new artifact?
00:16:49.528 --> 00:16:59.849
And the reason it's, it's important is because like, you know, like when you're talking about the legal aspect of ai, like contractual aspect of ai, so that's a really important question.
00:16:59.908 --> 00:17:00.239
Right?
00:17:00.649 --> 00:17:10.068
When, let's say your customer takes your data and creates some beddings out of it, is that a derivative output or is it basically the same data just in a different format?
00:17:10.519 --> 00:17:13.368
So I think Eli, so that's an example like lawyers.
00:17:13.743 --> 00:17:18.364
Who will be able to engage with these issues that didn't exist.
00:17:18.364 --> 00:17:20.344
Like even, you know, like two, three years ago.
00:17:21.064 --> 00:17:26.213
They will be, they'll have a lot of work and like, I think it's gonna be really interesting work.
00:17:26.463 --> 00:17:30.183
I think, like I said, like AI is creating a lot of these opportunities to be.
00:17:31.054 --> 00:17:33.713
Play in in, in multiple domains.
00:17:33.713 --> 00:17:38.213
And I think it's, you know, I think it's fun and, uh, there's not that many people who can do that.
00:17:38.213 --> 00:17:44.773
So, you know, that that's a pretty good way to, to get going with or accelerate your career rather.
00:17:45.689 --> 00:17:49.663
Yeah, I definitely think now the, the skill set is.
00:17:50.263 --> 00:17:51.253
Changing, isn't it?
00:17:51.294 --> 00:17:52.673
You need the blender skills now.
00:17:52.673 --> 00:17:55.078
I think you need obviously the technical skills.
00:17:55.679 --> 00:18:01.989
If you can talk, obviously there's so many different roles as well but you need, technical skills are getting less technical now'cause you've got the heavy lifting being done.
00:18:01.989 --> 00:18:07.269
So you need to now have that business knowledge strategy.
00:18:07.778 --> 00:18:10.868
It's some, so it's just evolving, isn't it?
00:18:10.868 --> 00:18:17.528
Now I think even if you're a data scientist now you and you're pure techie, I think you need to start learning them business skills.
00:18:17.558 --> 00:18:22.804
'cause you're gonna need to be able to communicate with the business and add value or you skill you.
00:18:23.628 --> 00:18:25.023
Roll's just gonna get void.
00:18:25.624 --> 00:18:26.523
Yeah, exactly.
00:18:26.554 --> 00:18:27.003
Exactly.
00:18:27.003 --> 00:18:41.314
And like, you know, the AI is blending so much, you know, all this age agentic all these age agentic capabilities, and, and in a few years, if you're a business leader who, who doesn't really understand these tools or these technologies.
00:18:41.983 --> 00:18:52.663
I think you're really, um, locking yourself out of a huge part of, like the problems that you can credibly try to solve.
00:18:53.003 --> 00:18:54.354
Yeah, I completely agree with you.
00:18:54.953 --> 00:18:57.689
And you men, oh, you mentioned you've got, you've got a blend of good skills.
00:18:57.689 --> 00:19:01.294
So is there, is there anything else that sort of helped you progress into leadership?
00:19:01.644 --> 00:19:05.419
I mean, like I said, like I, you know, I, I've been very, very lucky.
00:19:05.419 --> 00:19:18.963
I've, I've always worked for and with incredibly good people and the way I sort of like think about it if you're always the smartest guy or the smartest gal in the room.
00:19:19.743 --> 00:19:20.973
You're in the wrong room.
00:19:21.693 --> 00:19:32.644
You know, I, I, I think every person on teams that I've been, that I've had the privilege to lead is much better than me at many things.
00:19:33.153 --> 00:19:34.054
And that's great.
00:19:34.054 --> 00:19:39.243
That's how it should be if you ask me, like, so I don't, you know, I don't try to.
00:19:40.219 --> 00:19:43.249
To kind of like pretend that this is not the case.
00:19:43.519 --> 00:19:44.808
I lean into it.
00:19:44.858 --> 00:19:51.419
I want people to be better than me and learn thing, I mean, know things that I don't know and I can learn from them.
00:19:52.239 --> 00:19:55.088
I think that's been really critical for me.
00:19:55.388 --> 00:20:00.769
Then so the, the, the another thing is I'm pretty, uh.
00:20:01.068 --> 00:20:03.979
Pretty crazy about protecting my focus, right?
00:20:04.219 --> 00:20:09.858
There's just like so much stuff going on and, and the pace is just unrelenting, right?
00:20:09.858 --> 00:20:20.538
So even if days were 72 hour long, not 24 hours, if I engaged with everything that I can engage, I wouldn't get anything done.
00:20:21.138 --> 00:20:23.989
And that's one of the skills that I learned in consulting.
00:20:24.019 --> 00:20:25.114
'cause sometimes we would do.
00:20:25.868 --> 00:20:32.949
Projects where we had to help a private equity shop make a decision about buying a company and not buying a company.
00:20:32.949 --> 00:20:41.148
And we had like three weeks to to, learn about a target company and completely new space.
00:20:41.479 --> 00:20:50.723
So if that's, if that's the timelines that you're operating with, then you learn very quickly to separate things into those that are critical to understand in this context.
00:20:51.378 --> 00:20:54.469
And things that are nice to have to understand, right?
00:20:54.888 --> 00:21:00.638
So it's sort of like I've been applying the same mindset also to problems, right?
00:21:00.638 --> 00:21:06.038
Like, so I'm trying to do whatever I'm trying to do what is critical, what is not critical.
00:21:06.038 --> 00:21:12.479
And it's just like, I don't I don't, I don't have the, the luxury of engaging things that are not critical.
00:21:12.479 --> 00:21:14.068
And that's actually really helpful, right?
00:21:14.068 --> 00:21:17.848
So you end up, you end up doing more and you end up doing.
00:21:18.223 --> 00:21:22.183
Bigger things that drive bigger value for your company or your organization?
00:21:22.784 --> 00:21:38.713
Yeah, I think, I think the, you know, we talked about the ability to play in, in multiple worlds, and I guess, you know, like the last, but very much, not least, maybe it is the most important thing is build credibility before, before you seek influence, right?
00:21:39.044 --> 00:21:41.864
So just deliver, deliver, deliver, deliver.
00:21:41.864 --> 00:21:42.913
Because I think.
00:21:42.929 --> 00:21:56.064
If you, if you, if you get, if you gain a reputation of someone who can take on difficult challenges and just get things done influence will come, like, and, and progression will come as well.
00:21:56.364 --> 00:21:58.844
So that's probably the main thing.
00:21:59.443 --> 00:22:01.784
Yeah, I like comment about the focus.
00:22:01.814 --> 00:22:04.153
'cause I think that is key, isn't it?
00:22:04.153 --> 00:22:10.683
I mean, you need to, there's so much going on and you need to, what's gonna move the needle basically isn't what's gonna, what's the big thing.
00:22:11.013 --> 00:22:12.153
And again, that's the same with business.
00:22:12.153 --> 00:22:15.213
I think everyone's got bit loss, regenerative ai, haven't they?
00:22:15.213 --> 00:22:20.463
Again, tr there's so many projects you wanna do, you need to, what's gonna impact the business right now?
00:22:20.773 --> 00:22:23.114
'Cause there'll be so much you wanna do in the business now.
00:22:23.114 --> 00:22:24.673
There's so much opportunity now, but.
00:22:25.273 --> 00:22:27.433
What's the current challenge, isn't it really?
00:22:27.433 --> 00:22:30.884
You need to focus on what's gonna be the big impact for you right now.
00:22:31.723 --> 00:22:32.263
That's right.
00:22:32.324 --> 00:22:32.834
That's right.
00:22:33.433 --> 00:22:33.794
Okay.
00:22:34.173 --> 00:22:40.203
So if say, I wanted, I was an aspiring leader, what advice would you give myself?
00:22:40.354 --> 00:22:40.443
Give.
00:22:41.044 --> 00:22:42.604
Advice to give yourself?
00:22:43.144 --> 00:22:45.993
Um, that's a great question.
00:22:46.594 --> 00:22:58.144
I think, you know, ev everything, so you, you get, you're in a really interesting position because you get to talk to people from many different walks of life and different roles.
00:22:58.743 --> 00:23:00.844
And many different industries.
00:23:01.114 --> 00:23:05.733
So you kind of, you know, like to do successfully what you do on this podcast.
00:23:06.304 --> 00:23:13.443
So you must be curious and you must be willing to engage with things that you maybe don't know much about.
00:23:13.443 --> 00:23:15.624
And, sounds like you're doing really, really well with that.
00:23:16.223 --> 00:23:16.703
I hope so.
00:23:17.304 --> 00:23:17.963
That's fine.
00:23:18.304 --> 00:23:22.673
Okay, so let's tap into the question then on AI agents.
00:23:23.023 --> 00:23:38.943
Obviously this is, yeah, massive next evolution, obviously brings many opportunities and I guess there's obviously challenges and risks in the business environment, so they're obviously pushed as the next layer on top of LLMs.
00:23:39.574 --> 00:23:46.864
In your view, what capabilities do they unlock that base models cannot achieve on their own right now?
00:23:47.638 --> 00:23:48.959
This is an awesome question.
00:23:48.959 --> 00:23:52.019
This is really like truly foundational to understand.
00:23:52.229 --> 00:23:53.489
So there's a couple of things, right?
00:23:53.489 --> 00:24:01.769
So first of all, LLMs understand and agents do, right?
00:24:01.769 --> 00:24:12.028
So there, there's there's a huge progression from LLMs that are essentially like passive interlocutors or assistance.
00:24:12.588 --> 00:24:22.199
Where whereas agents are active problem solvers, you know, like a digital coworker that can reason, plan and execute things.
00:24:22.528 --> 00:24:29.398
So just to get like a tad more technical agents add state.
00:24:30.324 --> 00:24:33.894
Planning memory and tool usage, right?
00:24:33.894 --> 00:24:36.773
So like I planning memory and tool usage.
00:24:36.773 --> 00:24:38.304
I think that's pretty self-explanatory.
00:24:38.304 --> 00:24:40.044
Let me talk a little bit about state.
00:24:40.493 --> 00:24:49.943
So LLMs are, are awesome at understanding and, and generating language, but they're not inherently stateful, right?
00:24:50.243 --> 00:24:58.403
They don't retain goals, they don't retain memory or, or continuity beyond.
00:24:59.334 --> 00:25:01.104
The current context, right?
00:25:01.433 --> 00:25:07.794
And agents add that missing layer of planning and, and memory and action.
00:25:08.064 --> 00:25:10.223
They can decide what to do next.
00:25:10.223 --> 00:25:12.114
They can call external tools.
00:25:12.403 --> 00:25:16.114
They can coordinate steps over time.
00:25:16.473 --> 00:25:17.163
So.
00:25:17.763 --> 00:25:23.778
And, and, and all those things is just again, like, just to bring it back like LLMs talk.
00:25:23.808 --> 00:25:32.479
LLMs can converse with you, but agents can actually do things and like pretty, pretty complex things at that.
00:25:32.578 --> 00:25:38.969
So, and speaking of complexity, so the other thing is agents, I apologize, lost my voice.
00:25:39.568 --> 00:25:52.138
And so sorry agents can decompose large, large, complex problems into smaller tasks and orchestrate how those tasks are solved.
00:25:52.739 --> 00:25:57.659
Something that even smartest models, smartest LMS really struggle with, right?
00:25:57.898 --> 00:26:02.493
So you can, like, if you throw, if you ask even the smartest LLM.
00:26:03.094 --> 00:26:13.263
In the world to interact with petabytes of diverse data and ask it to, to derive conclusions from that data.
00:26:14.213 --> 00:26:15.294
It's too much, right?
00:26:15.294 --> 00:26:18.983
Like it's just too much data and there's too much variety and there's too much complexity.
00:26:18.983 --> 00:26:22.733
So LLMs in these situations, hallucinate like crazy.
00:26:23.064 --> 00:26:25.074
They make, they just make things up.
00:26:25.074 --> 00:26:28.463
Like their answers sound very confident, but factually.
00:26:29.439 --> 00:26:30.729
There are a lot of inaccuracies.
00:26:31.028 --> 00:26:34.538
So agents on the other hand work very differently.
00:26:34.538 --> 00:26:37.719
They kind of, they embrace tool use.
00:26:37.749 --> 00:26:45.249
They can call APIs, they can run database queries, they can delegate to specialists, sub-agents.
00:26:45.608 --> 00:26:50.979
And the beauty is, so those tools or, or subagents or, or APIs.
00:26:51.189 --> 00:26:56.169
You, they can be specialists like in just one thing, right?
00:26:56.588 --> 00:27:07.479
So you can have an agent tool that is only good at retrieving, like in, in our world, for example, data about ESG, right?
00:27:07.719 --> 00:27:12.909
And then you'll have ESG data, and then you'll have another tool that is exceptionally good at.
00:27:13.433 --> 00:27:19.324
Retrieving data about I don't know, like news about a company, right?
00:27:19.773 --> 00:27:23.763
And all those specialist tools, they'll be extremely good.
00:27:24.153 --> 00:27:29.074
They will use deterministic mechanisms of finding that information.
00:27:29.344 --> 00:27:39.013
Meaning, you know, like they, they will convert fluid conversational language from the user into whatever, like SQL calls or, or whatever, right?
00:27:39.723 --> 00:27:51.784
And by breaking the problem down into a bunch of smaller pieces for each piece, you have a specialist that is really, really good at doing that particular thing.
00:27:52.203 --> 00:28:02.223
So suddenly you can handle like enormous amounts of data and very complex, multifaceted user queries.
00:28:02.463 --> 00:28:03.364
And you can do that.
00:28:03.963 --> 00:28:05.493
Very successfully.
00:28:05.614 --> 00:28:05.943
Right.
00:28:06.223 --> 00:28:07.784
And, and that's just huge, right?
00:28:07.784 --> 00:28:15.253
Like you it's, even today, even like the lo the, the biggest, smartest lms, they hallucinate quite a bit.
00:28:15.614 --> 00:28:19.574
So it's like I said, like you can't just unleash them directly in the business context.
00:28:19.574 --> 00:28:21.864
Like, like you can't afford.
00:28:22.463 --> 00:28:25.854
You know, your tolerance for hallucination is extremely small.
00:28:25.854 --> 00:28:29.003
Like you can't even afford, like 5% of hallucinations.
00:28:29.374 --> 00:28:40.558
So that in those environments, like you really need to bring in agents that are exceptionally good at breaking those enormous problems down and just solving them one step at a time.
00:28:41.159 --> 00:28:45.314
And so what would be like a telltale sign of a high value use case for agents then?
00:28:45.663 --> 00:28:47.403
So I think there's a couple of things, right?
00:28:47.403 --> 00:28:54.463
So obviously, you know, everything, everything that you do, like it, it needs to have, clear value, right?
00:28:54.463 --> 00:28:59.364
Like, like if you're deploying an agent, you need to understand why you're doing it.
00:28:59.364 --> 00:29:02.449
What's the value that, that you want to drive, be it.
00:29:03.263 --> 00:29:13.173
Time savings or, or, or new revenue or efficiencies, or you can just, in some cases you can do something they were not able to do previously.
00:29:13.173 --> 00:29:16.534
So the first thing is like, what's the actual tangible impact, right?
00:29:17.433 --> 00:29:18.874
So that's sort of a given, right?
00:29:19.263 --> 00:29:24.243
That specifically for agents, I think repeatability is a big thing.
00:29:24.624 --> 00:29:28.824
If there is a workflow or a task that happens.
00:29:29.528 --> 00:29:35.409
Often enough, and it's sort of like more or less the same thing that is being done repeatedly over time.
00:29:35.798 --> 00:29:41.858
That's a really good signal that this is something where an agent can be quite impactful.
00:29:42.398 --> 00:29:51.699
Also complexity is actually like, I would say that complexity is, and by complexity I mean, a task may require.
00:29:52.104 --> 00:29:54.683
Reasoning and context, right?
00:29:54.683 --> 00:30:09.403
So you're not just like, not a single deterministic answer because like we said a little bit earlier, like, agents can actually break down problems, figure out what path to choose, which tools to use and which sequence.
00:30:09.763 --> 00:30:13.604
So if there's a little bit of complexity that's, that's also really good.
00:30:14.203 --> 00:30:18.763
And I guess, you know, like there's, there's also a dose of realism.
00:30:19.598 --> 00:30:24.068
Agents are very, very powerful tools, but it's not like it's a magic wand.
00:30:24.068 --> 00:30:38.544
So the sort of like the way to think about it is if you drop the brilliant person into the same task and ask them to do it, would they say, yep, I've got everything that I need to do this task.
00:30:38.933 --> 00:30:43.943
Or would they say, you didn't give me the data or the right tools to execute this task.
00:30:44.044 --> 00:30:54.453
If it's the latter, then agents are not going to be successful to execute again, like super, super powerful, but they can't create something out of nothing.
00:30:54.693 --> 00:31:01.374
It means that if that's the answer, then it means that you have to do some pre-work before deploying agents into the space.
00:31:01.673 --> 00:31:02.919
And of course, you know, in, in.
00:31:03.519 --> 00:31:13.038
In enterprise context, of course you have to be very mindful about regulatory compliance and sort of like government context.
00:31:13.608 --> 00:31:26.709
There are some use cases that inherently riskier than other use cases, and you just need to be, you need to understand very well the risk profiles of different use cases and judge accordingly.
00:31:26.709 --> 00:31:27.818
If this is something that.
00:31:28.419 --> 00:31:35.949
Um, you're very comfortable deploying agents into, or something that maybe you want to wait for a few months to get more clarity.
00:31:36.818 --> 00:31:50.769
So I would say just see, just to kind of like recap, impact repeatability, um, some, you know, like some complexity, some, like just realistic realism, right?
00:31:50.769 --> 00:31:55.429
And just remembering to think of regulatory and compliance stuff.
00:31:56.028 --> 00:32:01.878
So does that mean, oh, so this is a new, new thing for businesses now?
00:32:01.878 --> 00:32:04.858
So is this mean you need to get, when you're getting involved with this?
00:32:04.878 --> 00:32:10.159
This has gotta be bringing in multiple different departments within the business to cover all bases.
00:32:10.429 --> 00:32:12.709
So it's like not more team effort now.
00:32:13.683 --> 00:32:14.854
yeah, exactly right.
00:32:14.854 --> 00:32:19.709
So I I always say that Gen AI or agents, it's a team sport it like it truly is right.
00:32:20.509 --> 00:32:29.398
If you have a bunch of super smart people doing something in the basement I don't think it's gonna be I don't think it's going to scale.
00:32:29.398 --> 00:32:30.838
I don't think it's going to succeed.
00:32:30.919 --> 00:32:35.898
So you really need to have, you know, like different people call it differently.
00:32:35.989 --> 00:32:42.943
Like I've heard a name, gen AI pod like you, you have to have your product people, you have to have your technology people.
00:32:43.703 --> 00:32:50.624
You need to have legal, cybersecurity, um, you know, like all, all the, the folks who commercialize things.
00:32:50.624 --> 00:32:56.923
So all, all the sort of like, how do we price it, how do we bill it, how do we manage, uh, usage?
00:32:57.284 --> 00:33:01.993
So you gotta have, and I'm probably forgetting a bunch of important functions, right?
00:33:02.324 --> 00:33:05.384
So if you don't have all that.
00:33:05.693 --> 00:33:13.374
As a cohesive group or cohesive team, it's gonna be pretty hard to to, to deploy at scale.
00:33:13.973 --> 00:33:14.423
Okay.
00:33:14.844 --> 00:33:19.074
And then obviously many pilots fail to scale.
00:33:19.503 --> 00:33:26.433
So what are the hidden blockers that organizations overlook when deploying agents?
00:33:27.034 --> 00:33:28.459
So, you know, it's interesting.
00:33:28.808 --> 00:33:31.778
Complexity doesn't scale linearly.
00:33:32.439 --> 00:33:34.659
It kind of scales exponentially.
00:33:35.259 --> 00:33:38.318
I'm so sorry, I'm still having my voice issues.
00:33:38.769 --> 00:33:52.358
So, if you prototype an agent that works on a small, clean sliver of data for a pretty narrow use case, you can't assume that it will behave the same way.
00:33:52.959 --> 00:33:57.729
When it's exposed to the full scope of, of your data or workflows, right?
00:33:58.328 --> 00:34:18.789
So you, you, you may, something that works beautifully at small scale, you may discover that when you scaled up, you're running into memory limits context, window constraints, retrieval, quality orchestration load, and it's like all the things that don't show up in, in your demos.
00:34:19.614 --> 00:34:21.744
But they will show up in production.
00:34:22.103 --> 00:34:22.914
So that's one.
00:34:23.514 --> 00:34:26.844
Second, you gotta have agent ready data, right?
00:34:26.844 --> 00:34:32.003
Like agents can only reason as well as the data that they can see and interact with.
00:34:32.423 --> 00:34:36.534
So you need to make sure that you have the right data for the agents to work with.
00:34:37.114 --> 00:34:40.503
Meaning your data has to be trusted, timely, contextualized.
00:34:40.853 --> 00:34:54.184
If it's sort of like locked in silos or poorly mapped where you can just tr you can't trust your data, um, that that's, it's gonna be pretty hard to, to do anything at scale.
00:34:54.963 --> 00:35:01.143
And then there's this, this interesting concept of like signal to noise ratio, right?
00:35:01.623 --> 00:35:04.233
So let's say you do have agent ready data.
00:35:04.653 --> 00:35:07.893
So what's really important is to not overwhelm the agent.
00:35:08.688 --> 00:35:14.449
With data that it doesn't really need to solve the problems that you want it to solve.
00:35:14.958 --> 00:35:23.478
So it's sort of like, you know, if I go back to my, you know, like if you drop the really smart person into the problem, would they be able to do it?
00:35:24.079 --> 00:35:33.699
You know how like they say that most people can remember and repeat back only about seven to 10 digits if you tried to make people memorize.
00:35:34.298 --> 00:35:38.588
20 digits or 30 digits, almost everyone would fail, right?
00:35:38.588 --> 00:35:43.958
And there comes a point, you know, like you get to 50 digits, like 99.9% of people would fail.
00:35:44.438 --> 00:35:46.208
And it's the same thing with agents, right?
00:35:46.208 --> 00:35:53.378
Like the more data you throw at them, the higher the probability that they will get confused and will start hallucinating.
00:35:53.914 --> 00:35:58.418
So and so, the challenge is not to stuff more data in the context window.
00:35:58.778 --> 00:36:00.188
Like you really gotta know.
00:36:00.954 --> 00:36:07.164
What is the right data and the most impactful data that you want to bring into the problem.
00:36:07.583 --> 00:36:10.824
So this is not, you know, like the more, the better.
00:36:11.034 --> 00:36:17.778
Like you need to have, like this is the most impactful data that I can bring into this problem.
00:36:17.858 --> 00:36:23.648
So, and I think it's interesting that this concept of context engineering, it's really starting to evolve.
00:36:24.318 --> 00:36:28.068
It's, so, it's definitely an area to watch as you're scaling.
00:36:28.338 --> 00:36:32.568
And of course, you know, like I, it goes without saying, you gotta have the right infrastructure.
00:36:32.918 --> 00:36:43.778
Because I, you know, like some pilots often run on sort of like highly bes, bespoke setups, which don't translate into production, right?
00:36:43.778 --> 00:36:48.278
Because, you know, like, just because something runs.
00:36:48.878 --> 00:36:50.063
I, you know what?
00:36:50.094 --> 00:36:59.454
Well enough for, for your pilot, you may very quickly discover that that setup that you have to run the pilot is way too expensive to run in production, right?
00:36:59.844 --> 00:37:02.664
And like I said, you know, like some things just don't scale.
00:37:03.253 --> 00:37:09.284
So the question is not when you do your, your pilot, the question is not like, Hey, can I get it to work?
00:37:09.583 --> 00:37:15.974
The question really should be, can I get it to work 10,000 times a day reliably?
00:37:16.693 --> 00:37:19.034
At enterprise level.
00:37:19.123 --> 00:37:20.833
So I think these are the big things.
00:37:21.434 --> 00:37:27.018
I definitely hearing a lot around data quality is getting so key now, isn't it?
00:37:27.349 --> 00:37:27.588
Yeah.
00:37:27.923 --> 00:37:28.164
Absolutely.
00:37:28.548 --> 00:37:29.268
Yeah, exactly.
00:37:29.268 --> 00:37:39.759
It's sometimes what's really interesting, LLMs can do amazing things and then you throw agents into the mix and it's I look at what my team creates and I'm like.
00:37:40.344 --> 00:37:42.353
Wow, this is science fiction, right?
00:37:42.724 --> 00:37:46.773
I'm in it 24 7, but I still have those moments.
00:37:46.773 --> 00:37:49.623
Like I look at the stuff that they're building, I'm like, wow.
00:37:50.074 --> 00:37:52.954
Like I, I literally can't believe what I'm seeing.
00:37:53.463 --> 00:38:00.684
But so sometimes like we get really anchored on that, that we forget, like agents are still software, right?
00:38:01.023 --> 00:38:04.123
Like the old, garbage in, garbage out.
00:38:04.393 --> 00:38:09.884
As smart and capable as agents are like, you put bad data in, you're gonna get that stuff out.
00:38:10.454 --> 00:38:17.864
So yeah, having the right data, trusted, timely, contextualized well, accurate obviously.
00:38:18.304 --> 00:38:22.293
That is, you're going to fail if you don't have all that.
00:38:22.594 --> 00:38:32.884
A lot of people have said especially with lms, like building LM is easy, the easy part, easier part, the back, like the getting data is the hardest bit.
00:38:32.884 --> 00:38:36.873
Is that the same with building agents?
00:38:37.224 --> 00:38:39.764
So it, it's an interesting question, right?
00:38:40.608 --> 00:38:44.478
Um, building good agents is.
00:38:45.079 --> 00:38:46.639
Pretty hard, right?
00:38:46.639 --> 00:38:52.619
It's, it takes a lot of very specialized skills ability to innovate very, very quickly.
00:38:52.949 --> 00:38:54.688
So I think, I would say it's both right?
00:38:54.688 --> 00:39:03.398
Like you do need excellent teams and excellent infrastructure and like, just excellent sort of like, innovation process.
00:39:04.253 --> 00:39:13.193
But yeah it's, you can't do, even if you, you're exceptionally good at building stuff, you're not gonna be successful if you don't have great data.
00:39:13.193 --> 00:39:14.518
So it's, you really need both.
00:39:15.119 --> 00:39:15.989
Okay, perfect.
00:39:16.719 --> 00:39:22.059
And how do you separate orchestration complexity from genuine intelligence?
00:39:22.659 --> 00:39:26.559
See that, that's, that's an interesting, that's a really interesting question.
00:39:26.559 --> 00:39:27.548
Like intelligence is.
00:39:28.148 --> 00:39:30.278
Intelligence is a very loaded term.
00:39:30.818 --> 00:39:33.903
And, and I have, I have a degree in philosophy.
00:39:33.903 --> 00:39:42.403
Um, during my time at Yale, actually, I took a seminar with Nick Bostrom, who is one of the leading thinkers in, philosophy of ai.
00:39:42.943 --> 00:39:49.373
So I can debate intelligence and what that means and whether we have it or not until the cows come home.
00:39:49.974 --> 00:40:00.804
I think honestly, like I, I will punt a little bit on the intelligence thing, and I instead, I will say that in practice, the real metric is not intelligence, but value, right?
00:40:01.313 --> 00:40:12.653
So to me, the real question is, are agents creating value beyond, you know, like your traditional workflows or, you know, what, what automation could deliver?
00:40:12.653 --> 00:40:16.583
And to me the answer is an unequivocal yes.
00:40:16.684 --> 00:40:18.333
I think, you know, I, I think.
00:40:19.159 --> 00:40:31.048
So it's interesting, like sometimes when people say agents, they think of like early day agents that didn't really have agency or autonomy.
00:40:31.048 --> 00:40:41.318
They were really just like very sophisticated workflows wrapped around an LLM like executing static plans with, with style and flair, right?
00:40:41.378 --> 00:40:43.688
And that's sort of like that, that linear.
00:40:44.469 --> 00:40:48.398
Chains of things that agents were asked to ex execute.
00:40:48.608 --> 00:40:51.659
That's where, like the term chain of thought came from.
00:40:51.898 --> 00:40:56.159
There's just linearly, linear chains of, of execution.
00:40:56.489 --> 00:41:10.068
But now the agents that, that we have today with memory, like really good memory and planning and tool usage, they truly can, can act autonomously and decide how to approach a problem.
00:41:10.668 --> 00:41:12.528
Instead of following a fixed script.
00:41:12.528 --> 00:41:12.768
Right.
00:41:12.768 --> 00:41:15.139
So, and I mentioned agents are stateful.
00:41:15.768 --> 00:41:19.588
Um, they don't start from scratch with every request.
00:41:19.588 --> 00:41:23.789
Like they remember prior steps and they can track goals.
00:41:23.789 --> 00:41:31.349
They can track their execution against the goals, and they can adjust their next move based on.
00:41:32.009 --> 00:41:33.298
On outcomes, right?
00:41:33.568 --> 00:41:38.219
So they don't have to move linearly through predefined paths anymore, right?
00:41:38.548 --> 00:41:45.748
They can navigate dynamically through almost like a graph of actions that they can take.
00:41:46.228 --> 00:41:49.798
You know, like and then they can choose the most relevant path in real time.
00:41:50.099 --> 00:41:55.858
They have the autonomy to loop back, skip steps, like change course when new information comes in.
00:41:56.713 --> 00:42:01.333
It's, it's really like pretty similar to how humans solve problems, right?
00:42:01.704 --> 00:42:13.744
So I think, you know, like the agents that we have today absolutely add tremendous value on top of what those early day agents we're bringing to the table.
00:42:13.983 --> 00:42:15.514
And it's important to remember, like.
00:42:16.278 --> 00:42:22.458
It feels like there's so much has happened and the cha the pace of innovation so quick that it feels like these things have been around forever.
00:42:22.849 --> 00:42:27.389
We're still very early days of these, of this agentic revolution.
00:42:27.929 --> 00:42:32.128
We'll get to a place where these systems can learn and they can adapt.
00:42:32.719 --> 00:42:35.539
They can continuously improve outcomes.
00:42:35.969 --> 00:42:40.798
You know, I, I, I know that I totally punted on, on, uh, intelligence, but.
00:42:41.099 --> 00:42:41.418
Yeah.
00:42:41.429 --> 00:42:45.869
To me the value is unquestionably there.
00:42:45.869 --> 00:42:48.268
I see it every day many times.
00:42:48.699 --> 00:42:51.268
And the beauty is we barely scratch the surface.
00:42:52.188 --> 00:42:52.478
Yeah.
00:42:52.478 --> 00:42:53.173
Yeah, true.
00:42:53.173 --> 00:42:55.393
Like it's still early days, isn't it?
00:42:55.393 --> 00:42:55.844
Like it's.
00:42:56.248 --> 00:42:56.699
Yeah
00:42:56.963 --> 00:43:05.034
again, this is where I guess obvious you've got the technical knowledge, but also if you've got the business domain expertise here, like how can you actually, again, add value to the business?
00:43:05.034 --> 00:43:10.164
It's gonna, you're gonna add some small power to what you are trying to achieve.
00:43:11.043 --> 00:43:11.889
Yeah, exactly.
00:43:12.728 --> 00:43:13.268
Exactly.
00:43:13.869 --> 00:43:17.813
So you see like a big risk in over reliance agents for businesses.
00:43:18.414 --> 00:43:29.983
Um, I think you know it's a little bit of what I said earlier, unbelievably powerful tools, frameworks, um, when well designed, they can do amazing things.
00:43:30.583 --> 00:43:40.574
But agents are software, albeit software with autonomy to make very complex decisions and based on complex inputs, all that stuff, right?
00:43:41.114 --> 00:43:55.304
But you know, like it's really important not to get complacent and just like stay on top of how those environments evolve and how inputs shift over time.
00:43:55.414 --> 00:43:56.074
Um.
00:43:56.673 --> 00:44:03.514
So I would say, you know, like today, agents should augment human judgment.
00:44:04.023 --> 00:44:10.983
But you shouldn't just say, all right, like, we have an agent running, like we can go fishing for the rest of the month, right?
00:44:10.983 --> 00:44:15.378
Like, maybe we'll happen very, very quickly as models get better and these frameworks get better.
00:44:15.483 --> 00:44:19.923
Like maybe we'll be able to, to rely on these things more.
00:44:19.983 --> 00:44:22.173
But again, like unbelievably powerful tools.
00:44:22.173 --> 00:44:24.219
But it's important to remember that today they're tools.
00:44:24.568 --> 00:44:36.599
So yeah, I, I think it's really important to stay on top of, like to monitor what goes in monitor, what goes out track performance really, really closely.
00:44:36.599 --> 00:44:41.188
And there's some, you know, like really powerful tools that allow you to do that today, right?
00:44:41.188 --> 00:44:45.498
Like that, that can see all the like prompts, for example.
00:44:46.114 --> 00:44:58.143
Into an agent system and see all the outputs and do some like, extremely intelligent things like analyzing analyzing outputs from the quality perspective and like even governance, right?
00:44:58.143 --> 00:45:09.204
Like so I think, it, the, the risk is forgetting that we still need people very closely involved with these powerful tools.
00:45:09.804 --> 00:45:10.023
It.
00:45:10.623 --> 00:45:11.628
Yeah, definitely.
00:45:12.228 --> 00:45:12.619
Cool.
00:45:12.679 --> 00:45:24.469
So then if we look, say two to three years ahead, do you see agents becoming embedded into the enterprise architecture or is it gonna be more of a like a niche tactical tool for businesses?
00:45:24.498 --> 00:45:24.528
I.
00:45:25.128 --> 00:45:30.289
I think, um, I think it's definitely in the former, I think agents will become very embedded.
00:45:30.873 --> 00:45:35.974
And you know, like in a way they'll simply be how enterprise systems work, right?
00:45:36.514 --> 00:45:50.773
So you'll see agent capabilities built directly into CRMs, ERPs master data management platforms, often behind the scenes, sort of like similar to how APIs operates today.
00:45:51.224 --> 00:45:55.184
And I think, you know, the key enabler of that, of that embeddedness.
00:45:55.829 --> 00:45:57.599
It's going to be standardization, right?
00:45:57.599 --> 00:46:05.338
So if you take things like model context, protocol, MCP they will make a lot of data.
00:46:05.338 --> 00:46:08.938
Like MCP will make a lot of really, really good data sets.
00:46:09.358 --> 00:46:11.728
Enterprise, sorry.
00:46:11.728 --> 00:46:14.398
It will make really good data sets like agent ready.
00:46:14.728 --> 00:46:23.458
Um, they will make real good data sets more accessible, uh, well described, easier for agents to consume and understand.
00:46:23.938 --> 00:46:32.318
Um, like, so MCP will give agents a common language to, to use for communications across systems, right?
00:46:32.528 --> 00:46:36.128
So that's gonna be a very, very important driver of that embeddedness.
00:46:36.159 --> 00:46:41.798
The other one is going to be a to a, or, you know, like the protocol for agent to agent collaboration.
00:46:42.559 --> 00:46:59.378
I think, you know, like we're, I, maybe it hasn't happened at scale yet, but I think very quickly we're gonna get to a point where agents communicate with other agents across different enterprises, across different systems.
00:46:59.949 --> 00:47:06.219
So you kind of, you know, that, that will allow enterprises to move from having isolated.
00:47:06.818 --> 00:47:16.688
AI tools or agents to really this, these like connected collaborative ecosystems that reason and act together.
00:47:17.048 --> 00:47:24.268
So I I, I, I definitely think that the, the some of this embeddedness is happening, is happening already, right?
00:47:24.268 --> 00:47:34.849
So for example, like DDMV has pretty incredible MCP, servers that we already exposed to customers, we have a, to a capabilities.
00:47:34.849 --> 00:47:42.599
So we're starting to see that embeddedness already, and I think it's, it's only going to get much more pronounced very quickly.
00:47:42.838 --> 00:48:15.228
So I, I think like the vision is sort of like a lot of these interconnected agents powering workflows, powering important decisions, powering analytics behind the scenes, and sort of like seamlessly integrated through protocols like MCP and coordination layers, like eight to a again, I think this is still pretty early stages of that transformation, but I think we're starting to see the curve starting to bend up pretty quickly.
00:48:15.829 --> 00:48:16.489
Brilliant.
00:48:16.699 --> 00:48:16.969
Cool.
00:48:17.248 --> 00:48:17.579
Here we go.
00:48:17.579 --> 00:48:22.778
It's been a pleasure having you on shared some great insights and yeah, so obviously a hot topic.
00:48:23.148 --> 00:48:24.199
So thanks for joining us.
00:48:25.034 --> 00:48:25.963
Thank you very much.
00:48:25.963 --> 00:48:28.094
Thank you again for having me on your show.
00:48:28.094 --> 00:48:29.983
Like really interesting questions.
00:48:30.034 --> 00:48:38.364
There's a lot going on in the space and things are evolving very quickly, so it's definitely an area to watch and to learn about and to be excited about.
00:48:38.963 --> 00:48:39.264
Yep.
00:48:39.623 --> 00:48:39.804
I agree.
00:48:39.954 --> 00:48:43.523
And to our listeners, if you enjoyed today's episode Yep.
00:48:43.583 --> 00:48:47.934
If you could give us a follow, leave, a quick review, just really helps others discover the show.
00:48:48.233 --> 00:48:48.474
Thank you.
