Unveiling AI's Transformative Impact with Tom Marsh: A Microsoft Product Management Maestro's Journey Through Enhanced Business Applications and Personal Passions

WEBVTT
00:00:02.525 --> 00:00:04.371
Welcome to the Power 365 show.
00:00:04.371 --> 00:00:10.509
We're an interview staff at Microsoft across the Power Platform and Dynamics 365 Technology Stack.
00:00:10.509 --> 00:00:16.509
I hope you'll find this podcast educational and inspire you to do more with this great technology.
00:00:16.509 --> 00:00:18.907
Now let's get on with the show.
00:00:18.907 --> 00:00:25.768
In this episode, we'll be focusing on large language models and their role in Microsoft business applications.
00:00:25.768 --> 00:00:29.908
Today's guest is from Redmond Washington in the United States.
00:00:29.908 --> 00:00:32.966
He works at Microsoft as a principal group product manager.
00:00:32.966 --> 00:00:38.968
He's technical leader with over 25 years of experience in product management and engineering roles.
00:00:38.968 --> 00:00:41.302
You can find links to his bio, socials, etc.
00:00:41.302 --> 00:00:42.908
In the show notes for this episode.
00:00:42.908 --> 00:00:44.424
Welcome to the show.
00:00:44.424 --> 00:00:46.326
Tom Hi Mark, thanks for having me.
00:00:46.326 --> 00:00:48.326
Was the introduction correct?
00:00:48.860 --> 00:00:49.725
Yeah, it sounded right.
00:00:50.302 --> 00:00:51.049
Good, excellent.
00:00:51.049 --> 00:00:55.049
I'm excited to have you on because, at the moment, ai is my favorite topic.
00:00:55.049 --> 00:01:05.287
In fact, somebody pointed out that since 2018, it's been my favorite topic because that's when I really first started blogging a lot around AI and I was excited about where it's going.
00:01:05.287 --> 00:01:14.906
Then I just feel that Microsoft actually I'll tell you why I started getting into it then I felt that Microsoft was the only company out there doing what I call practical AI.
00:01:14.906 --> 00:01:21.612
Gpt has been in the power product for the last couple of years.
00:01:21.612 --> 00:01:32.549
It's not new, but, of course, since October 30th, when Sam Altman did what he did with OpenAI and everybody's got excited about large language models, the world has fundamentally changed.
00:01:32.549 --> 00:01:36.025
We're coming up 12 months, right, yeah, this month makes 12 months.
00:01:36.025 --> 00:01:38.768
Oh my gosh, what a whirlwind it's been.
00:01:38.768 --> 00:01:42.323
Before we unpack your role, what you do, etc.
00:01:42.323 --> 00:01:44.343
Tell us a bit about food, family and fun.
00:01:44.343 --> 00:01:45.246
What do they mean to you?
00:01:46.081 --> 00:01:47.621
Well, I live in Redmond, as you said.
00:01:47.621 --> 00:01:53.269
I have a wife and two elementary school kids, so most of my life definitely revolves around them.
00:01:53.269 --> 00:01:59.406
All my free time playing games, going to events, watching movies, taking them to classes, all that kind of stuff.
00:01:59.406 --> 00:02:00.905
So, yeah, that's fun.
00:02:00.905 --> 00:02:02.725
I especially like Minecraft.
00:02:02.725 --> 00:02:05.388
We get lost in hours playing that.
00:02:05.388 --> 00:02:08.269
Food-wise, I'm not a great cook.
00:02:08.269 --> 00:02:13.953
I do enjoy cooking, but my family has to put up with whatever I come up with.
00:02:13.953 --> 00:02:17.266
I guess a lot of omelettes, a lot of pancakes and things like that.
00:02:17.266 --> 00:02:18.586
But, yeah, have a lot of fun with that.
00:02:18.920 --> 00:02:20.668
Very cool Minecraft.
00:02:20.668 --> 00:02:22.606
Do you have your own server set up in a jour?
00:02:23.181 --> 00:02:27.407
No, I just have the regular bedrock edition and just play around in there.
00:02:28.502 --> 00:02:33.866
Okay, so my son is now 18 years old and I've lived around the world as in.
00:02:33.866 --> 00:02:44.710
He was from a former marriage, but one of the ways we stayed connected is I set up a Minecraft server in a jour and then we would play online no matter where I was in the world.
00:02:44.710 --> 00:02:48.508
Very cool as he grew up, and it was a really good way because you could have conversations etc.
00:02:48.508 --> 00:02:59.052
While you were playing, but you were playing the game, and so it was such a good tool, I reckon, to actually have good communication time.
00:02:59.721 --> 00:03:07.045
Yeah, I usually get my kids to lie on the floor in the room, so I'll be on my device and they're on theirs and we'll be sad in real time.
00:03:07.045 --> 00:03:09.328
But yeah, if they weren't local, then that would be a great way to do it.
00:03:09.840 --> 00:03:13.810
Yeah, I love it, and being able to set that all up in a jour was just epic.
00:03:13.810 --> 00:03:15.664
But yeah, probably haven't played for a couple of years.
00:03:15.664 --> 00:03:21.030
Now that he's got older and about to start studying law, his focus has changed a bit.
00:03:21.030 --> 00:03:21.610
Very cool.
00:03:21.610 --> 00:03:24.526
Tell me about how you got into Microsoft.
00:03:24.526 --> 00:03:30.070
What's that journey been and bring us right up to what your role is today and what you're doing.
00:03:30.280 --> 00:03:30.943
Yeah, absolutely.
00:03:30.943 --> 00:03:32.905
So I joined straight out of school.
00:03:32.905 --> 00:03:42.906
Actually, I first joined as an intern because my roommate across the hall from me had had an internship here and just raved about the experience that they had had and sounded like a lot of fun.
00:03:42.906 --> 00:03:44.445
So, all right, I'll go check that out.
00:03:44.445 --> 00:03:50.086
I also fell in love and came back for a second internship, actually, and then came back full time.
00:03:50.086 --> 00:03:53.329
So it's almost the only real job that I've ever had.
00:03:54.080 --> 00:04:04.170
I moved around in the company several times, but it always has that same bedrock culture and approach, so it feels different jobs, so it's one job, family, if you will.
00:04:04.170 --> 00:04:07.052
Yeah, so you were there from Gates, he was.
00:04:07.052 --> 00:04:11.969
Yeah, so this was my first internship was in 96, and then I started full time in 1998.
00:04:11.969 --> 00:04:20.007
Amazing, yeah, and so the theme throughout for me, most of my positions have involved insights of one type or another.
00:04:20.007 --> 00:04:34.423
So I started as a developer working on an online learning tool, so trying to bring students' insights back when online learning was not so common as it is today, doing some of these kind of distance universities and corporate training, things like that.
00:04:34.423 --> 00:04:49.036
Then I spent a number of years doing insights for developers in Visual Studio, doing unit test generation and ethics, cop and pre-fast, basically helping you find issues in your software and correct them so that we can all enjoy more reliable software.
00:04:49.800 --> 00:04:54.526
And then I spent about a decade in Bing and Bing ads, most of that time working on the knowledge graph.
00:04:54.526 --> 00:04:58.266
There I used to joke that we were trying to learn everything there was to know about everything.
00:04:58.266 --> 00:05:05.024
So no matter what question you asked, we'd be able to answer Kind of before ChatGPT came along and gave you a natural language way of doing it.
00:05:05.024 --> 00:05:07.805
We were a little bit more structured, but that was a really fun time.
00:05:07.805 --> 00:05:20.567
And then, about two and a half years ago or so, I came over to business applications and thought there's a real opportunity to help people get more insights, be more efficient in their work.
00:05:20.567 --> 00:05:29.045
We all spend so much of our time working and so if we can have some impact on that, I felt it would be a really important thing to do, and I love it.
00:05:29.045 --> 00:05:40.346
I feel like every day I'm getting to hopefully get a little bit less toil in people's day-to-day life and help them get more satisfaction, get more enjoyment, get more done in their work.
00:05:40.346 --> 00:05:41.665
So that's what I do these days.
00:05:42.519 --> 00:05:48.709
So how does that practically hit the technology that we might see or use in our day-to-day?
00:05:49.199 --> 00:06:10.648
Yeah, so our team is called Co-Pilot AI team, so we build the AI that is sitting between the foundation models and all of the applications, whether you're talking about Power Apps, power, automate and many others, or all the Dynamics ones and so the manifestation really is this idea of Co-Pilot, and Microsoft's vision for Co-Pilot is this idea of a real-time collaborator.
00:06:10.648 --> 00:06:15.406
That's not something that you just automate everything and go on vacation for a month.
00:06:15.406 --> 00:06:23.007
It's really about sitting there with you, embedded in your flow, understanding your context and helping you get done whatever task you have.
00:06:23.007 --> 00:06:34.548
And so that manifests different ways for us depending upon whether you're working on an email, working on a sales deal you know will help you write the email or whether you're working on building a Power App.
00:06:34.548 --> 00:06:50.168
Then it will help you either create the table behind that, give the right instructions to GPT to help build that table, or maybe arrange the controls on the screen of the app and edit those as well, and even let you add Co-Pilot to the app that you're building so that your users can also get a Co-Pilot.
00:06:50.639 --> 00:06:57.846
Yeah, so you're saying that this is the intermediary between if I was using Co-Pilot in Power Automate or if I was using it in Power Apps.
00:06:57.846 --> 00:07:02.766
It's the working piece that applies across the Power a suite of tools.
00:07:03.007 --> 00:07:03.369
That's right.
00:07:03.369 --> 00:07:05.706
Yeah, it's not quite as unified as that just yet.
00:07:05.706 --> 00:07:06.583
We're getting there.
00:07:06.583 --> 00:07:15.567
It started out by a lot more pieces, right, okay, let's go and build one thing for Power Automate and now let's do something slightly different for Power Apps.
00:07:15.567 --> 00:07:23.103
But yes, over time those are becoming more unified and kind of a more common layer of abstraction that sits in between the foundation models and the apps themselves.
00:07:24.220 --> 00:07:27.449
So does it have awareness of me in that context?
00:07:27.449 --> 00:07:39.425
And what I'm alluding to here is that, you know, the Office Graph kind of knows what I'm working on, what documents, teams et cetera, knows the calls that I'm having, potentially with customers, things like that.
00:07:39.425 --> 00:07:52.968
Is it feeding that into the awareness or is it only purely based on what you're prompting to do, or is it understanding me and my working behaviors and how I could be more efficient?
00:07:53.360 --> 00:07:56.088
Yeah, it's getting smarter and smarter over time.
00:07:56.088 --> 00:08:05.826
So the first versions were fairly unaware of context, right, you know, think static prompts that give you some pre-canned output.
00:08:05.826 --> 00:08:07.946
And then it got more advanced.
00:08:07.946 --> 00:08:19.449
We started doing things like fine tune model for Power Automate to create a flow right, where we trained that model specifically on many, many examples of flows and descriptions of flows.
00:08:19.449 --> 00:08:26.545
And so now it started to feel a little more contextual, because you're describing what you want and you're getting that flow out.
00:08:26.699 --> 00:08:30.786
But the reality is the underlying model is the same for you as it is for me.
00:08:30.786 --> 00:08:34.769
And then over time we're starting to add more and more context.
00:08:34.769 --> 00:08:45.027
So now, as you go in and you're editing a Power App, looking at what edits you are currently making, what is the app in its current state, and using that to feed into the prompt.
00:08:45.027 --> 00:08:47.506
We also use examples.
00:08:47.506 --> 00:09:15.205
Currently they're not specific examples per customer, but over time they will start to be more specific examples per customer and you can imagine that being at a tenant level or even at a user level, so that those then get fed into the prompt, is not that the underlying foundation model itself is changing to know you, but the business applications are getting to know you by injecting these examples and kind of teaching it and call it in context learning, so that it gets smarter at the time that you make the call.
00:09:16.220 --> 00:09:17.706
Yeah, interesting.
00:09:17.706 --> 00:09:23.048
Tell you, the favorite feature of this for me and I tried it must have been back in March this year.
00:09:23.048 --> 00:09:36.489
I was on a live session with Charles Lamana and he announced of course, this is the MVP, session was a big, the co-pilots you know is coming, and that you could build your own app.
00:09:36.489 --> 00:09:46.013
And so, while he's live talking, I jumped into my machine and I built an app, a Power App, for managing my seed library.
00:09:46.013 --> 00:09:51.307
I plant lots of, I'm into gardening lots of seeds and I've never found a tool that does it well.
00:09:51.307 --> 00:10:02.024
And then, if I put my day job hat on, I mainly in pre-sales right, so I am taking a customer scenarios and wanting to show how they light up on Microsoft Technologies.
00:10:02.024 --> 00:10:20.850
What I loved about it was the ability to build sample data that was on point with what my situation was, and that alone that feature alone, for me and my role and what I've done for the last 20 years in pre-sales, was massive Exactly Massive game changer.
00:10:21.059 --> 00:10:27.826
And I think a lot of people have these stories where it's just that one really surprising, delightful experience that they have with these models.
00:10:27.826 --> 00:10:30.407
Right, they're still imperfect.
00:10:30.407 --> 00:10:39.333
Right, there's a lot of things they can't do, but I think we've all experienced that kind of magical example, not just once, but repeated magical examples that just make them a delight to work with.
00:10:40.360 --> 00:11:01.710
Yeah, I like the metaphor that Mo X Microsofty, who's written a book on AI and whether we're going to have a utopian or dystopian future within the next 40 years, and I like his concept and it really grounds me is that where we are at with AI right now is really an infant child, almost just a toddler right.
00:11:01.710 --> 00:11:17.221
And like I see, last week Sam Altman has started to talk about GTP5, gpt5, right and some of the challenges and stuff around that and like we've had one year GPT 3.5 Turbo and then four and that's just in the first 12 months, right.
00:11:17.241 --> 00:11:19.448
So this baby is really starting to grow.
00:11:19.570 --> 00:11:31.208
right Now, back to what I set off air to you about Orion Cunningham and his question to the audience yesterday around where will you be in three years time?
00:11:31.208 --> 00:11:32.232
Or what will you be doing?
00:11:32.232 --> 00:11:35.927
Or, and I go back to where we are in our journey of AI.
00:11:35.927 --> 00:11:52.528
That's become although you know, we know AI has been around since the 60s odd and maybe even before, but our concept of it now in the context of large language models and really it becoming highly practical, definitely changed the workflow of my day to day in so many areas.
00:11:52.528 --> 00:11:53.731
In fact.
00:11:53.731 --> 00:12:06.227
Sneak thing here about the podcast that's been running for six years and I'm not the best writer of titles for each of my podcast, right they after six years twice a week, it gets pretty generic.
00:12:06.227 --> 00:12:22.135
Since I have now AI read the entire transcript and come up with five potential titles for the show, I've seen over a third increase in traffic to the podcast because of the title rewrites.
00:12:22.596 --> 00:12:34.166
Yeah, and, can you imagine, would have been like to do that just a year or two ago right, you would have had to go and create a custom model, you would have had to go and get a bunch of examples of transcripts and a bunch of examples of labels.
00:12:34.166 --> 00:12:36.664
Like you'd never do it, it was just be way too much work.
00:12:36.664 --> 00:12:49.182
But that's the magic of these models is that they're finally big enough, general purpose enough and easy enough to deploy that you can use them all over the place, right, yeah, and that's the explosion that we're seeing right now and what makes it so exciting to me.
00:12:50.105 --> 00:12:59.278
When you look from your position inside Microsoft, looking out, and you're seeing the data sets that you're analyzing and perhaps the industries.
00:12:59.278 --> 00:13:04.249
Are you seeing any kind of industry and market adopting this quicker than others?
00:13:04.249 --> 00:13:08.892
And particularly thinking what are the use cases that are coming to the surface?
00:13:08.892 --> 00:13:18.466
I love I think there was a stat yesterday that showed it's a 2x improvement on writing power automate, so creating a flow is two times as fast now.
00:13:18.466 --> 00:13:19.952
That's massive, right.
00:13:19.952 --> 00:13:23.792
If you imagine everybody could 2x their day from a productivity perspective.
00:13:23.792 --> 00:13:24.999
What are the things that you're seeing?
00:13:24.999 --> 00:13:25.760
What are the trends?
00:13:26.523 --> 00:13:28.607
Yeah, it's really across the board, I think.
00:13:28.607 --> 00:13:32.264
We see customers from all walks of life who are approaching this.
00:13:32.264 --> 00:13:48.445
I mean we even see some financial services companies that you might think, oh, that's highly regulated, maybe they might take a while to jump in, and they are, of course, more cautious, but they are also finding ways to bring value, let's say, to their own employees.
00:13:48.445 --> 00:13:50.072
So how do they deploy?
00:13:50.072 --> 00:13:58.282
Let's go and use Co-Pilot to create a bot that you used for employee facing work as a way to get started and still get a ton of value out of this.
00:13:59.043 --> 00:14:15.149
We see communications companies also looking at how to improve, kind of get more people coming and building automation, and how Co-Pilot makes it easier for them to do that, and so it's less time, it's more people being able to access these things.
00:14:15.149 --> 00:14:17.844
We see customer service.
00:14:17.844 --> 00:14:33.884
We see our own customer service group getting significant wins in terms of how long it takes to complete a case and, of course, you see, also customer satisfaction goes up there as well, both because of the accuracy but also because, hey, and who wants to wait longer for their case to be resolved?
00:14:33.884 --> 00:14:37.003
Just by getting it faster, the satisfaction also goes up.
00:14:37.003 --> 00:14:41.480
So it's really it's kind of all over the place that we see this taking off.
00:14:42.342 --> 00:15:05.624
How do you your role, because, as somebody outside Microsoft, they feel like I'm drinking from a fire hose right this year, more than any other, you know, how are you maintaining the level of information, continuous learning, that you must be on because of the speed that things are going at?
00:15:06.195 --> 00:15:08.903
Yeah, it is definitely feels like a fire hose, right.
00:15:08.903 --> 00:15:17.438
I mean, I think we're very privileged to sit where we are because we have so many different sources of that information.
00:15:17.438 --> 00:15:24.638
Right, we're learning from our customers every day, talking about what they're trying to accomplish, talking about what they're seeing or hearing from other vendors.
00:15:24.638 --> 00:15:30.320
Obviously, we have the news itself constantly bringing us all kinds of information.
00:15:30.320 --> 00:15:37.777
We have partnerships, and so we hear from our partners on a regular basis what they're thinking about, where they're going and have conversations with them about that.
00:15:37.777 --> 00:15:40.503
We have our own research organization.
00:15:40.503 --> 00:15:42.301
We have our own product teams.
00:15:42.914 --> 00:16:01.823
Obviously, microsoft's a very large company, so there's a lot of different people, and Co-Pilot, I think, is really the only example in the last 25 years that I can remember where it felt like a whole company thing, and Microsoft is probably almost an order of magnitude larger than when I joined, and so the whole company means something different now.
00:16:01.823 --> 00:16:15.423
But almost everybody I run into, almost every team, has some aspect of Co-Pilot that they're working on, and so there's a lot of this kind of cross group exchange of ideas and it feels very collegiate.
00:16:15.423 --> 00:16:23.722
It's all these brown bags and internal events and things like that on top of all of the external ones, and so you just have to pick and choose.
00:16:23.722 --> 00:16:25.822
There's plenty of information out there.
00:16:25.822 --> 00:16:29.081
It's not like, oh, I can't find it, oh, I've never heard of what's going on.
00:16:29.081 --> 00:16:32.921
It's more a question of how do you prioritize and how do you choose where.
00:16:32.941 --> 00:16:58.940
To go a little bit deeper that's an interesting concept of you talking about this very collegiate type of environment, because I've worked with Microsoft for 30 years now in the ecosystem and of course it's not always particularly between product groups had a love on each other type relationship and it's interesting how the spirit of technology doesn't matter with the M365, whether you're in Azure, whether you're in, I assume, even the gaming and the product divisions, r&d, et cetera.
00:16:58.940 --> 00:17:14.541
It really is permeating every part of the business and I assume that the collective knowledge, the cross pollination, right, and so they'll be looking at it from a very specific area and you see that you're like you know what we could use that Are you seeing a lot of that?
00:17:14.541 --> 00:17:18.201
Like I hadn't thought about it like that and therefore we need to bring this to our group.
00:17:18.682 --> 00:17:22.801
Yeah, absolutely, we see that all the time right Learning from what Office is doing.
00:17:22.801 --> 00:17:25.162
In our early days we took a lot of inspiration.
00:17:25.162 --> 00:17:45.400
Early days, a year or two ago, we took a lot of inspiration from what GitHub Copilot had done, because they were sort of the first on the scene with this and I still am inspired by a lot of the work that they did in terms of how they determined what aspects of Copilot made the biggest difference in how much people loved the product.
00:17:46.236 --> 00:17:58.461
They did some really interesting things, comparing qualitative studies or surveys and the quantitative behaviors that those people were doing, so that they could figure out one that matched the other, and so we took inspiration from that.
00:17:58.461 --> 00:18:15.221
But there's a very concerted effort to create these forums across the company, at the very senior levels, but also through things like internal machine learning conferences and other more Copilot specific forums, where you could probably spend a good part of your week attending these things if you had the time.
00:18:15.221 --> 00:18:19.038
Yes, we definitely do get a lot of inspiration out of those forums.
00:18:19.994 --> 00:18:22.442
And what you're doing is hardware slowing you down at all.
00:18:23.454 --> 00:18:24.318
Not too much.
00:18:24.318 --> 00:18:26.661
I would say there's two aspects of that.
00:18:26.661 --> 00:18:30.138
There's one is what's the throughput, and the second one is can you get the hardware?
00:18:30.138 --> 00:18:35.401
Yeah, so far I think our supply chain folks have been challenged for sure.
00:18:35.401 --> 00:18:45.362
But I think that our leadership saw the opportunity early enough that there's been enough hardware to go around and at least in my team, where I am, we've been fortunate enough to have what we need.
00:18:46.275 --> 00:18:50.117
In terms of speed, it's hard to get too caught up in it, for two reasons.
00:18:50.117 --> 00:18:56.063
One is the models just keep on getting faster and faster, and not by 10% or 20%, but by 2% and 3x.
00:18:56.063 --> 00:19:06.499
And the other thing is that when you're saving somebody minutes or possibly hours of time, if the response is a little bit slower, it's not such a big deal.
00:19:06.499 --> 00:19:11.019
I find myself sometimes going and asking Copilot something, and then I'll go switch.
00:19:11.019 --> 00:19:21.525
I'm not the most patient person in the world, so I'll go switch and maybe start reading an email or something and I'll come back, and it's a perfectly reasonable interaction model to me.
00:19:21.525 --> 00:19:28.179
If you'd asked me whether a multi-second delay was acceptable when I was working and being out, I told you you're crazy.
00:19:28.179 --> 00:19:30.740
Nobody's going to come to our site anymore.
00:19:30.740 --> 00:19:46.357
Yes, yes, yes, but here it's a very different thing, because even with those kind of delays, the time it would have taken me to do the equivalent without Copilot was so long that I'm still saving a huge amount of time and I keep on coming back.
00:19:47.171 --> 00:19:57.458
Yeah, amazing what challenges, and I want to put the ethics and responsible AI to ones like I think a lot of people have indexed on that.
00:19:57.458 --> 00:20:18.955
But more from a whether it be speed, whether it be adoption, whether it be technical blockers, what are some of the things that you know are some of the challenges that you're going to need to solution for and I'm alluding slightly to Sam Altman he was talking about there needs to be changes in the hardware and stuff.
00:20:18.955 --> 00:20:20.414
Do five needs to do.
00:20:20.414 --> 00:20:25.301
There's some scientific research.
00:20:25.301 --> 00:20:26.836
It's not there yet.
00:20:26.836 --> 00:20:35.519
Are you thinking about these things in the context of the business applications ecosystem that are like guys, like when we look at backlogs and things?
00:20:35.519 --> 00:20:36.615
This needs to be a priority.
00:20:36.615 --> 00:20:40.099
This will become a roadblocker if we do not solution for it.
00:20:40.810 --> 00:20:46.874
Yeah, I think a lot of the energy is actually going into what I might call a block in tackling.
00:20:46.874 --> 00:20:58.314
It's about how do you get rid of the grounding errors, for example, yes, the default models have still a significant issue with poorly grounded responses, which is not okay.
00:20:58.314 --> 00:21:08.740
If you're creating a unparsable flow or unparsable app or you're giving somebody just the wrong answer and they're relying on that in some financial transaction or something.
00:21:08.740 --> 00:21:25.778
A lot of energy goes into how do you get the right retrieval, augmented generation or other techniques to get the right examples, the right data and then validate into the model and then validate that the output actually matches those examples so that these can be used in a business context.
00:21:25.778 --> 00:21:31.957
If you just go over to a regular model without that grounding, you're going to have some pretty serious issues.
00:21:31.957 --> 00:21:34.898
That's a big part of it that we look into.
00:21:34.898 --> 00:21:43.874
Then, even once you fix that, there's still another layer of I don't know how to describe it, but there's quality issues.
00:21:43.874 --> 00:21:45.655
Right, the response is not quite right.
00:21:45.655 --> 00:22:04.317
Additional work goes into things like how do we rewrite the queries to give us better results or how do we iteratively fix the results that come out and say, okay, this is wrong, please go and fix this, instruct the model to go and fix this aspect or that aspect and kind of iterate until we see a better result.
00:22:04.931 --> 00:22:07.819
We do a lot of experimenting with different ways of prompting.
00:22:07.819 --> 00:22:10.858
There's a great doc that one of our data scientists wrote.
00:22:10.858 --> 00:22:12.031
That's basically.
00:22:12.031 --> 00:22:19.478
It's like the prompting equivalent of coding guidelines People talk about hey, english is the new coding language.
00:22:19.478 --> 00:22:21.316
Well, great, we need some coding guidelines for it.
00:22:21.316 --> 00:22:23.936
How do you word things?
00:22:23.936 --> 00:22:25.655
What order do you put the instructions?
00:22:25.655 --> 00:22:30.401
What kinds of additional guardrail instructions do you add onto it?
00:22:30.401 --> 00:22:32.414
Is it big, important piece as well?
00:22:32.414 --> 00:22:33.417
As we try to push for this?
00:22:33.417 --> 00:22:34.019
Accuracy.
00:22:34.019 --> 00:22:37.440
Accuracy is really the number one thing that we're after.
00:22:37.440 --> 00:22:47.381
It's like accuracy, and not just on simple things, but being able to handle very complex tasks, where you start to give a vague instruction for what you'd like your application to do or something like that.
00:22:49.230 --> 00:22:55.513
I just started playing recently with Dali and the reason is it's now in the GPT-4.
00:22:55.513 --> 00:23:04.561
And I noticed that after it created the image, I then would have an information and actually had fully rewritten my prompt.
00:23:04.561 --> 00:23:04.942
Yep.
00:23:04.942 --> 00:23:12.201
I was like, if we dial back nine months ago, everyone was like, hey, you got to become a prompt engineer.
00:23:12.201 --> 00:23:17.320
I just feel that AI will be the best prompt engineer itself.
00:23:17.320 --> 00:23:29.643
In other words, one of the things I instruct my AI to do is always ask me qualifying questions and never just reply qualify me, qualify me until you're satisfied that you're understanding what I'm asking.
00:23:30.519 --> 00:23:31.568
It's interesting what you say.
00:23:31.568 --> 00:23:37.000
There is really understanding those, then queries behind and how they could be written better and perform better.
00:23:37.000 --> 00:23:43.522
I'm probably asking you to speculate just from an observational perspective, not from your professional perspective.
00:23:43.522 --> 00:23:45.281
But low code, no code.
00:23:46.237 --> 00:24:15.701
If I look forward, when I saw that Ryan Cunningham had that up there, I truly feel that the idea of low code would be gone within five years and it will be no code from the context of how we look at the Power Platform, because I assume we're going to get to the point that I'll be able to give a descriptive instruction, get it qualified and then a very sophisticated app could be built Absolutely, or would it be an app?
00:24:15.701 --> 00:24:21.682
It might suggest that this should be a chat bot or the interfaces, etc.
00:24:21.682 --> 00:24:26.981
There would be a certain choice based on the targeting personas and things like that.
00:24:26.981 --> 00:24:41.027
Do you think that, and what you've seen in 12 months and you extrapolate that out over the next five years three to five do you see that we almost can't predict where we will be there?
00:24:41.027 --> 00:24:44.400
Do you think it'll be that quantum leap different than where we are now?
00:24:45.155 --> 00:25:17.561
I do think it's very hard to predict where we're going to be in the future, but I also agree that with what you said about low-code going away at least I mostly agree with it I think that it is very likely that in that time frame, or even sooner, we'll be able to build fairly complex experiences using just natural language and probably other modalities too right, speech, gesture, images, videos, things like this to communicate, just as I would communicate to you if you were some magician building something for me.
00:25:18.124 --> 00:25:18.263
Yeah.
00:25:18.954 --> 00:25:38.799
I do think that will work, but I think that the other piece that needs to be retained is there needs to be some way for the machine to tell me what it is doing, or show me what it is doing and I don't know whether you call that low-code or not, because I don't think that necessarily showing me a flow diagram with a bunch of technical jargon on it is how I want to see that.
00:25:38.799 --> 00:25:45.123
But nor do I want to listen to one hour lecture that is painstakingly describing to me.
00:25:45.123 --> 00:25:48.423
Okay, this is an app that has five buttons.
00:25:48.423 --> 00:25:57.103
The first button is called like no, just show me, give me a walkthrough, give me like, what a YouTube video on this app would show me about how this app works.
00:25:57.714 --> 00:26:12.579
I think that's maybe the future so-called low-code experience is that you describe how you want and you get a multimodal description back of what it has actually created for you and probably also even some ideas on how you might want to make it better.
00:26:12.579 --> 00:26:19.722
But truly, it's collaborating right, because there's this idea of sparking creativity that is a really important part of co-pilot.
00:26:19.722 --> 00:26:33.317
It's not supposed to be this, just this kind of instruction follow-over, where you tell it exactly what you want, it does just that thing, and then it kind of like dumbly, sits there right, it's supposed to anticipate and spark oh, here's more that you might want to do and give you that idea.
00:26:33.317 --> 00:26:39.901
So you get excited and like, oh yes, that would be great, I want to do that and I want to do this other thing, and you come up with something much better than you would have on your own.
00:26:40.575 --> 00:26:52.523
Brad Smith a couple of years ago, made a 2019 release to book and, of course, long before where we are now, and he said some people liken data to oil.
00:26:52.523 --> 00:26:55.701
Right, and he was like it's wrong, it's more like air.
00:26:55.701 --> 00:26:59.462
Data is more the air we breathe.
00:26:59.462 --> 00:27:08.365
And what's your thinking around the importance of the data estate that you're operating over?
00:27:08.365 --> 00:27:10.119
You talked about grounding before.
00:27:10.119 --> 00:27:20.147
What's the importance of one access to the data that needs to inform decisions to the robustness of that?
00:27:20.147 --> 00:27:21.817
Is there enough?
00:27:21.817 --> 00:27:23.299
Is it the most recent?
00:27:23.299 --> 00:27:24.436
Is it?
00:27:24.436 --> 00:27:33.363
What's your thoughts on the importance of data in an AI future, for the way any kind of business should be considering how they adopt AI.
00:27:33.934 --> 00:27:35.000
I mean, it's absolutely essential.
00:27:35.000 --> 00:27:41.239
Without accurate and complete and up-to-date business data, the AI can only guess.
00:27:41.239 --> 00:27:47.564
Now, it can guess pretty well, maybe it can guess better than a human would, but it's still guessing.
00:27:47.564 --> 00:27:48.798
It's not gonna guess exactly.
00:27:48.798 --> 00:27:50.721
How many widgets do you have on hand?
00:27:50.721 --> 00:27:59.244
How are you going to know that if you don't just go and look it up, right, and so that information being there is really important.
00:27:59.244 --> 00:28:07.554
I think the good news, though, is that AI not only depends on that data, but it can also help you create that data Right.
00:28:07.554 --> 00:28:09.147
So it can help you create like.
00:28:09.167 --> 00:28:14.194
I don't know if you played around with the summary and teams, but now almost every meeting I go to has good meeting notes.
00:28:15.048 --> 00:28:15.912
When was that ever true?
00:28:15.912 --> 00:28:27.328
Like, I mean, there was always that one person in my team who was very diligent and would take good notes, and all the other meetings were just junk right, and now Co-Pilot is actually pretty darn good at it, and so every meeting has notes.
00:28:27.328 --> 00:28:35.513
That is a form of data that is very important to capture, because it can also start pulling out action items, but you look at it in that business context.
00:28:35.513 --> 00:28:44.428
You know we're pulling out contacts and price requests and things like that out of emails and actually get your CRM into good shape right and allow you to.
00:28:44.428 --> 00:28:48.076
You know we can normalize the data much better because it's getting AI created.
00:28:48.076 --> 00:28:51.204
It'll be much more comprehensive because you're not having to type it all in.
00:28:51.204 --> 00:29:06.894
It'll be much more consistent because it does it for you, and so there's a virtuous cycle that is starting to be created where the AI is creating better data, consuming that data and getting better at creating that data, and I think we'll see that flywheel continue to its kind of natural conclusion.
00:29:07.727 --> 00:29:07.928
Yeah.
00:29:08.368 --> 00:29:08.730
I like it.
00:29:08.730 --> 00:29:18.188
In closing, tom, is there any advice I suppose that you could give to kind of three audiences that listen to this show that?
00:29:18.188 --> 00:29:26.244
My three audiences are people using Microsoft business applications, that's the Power Platform as well as the Dynamics 365 Suite.
00:29:26.244 --> 00:29:33.138
They are then consultants that generally working for partners implementing these technologies as part of their career.
00:29:33.138 --> 00:29:38.923
And then the third group, often Microsoft sellers in the field that want to understand more.
00:29:38.923 --> 00:29:41.874
You know they can do it on their commute and things like that.
00:29:41.874 --> 00:29:54.224
With that audience in mind, what kind of recommendations would you give them around how they can keep themselves or what should they be focusing on when it comes to that fire hose?
00:29:54.224 --> 00:30:03.960
You know, because they want to stay current, they want to be, they see a future where they're going to be fully fill in with AI and it's going to be part of their life and they want to adopt it.
00:30:03.960 --> 00:30:05.224
Yeah, what advice.
00:30:06.088 --> 00:30:09.244
Yeah, I mean, I don't think there's any substitute for just using it right.
00:30:09.244 --> 00:30:19.780
So try and figure out how to get your hands on this and make time in your day to go and use it, because it'll help you get past the hype and understand okay great.
00:30:19.780 --> 00:30:36.219
Yes, it does have some flaws here, but it also is really amazing in these cases and it'll allow you to make better decisions when you're talking to your customers or your colleagues and say, hey, I think you should use it in these 10 ways and maybe not in this one or two other ways that are not quite ready yet.
00:30:36.219 --> 00:30:45.743
Because it is, the space is moving so fast that if you're just trying to kind of like stay on the sidelines and watch it go by, you'll miss it.
00:30:45.743 --> 00:30:49.836
You need to jump in, get in the stream, play with it.
00:30:49.957 --> 00:30:58.244
Especially in Power Platform is really easy to get started, but also in Dynamics as well, if you can get yourself an instance and try it out and watch that.
00:30:58.244 --> 00:31:04.244
And then, besides that, we also have, you know we're doing more and more content creation for Microsoft partners.
00:31:04.244 --> 00:31:07.224
A lot of the folks you just listed would have access to that content.
00:31:07.224 --> 00:31:12.596
We have this site akams slash BA, copilot, as in business applications copilot.
00:31:12.596 --> 00:31:15.830
So many demos, presentations, documentation.
00:31:15.830 --> 00:31:24.839
We're constantly putting information up there, so I would watch that as well, but I think there's really no substitute for just trying it, using it day to day yourself.
00:31:24.859 --> 00:31:26.224
Hey thanks for listening.
00:31:26.224 --> 00:31:31.717
I'm your host business application MVP, Mark Smith, otherwise known as the NZ365 guy.
00:31:31.717 --> 00:31:36.715
If there's a guest you'd like to see on the show from Microsoft, please message me on LinkedIn.
00:31:36.715 --> 00:31:40.592
If you want to be a supporter of the show, please check out buymeacoffeecom.
00:31:40.592 --> 00:31:46.374
How will you create on the Power Platform today?
00:31:46.374 --> 00:31:47.566
Ciao?