00:00:02Power Virtual Agents and AI
00:12:08Democratization of Data Science and AI
00:25:49Understanding Azure OpenAI and Generative Answers
00:35:13Challenges of Using Bing for Answers
00:40:14Exciting Updates and Innovations Await
درباره این اپیزود
- Ever wondered about the vast universe of Power Virtual Agents and AI? Brace yourselves for an enlightening conversation with our guest, Dewain Robinson, straight from Nashville, Tennessee, the Principal Program Manager for Power, Virtual Agents, and Conversational AI at Microsoft. We dive into the intricate workings of Power Virtual Agents and how Azure OpenAI service can revolutionize data accessibility by creating an easy-to-navigate knowledge base.
- Our discourse traverses the democratization of data science and AI, revealing how co-pilot is opening new doors for people without coding backgrounds, and how large language models can extract knowledge from data. We navigate through the multifaceted world of Azure OpenAI, its significance and the necessity of recognizing the loopholes when training a model. Dwayne also shares insights on how Azure OpenAI in Microsoft Teams can make data access more efficient.
- As we advance, we tackle the challenges of using large language models and search engine optimization to help customers identify data issues. The importance of starting with public data before using internal data is emphasized, alongside the benefits of publishing content on a web page. We wind up with a sneak peek into the upcoming innovations with Azure Cognitive Services and their potential to create more powerful virtual agents and conversation AI. Prepare to be amazed by the technological advances that are just around the corner!
OTHER RESOURCES:
Power Platform/Power Users: https://powerusers.microsoft.com/t5/user/viewprofilepage/user-id/399912
GitHub: https://github.com/Dewain27
Power Platform/Power Users: https://powerusers.microsoft.com/t5/user/viewprofilepage/user-id/399912
GitHub: https://github.com/Dewain27
If you want to get in touch with me, you can message me here on Linkedin.
Thanks for listening 🚀 - Mark Smith
انگلیسی
ایالات متحده آمریکا
یادداشت ها را نشان دهید 🔗
رونوشت 🔗
WEBVTT
00:00:02.726 --> 00:00:04.551
Welcome to the Power 365 show.
00:00:04.551 --> 00:00:10.689
We're an interview staff at Microsoft across the Power Platform and Dynamics 365 Technology Stack.
00:00:10.689 --> 00:00:16.687
I hope you'll find this podcast educational and inspire you to do more with this great technology.
00:00:16.687 --> 00:00:19.126
Now let's get on with the show.
00:00:19.126 --> 00:00:23.966
Today's guest is from Nashville, tennessee in the United States.
00:00:23.966 --> 00:00:29.689
He works at Microsoft as a principal program manager for Power, virtual Agents and conversational AI.
00:00:29.689 --> 00:00:36.289
He's been working with conversational AI for eight years and with Microsoft Technologies in IT for over 25 years.
00:00:36.289 --> 00:00:40.085
He combined links to his bio, social media etc.
00:00:40.085 --> 00:00:49.670
In the show notes for the episode as well, as we'll discuss a couple of other topics and we'll put links in the show notes so you can jump straight to what we refer to in this episode.
00:00:49.670 --> 00:00:50.725
Welcome to the show, Dewain.
00:00:51.161 --> 00:00:51.582
Thanks, Mark.
00:00:51.582 --> 00:00:52.567
Thanks for having me today.
00:00:52.799 --> 00:00:53.645
Good to have you on the show.
00:00:53.645 --> 00:00:59.308
I always like to bridge the personal with the professional to kick off an episode.
00:00:59.308 --> 00:01:02.128
So just to start with food, family and fun.
00:01:02.128 --> 00:01:06.406
What do they mean to you, all those things, when you're not focused on your day job?
00:01:07.820 --> 00:01:16.347
Yeah, so I got married, have two kids, one 16-year-old and one 13-year-old, and one of them is a field goal kicker.
00:01:16.347 --> 00:01:21.165
It's being professionally trained and it's actually pretty good at that.
00:01:21.165 --> 00:01:29.271
Then a daughter who just won a national championship on in cheering for competitive cheerleading.
00:01:29.271 --> 00:01:36.990
And then, as far as food is concerned, unfortunately it's a little bit of a too heavy of a sin for me.
00:01:36.990 --> 00:01:45.989
I like a lot of Southern US type of foods, which is generally bad for me but it's enjoyable on the back end.
00:01:45.989 --> 00:01:52.527
But outside of that, I do a lot of DJing and stage and lighting design.
00:01:52.527 --> 00:01:58.868
I actually even do that for a friend of mine who used to be the lead singer of Creed, Scott Stapp.
00:01:58.868 --> 00:02:04.427
So I do a lot of work with him and his charity as well to do stuff like that.
00:02:04.427 --> 00:02:06.721
So lots of stuff Everybody, I think.
00:02:06.721 --> 00:02:21.824
This week, Mark, I even learned how to stream from a drone through an live stream so that my son's football games I can stream it out over the internet and do the cool stuff that you do.
00:02:22.620 --> 00:02:23.002
Wicked?
00:02:23.002 --> 00:02:24.907
Are we talking about a DJI drone?
00:02:26.000 --> 00:02:27.445
I am talking about a DJI drone.
00:02:28.039 --> 00:02:28.360
Awesome.
00:02:28.360 --> 00:02:30.449
Can you see that in the background there on mine?
00:02:31.800 --> 00:02:32.783
Oh, yes, you.
00:02:32.783 --> 00:02:37.725
In my opinion, I had to buy that same controller that you have in order to be able to live stream.
00:02:38.360 --> 00:02:41.643
Yeah, so the funny thing is I virtually never used that controller.
00:02:41.643 --> 00:02:45.967
I tend to use the one that I plug my phone into rather than the one with the onboard screen.
00:02:45.967 --> 00:02:48.927
But, yeah, okay, so that's awesome.
00:02:48.927 --> 00:03:05.289
And the other thing that caught my attention in what you said there is lighting design, and, for Parsons, in my life, I've been involved in that area myself as well, and isn't it just an amazing area and what you can do with software and lighting these days.
00:03:06.111 --> 00:03:06.693
Oh, it's crazy.
00:03:06.693 --> 00:03:30.828
I actually have like two of the X lasers that you have to have FDA variances for, so I can do like atmospheric or laser shows on the side, and so being able to learn how to do that and then like computers coming into DMX control and all of the it's just it's a super amazing topic and who knows, Mark, maybe we'll end up having a podcast conversation about that one day.
00:03:31.659 --> 00:03:58.693
It's so cool, so cool and it's just, it's such a creative area, I feel, because it can bring you know concerts, events to life, like yeah, I mean the last, and I'm probably gonna get some flak for this the last big concert I went to was Taylor Swift with my wife, and what I couldn't get over, just the amount of theatrics now involved in a concert.
00:03:58.693 --> 00:04:08.902
You know it's everything from in the audience having bands on your arm, that light up, or you know things that as you enter at the gates that they give you to.
00:04:08.902 --> 00:04:21.646
I can't even remember how it was done because that was actually another concert I was at in Sydney as well and just the whole thing that plays in the audience plays into the theatrics now of big stage events.
00:04:21.646 --> 00:04:24.226
It's pretty phenomenal where tech has enabled this.
00:04:25.300 --> 00:04:26.103
Oh yeah, totally.
00:04:26.103 --> 00:04:35.190
Could you imagine, as we kind of start, as we talk about like conversational, could you imagine just programming lights by just telling it the mood that you wanted?
00:04:35.653 --> 00:04:37.122
Yeah, amazing, right and things like that.
00:04:37.122 --> 00:04:38.788
We're not far from that right.
00:04:38.788 --> 00:04:40.264
Yeah, yeah, totally.
00:04:40.899 --> 00:04:54.627
Now we're gonna talk today a bit about Power Virtual Agents and AI, and I hope I don't come across as too excitable about this, because I honestly think this is the biggest opportunity in the power platform space bar none.
00:04:55.079 --> 00:04:58.887
I mean co-pilots awesome, don't get me wrong and where that's ultimately gonna play out.
00:05:00.360 --> 00:05:22.749
You know I'm loving the co-pilot, particularly in Power Automate and the signs of what's gonna be possible from app development, but when I think of organizations and the amount of data that they have tied up inside them, that isn't so data silos, et cetera, that they obviously can't in many situations.
00:05:22.879 --> 00:05:29.665
Most a lot of companies in the enterprise space have banned their staff using chat GPT right, the public accessible.
00:05:29.704 --> 00:06:08.045
Because why there's been already a bunch of use cases around people leaking confidential data out to train those models and, of course, microsoft and why I just wanna dwell on this a second, because it took me awhile to get my head round it, even back at MVP Summit, because I couldn't figure how you could hand your data to chat GPT and, if you're like, not transfer across to a public system, until I understood, of course, the whole concept of Azure OpenAI service, which means the large language model sits in your Azure tenant and does your bidding on your data as you make it available.
00:06:08.045 --> 00:06:20.622
Why this excites me is because really anybody can bring their data inside their Azure tenant and then, with the power virtual agent interface, they have a way of tapping into current data, historic data.
00:06:20.622 --> 00:06:26.848
The knowledge of that organization becomes available in a fully secure, trusted manner.
00:06:26.848 --> 00:06:33.723
So, as I said, hopefully I don't get too excited about this, but I just think this is the most amazing area of tech at the moment.
00:06:34.505 --> 00:06:37.788
Yeah, totally, I think, the shocking part of this.
00:06:37.959 --> 00:07:04.035
So, mark, you gotta realize I've been in this whole conversation in the AI space with bot framework and everything, and it's just been Kind of like it is going along and people were interested and you have these implementations here and there, but the just I think it was around Christmas time this past year when this just exploded and then we double down in the power of agents team.
00:07:04.860 --> 00:07:21.475
I remember the day that my leader came in and he's like no, this is the thing, like we have to get on this, and everybody Just went heads down to make sure that we could really deliver some really interesting experiences that were really differentiating in the market space.
00:07:22.261 --> 00:07:37.435
And this is where you saw the other generative actions, the co-pilots, the Generative answers type of feature sets coming into the product and how can we make these things work together and such.
00:07:37.435 --> 00:07:52.225
And you just saw this massive increase in the in that direction and over time, it's just really started to make a lot of sense why, why conversational applications are we gonna become really a thing of the future?
00:07:52.225 --> 00:08:00.336
And we just didn't think it was gonna come quite as quick as I was still worried about, like in L U and how is the best way to train the NLU.
00:08:00.336 --> 00:08:09.692
And now we're at a point where I would say, in twelve to eighteen months you probably would never train an NLU ever again because of this type of technology.
00:08:09.692 --> 00:08:15.389
So it's just an interesting innovation, you know, and so quick, that's just the craziest part.
00:08:16.543 --> 00:08:18.870
NLU, just for the audience meaning.
00:08:19.701 --> 00:08:20.964
Natural language understanding.
00:08:20.964 --> 00:08:23.752
For the excellent brilliant, brilliant.
00:08:24.841 --> 00:08:28.865
Tell me what does what in your words.
00:08:28.865 --> 00:08:31.870
You know you, cuz you're living and breathing this every day.
00:08:31.870 --> 00:08:35.654
You know for the last eight years, but particularly in what's happened since Christmas.
00:08:35.654 --> 00:08:38.504
As you said, power virtual agents and I.
00:08:38.504 --> 00:08:40.528
How do you explain it to people?
00:08:40.528 --> 00:08:41.990
How do you get them excited?
00:08:41.990 --> 00:08:42.913
What are your thoughts?
00:08:43.559 --> 00:08:45.543
Yeah, that's a interesting challenge.
00:08:45.543 --> 00:08:51.975
So you know, if you're talking to just the general layman, right, you kind of have to explain to them.
00:08:51.975 --> 00:08:54.745
There's a smart speaker you talk to the Thomas.
00:08:54.745 --> 00:09:10.953
People think of conversational I or they think of Imagine, go to a website, you have a chat experience or something like that, and so what people will tell you is, yeah, I hate those things, right, or are they're either really passionate about the use of all the time.
00:09:10.953 --> 00:09:15.908
What you you basically have to get people to start to understand is the concept.
00:09:16.109 --> 00:09:31.043
Imagine if that smart speaker that sat on your Desk had access to knowledge and information everyone's experience issue, ask a question to your phone's assistant or anything of that nature and you don't get.
00:09:31.043 --> 00:09:34.527
It really isn't program to do that thing.
00:09:34.527 --> 00:09:41.216
So what you end up with is you end up with a result that sub par, it's like, falls back to search or something like that.
00:09:41.216 --> 00:09:48.370
What we're starting to see now is the ability for those things to really actually understand the words you're saying.
00:09:48.370 --> 00:09:52.075
Before we're doing like keyword matching and things of this nature.
00:09:52.075 --> 00:10:05.216
Now it actually understands what you said and can understand the context of a conversation, but then it can also consume data in the same way and look for relevancy and be able to match those things up.
00:10:05.216 --> 00:10:06.942
I want you do that.
00:10:06.942 --> 00:10:15.754
You start unlocking the power of data and the natural language in a way that's never been done before, and so that's really where it becomes.
00:10:16.441 --> 00:10:19.488
The massive innovation is your teaching.
00:10:19.488 --> 00:10:25.847
You're telling the computer what to do In natural language and it truly understands what you mean.
00:10:25.847 --> 00:10:30.844
And it also truly understands the data and the things that are available in the back end.
00:10:30.844 --> 00:10:52.744
And when you start thinking like, oh, well, then you can even explain to it in natural language what an API does right or an application programming interface for those that don't know, like, how to Submit a form for something to a back end or help you find the best you know product that it meets your needs.
00:10:52.744 --> 00:11:01.596
You know you can pull that out of the words and pull it from the data and then map it and automatically get answers.
00:11:01.596 --> 00:11:05.385
The things just like is if you were talking to a live person.
00:11:05.385 --> 00:11:07.009
It's really innovative.
00:11:07.009 --> 00:11:16.248
And then you start getting into like, oh, and I can make it say it in the way you want, make it talk a travel agent or something like that.
00:11:16.248 --> 00:11:18.010
So it's really interesting.
00:11:18.510 --> 00:11:25.388
I think we're just at the tip of an iceberg Playing with the stuff right now, and I think we will get.
00:11:25.388 --> 00:11:47.573
It would just get better and better over time, because I think part of the problem is that the language models are in major major league Innovative right now, but it's the surrounding tech around it that makes it safe and makes it usable For the average person, because right now, what you see is a lot of people going back and forth and trying to figure out how to apply it.
00:11:47.573 --> 00:11:55.328
I think this will normalize itself and commoditize itself over time, and that's going to be where and that's what we're doing right.
00:11:55.328 --> 00:12:06.234
We're trying to take it and make it commoditize for use for the average person and unlock that power without having to have a team of data scientists sitting on your own staff.
00:12:06.234 --> 00:12:06.514
Right?
00:12:06.514 --> 00:12:07.534
That's kind of the key.
00:12:08.861 --> 00:12:23.972
Yeah, and it's interesting that you say that about data scientists, because I was always in the past, up to December January, the that for an organization to be massively successful, it's going to need data science as scientists on their team.
00:12:24.013 --> 00:12:35.989
And and dating back to who twenty fifteen, twenty sixteen, the organization work for, we had data scientists and yet they never saw the breakthroughs that we're seeing now.
00:12:36.089 --> 00:13:10.030
But also what I'm meaning is that when that was kind of a wow, that is a phenomenal skillset and and and out of the reach, now with prompt engineering and things like that, a lot of lay people are starting to get high value responses from data is just talking to somebody you say another Microsofty and they were saying that a person needed to build a connector to the power platform and to load a whole bunch of data and, based on the size of it, that was the preferred method of a power automate been the method.
00:13:10.572 --> 00:13:11.813
This person didn't know how to code.
00:13:11.813 --> 00:13:26.595
They use co pilot, you know, and your studio got the got it, got it set up and in ten hours, build a full working API connector and was able to load the sixty thousand records that being quoted by a partner.
00:13:26.595 --> 00:13:38.975
Fifty hours to do that and it's kind of like it's it's boarded into the realms of people that had the will to do it but didn't have the tech training, background resources to be able to make that a reality.
00:13:38.975 --> 00:13:45.423
And I think that's the you know the game changing kind of situation that we're seeing in a lot of organizations.
00:13:45.423 --> 00:13:48.924
It's, it is democratization right, it's making it available to everybody.
00:13:49.946 --> 00:14:03.049
Yeah, for those that aren't familiar with how to do a connector or whatever, you go, take a look at this, and I actually have a set of videos that I've provided Mark and it'll be in the show notes.
00:14:03.049 --> 00:14:11.557
There's like a I think I'll bake it down to like 15 minutes teach you how to build a custom connector to consume any API that you want.
00:14:11.557 --> 00:14:29.256
So if it's not already there or you have, like this custom endpoint that you want to bring in, it's super, super key, and if you start really thinking about it and, mark, I'm going to blow your mind for a second right, because we will get to talking about generative actions as we go into this conversation.
00:14:29.256 --> 00:14:33.916
And for those that don't know, generative action is kind of like lane chain.
00:14:33.916 --> 00:14:43.057
If you want to go look up lane chain or things of that nature, what you're going to see is how is a plug-in and a connector really different in this whole world?
00:14:43.057 --> 00:14:47.936
So people don't realize that I've been building connectors for the Power Platform forever.
00:14:47.936 --> 00:14:55.131
You've been basically defining an API and you're writing descriptions on what it is and you're explaining in natural language what it does.
00:14:55.131 --> 00:15:01.376
So what keeps you from using a large language model to be able to understand how to use it.
00:15:02.184 --> 00:15:08.592
And so when you start getting to that world, it really starts blowing people's minds like the stuff we showed at Build around generative actions.
00:15:08.592 --> 00:15:10.873
They're like, how in the world is it possible?
00:15:10.873 --> 00:15:26.889
And it's like when you have over a thousand connectors in a library that say what they do and you just simply explain what it does and plug it into your thing and allow it and give it how to connect to the service, it can just figure it out.
00:15:26.889 --> 00:15:31.623
And that's really it's almost scary in some ways.
00:15:31.623 --> 00:15:36.804
Mark right, it's, but I will say that we're not 100% there yet.
00:15:36.804 --> 00:15:47.796
I feel like there's a lot of, like I said, the wrapper around this stuff that you got to get down to be able to make sure that it does it with a high level of accuracy at the orchestration layer.
00:15:47.796 --> 00:15:50.711
But it's going to, it's not.
00:15:50.711 --> 00:15:52.096
It's not far away, right?
00:15:52.096 --> 00:16:00.255
I think a lot of people think, oh, that's six years down the road and I'm like, no, I think that's like six months down the road type of thing.
00:16:01.065 --> 00:16:13.083
Five to 10 years ago I would go around Australia and I would speak at events for Microsoft on big data the four mega trends that we're going to transform, and big data was one of them.
00:16:13.083 --> 00:16:23.335
Right, and I felt that what we have seen is organizations get really good at collecting data, but they haven't got really good at making it available.
00:16:23.335 --> 00:16:25.288
So they collect it, collect it.
00:16:25.288 --> 00:16:51.451
You know, things like GDPR et cetera have gone and put some governance around it, but they've not really good at making all these data silos across their data estate available for consumption, to grow a more informed staff to do the role of whatever they're doing inside that organization, and so I feel like the payback for all that data collection is really AI is going to make it possible to turn it into something practical.
00:16:52.524 --> 00:16:54.950
Yeah, I think there's still a.
00:16:54.950 --> 00:17:05.357
There's still a piece, and this is one of the things that's as we start working with customers in trying to unlock the knowledge out of the data.
00:17:05.357 --> 00:17:20.771
Right, you have to keep in mind that a large language model, for example, can understand the data, can understand what you asked about the data and can read tons of data and come back and give you an answer, summarize to offer the information.
00:17:20.771 --> 00:17:24.815
But I'll give you a good example of where there's still a challenge in the world, right?
00:17:24.815 --> 00:17:29.660
So there's this for those that don't know, there's this feature called generative answers.
00:17:29.660 --> 00:17:32.210
That's inside of Power Virtual Agents and what it is is.
00:17:32.309 --> 00:17:35.724
It's basically what we refer to as search and summarization.
00:17:35.724 --> 00:17:47.217
So it searches across data that you pointed at and then it understands the question that you asked and without you having to have data scientists to make sure that it doesn't cuss at you or say something it shouldn't say.
00:17:47.217 --> 00:18:01.025
What it does is it will search and summarize the answers over that data set that you have, that you pointed it at, and if that data set happens to be stored in a Microsoft place, we figure out where's the best place to search to get that content.
00:18:01.025 --> 00:18:02.086
So, if it's a website.
00:18:02.086 --> 00:18:04.873
It will use Bing and the Prometheus engine to share a point.
00:18:04.873 --> 00:18:07.317
It will use the graph, things like this, right?
00:18:07.964 --> 00:18:17.993
So what we found, though, is we went and pointed it at all of the different financial reports from Microsoft, and then we asked it who's the CEO of Microsoft?
00:18:17.993 --> 00:18:23.496
The problem is is that the answer you get would be different.
00:18:23.496 --> 00:18:39.036
Sometimes it would say Sacha Nadella, sometimes it would say Bill Gates, sometimes it would say Steve Ballmer right, and the reason is because I think where we're going to start seeing people to really unlock their data, they're going to have to start thinking data equivalent to code, in a way.
00:18:39.036 --> 00:18:54.076
So the same way you think about you have data and you develop the data for your website and you go through a publishing process with the data and the content, to say this is the authoritative answer this is that article that says how to do this or this?
00:18:54.116 --> 00:18:55.843
is that knowledge-based update?
00:18:55.843 --> 00:19:05.290
Right, you might have versions of it, but what you want to present to the model and to the responder or the response system?
00:19:05.290 --> 00:19:20.327
Right, because you want to present the authoritative answer data, right, you don't want all the drafts, you don't want all the old stuff, and so what we're seeing is people having to realize that they need to think about their data publishing process and how they're doing it.
00:19:20.327 --> 00:19:31.704
Like pointing it at a SharePoint library that you have everybody's making a copy of the same file and coming up with different answers to the question will just get you inconsistent results.
00:19:31.704 --> 00:19:33.284
It will be more results.
00:19:33.284 --> 00:19:36.492
So it's kind of like if you put garbage in, you'll get garbage out.
00:19:36.492 --> 00:19:37.655
Everybody always says that.
00:19:38.144 --> 00:19:41.232
So now what we're seeing is people, especially with PVA.
00:19:41.232 --> 00:19:46.613
What our hope is is that people will stop trying to build models to do search and summarization.
00:19:46.613 --> 00:19:56.144
Just use the commodity for that and focus on data quality to the model, and that's where you will get higher quality results out of it.
00:19:56.144 --> 00:20:15.194
It's actually not necessarily in the prompt engineering, which is where everybody seems to want to play right now, but that stuff's going to get commoditized pretty darn quick, so I don't think you really need to play in that space too much just to get knowledge summarization and things from content.
00:20:15.444 --> 00:20:20.224
That alone is a phenomenal cost savings to organizations.
00:20:20.305 --> 00:20:25.096
I'll give you a use case working with a bank, a large bank, across Australia, new Zealand.
00:20:25.884 --> 00:20:40.769
They have a call center, internal call center that costs them between three and four million per annum to run and its job is at a bank teller if they have a customer in front of them and they need to know the 90-day fixed term rate.
00:20:41.721 --> 00:20:59.880
Now when they type that into their data repository, like you said with the example of the who's, the CEO of Microsoft, they get 300 responses of that fixed rate could have been from five years ago, right, and what they want is the.
00:20:59.880 --> 00:21:15.670
So what they're doing at the moment is that that teller, it's calling up somebody in the internal contact center getting them to search and filter down and give them it's 17.9 or 5.3, right, percent, because that's today's term deposit rate.
00:21:15.670 --> 00:21:18.903
And of course we're going hey, this is.
00:21:18.903 --> 00:21:34.169
You know that teller could easily, with a chat experience on the device, key in what was requested and it gives the answer based on the location, the date of today, etc.
00:21:34.169 --> 00:21:41.461
And provide the most relevant answer with links to if there's a document that needs to be handed out with it, etc.
00:21:41.461 --> 00:21:43.106
Disclosure statement, that type of thing.
00:21:43.106 --> 00:21:48.204
If that can be solved, that's a massive savings right that's to the business.
00:21:48.907 --> 00:22:05.932
Yeah, totally, and see, and that's the difference between so I'll put it this way, so I'll give you a great example, mark, where we see people getting really confused is the difference between knowledge extraction, or what we call generative answers, and generative actions.
00:22:05.932 --> 00:22:06.983
And why are they different?
00:22:06.983 --> 00:22:07.425
Right?
00:22:07.425 --> 00:22:17.265
Yes, generative answers is the fact of I have a whole bunch of knowledge that is in documents and things like that, and that information.
00:22:17.265 --> 00:22:30.868
I compare this, if I talk about it in, like a cruise line example right, whether or not Jack Daniels is included in the premium drink package or not, it's generally something that's in an article somewhere.
00:22:30.868 --> 00:22:33.929
That's something you're going to extract the data off of that.
00:22:33.929 --> 00:22:44.871
But if I want to know what cruises leave out of a certain port within a certain price range, that's probably not something you're going to pull from content.
00:22:44.871 --> 00:22:52.229
People confuse that in a website because they go well, no, that's on the website, but it's not stored in the website as an article or something.
00:22:52.229 --> 00:22:55.608
Right, it's actually an API that's being called on the back yet.
00:22:56.220 --> 00:23:12.346
So, in the case for which you were talking about, this is where generative actions would play, because you would have an API that understands what are the pieces of information I need to be able to go get the answer to that question with that rate is, and then I will return it.
00:23:12.346 --> 00:23:31.009
And so the cool thing, what we're going to see happen and this is going to be happening very soon is the ability for you to be able to just plug in that connector to that API in the Power Platform and just say if someone asks this question, here's the API to get it from.
00:23:31.009 --> 00:23:40.903
And if they provide the information, if they say what is the rate in Australia, then it will automatically fill in the API to say I need the Australia one.
00:23:40.903 --> 00:23:47.508
If you don't say that, then what it will say is well, what country are you looking for the rate for?
00:23:47.508 --> 00:23:49.406
And then you'll fill it in.
00:23:49.406 --> 00:23:58.124
So, whatever you don't provide as required information in that connector, it would generate a question to get the answer from the user and then provide it.
00:23:59.343 --> 00:24:33.008
But when it starts getting really crazy is when you get into the planner, and the planner is the ability to say, oh, the information you gave me through this conversation, I already know who you are because you authenticated, so I can go to your profile through a different API to get what location you're from and provide that location in so that I don't have to ask you that question and that's the difference between the planner and the connector component of it, and you'll see generative actions.
00:24:33.008 --> 00:24:53.904
One of the things that we're looking at is breaking it into those two different major components, where there's one component, that's the planner, that helps orchestrate it and figure out a plan to answer the question, and another one, which is how does it consume an API or a plug-in just to be able to make it, where you don't have to author all of the different ways that somebody might interact with that API.
00:24:53.904 --> 00:24:57.308
So it's kind of really you start getting into this.
00:24:57.308 --> 00:24:59.787
It's really starts to make your head hurt in a way.
00:25:01.161 --> 00:25:04.487
And it makes you want to get your hands on and really start experiences.
00:25:04.487 --> 00:25:12.810
Now, one of the resources we're going to provide in the show notes was to a blog post that was written by Sarah Critchley on August, the 7th, 2023.
00:25:12.810 --> 00:25:30.627
Do you want to talk a bit about what was announced in that and really what the impact it is for people that really want to get going now on Power Virtual Agents, get up their skill level, because this is really, I see as a practical way to use AI generative AI in your organization.
00:25:31.000 --> 00:25:49.066
Yeah, so there's a lot of confusion in two different key areas that it's going on with people in this, and when we get into, like when we're talking about extracting information out of knowledge, which was generative answers right, a lot of people kept coming to us and saying we're totally confused.
00:25:49.066 --> 00:25:55.146
Generative answers is this thing inside of PVA that will answer stuff over your data.
00:25:55.146 --> 00:26:00.905
Then I have Azure Open AI on your data what is that?
00:26:00.905 --> 00:26:06.152
And then you have Azure Open AI which is what?
00:26:06.152 --> 00:26:08.469
And so I had to start liking this.
00:26:08.469 --> 00:26:11.461
I like to use analogies that people can relate to.
00:26:11.461 --> 00:26:14.104
So Azure Open AI is like buying a brick.
00:26:14.104 --> 00:26:20.178
Azure OpenAI on your data is like buying a wall.
00:26:20.178 --> 00:26:26.289
Then, when you look at generative answers inside of PVA, it's like buying a house.
00:26:26.289 --> 00:26:28.811
What do I mean by that?
00:26:28.811 --> 00:26:56.471
If you look at it, you say, well, generative answers actually includes an OpenAI implementation, an Azure OpenAI implementation that's trained by data scientists and a whole bunch of services to do query optimization, moderation and make sure that you can ask a question, you can point it at a set of data and it will respond to you with an answer, with reference links and things like that.
00:26:56.471 --> 00:27:02.172
You don't need to know anything about that, except here's the data I want to give it.
00:27:02.172 --> 00:27:05.934
I provided the question that someone asked.
00:27:05.934 --> 00:27:10.489
I provided where the data is, that I want the answer done and it just handles it.
00:27:10.489 --> 00:27:15.070
But what if your data, so this thing, mark?
00:27:15.070 --> 00:27:19.375
What about if my data is stored across multiple locations?
00:27:19.375 --> 00:27:31.073
And maybe my data it's stored inside a PDF and I need to be able to vectorize, search over it to be able to improve the search results that are being given back?
00:27:31.073 --> 00:27:41.955
Well, that's what you get with Azure OpenAI over your data, because now you are in control of the index or the query itself that is going in.
00:27:41.955 --> 00:27:43.750
But you still didn't have to.
00:27:43.750 --> 00:27:52.513
You're still taking advantage of all of that tuning that's been done on the Azure OpenAI model without having to hire the data scientists to go do that.
00:27:52.585 --> 00:28:05.173
You're not managing prompts at this point, and if you're not familiar with prompts, for people who don't know what that is, it's simply think of it as the words in natural language that explain to it.
00:28:05.173 --> 00:28:08.875
You should never do this and you should do that.
00:28:08.875 --> 00:28:11.333
This is where we get into Azure OpenAI.
00:28:11.333 --> 00:28:20.394
Azure OpenAI is an implementation where you're in control of the prompts and there's two levels of prompts that you have to worry about.
00:28:20.394 --> 00:28:24.415
Think of it like if you've ever heard of the laws of robotics.
00:28:25.006 --> 00:28:30.230
There's a base set of rules that you want to give, give that can never be broken.
00:28:30.230 --> 00:28:33.532
Or it's like, hey, these are the things that govern the system.
00:28:33.532 --> 00:28:41.471
Then there are ones that come in behind the scenes, that are part of the query itself, that allow it to be able to do it.
00:28:41.471 --> 00:29:06.653
If you wanted to make it talk, when you say, hey, make me a poem about Tom Brady in the style of old English, the way that it's doing that is, it's actually in the prompt, in the query that you're sending, that the fact that you want a poem, the fact that you want it about Tom Brady, the fact that you want it in the style of old English, is actually part of the prompt that you're passing in as the query.
00:29:06.653 --> 00:29:16.657
But the base training of the model is don't allow someone to ask a question that results in insulting them or cursing at them.
00:29:17.038 --> 00:29:17.278
Yes.
00:29:17.665 --> 00:29:18.711
Things of this nature.
00:29:18.711 --> 00:29:36.449
If you look at the Bing thing that happened inside of Microsoft, where people were talking to the bot for hours and hours and then they were able to get it to do stuff that it shouldn't do, this is an example of the worry that you have to have as a data scientist.
00:29:36.449 --> 00:29:42.970
When you are doing Azure Open AI by itself, those prompts and things like that you have to come up with.
00:29:42.970 --> 00:29:43.686
It's almost.
00:29:43.686 --> 00:29:44.469
I explain this.
00:29:44.469 --> 00:29:49.294
A bunch of my friends are lawyers and I said a data scientist has to think like a lawyer.
00:29:49.294 --> 00:29:54.876
What are all the loopholes that someone might find in the training set?
00:29:54.876 --> 00:30:01.230
One of the number one things you need to do is tell it never to divulge its training, which is one of the things that we learned.
00:30:01.230 --> 00:30:05.993
So it's just stuff like this that it helps people understand.
00:30:06.595 --> 00:30:08.238
How much control do you want?
00:30:08.424 --> 00:30:41.154
But with each control level that you release as you move down that stack from generative answers and down, while you're gaining control, you're also gaining the need for you to really know what you're doing, because if you index content that people don't have rights to and you expose it externally and allow people to ask questions over it that was your decision on the Azure Open AI over your data Because you own the index, you own the control.
00:30:42.067 --> 00:30:46.796
So these type of things you just have to make sure that you understand, and I think that's the biggest risk mark.
00:30:46.796 --> 00:30:57.470
I think a lot of people find it because they can do stuff really easy and really quickly, but they don't necessarily think about what all do you need to do and a lot of people don't understand that.
00:30:57.470 --> 00:31:14.237
The reason why it takes a little while for us to get stuff out in PVA, for example, is we spend a lot of time with our applied AI team and our responsible AI team to make sure that what we're doing is safe, because we want to make sure that it's safe.
00:31:14.237 --> 00:31:18.496
It's not just because it's cool and you can do it and it's awesome in a demo.
00:31:18.496 --> 00:31:22.035
It takes a lot to make sure that it's good enough to go out in the public.
00:31:22.464 --> 00:31:30.372
One of the game changes I've seen is in the area of SAP, and a great percentage of business customers globally are using Microsoft Teams right now.
00:31:30.372 --> 00:31:43.674
These big enterprises will often have data locked up in SAP systems and you never find a general person inside an organization that says, oh man, I love working in SAP.
00:31:43.674 --> 00:31:47.772
Brilliant, I could spend my entire life.
00:31:47.772 --> 00:32:05.617
Most people are like, oh my gosh, it takes so much effort to get to the information that I need, and one of the use cases that are becoming up consistently is that you put Power Virtual Agents inside Teams and you can go, hey, are there any outstanding shipments for this week?
00:32:05.617 --> 00:32:07.128
And it can go and query that.
00:32:07.128 --> 00:32:08.272
Come back with an answer.
00:32:08.272 --> 00:32:11.952
That would have been 15 clicks in the past.
00:32:11.952 --> 00:32:16.614
Within the SAP experience, people are really stuck, but you give them that data model.
00:32:16.614 --> 00:32:22.712
One of the things you're not wanting to do is go hey, you have access to Workday and now I can query people's salaries inside the business.
00:32:23.204 --> 00:32:37.131
You're making sure that that kind of protection is in place is so important and that the what data is accessible, because a lot of companies have, of course, confidential data that's even confidential internally.
00:32:37.131 --> 00:32:40.513
It's not that you're an employee and therefore you've accessed everything.
00:32:40.513 --> 00:32:43.132
There's need to know in many organizations.
00:32:43.132 --> 00:32:46.894
How do you see companies getting that bit right?
00:32:46.894 --> 00:33:04.819
Is it going to still need the data scientists inside the organization that says, listen, even though we're giving it access to this data set, we've got to make sure it can't access this portion here, because that's confidential and it should be based on someone's security role as to whether they can query.
00:33:04.819 --> 00:33:06.000
That's it there.
00:33:06.000 --> 00:33:11.997
So that whole identifying who they are is important to the questions that they can ask.
00:33:13.910 --> 00:33:47.845
Yeah, it's an interesting challenge because, by the way I was talking about before, I actually see this function coming before too much longer within organizations that are going to need to be able to think about, and you already sort of have it today in the form of data architecture, and so what I anticipate is that this is going to become the instrumental component that organizations have to go make sure they get right before they can really extract the value out of large language models, and so it's interesting.
00:33:48.426 --> 00:34:01.994
It's like I said before people are really worried about how they go do prompt engineering and how they go do all this stuff, but what you're going to see is every vendor in the world is already figuring out how to do that over there on where they store their data.
00:34:01.994 --> 00:34:18.068
The question becomes is how are you going to publish that data in a secure way and in an authoritative way, and what are the key triggers to be able to make sure that the system knows One?
00:34:18.068 --> 00:34:26.110
Do you have rights to ask this question over that data, like you can ask a question, but I could also come back and say I'm sorry, I can't answer that for you.
00:34:26.351 --> 00:34:26.793
Yeah, yeah, yeah.
00:34:27.505 --> 00:34:33.434
Maybe I say that to one person in the organization, but I don't say it to the CEO because he has access right.
00:34:33.434 --> 00:34:33.855
Yes, yes.
00:34:33.855 --> 00:35:12.791
So when you start looking at things like this data access, data quality, data governance all of these are going to be massive places that people are really going to need help, and I think, if I look across like our partners and things of that nature, or you start looking at organizations, they're so busy worrying about how to get the response off the data that they, most of the time, what we've found is they can't go forward because they realize very quickly that the data that they have access to hasn't been managed properly and therefore they've got a data quality issue.
00:35:13.414 --> 00:35:13.815
Exactly.
00:35:14.425 --> 00:35:15.469
And I will tell you that.
00:35:15.469 --> 00:35:15.989
Here's a.
00:35:15.989 --> 00:35:17.253
I'll give you a funny story, mark.
00:35:17.253 --> 00:35:26.577
So we launched generative answers with PVA and the first feature that we did is we launched it with the Bing Prometheus engine.
00:35:26.577 --> 00:35:36.516
So if you pointed out a website and what we ended up having was we had a bunch of customers come back to us and they said, hey, why did you answer this question?
00:35:36.516 --> 00:35:37.650
There's a problem here.
00:35:37.650 --> 00:35:39.932
You shouldn't be able to answer that question.
00:35:39.932 --> 00:35:43.394
And how did you get that reference link to where that is?
00:35:43.394 --> 00:35:47.009
That's not supposed to be where people can see it, and you know what?
00:35:47.048 --> 00:35:47.329
it was.
00:35:47.329 --> 00:35:50.092
It was the fact that they put it out onto the internet.
00:35:50.092 --> 00:35:54.773
They never did anything with the SEO or search engine optimization.
00:35:54.773 --> 00:35:59.092
Yes, yes, they didn't say don't allow a search engine to crawl this thing, don't crawl it.
00:35:59.605 --> 00:36:07.050
And they ended up finding all kinds of things that they had basically put in obscurity, thinking that it wasn't going to get crawled by search engines.
00:36:07.050 --> 00:36:24.050
But the search engines found it and therefore it could answer the question, and so we ended up turning it around and saying, hey, we just helped you figure out where you have data issues, so it's really funny that you will find every data issue you have ever had in your life by using a large language model.
00:36:25.085 --> 00:36:28.028
It's pretty interesting and that's why I tell people start with.
00:36:28.028 --> 00:36:36.954
I think a lot of people think, oh, I just pointed out my entire SharePoint or my entire workday and that's probably not the right first step.
00:36:36.954 --> 00:36:39.228
The first step is crawl, Give it a few things.
00:36:39.228 --> 00:36:44.476
Maybe give it a few product manuals or something like that or something of that nature.
00:36:44.476 --> 00:36:49.876
And also it's also interesting that people seem to want to start with internal data.
00:36:49.876 --> 00:36:52.793
But is that not the scariest place mark?
00:36:52.793 --> 00:37:09.813
I would actually start with public data and work my way to private data, because it's so much easier to take something that's public and just say here's the quality of it, Make sure it's pointing at the right quality stuff, versus also trying to figure out data access rates and the risk that comes along with that.
00:37:09.813 --> 00:37:11.369
It's very interesting problems.
00:37:12.445 --> 00:37:14.992
For a public data set is what it's doing.
00:37:14.992 --> 00:37:22.813
Is it basically crawling all the links on that public data set or do you need an RSS feed in that public data set?
00:37:22.813 --> 00:37:29.496
What's its method of gathering the body of knowledge before it starts answering?
00:37:30.016 --> 00:37:38.554
Yeah, so in most cases, if you're using the Bing Prometheus engine, what it's doing is it's using the index off of Bing.
00:37:38.554 --> 00:37:57.213
So when Bing goes through and does its stuff, but when it finds results, it goes out and grabs the actual HTTP page and pulls that content in so that it gets the latest version of that information, and then that's what it does for the search and summarization action.
00:37:57.213 --> 00:38:05.773
But one of the things that a lot of people don't know, mark, is that you can actually use Bing with a custom query.
00:38:05.773 --> 00:38:22.753
What that means is, for those that don't know, you can go to Bing and there's a way to say I want to build my own custom query, and so whenever you pass it in and you say here's the URL, you pass in also this unique identifier as part of it and the result will be is it will scope the data down to what you want.
00:38:22.753 --> 00:38:26.601
So that's a way for you to be able to kind of control the data set.
00:38:26.601 --> 00:38:28.047
That's kind of being in return with Bing.
00:38:28.585 --> 00:38:34.650
Otherwise, you have to start thinking about the indexer and where is the index content and what's the ability.
00:38:34.650 --> 00:38:37.894
So, like a lot of people think well, I've got a PDF on my website.
00:38:37.894 --> 00:38:42.972
Well, bing doesn't, doesn't crack PDFs, so you've got to come up with how do you index that.
00:38:42.972 --> 00:38:52.976
And then if you have a document and let's say it's 150 pages, you can't afford to run that back in through the tokens that you would have to buy for a large language model.
00:38:52.976 --> 00:39:02.153
So that's where you start getting into vector based search being pretty important so that you can basically chunk the file.
00:39:02.153 --> 00:39:09.034
So maybe I only want to return a paragraph within a document, not the entire content of the document whenever I return it.
00:39:09.034 --> 00:39:19.679
So these are kind of the challenges that you start to run into and I think this is the part that when people first get into this, they're like, oh, it's going to be super cool and I'm going to be able to do it.
00:39:19.679 --> 00:39:23.811
And then you go oh, I got to get search right, I got to figure out how to index that content.
00:39:23.811 --> 00:39:27.173
Once you get that down, the results will blow your mind.
00:39:27.605 --> 00:39:30.894
But it's really, it's really kind of the key thing.
00:39:30.894 --> 00:39:36.170
So depends on the kind of where your data is and how you want to access it and what it, how is it stored.
00:39:36.170 --> 00:39:37.931
But most people for public data.
00:39:37.931 --> 00:39:43.632
I tell them the best way is to put it directly so you don't have to pay for your own indexing.
00:39:43.632 --> 00:39:45.590
The engines do a great job.
00:39:45.590 --> 00:39:58.237
So take your content and publish it in an ASVX page or an HTML page with the content, and as long as you don't make it noncrawlable, then the systems will do the work for you.
00:39:58.237 --> 00:40:00.291
You don't have to necessarily do it.
00:40:00.291 --> 00:40:03.112
You just have to make sure the content's up to date and it's right.
00:40:03.112 --> 00:40:04.708
Rest of it solves itself.
00:40:05.329 --> 00:40:05.931
That's awesome.
00:40:05.931 --> 00:40:07.730
We're already at time.
00:40:07.730 --> 00:40:08.708
Well, well past time.
00:40:08.708 --> 00:40:12.090
We could talk for ages on this.
00:40:12.090 --> 00:40:13.519
Anything you want to say in closing?
00:40:13.945 --> 00:40:14.646
No, I think.
00:40:14.646 --> 00:40:19.195
In general, for those that are interested in this, definitely stay tuned.
00:40:19.195 --> 00:40:19.918
You can go out.
00:40:19.918 --> 00:40:24.155
If you want to play with a trial, you can go to akams slash trypva.
00:40:24.155 --> 00:40:28.215
You can also follow me on YouTube.
00:40:28.215 --> 00:40:34.853
I think it's DeWayne 27 and I spelled DeWayne really weird, so make sure you look at the show notes on how it's spelled.
00:40:34.853 --> 00:40:46.135
Also, if you follow me on LinkedIn or join up on LinkedIn, what I try to do is make sure that the things that are relevant are published for people so that they can see.
00:40:46.746 --> 00:40:49.034
I try to focus on educational type of things.
00:40:49.034 --> 00:40:51.090
I'm sorry, just product announcements.
00:40:51.090 --> 00:41:05.775
The blog is a good place to get the product announcements and the show notes as well, but in general, get in there, get your hands dirty, start playing with this and enjoy the innovation that's there.
00:41:05.775 --> 00:41:16.467
Just be aware that my key guidance to people is don't try to be the wave ride, the wave of innovation.
00:41:16.467 --> 00:41:21.255
Let the big players commoditize where they're going to commoditize.
00:41:21.364 --> 00:41:31.326
Make sure you understand where they're going and just make sure that what you're doing is you're not trying to duplicate them or jump far ahead, because the reason why you might get ahead in the short terms you're going to cut some corners.
00:41:31.326 --> 00:41:39.338
It's likely to cost you in quality or some sort of hard lesson you're going to learn.
00:41:39.338 --> 00:41:43.295
So I would just say in general stay tuned.
00:41:43.295 --> 00:41:49.835
We've got a lot of cool stuff coming and, mark, you don't even want to know what all we're going to announce in the coming months.
00:41:49.835 --> 00:41:51.422
It's going to blow your mind even more.
00:41:51.422 --> 00:41:55.885
So please go to the Power Platform conference, if you can, or show up at Ignite.
00:41:55.885 --> 00:41:59.606
There will be plenty of new cool stuff that we're going to talk about as we get closer.
00:00:02.726 --> 00:00:04.551
Welcome to the Power 365 show.
00:00:04.551 --> 00:00:10.689
We're an interview staff at Microsoft across the Power Platform and Dynamics 365 Technology Stack.
00:00:10.689 --> 00:00:16.687
I hope you'll find this podcast educational and inspire you to do more with this great technology.
00:00:16.687 --> 00:00:19.126
Now let's get on with the show.
00:00:19.126 --> 00:00:23.966
Today's guest is from Nashville, tennessee in the United States.
00:00:23.966 --> 00:00:29.689
He works at Microsoft as a principal program manager for Power, virtual Agents and conversational AI.
00:00:29.689 --> 00:00:36.289
He's been working with conversational AI for eight years and with Microsoft Technologies in IT for over 25 years.
00:00:36.289 --> 00:00:40.085
He combined links to his bio, social media etc.
00:00:40.085 --> 00:00:49.670
In the show notes for the episode as well, as we'll discuss a couple of other topics and we'll put links in the show notes so you can jump straight to what we refer to in this episode.
00:00:49.670 --> 00:00:50.725
Welcome to the show, Dewain.
00:00:51.161 --> 00:00:51.582
Thanks, Mark.
00:00:51.582 --> 00:00:52.567
Thanks for having me today.
00:00:52.799 --> 00:00:53.645
Good to have you on the show.
00:00:53.645 --> 00:00:59.308
I always like to bridge the personal with the professional to kick off an episode.
00:00:59.308 --> 00:01:02.128
So just to start with food, family and fun.
00:01:02.128 --> 00:01:06.406
What do they mean to you, all those things, when you're not focused on your day job?
00:01:07.820 --> 00:01:16.347
Yeah, so I got married, have two kids, one 16-year-old and one 13-year-old, and one of them is a field goal kicker.
00:01:16.347 --> 00:01:21.165
It's being professionally trained and it's actually pretty good at that.
00:01:21.165 --> 00:01:29.271
Then a daughter who just won a national championship on in cheering for competitive cheerleading.
00:01:29.271 --> 00:01:36.990
And then, as far as food is concerned, unfortunately it's a little bit of a too heavy of a sin for me.
00:01:36.990 --> 00:01:45.989
I like a lot of Southern US type of foods, which is generally bad for me but it's enjoyable on the back end.
00:01:45.989 --> 00:01:52.527
But outside of that, I do a lot of DJing and stage and lighting design.
00:01:52.527 --> 00:01:58.868
I actually even do that for a friend of mine who used to be the lead singer of Creed, Scott Stapp.
00:01:58.868 --> 00:02:04.427
So I do a lot of work with him and his charity as well to do stuff like that.
00:02:04.427 --> 00:02:06.721
So lots of stuff Everybody, I think.
00:02:06.721 --> 00:02:21.824
This week, Mark, I even learned how to stream from a drone through an live stream so that my son's football games I can stream it out over the internet and do the cool stuff that you do.
00:02:22.620 --> 00:02:23.002
Wicked?
00:02:23.002 --> 00:02:24.907
Are we talking about a DJI drone?
00:02:26.000 --> 00:02:27.445
I am talking about a DJI drone.
00:02:28.039 --> 00:02:28.360
Awesome.
00:02:28.360 --> 00:02:30.449
Can you see that in the background there on mine?
00:02:31.800 --> 00:02:32.783
Oh, yes, you.
00:02:32.783 --> 00:02:37.725
In my opinion, I had to buy that same controller that you have in order to be able to live stream.
00:02:38.360 --> 00:02:41.643
Yeah, so the funny thing is I virtually never used that controller.
00:02:41.643 --> 00:02:45.967
I tend to use the one that I plug my phone into rather than the one with the onboard screen.
00:02:45.967 --> 00:02:48.927
But, yeah, okay, so that's awesome.
00:02:48.927 --> 00:03:05.289
And the other thing that caught my attention in what you said there is lighting design, and, for Parsons, in my life, I've been involved in that area myself as well, and isn't it just an amazing area and what you can do with software and lighting these days.
00:03:06.111 --> 00:03:06.693
Oh, it's crazy.
00:03:06.693 --> 00:03:30.828
I actually have like two of the X lasers that you have to have FDA variances for, so I can do like atmospheric or laser shows on the side, and so being able to learn how to do that and then like computers coming into DMX control and all of the it's just it's a super amazing topic and who knows, Mark, maybe we'll end up having a podcast conversation about that one day.
00:03:31.659 --> 00:03:58.693
It's so cool, so cool and it's just, it's such a creative area, I feel, because it can bring you know concerts, events to life, like yeah, I mean the last, and I'm probably gonna get some flak for this the last big concert I went to was Taylor Swift with my wife, and what I couldn't get over, just the amount of theatrics now involved in a concert.
00:03:58.693 --> 00:04:08.902
You know it's everything from in the audience having bands on your arm, that light up, or you know things that as you enter at the gates that they give you to.
00:04:08.902 --> 00:04:21.646
I can't even remember how it was done because that was actually another concert I was at in Sydney as well and just the whole thing that plays in the audience plays into the theatrics now of big stage events.
00:04:21.646 --> 00:04:24.226
It's pretty phenomenal where tech has enabled this.
00:04:25.300 --> 00:04:26.103
Oh yeah, totally.
00:04:26.103 --> 00:04:35.190
Could you imagine, as we kind of start, as we talk about like conversational, could you imagine just programming lights by just telling it the mood that you wanted?
00:04:35.653 --> 00:04:37.122
Yeah, amazing, right and things like that.
00:04:37.122 --> 00:04:38.788
We're not far from that right.
00:04:38.788 --> 00:04:40.264
Yeah, yeah, totally.
00:04:40.899 --> 00:04:54.627
Now we're gonna talk today a bit about Power Virtual Agents and AI, and I hope I don't come across as too excitable about this, because I honestly think this is the biggest opportunity in the power platform space bar none.
00:04:55.079 --> 00:04:58.887
I mean co-pilots awesome, don't get me wrong and where that's ultimately gonna play out.
00:05:00.360 --> 00:05:22.749
You know I'm loving the co-pilot, particularly in Power Automate and the signs of what's gonna be possible from app development, but when I think of organizations and the amount of data that they have tied up inside them, that isn't so data silos, et cetera, that they obviously can't in many situations.
00:05:22.879 --> 00:05:29.665
Most a lot of companies in the enterprise space have banned their staff using chat GPT right, the public accessible.
00:05:29.704 --> 00:06:08.045
Because why there's been already a bunch of use cases around people leaking confidential data out to train those models and, of course, microsoft and why I just wanna dwell on this a second, because it took me awhile to get my head round it, even back at MVP Summit, because I couldn't figure how you could hand your data to chat GPT and, if you're like, not transfer across to a public system, until I understood, of course, the whole concept of Azure OpenAI service, which means the large language model sits in your Azure tenant and does your bidding on your data as you make it available.
00:06:08.045 --> 00:06:20.622
Why this excites me is because really anybody can bring their data inside their Azure tenant and then, with the power virtual agent interface, they have a way of tapping into current data, historic data.
00:06:20.622 --> 00:06:26.848
The knowledge of that organization becomes available in a fully secure, trusted manner.
00:06:26.848 --> 00:06:33.723
So, as I said, hopefully I don't get too excited about this, but I just think this is the most amazing area of tech at the moment.
00:06:34.505 --> 00:06:37.788
Yeah, totally, I think, the shocking part of this.
00:06:37.959 --> 00:07:04.035
So, mark, you gotta realize I've been in this whole conversation in the AI space with bot framework and everything, and it's just been Kind of like it is going along and people were interested and you have these implementations here and there, but the just I think it was around Christmas time this past year when this just exploded and then we double down in the power of agents team.
00:07:04.860 --> 00:07:21.475
I remember the day that my leader came in and he's like no, this is the thing, like we have to get on this, and everybody Just went heads down to make sure that we could really deliver some really interesting experiences that were really differentiating in the market space.
00:07:22.261 --> 00:07:37.435
And this is where you saw the other generative actions, the co-pilots, the Generative answers type of feature sets coming into the product and how can we make these things work together and such.
00:07:37.435 --> 00:07:52.225
And you just saw this massive increase in the in that direction and over time, it's just really started to make a lot of sense why, why conversational applications are we gonna become really a thing of the future?
00:07:52.225 --> 00:08:00.336
And we just didn't think it was gonna come quite as quick as I was still worried about, like in L U and how is the best way to train the NLU.
00:08:00.336 --> 00:08:09.692
And now we're at a point where I would say, in twelve to eighteen months you probably would never train an NLU ever again because of this type of technology.
00:08:09.692 --> 00:08:15.389
So it's just an interesting innovation, you know, and so quick, that's just the craziest part.
00:08:16.543 --> 00:08:18.870
NLU, just for the audience meaning.
00:08:19.701 --> 00:08:20.964
Natural language understanding.
00:08:20.964 --> 00:08:23.752
For the excellent brilliant, brilliant.
00:08:24.841 --> 00:08:28.865
Tell me what does what in your words.
00:08:28.865 --> 00:08:31.870
You know you, cuz you're living and breathing this every day.
00:08:31.870 --> 00:08:35.654
You know for the last eight years, but particularly in what's happened since Christmas.
00:08:35.654 --> 00:08:38.504
As you said, power virtual agents and I.
00:08:38.504 --> 00:08:40.528
How do you explain it to people?
00:08:40.528 --> 00:08:41.990
How do you get them excited?
00:08:41.990 --> 00:08:42.913
What are your thoughts?
00:08:43.559 --> 00:08:45.543
Yeah, that's a interesting challenge.
00:08:45.543 --> 00:08:51.975
So you know, if you're talking to just the general layman, right, you kind of have to explain to them.
00:08:51.975 --> 00:08:54.745
There's a smart speaker you talk to the Thomas.
00:08:54.745 --> 00:09:10.953
People think of conversational I or they think of Imagine, go to a website, you have a chat experience or something like that, and so what people will tell you is, yeah, I hate those things, right, or are they're either really passionate about the use of all the time.
00:09:10.953 --> 00:09:15.908
What you you basically have to get people to start to understand is the concept.
00:09:16.109 --> 00:09:31.043
Imagine if that smart speaker that sat on your Desk had access to knowledge and information everyone's experience issue, ask a question to your phone's assistant or anything of that nature and you don't get.
00:09:31.043 --> 00:09:34.527
It really isn't program to do that thing.
00:09:34.527 --> 00:09:41.216
So what you end up with is you end up with a result that sub par, it's like, falls back to search or something like that.
00:09:41.216 --> 00:09:48.370
What we're starting to see now is the ability for those things to really actually understand the words you're saying.
00:09:48.370 --> 00:09:52.075
Before we're doing like keyword matching and things of this nature.
00:09:52.075 --> 00:10:05.216
Now it actually understands what you said and can understand the context of a conversation, but then it can also consume data in the same way and look for relevancy and be able to match those things up.
00:10:05.216 --> 00:10:06.942
I want you do that.
00:10:06.942 --> 00:10:15.754
You start unlocking the power of data and the natural language in a way that's never been done before, and so that's really where it becomes.
00:10:16.441 --> 00:10:19.488
The massive innovation is your teaching.
00:10:19.488 --> 00:10:25.847
You're telling the computer what to do In natural language and it truly understands what you mean.
00:10:25.847 --> 00:10:30.844
And it also truly understands the data and the things that are available in the back end.
00:10:30.844 --> 00:10:52.744
And when you start thinking like, oh, well, then you can even explain to it in natural language what an API does right or an application programming interface for those that don't know, like, how to Submit a form for something to a back end or help you find the best you know product that it meets your needs.
00:10:52.744 --> 00:11:01.596
You know you can pull that out of the words and pull it from the data and then map it and automatically get answers.
00:11:01.596 --> 00:11:05.385
The things just like is if you were talking to a live person.
00:11:05.385 --> 00:11:07.009
It's really innovative.
00:11:07.009 --> 00:11:16.248
And then you start getting into like, oh, and I can make it say it in the way you want, make it talk a travel agent or something like that.
00:11:16.248 --> 00:11:18.010
So it's really interesting.
00:11:18.510 --> 00:11:25.388
I think we're just at the tip of an iceberg Playing with the stuff right now, and I think we will get.
00:11:25.388 --> 00:11:47.573
It would just get better and better over time, because I think part of the problem is that the language models are in major major league Innovative right now, but it's the surrounding tech around it that makes it safe and makes it usable For the average person, because right now, what you see is a lot of people going back and forth and trying to figure out how to apply it.
00:11:47.573 --> 00:11:55.328
I think this will normalize itself and commoditize itself over time, and that's going to be where and that's what we're doing right.
00:11:55.328 --> 00:12:06.234
We're trying to take it and make it commoditize for use for the average person and unlock that power without having to have a team of data scientists sitting on your own staff.
00:12:06.234 --> 00:12:06.514
Right?
00:12:06.514 --> 00:12:07.534
That's kind of the key.
00:12:08.861 --> 00:12:23.972
Yeah, and it's interesting that you say that about data scientists, because I was always in the past, up to December January, the that for an organization to be massively successful, it's going to need data science as scientists on their team.
00:12:24.013 --> 00:12:35.989
And and dating back to who twenty fifteen, twenty sixteen, the organization work for, we had data scientists and yet they never saw the breakthroughs that we're seeing now.
00:12:36.089 --> 00:13:10.030
But also what I'm meaning is that when that was kind of a wow, that is a phenomenal skillset and and and out of the reach, now with prompt engineering and things like that, a lot of lay people are starting to get high value responses from data is just talking to somebody you say another Microsofty and they were saying that a person needed to build a connector to the power platform and to load a whole bunch of data and, based on the size of it, that was the preferred method of a power automate been the method.
00:13:10.572 --> 00:13:11.813
This person didn't know how to code.
00:13:11.813 --> 00:13:26.595
They use co pilot, you know, and your studio got the got it, got it set up and in ten hours, build a full working API connector and was able to load the sixty thousand records that being quoted by a partner.
00:13:26.595 --> 00:13:38.975
Fifty hours to do that and it's kind of like it's it's boarded into the realms of people that had the will to do it but didn't have the tech training, background resources to be able to make that a reality.
00:13:38.975 --> 00:13:45.423
And I think that's the you know the game changing kind of situation that we're seeing in a lot of organizations.
00:13:45.423 --> 00:13:48.924
It's, it is democratization right, it's making it available to everybody.
00:13:49.946 --> 00:14:03.049
Yeah, for those that aren't familiar with how to do a connector or whatever, you go, take a look at this, and I actually have a set of videos that I've provided Mark and it'll be in the show notes.
00:14:03.049 --> 00:14:11.557
There's like a I think I'll bake it down to like 15 minutes teach you how to build a custom connector to consume any API that you want.
00:14:11.557 --> 00:14:29.256
So if it's not already there or you have, like this custom endpoint that you want to bring in, it's super, super key, and if you start really thinking about it and, mark, I'm going to blow your mind for a second right, because we will get to talking about generative actions as we go into this conversation.
00:14:29.256 --> 00:14:33.916
And for those that don't know, generative action is kind of like lane chain.
00:14:33.916 --> 00:14:43.057
If you want to go look up lane chain or things of that nature, what you're going to see is how is a plug-in and a connector really different in this whole world?
00:14:43.057 --> 00:14:47.936
So people don't realize that I've been building connectors for the Power Platform forever.
00:14:47.936 --> 00:14:55.131
You've been basically defining an API and you're writing descriptions on what it is and you're explaining in natural language what it does.
00:14:55.131 --> 00:15:01.376
So what keeps you from using a large language model to be able to understand how to use it.
00:15:02.184 --> 00:15:08.592
And so when you start getting to that world, it really starts blowing people's minds like the stuff we showed at Build around generative actions.
00:15:08.592 --> 00:15:10.873
They're like, how in the world is it possible?
00:15:10.873 --> 00:15:26.889
And it's like when you have over a thousand connectors in a library that say what they do and you just simply explain what it does and plug it into your thing and allow it and give it how to connect to the service, it can just figure it out.
00:15:26.889 --> 00:15:31.623
And that's really it's almost scary in some ways.
00:15:31.623 --> 00:15:36.804
Mark right, it's, but I will say that we're not 100% there yet.
00:15:36.804 --> 00:15:47.796
I feel like there's a lot of, like I said, the wrapper around this stuff that you got to get down to be able to make sure that it does it with a high level of accuracy at the orchestration layer.
00:15:47.796 --> 00:15:50.711
But it's going to, it's not.
00:15:50.711 --> 00:15:52.096
It's not far away, right?
00:15:52.096 --> 00:16:00.255
I think a lot of people think, oh, that's six years down the road and I'm like, no, I think that's like six months down the road type of thing.
00:16:01.065 --> 00:16:13.083
Five to 10 years ago I would go around Australia and I would speak at events for Microsoft on big data the four mega trends that we're going to transform, and big data was one of them.
00:16:13.083 --> 00:16:23.335
Right, and I felt that what we have seen is organizations get really good at collecting data, but they haven't got really good at making it available.
00:16:23.335 --> 00:16:25.288
So they collect it, collect it.
00:16:25.288 --> 00:16:51.451
You know, things like GDPR et cetera have gone and put some governance around it, but they've not really good at making all these data silos across their data estate available for consumption, to grow a more informed staff to do the role of whatever they're doing inside that organization, and so I feel like the payback for all that data collection is really AI is going to make it possible to turn it into something practical.
00:16:52.524 --> 00:16:54.950
Yeah, I think there's still a.
00:16:54.950 --> 00:17:05.357
There's still a piece, and this is one of the things that's as we start working with customers in trying to unlock the knowledge out of the data.
00:17:05.357 --> 00:17:20.771
Right, you have to keep in mind that a large language model, for example, can understand the data, can understand what you asked about the data and can read tons of data and come back and give you an answer, summarize to offer the information.
00:17:20.771 --> 00:17:24.815
But I'll give you a good example of where there's still a challenge in the world, right?
00:17:24.815 --> 00:17:29.660
So there's this for those that don't know, there's this feature called generative answers.
00:17:29.660 --> 00:17:32.210
That's inside of Power Virtual Agents and what it is is.
00:17:32.309 --> 00:17:35.724
It's basically what we refer to as search and summarization.
00:17:35.724 --> 00:17:47.217
So it searches across data that you pointed at and then it understands the question that you asked and without you having to have data scientists to make sure that it doesn't cuss at you or say something it shouldn't say.
00:17:47.217 --> 00:18:01.025
What it does is it will search and summarize the answers over that data set that you have, that you pointed it at, and if that data set happens to be stored in a Microsoft place, we figure out where's the best place to search to get that content.
00:18:01.025 --> 00:18:02.086
So, if it's a website.
00:18:02.086 --> 00:18:04.873
It will use Bing and the Prometheus engine to share a point.
00:18:04.873 --> 00:18:07.317
It will use the graph, things like this, right?
00:18:07.964 --> 00:18:17.993
So what we found, though, is we went and pointed it at all of the different financial reports from Microsoft, and then we asked it who's the CEO of Microsoft?
00:18:17.993 --> 00:18:23.496
The problem is is that the answer you get would be different.
00:18:23.496 --> 00:18:39.036
Sometimes it would say Sacha Nadella, sometimes it would say Bill Gates, sometimes it would say Steve Ballmer right, and the reason is because I think where we're going to start seeing people to really unlock their data, they're going to have to start thinking data equivalent to code, in a way.
00:18:39.036 --> 00:18:54.076
So the same way you think about you have data and you develop the data for your website and you go through a publishing process with the data and the content, to say this is the authoritative answer this is that article that says how to do this or this?
00:18:54.116 --> 00:18:55.843
is that knowledge-based update?
00:18:55.843 --> 00:19:05.290
Right, you might have versions of it, but what you want to present to the model and to the responder or the response system?
00:19:05.290 --> 00:19:20.327
Right, because you want to present the authoritative answer data, right, you don't want all the drafts, you don't want all the old stuff, and so what we're seeing is people having to realize that they need to think about their data publishing process and how they're doing it.
00:19:20.327 --> 00:19:31.704
Like pointing it at a SharePoint library that you have everybody's making a copy of the same file and coming up with different answers to the question will just get you inconsistent results.
00:19:31.704 --> 00:19:33.284
It will be more results.
00:19:33.284 --> 00:19:36.492
So it's kind of like if you put garbage in, you'll get garbage out.
00:19:36.492 --> 00:19:37.655
Everybody always says that.
00:19:38.144 --> 00:19:41.232
So now what we're seeing is people, especially with PVA.
00:19:41.232 --> 00:19:46.613
What our hope is is that people will stop trying to build models to do search and summarization.
00:19:46.613 --> 00:19:56.144
Just use the commodity for that and focus on data quality to the model, and that's where you will get higher quality results out of it.
00:19:56.144 --> 00:20:15.194
It's actually not necessarily in the prompt engineering, which is where everybody seems to want to play right now, but that stuff's going to get commoditized pretty darn quick, so I don't think you really need to play in that space too much just to get knowledge summarization and things from content.
00:20:15.444 --> 00:20:20.224
That alone is a phenomenal cost savings to organizations.
00:20:20.305 --> 00:20:25.096
I'll give you a use case working with a bank, a large bank, across Australia, new Zealand.
00:20:25.884 --> 00:20:40.769
They have a call center, internal call center that costs them between three and four million per annum to run and its job is at a bank teller if they have a customer in front of them and they need to know the 90-day fixed term rate.
00:20:41.721 --> 00:20:59.880
Now when they type that into their data repository, like you said with the example of the who's, the CEO of Microsoft, they get 300 responses of that fixed rate could have been from five years ago, right, and what they want is the.
00:20:59.880 --> 00:21:15.670
So what they're doing at the moment is that that teller, it's calling up somebody in the internal contact center getting them to search and filter down and give them it's 17.9 or 5.3, right, percent, because that's today's term deposit rate.
00:21:15.670 --> 00:21:18.903
And of course we're going hey, this is.
00:21:18.903 --> 00:21:34.169
You know that teller could easily, with a chat experience on the device, key in what was requested and it gives the answer based on the location, the date of today, etc.
00:21:34.169 --> 00:21:41.461
And provide the most relevant answer with links to if there's a document that needs to be handed out with it, etc.
00:21:41.461 --> 00:21:43.106
Disclosure statement, that type of thing.
00:21:43.106 --> 00:21:48.204
If that can be solved, that's a massive savings right that's to the business.
00:21:48.907 --> 00:22:05.932
Yeah, totally, and see, and that's the difference between so I'll put it this way, so I'll give you a great example, mark, where we see people getting really confused is the difference between knowledge extraction, or what we call generative answers, and generative actions.
00:22:05.932 --> 00:22:06.983
And why are they different?
00:22:06.983 --> 00:22:07.425
Right?
00:22:07.425 --> 00:22:17.265
Yes, generative answers is the fact of I have a whole bunch of knowledge that is in documents and things like that, and that information.
00:22:17.265 --> 00:22:30.868
I compare this, if I talk about it in, like a cruise line example right, whether or not Jack Daniels is included in the premium drink package or not, it's generally something that's in an article somewhere.
00:22:30.868 --> 00:22:33.929
That's something you're going to extract the data off of that.
00:22:33.929 --> 00:22:44.871
But if I want to know what cruises leave out of a certain port within a certain price range, that's probably not something you're going to pull from content.
00:22:44.871 --> 00:22:52.229
People confuse that in a website because they go well, no, that's on the website, but it's not stored in the website as an article or something.
00:22:52.229 --> 00:22:55.608
Right, it's actually an API that's being called on the back yet.
00:22:56.220 --> 00:23:12.346
So, in the case for which you were talking about, this is where generative actions would play, because you would have an API that understands what are the pieces of information I need to be able to go get the answer to that question with that rate is, and then I will return it.
00:23:12.346 --> 00:23:31.009
And so the cool thing, what we're going to see happen and this is going to be happening very soon is the ability for you to be able to just plug in that connector to that API in the Power Platform and just say if someone asks this question, here's the API to get it from.
00:23:31.009 --> 00:23:40.903
And if they provide the information, if they say what is the rate in Australia, then it will automatically fill in the API to say I need the Australia one.
00:23:40.903 --> 00:23:47.508
If you don't say that, then what it will say is well, what country are you looking for the rate for?
00:23:47.508 --> 00:23:49.406
And then you'll fill it in.
00:23:49.406 --> 00:23:58.124
So, whatever you don't provide as required information in that connector, it would generate a question to get the answer from the user and then provide it.
00:23:59.343 --> 00:24:33.008
But when it starts getting really crazy is when you get into the planner, and the planner is the ability to say, oh, the information you gave me through this conversation, I already know who you are because you authenticated, so I can go to your profile through a different API to get what location you're from and provide that location in so that I don't have to ask you that question and that's the difference between the planner and the connector component of it, and you'll see generative actions.
00:24:33.008 --> 00:24:53.904
One of the things that we're looking at is breaking it into those two different major components, where there's one component, that's the planner, that helps orchestrate it and figure out a plan to answer the question, and another one, which is how does it consume an API or a plug-in just to be able to make it, where you don't have to author all of the different ways that somebody might interact with that API.
00:24:53.904 --> 00:24:57.308
So it's kind of really you start getting into this.
00:24:57.308 --> 00:24:59.787
It's really starts to make your head hurt in a way.
00:25:01.161 --> 00:25:04.487
And it makes you want to get your hands on and really start experiences.
00:25:04.487 --> 00:25:12.810
Now, one of the resources we're going to provide in the show notes was to a blog post that was written by Sarah Critchley on August, the 7th, 2023.
00:25:12.810 --> 00:25:30.627
Do you want to talk a bit about what was announced in that and really what the impact it is for people that really want to get going now on Power Virtual Agents, get up their skill level, because this is really, I see as a practical way to use AI generative AI in your organization.
00:25:31.000 --> 00:25:49.066
Yeah, so there's a lot of confusion in two different key areas that it's going on with people in this, and when we get into, like when we're talking about extracting information out of knowledge, which was generative answers right, a lot of people kept coming to us and saying we're totally confused.
00:25:49.066 --> 00:25:55.146
Generative answers is this thing inside of PVA that will answer stuff over your data.
00:25:55.146 --> 00:26:00.905
Then I have Azure Open AI on your data what is that?
00:26:00.905 --> 00:26:06.152
And then you have Azure Open AI which is what?
00:26:06.152 --> 00:26:08.469
And so I had to start liking this.
00:26:08.469 --> 00:26:11.461
I like to use analogies that people can relate to.
00:26:11.461 --> 00:26:14.104
So Azure Open AI is like buying a brick.
00:26:14.104 --> 00:26:20.178
Azure OpenAI on your data is like buying a wall.
00:26:20.178 --> 00:26:26.289
Then, when you look at generative answers inside of PVA, it's like buying a house.
00:26:26.289 --> 00:26:28.811
What do I mean by that?
00:26:28.811 --> 00:26:56.471
If you look at it, you say, well, generative answers actually includes an OpenAI implementation, an Azure OpenAI implementation that's trained by data scientists and a whole bunch of services to do query optimization, moderation and make sure that you can ask a question, you can point it at a set of data and it will respond to you with an answer, with reference links and things like that.
00:26:56.471 --> 00:27:02.172
You don't need to know anything about that, except here's the data I want to give it.
00:27:02.172 --> 00:27:05.934
I provided the question that someone asked.
00:27:05.934 --> 00:27:10.489
I provided where the data is, that I want the answer done and it just handles it.
00:27:10.489 --> 00:27:15.070
But what if your data, so this thing, mark?
00:27:15.070 --> 00:27:19.375
What about if my data is stored across multiple locations?
00:27:19.375 --> 00:27:31.073
And maybe my data it's stored inside a PDF and I need to be able to vectorize, search over it to be able to improve the search results that are being given back?
00:27:31.073 --> 00:27:41.955
Well, that's what you get with Azure OpenAI over your data, because now you are in control of the index or the query itself that is going in.
00:27:41.955 --> 00:27:43.750
But you still didn't have to.
00:27:43.750 --> 00:27:52.513
You're still taking advantage of all of that tuning that's been done on the Azure OpenAI model without having to hire the data scientists to go do that.
00:27:52.585 --> 00:28:05.173
You're not managing prompts at this point, and if you're not familiar with prompts, for people who don't know what that is, it's simply think of it as the words in natural language that explain to it.
00:28:05.173 --> 00:28:08.875
You should never do this and you should do that.
00:28:08.875 --> 00:28:11.333
This is where we get into Azure OpenAI.
00:28:11.333 --> 00:28:20.394
Azure OpenAI is an implementation where you're in control of the prompts and there's two levels of prompts that you have to worry about.
00:28:20.394 --> 00:28:24.415
Think of it like if you've ever heard of the laws of robotics.
00:28:25.006 --> 00:28:30.230
There's a base set of rules that you want to give, give that can never be broken.
00:28:30.230 --> 00:28:33.532
Or it's like, hey, these are the things that govern the system.
00:28:33.532 --> 00:28:41.471
Then there are ones that come in behind the scenes, that are part of the query itself, that allow it to be able to do it.
00:28:41.471 --> 00:29:06.653
If you wanted to make it talk, when you say, hey, make me a poem about Tom Brady in the style of old English, the way that it's doing that is, it's actually in the prompt, in the query that you're sending, that the fact that you want a poem, the fact that you want it about Tom Brady, the fact that you want it in the style of old English, is actually part of the prompt that you're passing in as the query.
00:29:06.653 --> 00:29:16.657
But the base training of the model is don't allow someone to ask a question that results in insulting them or cursing at them.
00:29:17.038 --> 00:29:17.278
Yes.
00:29:17.665 --> 00:29:18.711
Things of this nature.
00:29:18.711 --> 00:29:36.449
If you look at the Bing thing that happened inside of Microsoft, where people were talking to the bot for hours and hours and then they were able to get it to do stuff that it shouldn't do, this is an example of the worry that you have to have as a data scientist.
00:29:36.449 --> 00:29:42.970
When you are doing Azure Open AI by itself, those prompts and things like that you have to come up with.
00:29:42.970 --> 00:29:43.686
It's almost.
00:29:43.686 --> 00:29:44.469
I explain this.
00:29:44.469 --> 00:29:49.294
A bunch of my friends are lawyers and I said a data scientist has to think like a lawyer.
00:29:49.294 --> 00:29:54.876
What are all the loopholes that someone might find in the training set?
00:29:54.876 --> 00:30:01.230
One of the number one things you need to do is tell it never to divulge its training, which is one of the things that we learned.
00:30:01.230 --> 00:30:05.993
So it's just stuff like this that it helps people understand.
00:30:06.595 --> 00:30:08.238
How much control do you want?
00:30:08.424 --> 00:30:41.154
But with each control level that you release as you move down that stack from generative answers and down, while you're gaining control, you're also gaining the need for you to really know what you're doing, because if you index content that people don't have rights to and you expose it externally and allow people to ask questions over it that was your decision on the Azure Open AI over your data Because you own the index, you own the control.
00:30:42.067 --> 00:30:46.796
So these type of things you just have to make sure that you understand, and I think that's the biggest risk mark.
00:30:46.796 --> 00:30:57.470
I think a lot of people find it because they can do stuff really easy and really quickly, but they don't necessarily think about what all do you need to do and a lot of people don't understand that.
00:30:57.470 --> 00:31:14.237
The reason why it takes a little while for us to get stuff out in PVA, for example, is we spend a lot of time with our applied AI team and our responsible AI team to make sure that what we're doing is safe, because we want to make sure that it's safe.
00:31:14.237 --> 00:31:18.496
It's not just because it's cool and you can do it and it's awesome in a demo.
00:31:18.496 --> 00:31:22.035
It takes a lot to make sure that it's good enough to go out in the public.
00:31:22.464 --> 00:31:30.372
One of the game changes I've seen is in the area of SAP, and a great percentage of business customers globally are using Microsoft Teams right now.
00:31:30.372 --> 00:31:43.674
These big enterprises will often have data locked up in SAP systems and you never find a general person inside an organization that says, oh man, I love working in SAP.
00:31:43.674 --> 00:31:47.772
Brilliant, I could spend my entire life.
00:31:47.772 --> 00:32:05.617
Most people are like, oh my gosh, it takes so much effort to get to the information that I need, and one of the use cases that are becoming up consistently is that you put Power Virtual Agents inside Teams and you can go, hey, are there any outstanding shipments for this week?
00:32:05.617 --> 00:32:07.128
And it can go and query that.
00:32:07.128 --> 00:32:08.272
Come back with an answer.
00:32:08.272 --> 00:32:11.952
That would have been 15 clicks in the past.
00:32:11.952 --> 00:32:16.614
Within the SAP experience, people are really stuck, but you give them that data model.
00:32:16.614 --> 00:32:22.712
One of the things you're not wanting to do is go hey, you have access to Workday and now I can query people's salaries inside the business.
00:32:23.204 --> 00:32:37.131
You're making sure that that kind of protection is in place is so important and that the what data is accessible, because a lot of companies have, of course, confidential data that's even confidential internally.
00:32:37.131 --> 00:32:40.513
It's not that you're an employee and therefore you've accessed everything.
00:32:40.513 --> 00:32:43.132
There's need to know in many organizations.
00:32:43.132 --> 00:32:46.894
How do you see companies getting that bit right?
00:32:46.894 --> 00:33:04.819
Is it going to still need the data scientists inside the organization that says, listen, even though we're giving it access to this data set, we've got to make sure it can't access this portion here, because that's confidential and it should be based on someone's security role as to whether they can query.
00:33:04.819 --> 00:33:06.000
That's it there.
00:33:06.000 --> 00:33:11.997
So that whole identifying who they are is important to the questions that they can ask.
00:33:13.910 --> 00:33:47.845
Yeah, it's an interesting challenge because, by the way I was talking about before, I actually see this function coming before too much longer within organizations that are going to need to be able to think about, and you already sort of have it today in the form of data architecture, and so what I anticipate is that this is going to become the instrumental component that organizations have to go make sure they get right before they can really extract the value out of large language models, and so it's interesting.
00:33:48.426 --> 00:34:01.994
It's like I said before people are really worried about how they go do prompt engineering and how they go do all this stuff, but what you're going to see is every vendor in the world is already figuring out how to do that over there on where they store their data.
00:34:01.994 --> 00:34:18.068
The question becomes is how are you going to publish that data in a secure way and in an authoritative way, and what are the key triggers to be able to make sure that the system knows One?
00:34:18.068 --> 00:34:26.110
Do you have rights to ask this question over that data, like you can ask a question, but I could also come back and say I'm sorry, I can't answer that for you.
00:34:26.351 --> 00:34:26.793
Yeah, yeah, yeah.
00:34:27.505 --> 00:34:33.434
Maybe I say that to one person in the organization, but I don't say it to the CEO because he has access right.
00:34:33.434 --> 00:34:33.855
Yes, yes.
00:34:33.855 --> 00:35:12.791
So when you start looking at things like this data access, data quality, data governance all of these are going to be massive places that people are really going to need help, and I think, if I look across like our partners and things of that nature, or you start looking at organizations, they're so busy worrying about how to get the response off the data that they, most of the time, what we've found is they can't go forward because they realize very quickly that the data that they have access to hasn't been managed properly and therefore they've got a data quality issue.
00:35:13.414 --> 00:35:13.815
Exactly.
00:35:14.425 --> 00:35:15.469
And I will tell you that.
00:35:15.469 --> 00:35:15.989
Here's a.
00:35:15.989 --> 00:35:17.253
I'll give you a funny story, mark.
00:35:17.253 --> 00:35:26.577
So we launched generative answers with PVA and the first feature that we did is we launched it with the Bing Prometheus engine.
00:35:26.577 --> 00:35:36.516
So if you pointed out a website and what we ended up having was we had a bunch of customers come back to us and they said, hey, why did you answer this question?
00:35:36.516 --> 00:35:37.650
There's a problem here.
00:35:37.650 --> 00:35:39.932
You shouldn't be able to answer that question.
00:35:39.932 --> 00:35:43.394
And how did you get that reference link to where that is?
00:35:43.394 --> 00:35:47.009
That's not supposed to be where people can see it, and you know what?
00:35:47.048 --> 00:35:47.329
it was.
00:35:47.329 --> 00:35:50.092
It was the fact that they put it out onto the internet.
00:35:50.092 --> 00:35:54.773
They never did anything with the SEO or search engine optimization.
00:35:54.773 --> 00:35:59.092
Yes, yes, they didn't say don't allow a search engine to crawl this thing, don't crawl it.
00:35:59.605 --> 00:36:07.050
And they ended up finding all kinds of things that they had basically put in obscurity, thinking that it wasn't going to get crawled by search engines.
00:36:07.050 --> 00:36:24.050
But the search engines found it and therefore it could answer the question, and so we ended up turning it around and saying, hey, we just helped you figure out where you have data issues, so it's really funny that you will find every data issue you have ever had in your life by using a large language model.
00:36:25.085 --> 00:36:28.028
It's pretty interesting and that's why I tell people start with.
00:36:28.028 --> 00:36:36.954
I think a lot of people think, oh, I just pointed out my entire SharePoint or my entire workday and that's probably not the right first step.
00:36:36.954 --> 00:36:39.228
The first step is crawl, Give it a few things.
00:36:39.228 --> 00:36:44.476
Maybe give it a few product manuals or something like that or something of that nature.
00:36:44.476 --> 00:36:49.876
And also it's also interesting that people seem to want to start with internal data.
00:36:49.876 --> 00:36:52.793
But is that not the scariest place mark?
00:36:52.793 --> 00:37:09.813
I would actually start with public data and work my way to private data, because it's so much easier to take something that's public and just say here's the quality of it, Make sure it's pointing at the right quality stuff, versus also trying to figure out data access rates and the risk that comes along with that.
00:37:09.813 --> 00:37:11.369
It's very interesting problems.
00:37:12.445 --> 00:37:14.992
For a public data set is what it's doing.
00:37:14.992 --> 00:37:22.813
Is it basically crawling all the links on that public data set or do you need an RSS feed in that public data set?
00:37:22.813 --> 00:37:29.496
What's its method of gathering the body of knowledge before it starts answering?
00:37:30.016 --> 00:37:38.554
Yeah, so in most cases, if you're using the Bing Prometheus engine, what it's doing is it's using the index off of Bing.
00:37:38.554 --> 00:37:57.213
So when Bing goes through and does its stuff, but when it finds results, it goes out and grabs the actual HTTP page and pulls that content in so that it gets the latest version of that information, and then that's what it does for the search and summarization action.
00:37:57.213 --> 00:38:05.773
But one of the things that a lot of people don't know, mark, is that you can actually use Bing with a custom query.
00:38:05.773 --> 00:38:22.753
What that means is, for those that don't know, you can go to Bing and there's a way to say I want to build my own custom query, and so whenever you pass it in and you say here's the URL, you pass in also this unique identifier as part of it and the result will be is it will scope the data down to what you want.
00:38:22.753 --> 00:38:26.601
So that's a way for you to be able to kind of control the data set.
00:38:26.601 --> 00:38:28.047
That's kind of being in return with Bing.
00:38:28.585 --> 00:38:34.650
Otherwise, you have to start thinking about the indexer and where is the index content and what's the ability.
00:38:34.650 --> 00:38:37.894
So, like a lot of people think well, I've got a PDF on my website.
00:38:37.894 --> 00:38:42.972
Well, bing doesn't, doesn't crack PDFs, so you've got to come up with how do you index that.
00:38:42.972 --> 00:38:52.976
And then if you have a document and let's say it's 150 pages, you can't afford to run that back in through the tokens that you would have to buy for a large language model.
00:38:52.976 --> 00:39:02.153
So that's where you start getting into vector based search being pretty important so that you can basically chunk the file.
00:39:02.153 --> 00:39:09.034
So maybe I only want to return a paragraph within a document, not the entire content of the document whenever I return it.
00:39:09.034 --> 00:39:19.679
So these are kind of the challenges that you start to run into and I think this is the part that when people first get into this, they're like, oh, it's going to be super cool and I'm going to be able to do it.
00:39:19.679 --> 00:39:23.811
And then you go oh, I got to get search right, I got to figure out how to index that content.
00:39:23.811 --> 00:39:27.173
Once you get that down, the results will blow your mind.
00:39:27.605 --> 00:39:30.894
But it's really, it's really kind of the key thing.
00:39:30.894 --> 00:39:36.170
So depends on the kind of where your data is and how you want to access it and what it, how is it stored.
00:39:36.170 --> 00:39:37.931
But most people for public data.
00:39:37.931 --> 00:39:43.632
I tell them the best way is to put it directly so you don't have to pay for your own indexing.
00:39:43.632 --> 00:39:45.590
The engines do a great job.
00:39:45.590 --> 00:39:58.237
So take your content and publish it in an ASVX page or an HTML page with the content, and as long as you don't make it noncrawlable, then the systems will do the work for you.
00:39:58.237 --> 00:40:00.291
You don't have to necessarily do it.
00:40:00.291 --> 00:40:03.112
You just have to make sure the content's up to date and it's right.
00:40:03.112 --> 00:40:04.708
Rest of it solves itself.
00:40:05.329 --> 00:40:05.931
That's awesome.
00:40:05.931 --> 00:40:07.730
We're already at time.
00:40:07.730 --> 00:40:08.708
Well, well past time.
00:40:08.708 --> 00:40:12.090
We could talk for ages on this.
00:40:12.090 --> 00:40:13.519
Anything you want to say in closing?
00:40:13.945 --> 00:40:14.646
No, I think.
00:40:14.646 --> 00:40:19.195
In general, for those that are interested in this, definitely stay tuned.
00:40:19.195 --> 00:40:19.918
You can go out.
00:40:19.918 --> 00:40:24.155
If you want to play with a trial, you can go to akams slash trypva.
00:40:24.155 --> 00:40:28.215
You can also follow me on YouTube.
00:40:28.215 --> 00:40:34.853
I think it's DeWayne 27 and I spelled DeWayne really weird, so make sure you look at the show notes on how it's spelled.
00:40:34.853 --> 00:40:46.135
Also, if you follow me on LinkedIn or join up on LinkedIn, what I try to do is make sure that the things that are relevant are published for people so that they can see.
00:40:46.746 --> 00:40:49.034
I try to focus on educational type of things.
00:40:49.034 --> 00:40:51.090
I'm sorry, just product announcements.
00:40:51.090 --> 00:41:05.775
The blog is a good place to get the product announcements and the show notes as well, but in general, get in there, get your hands dirty, start playing with this and enjoy the innovation that's there.
00:41:05.775 --> 00:41:16.467
Just be aware that my key guidance to people is don't try to be the wave ride, the wave of innovation.
00:41:16.467 --> 00:41:21.255
Let the big players commoditize where they're going to commoditize.
00:41:21.364 --> 00:41:31.326
Make sure you understand where they're going and just make sure that what you're doing is you're not trying to duplicate them or jump far ahead, because the reason why you might get ahead in the short terms you're going to cut some corners.
00:41:31.326 --> 00:41:39.338
It's likely to cost you in quality or some sort of hard lesson you're going to learn.
00:41:39.338 --> 00:41:43.295
So I would just say in general stay tuned.
00:41:43.295 --> 00:41:49.835
We've got a lot of cool stuff coming and, mark, you don't even want to know what all we're going to announce in the coming months.
00:41:49.835 --> 00:41:51.422
It's going to blow your mind even more.
00:41:51.422 --> 00:41:55.885
So please go to the Power Platform conference, if you can, or show up at Ignite.
00:41:55.885 --> 00:41:59.606
There will be plenty of new cool stuff that we're going to talk about as we get closer.
جستجوی اپیزودهای گذشته
اپیزودهای قبلی The Intelligence Age را جستجو کن.
سلب مسئولیت: پادکست و آثار هنری تعبیه شده در این صفحه متعلق به Mark Smith [nz365guy] است که متعلق به صاحب آن است و به Listen Notes، Inc وابسته یا تایید نشده است.
ویرایش
از کمک شما برای بروز نگهداشتن پایگاهدادههای پادکست سپاسگزاریم
