Episode 55 - MCP Apps for App Service with Jordan Selig

WEBVTT
00:00:08.320 --> 00:00:14.080
Welcome to Microsoft Community Insight Podcast where we share insights and storage from community experts.
00:00:14.240 --> 00:00:19.839
Today we were diving into MCP Apps and App Service with our guest Jordan Slay.
00:00:20.000 --> 00:00:22.000
Could you please introduce yourself?
00:00:22.800 --> 00:00:23.120
Sure.
00:00:23.440 --> 00:00:25.039
Yeah, thank you for having me today.
00:00:25.440 --> 00:00:26.800
My name is Jordan Selig.
00:00:26.960 --> 00:00:29.679
I'm a product manager with Azure App Service.
00:00:29.839 --> 00:00:31.760
I've been on the team for five years now.
00:00:31.839 --> 00:00:33.600
I just hit my anniversary last month.
00:00:33.759 --> 00:00:35.759
So uh big exciting accomplishment.
00:00:35.840 --> 00:00:39.280
I'm getting my my crystal in the mail in a couple months, I think.
00:00:39.520 --> 00:00:43.280
Um so yeah, I mean I started on the team five years ago.
00:00:43.439 --> 00:00:53.520
Um I previously worked at a large bank for four years, so I had experience working um with large enterprise customers and what that demanded.
00:00:53.600 --> 00:00:58.880
Um I was in the cloud security space, and that's what really introduced me um to the cloud.
00:00:59.039 --> 00:01:00.640
Um, it was time for a change.
00:01:00.719 --> 00:01:04.239
So I moved over to Microsoft and App Service.
00:01:04.400 --> 00:01:11.760
Um I was building a lot of apps in my free time um on services like Google Firebase and AWS Amplify.
00:01:11.920 --> 00:01:19.920
And you know, I wanted to support a product that allowed people like me to build and share my ideas with you know with others.
00:01:20.079 --> 00:01:26.159
So um got a role on App Service and I work on a number of different product areas.
00:01:26.239 --> 00:01:31.840
Um our networking, uh App Service environments are a single tenant offering.
00:01:32.079 --> 00:01:38.879
Um I was leading all of our app service CLI efforts, but that's kind of been delegated across different um teams now.
00:01:38.959 --> 00:01:43.840
Um I lead Azure policy, so security and governance for App Service.
00:01:44.000 --> 00:01:51.920
And more recently, the last about year and a half or so, I've been the lead for basically all of the um AI investments for App Service.
00:01:52.239 --> 00:02:02.560
So um my goal and my my mission that I was told to achieve was to make App Service a premier platform for AI and you know agentic workloads.
00:02:02.719 --> 00:02:06.079
So um that's really what I've been working on the last year and a half.
00:02:06.319 --> 00:02:13.039
And um, you know, I continue to work on that, and I have a lot of features in the roadmap that I'm hoping to land um in the next few months.
00:02:13.280 --> 00:02:20.080
So instead of like customers always go to Kubernetes, they can choice choose between go to container apps or app service.
00:02:20.400 --> 00:02:28.080
As a way you're making it more like uh agentic, when I was putting MCP for capabilities as well.
00:02:28.639 --> 00:02:28.879
Right.
00:02:29.039 --> 00:02:52.000
So if you're looking for you know a full-fledged managed service, uh platform as a service pass offering, um, you're looking at you know, app service, um, Azure container apps, functions, these are the services on um on Azure that manage all of the you know operational work that you would typically have to do if you were running your apps off of a VM.
00:02:52.080 --> 00:02:58.240
And um AKS is like a semi-pass just because you still need to do all the the Kubernetes work.
00:02:58.639 --> 00:03:04.719
Um so ACA is the option if you want to you know completely abstract all that Kubernetes um details.
00:03:04.960 --> 00:03:09.919
So um, you know, we're trying to make app service, you know, at least personally, that's my goal.
00:03:10.000 --> 00:03:15.360
You know, my partner teams on apps on Azure container apps are doing you know all the AI work for them.
00:03:15.520 --> 00:03:17.520
I'm the the person responsible for app service.
00:03:17.599 --> 00:03:25.840
So uh building out new features and building out samples and demos just to show like, hey, you could do all of these agentic and AI workloads on app service.
00:03:26.639 --> 00:03:26.960
Okay.
00:03:27.680 --> 00:03:33.280
So what's what's currently new on MCP apps at the moment?
00:03:34.159 --> 00:03:43.120
So on App Service, um, if you saw some of our build announcements a couple weeks ago, we released a new feature called built-in MCP.
00:03:43.439 --> 00:04:01.680
And this is a really cool feature because um it aligns with this overall work stream that I've created called Easy AI, which is basically taking your existing workloads on App Service and adding agents, some sort of AI functionality to them.
00:04:01.840 --> 00:04:05.759
Um App Service has been around for a long time, 10, 15 years.
00:04:06.000 --> 00:04:22.800
We have tons, you know, tons and tons of workloads already on here, a lot of workloads that were deployed, you know, 10, 15 years ago, and modernizing those to you know enable AI and agents, re-architecting those is a huge task for some of these larger companies, larger enterprises.
00:04:23.120 --> 00:04:35.360
So one of my work streams is how do we take these existing potentially legacy apps where you know the current um operations teams that manage these apps maybe don't even know what the code does.
00:04:35.439 --> 00:04:36.480
They don't even know where the code is.
00:04:36.639 --> 00:04:39.040
In some cases, we have some customers like that.
00:04:39.199 --> 00:04:47.040
So, how do we allow them to enable AI in these apps while at the same time they're they're probably re-architecting, they're probably innovating somewhere.
00:04:47.199 --> 00:04:53.759
But you know, if they want to quickly get AI agents into their into their work, you know, how do we do that for them?
00:04:54.000 --> 00:04:59.439
So the first feature was this built-in MCP feature, and it's live, it's in the portal today.
00:04:59.519 --> 00:05:01.600
We announced it at build a couple weeks ago.
00:05:01.759 --> 00:05:22.079
And what it is is um if your app has an API service, so if it has an API that can be defined with an open API spec, you give us that spec, and then we will host an MCP server on your app service and expose all of those operations, those API operations as MCP tools.
00:05:22.240 --> 00:05:39.920
So without any code changes, without any you know, changes to your app, you now have an MCP server that um allows agents to connect to your app and interact with your app following all the same networking and authentication protocols that you have already potentially implemented in your app already.
00:05:40.000 --> 00:05:42.399
So it's a really cool feature, it's really easy to use.
00:05:42.639 --> 00:05:46.399
And um I do have a demo of it a little later on if we want to go into that.
00:05:46.720 --> 00:05:51.600
I was wondering could you like call it from like agile scales?
00:05:51.759 --> 00:05:53.439
You know, the new things called Agile Scales.
00:05:53.600 --> 00:05:57.120
Could you call the that new MCP apps from it?
00:05:58.480 --> 00:05:58.879
Yeah.
00:05:59.199 --> 00:06:03.839
So what this feature does is it literally just creates an MCP server.
00:06:04.000 --> 00:06:10.000
So you tell us the the endpoint that you want to use for this, and it's just an MCP server.
00:06:10.079 --> 00:06:17.920
So you can call that with any, you know, any agent, any client that wants to call an MCP server, it's you know open to using this.
00:06:18.079 --> 00:06:19.759
So uh no difference.
00:06:19.839 --> 00:06:21.439
We just make it really easy to create this.
00:06:21.519 --> 00:06:23.040
You don't have to build it from scratch.
00:06:23.600 --> 00:06:24.240
Uh okay.
00:06:24.399 --> 00:06:26.160
And it would just hose in there.
00:06:26.319 --> 00:06:26.720
It does it.
00:06:27.040 --> 00:06:30.079
I take it it just you just need to endpoint to recall it.
00:06:30.560 --> 00:06:30.800
Yeah.
00:06:31.199 --> 00:06:36.800
The endpoint, and if there's any authentication on the app service, um, you'll have to account for that in your client.
00:06:36.959 --> 00:06:39.199
But there's nothing proprietary or special here.
00:06:39.279 --> 00:06:49.519
It's just, you know, we have implemented the you know the latest from you know the MCP standards and uh made it available and easy for uh our customers to use.
00:06:50.480 --> 00:06:50.800
Okay.
00:06:51.040 --> 00:06:54.639
Uh yeah, you say you you can show us some demos, you say got some demos.
00:06:55.680 --> 00:06:57.279
So how about this thing in action?
00:06:58.399 --> 00:07:02.079
Yeah, so let me just get my uh monitors here and I'll set up.
00:07:02.240 --> 00:07:06.560
I'll put you here and I will present this.
00:07:15.439 --> 00:07:19.920
So what's so you're constantly building now towards the MTP apps?
00:07:20.160 --> 00:07:21.360
What's after mcp apps?
00:07:21.519 --> 00:07:26.160
Is just adding new AI features between app service?
00:07:26.800 --> 00:07:34.639
Yeah, so so I'll get a little bit into that and it'll I think it'll be more clear when I show you some of the um some of the work we're doing inside the Azure portal.
00:07:34.879 --> 00:07:34.959
Yeah.
00:07:35.120 --> 00:07:37.279
So I'll make this a little bigger so it's a little easier to see.
00:07:40.319 --> 00:07:40.639
Okay.
00:07:41.600 --> 00:07:47.360
Um so this is an app that we built for Microsoft Build a couple weeks ago.
00:07:47.439 --> 00:07:48.959
I'll show you what that is really quickly.
00:07:49.120 --> 00:07:51.279
Um it's just an online clothing store.
00:07:51.360 --> 00:07:59.199
It's something we use for a lot of demos just because it can demonstrate a lot of the features that we um you know have been building.
00:07:59.439 --> 00:08:04.959
Um one of the things that we actually did with this app, and I don't know if you've heard of the managed instance on app service feature.
00:08:05.519 --> 00:08:06.720
Are you familiar with that?
00:08:07.199 --> 00:08:08.240
Manage instance?
00:08:08.319 --> 00:08:11.120
No, I've seen the managed identity thing, but no.
00:08:11.519 --> 00:08:18.639
So manage instance, and I'm not gonna spend too much time on it today just because I'm I'm not the expert and it's not really AI focused.
00:08:18.720 --> 00:08:20.720
Um, but some of my teammates are working on this.
00:08:20.800 --> 00:08:26.160
And if you're interested in learning more about managed instance, uh I can connect you and send you some details about that.
00:08:26.240 --> 00:08:44.480
Um but manage instance is a way of you know, if you've had these, you know, these applications hosted on-prem that you haven't migrated to the cloud yet because of the you know, the amount of work it would take, because you need certain um controls that the cloud or app service doesn't let you control traditionally.
00:08:44.639 --> 00:08:50.399
Um Manage Instance allows you to have that OS level control um for your apps.
00:08:50.480 --> 00:08:51.840
So it's it's a really cool feature.
00:08:51.919 --> 00:08:58.399
It's gonna help a lot of customers migrate these applications that they just haven't migrated yet and migrate them quickly.
00:08:58.559 --> 00:08:59.600
So okay.
00:08:59.840 --> 00:09:17.679
So it's typically for legacy application for typically for legacy applications, but it can also be used for newer applications that need more control than you know, somewhere in between what app service gives today and a VM where you have to control, you know, everything.
00:09:17.919 --> 00:09:21.679
Um so um, like I said, I can definitely share more details about that.
00:09:21.759 --> 00:09:25.840
I'm yeah, to be an expert about that, but it's a really cool feature.
00:09:25.919 --> 00:09:32.559
And um, you know, a lot of our customers have expressed interest in it because you know, migrating takes a long time.
00:09:32.720 --> 00:09:36.720
It it requires a lot of changes to your apps if you know to be cloud ready.
00:09:36.879 --> 00:09:40.799
And this feature is something that will help a lot of customers get to the cloud quickly.
00:09:41.120 --> 00:09:41.679
Yeah, thanks.
00:09:42.399 --> 00:09:42.960
Cool.
00:09:43.360 --> 00:09:47.200
So I'm just gonna give you some scope to what this demo was.
00:09:47.360 --> 00:09:54.960
So this was an app that we migrated to um app service using the the managed instance feature.
00:09:55.120 --> 00:09:58.320
So this is a managed instance app service.
00:09:58.879 --> 00:10:01.200
And it it looks just like a standard app service in the portal.
00:10:01.360 --> 00:10:08.000
There's not really much here that you know will indicate that it's it's different, um, but it is uh a managed instance.
00:10:08.080 --> 00:10:14.720
So that that was the first part of this demo was saying, you know, here's how you take this old legacy app and bring it to the cloud.
00:10:14.960 --> 00:10:19.600
The next part was um, you know, this was uh an online shopping store.
00:10:19.759 --> 00:10:23.279
So you have like, you know, how do I, you know, I want to buy some of these items.
00:10:23.440 --> 00:10:26.559
I want to add them to my cart, I want to remove them from my cart.
00:10:26.639 --> 00:10:33.919
So, you know, the APIs that are backing this are um just the traditional APIs that you would see in an online shopping store.
00:10:34.399 --> 00:10:40.559
So um what we wanted to do is enable the um built-in MCP feature.
00:10:40.720 --> 00:10:44.080
So we have this new blade in the Azure portal.
00:10:44.159 --> 00:10:49.200
It's called AI and has preview in there because we just released it, and um we'll remove that soon.
00:10:49.440 --> 00:10:56.320
But um this is what we'll be building out over the next few months as we start um releasing more of these AI features.
00:10:56.639 --> 00:11:03.840
So uh the first tab here is this MCP servers tab, and this is where you enable the built-in MCP feature.
00:11:04.000 --> 00:11:09.279
So it's as simple as, and I've already done it just in the interest of time, um, even though it only takes a minute.
00:11:09.360 --> 00:11:13.919
Um, you all you do is upload the open API spec and then provide.
00:11:14.080 --> 00:11:15.919
Let's see if I can show you some of this.
00:11:17.200 --> 00:11:26.000
You just provide details like the endpoint if you want to add a description, um and then here's where you provide any of the the off details.
00:11:26.240 --> 00:11:28.720
So it's it's super simple, it just takes a minute.
00:11:29.039 --> 00:11:29.200
Yeah.
00:11:29.759 --> 00:11:32.240
Can you using VS Code as well?
00:11:32.480 --> 00:11:34.159
The new AI features.
00:11:35.039 --> 00:11:39.200
You can do this um with uh API.
00:11:39.360 --> 00:11:43.600
Uh we don't have dedicated CLI commands, but there's probably a way to hook it up with the CLI.
00:11:43.919 --> 00:11:44.080
Okay.
00:11:44.320 --> 00:11:47.039
Um so ARM API CLI.
00:11:47.120 --> 00:11:50.000
Um, so if you're in VS Code, you know there's plenty of options.
00:11:50.159 --> 00:11:58.960
Um I we are working on releasing um, and we've already done a bunch, but we're continually iterating um with the Azure skills.
00:11:59.200 --> 00:12:07.919
Um so we are continuously contributing to that to make sure that the Azure skills always have the latest information about app service and you know what we as the product group recommends.
00:12:08.240 --> 00:12:13.759
So this, if it's not already in there, will be in there soon just to say, like, if you want to enable this, here's how to do it.
00:12:14.480 --> 00:12:16.879
I think I should probably do this in bicep, right?
00:12:16.960 --> 00:12:18.960
Because I think it will be out already.
00:12:21.200 --> 00:12:22.159
For sure.
00:12:22.480 --> 00:12:26.399
So so anyway, um, this is now an MCP server.
00:12:26.480 --> 00:12:28.240
You can see the the endpoint right here.
00:12:28.480 --> 00:12:29.120
Yeah.
00:12:29.360 --> 00:12:34.480
Um, and I can connect to this through whatever client I want to connect to it, you know, from.
00:12:34.639 --> 00:12:36.000
So very simple.
00:12:36.159 --> 00:12:39.519
Um so so that was the first part of this demo.
00:12:39.600 --> 00:12:46.559
It was showing, you know, how do you take an old app and then now just add some sort of capability that allows agents to easily connect to it?
00:12:46.639 --> 00:12:48.159
And that was this feature.
00:12:49.039 --> 00:12:57.840
Now the the the next part of this demo, which um I did not go into too much detail for build, um, was the second part here.
00:12:58.159 --> 00:13:03.519
I'm curious to know, are you allowed to add custom tools or is it just predefined tools that's already selected?
00:13:03.840 --> 00:13:06.080
That you have to select from the application.
00:13:06.399 --> 00:13:12.159
So these tools, so these tools are the same tools or the same operations from your API.
00:13:12.240 --> 00:13:18.720
So it literally just takes your API, it maps each operation and creates a tool for that.
00:13:18.960 --> 00:13:23.360
So this specific open API spec, and I'll make more detail, make more sense in a minute.
00:13:23.600 --> 00:13:29.360
Um, I built some automation around the inventory management for this online store.
00:13:29.440 --> 00:13:35.120
So you'll see things like you know, get the inventory, get product, um, check stock, update stock.
00:13:35.279 --> 00:13:36.720
So that's what these tools are.
00:13:36.879 --> 00:13:37.919
So, you know.
00:13:38.639 --> 00:13:38.879
Okay.
00:13:39.279 --> 00:13:39.919
Nice.
00:13:40.480 --> 00:13:40.799
Okay.
00:13:41.440 --> 00:13:44.320
So um the next part is this agents tab.
00:13:44.559 --> 00:13:48.000
So that'll make more sense when I show you the next part of the demo.
00:13:49.120 --> 00:13:54.559
So um for this for this demo, we wanted to show, you know, we we added MCP.
00:13:54.639 --> 00:14:02.159
So now how do we add some agentic uh process into this app without actually changing the app code?
00:14:02.320 --> 00:14:09.360
So we thought um, can we use agents to manage the inventory for this online shopping store?
00:14:10.720 --> 00:14:14.240
So um you want to say something?
00:14:14.639 --> 00:14:26.080
Yeah, so would the agent just be having that B you been like constantly running, or do you need him in a loop to like so so yeah, that's a great question.
00:14:26.240 --> 00:14:28.960
Um it that's completely up to you.
00:14:29.039 --> 00:14:33.600
Uh but for the purposes of this demo, I built an agent hosted on App Service.
00:14:33.759 --> 00:14:45.120
But you know, this agent could be anything, it could be you know uh a managed agent, it could be some sort of custom agent, um, anything you want that can consume the the MCP endpoint that you created.
00:14:45.600 --> 00:14:56.480
So for the purposes of this demo, and this is you know just a demo, so um I built um I actually have a little bit of a slideshow that has an architecture diagram.
00:15:05.840 --> 00:15:11.840
Okay, so the um the the box on the right is the existing app.
00:15:11.919 --> 00:15:14.639
It's just you know our legacy app, it's got our MCP server.
00:15:14.879 --> 00:15:18.720
Yeah and on the left side, this is the agentic app that I built.
00:15:18.799 --> 00:15:24.639
It's using Microsoft Agent Framework, and it has six um agents running on App Service.
00:15:24.720 --> 00:15:31.120
So these are hosted agents on App Service, and they connect to Microsoft Foundry for um the LLM.
00:15:32.879 --> 00:15:41.759
And the six agents that I built were one that that monitors inventory, so it's just checking the stock levels of the different items in the store.
00:15:41.919 --> 00:15:43.919
It's using MCP to do this.
00:15:44.080 --> 00:15:54.480
Um, and then I have a bunch of other agents that um simulate uh ordering stock and fulfilling those orders and then updating the inventory with MCP again.
00:15:54.639 --> 00:15:56.960
So um there's a lot going on behind the scenes here.
00:15:57.039 --> 00:16:00.320
Um this is pure Microsoft agent framework.
00:16:00.480 --> 00:16:08.639
Um, you know, you can see for uh the the agents that use the MCP server um to check inventory.
00:16:08.720 --> 00:16:14.879
It's just as simple as adding an endpoint and then using the the framework to build these agents, it's it's super simple.
00:16:16.639 --> 00:16:20.000
Okay, and you can call any like agent in Azure.
00:16:20.240 --> 00:16:26.559
You can call Find your agent, you can call like SharePoint agent, different agents, even co-pilots as well.
00:16:26.960 --> 00:16:27.279
Exactly.
00:16:27.360 --> 00:16:32.240
Yeah, so so the agents for this demo are all agents hosted on app service.
00:16:32.559 --> 00:16:42.639
But if I wanted to use a copilot agent, if I wanted to use like some sort of SaaS product that uses you know MCP to you know get information, I could do that.
00:16:42.720 --> 00:16:49.919
Um the purpose of this demo is using an app service agent, um, just to show that you know you can host agents on app service, it's very easy.
00:16:50.000 --> 00:16:58.080
We support Microsoft Agent Framework as well as all the other um you know traditional and and common frameworks that people are using these days.
00:16:58.720 --> 00:17:06.240
I think you define the agent the same as you define it in like in Foundry, like in Python and JavaScript.
00:17:06.400 --> 00:17:06.559
Okay.
00:17:07.039 --> 00:17:07.359
Exactly.
00:17:07.519 --> 00:17:09.279
I'm just using Microsoft Agent Framework.
00:17:09.359 --> 00:17:16.559
It's platform agnostic, so it doesn't care that I'm running on App Service, it's just like five or six lines of code to define an agent.
00:17:16.640 --> 00:17:17.680
It's it's super simple.
00:17:17.839 --> 00:17:20.720
If you haven't seen Microsoft Agent Framework, you know, check it out.
00:17:20.799 --> 00:17:22.640
It runs really well on App Service.
00:17:22.720 --> 00:17:28.480
Um, I have a bunch of demos showing how to do this exact um scenario that I'm showing today.
00:17:29.119 --> 00:17:29.440
Okay.
00:17:30.319 --> 00:17:43.279
So um before we get back to the demo, the other piece that I'm gonna call out is um observability, monitoring your agents is also something that you know is just becoming more and more important as people start using more and more agents.
00:17:43.440 --> 00:17:46.799
And doing that on app service is also very simple.
00:17:47.200 --> 00:17:56.079
Um, if you enable um open telemetry and configure the gen AI semantics, so this is very straightforward.
00:17:56.240 --> 00:17:58.559
As you can see, it's it's like six lines of code.
00:17:58.720 --> 00:18:02.799
Um, connect your app to app insights and enable open telemetry.
00:18:02.880 --> 00:18:06.240
And the gen AI semantics are just like the agent name, the agent ID.
00:18:06.400 --> 00:18:07.759
So very simple.
00:18:08.640 --> 00:18:16.880
We will be able to provide a level of monitoring and observability for your agents that previously you would have to custom build somewhere.
00:18:17.440 --> 00:18:19.839
So um I'll show you all this.
00:18:20.319 --> 00:18:24.799
How about tracing, like tracing the agent, like input prompt and everything?
00:18:25.200 --> 00:18:28.400
Yeah, so I'll telemetry as well, the logs.
00:18:28.720 --> 00:18:36.000
Yeah, so if you add this to your code for your agents on app service, we have a built-in process for for showing all that.
00:18:36.079 --> 00:18:41.279
So I will show you that um in just a minute um as I take you through the next part of the demo.
00:18:41.920 --> 00:18:46.799
So um this right here is another app on App Service.
00:18:47.119 --> 00:18:54.160
Um, I'll take you into the environment variables just so you can see that it is connected to the MCP server.
00:18:54.640 --> 00:18:58.640
Um, this is that uh store app that I was showing you.
00:18:58.960 --> 00:19:00.240
An app service.
00:19:00.960 --> 00:19:06.000
So you can't really see the agent, but it just hosting app service behind the back end.
00:19:06.559 --> 00:19:06.799
Right.
00:19:06.880 --> 00:19:13.599
So in reality, when you're you know running agents, you don't actually see the agents working, you just see the output.
00:19:13.759 --> 00:19:18.799
So behind the scenes, this is a very complicated you know process with six different agents doing a lot of different things.
00:19:18.960 --> 00:19:25.519
But from like an end user perspective, you know, an operator of you know a tool like this, they don't know how to use agents typically.
00:19:25.599 --> 00:19:28.880
That's for the IT teams, the operations teams just want a tool that works.
00:19:29.200 --> 00:19:30.319
Yeah, that's true.
00:19:30.640 --> 00:19:36.640
So um this right here is the inventory for the app that we were just looking at.
00:19:36.799 --> 00:19:41.920
Um, I'm just gonna go ahead and simulate a sale just so we can see one of these items go out of stock.
00:19:42.000 --> 00:19:46.319
I have a script on the side here that just simulates a sale, so I'm gonna let that run for a second.
00:19:46.480 --> 00:19:50.079
And it's gonna randomly pick an item and a size to go out of stock.
00:19:53.440 --> 00:19:54.559
It's thinking.
00:19:56.240 --> 00:19:57.119
Okay, there we go.
00:19:57.279 --> 00:19:59.440
So I'll just refresh the page so you can see.
00:20:00.960 --> 00:20:02.160
Oh, you're gonna buy a new one.
00:20:02.319 --> 00:20:02.880
Oh, okay.
00:20:03.200 --> 00:20:07.119
So yeah, so it it picked the size medium of this, you know, this navy blazer.
00:20:07.279 --> 00:20:10.720
So this is one of the items in the store, and it decided to sell out.
00:20:11.039 --> 00:20:17.920
So um, in reality, that this should be automated, and you know, the agent should be polling and checking to see when things go out of stock.
00:20:18.000 --> 00:20:20.799
Um, for demo purposes, I have a button here.
00:20:22.240 --> 00:20:25.279
I have a button here that just says to trigger that process.
00:20:25.440 --> 00:20:28.240
So I'm gonna let this this um this start.
00:20:28.400 --> 00:20:32.160
And what it's doing is it's running all of our six agents here.
00:20:32.240 --> 00:20:36.079
So it's checking inventory, it's using MCP again to see what happened.
00:20:36.240 --> 00:20:41.200
Um, there's another agent that is connected to historical sales data.
00:20:41.279 --> 00:20:45.039
Um, so I have just a blob in Azure Storage that has all this historical sales data.
00:20:45.119 --> 00:20:49.599
So it's using that to just to say how much stock should we reorder.
00:20:49.680 --> 00:20:57.200
Um, so this is another intelligent agent that's using um reasoning to decide what you know our order should look like.
00:20:57.519 --> 00:21:00.559
And you can see we paused right here at our reorder recommendation.
00:21:00.720 --> 00:21:06.160
It's recommending that I order 20 units of this particular item because of the historical sales data.
00:21:06.240 --> 00:21:09.440
Um, see, it's giving us some explanation here for what we need.
00:21:09.519 --> 00:21:17.519
And it's also saying, you know, June is not peak season for a thick navy blazer, but here's what I think you should get.
00:21:17.759 --> 00:21:20.400
So um, you know, it's a smart process.
00:21:20.960 --> 00:21:24.640
I'm gonna approve this just so we can move forward.
00:21:24.880 --> 00:21:33.279
So now it's just running, you know, like a simulated order and then fulfillment of that order, and then it went ahead and updated our inventory.
00:21:33.359 --> 00:21:38.960
So if we go back up here, you can see that our Blazor now has 20 um items in stock.
00:21:39.119 --> 00:21:41.920
So uh really simple process.
00:21:42.160 --> 00:21:49.759
We took our app, our legacy app, and now we've added this automation to it without modifying that existing app at all to manage our inventory.
00:21:50.400 --> 00:22:12.880
And when it comes to tracing and observability, like you asked, if we go back to our AI blade and we Go over to agents, we have this new view, and this view has all six of the agents that just ran and shows you all the calls, all the tokens that they were using, if there were any errors.
00:22:13.759 --> 00:22:16.319
And then you could even go into App Insights.
00:22:16.799 --> 00:22:18.400
So I'll just start with this one.
00:22:20.400 --> 00:22:28.400
And get even deeper into each of those agent calls, view all the traces, and you know, explore anything you need to with these different agents.
00:22:28.559 --> 00:22:36.880
So uh we've made it really easy to enable this feature and get this visibility um for any agentic app that you host on App Service.
00:22:38.640 --> 00:22:41.680
Okay, yeah, it's quite easy to you see.
00:22:41.759 --> 00:22:47.359
Yeah, you go to you can find like the logs of it to see if there's any error rates and stuff.
00:22:47.839 --> 00:22:53.440
Yeah, this is showing 24 hours, but if I move it back, you know, like 60 days, you can see a lot more activity.
00:22:53.759 --> 00:22:56.960
Um additional traces.
00:22:57.119 --> 00:22:59.039
So I mean, this is all features in App Insights.
00:22:59.200 --> 00:23:02.319
So if you're familiar with App Insights, there's a ton you could do here.
00:23:02.480 --> 00:23:06.720
Um, and we we just made it really easy to set this up and get started with this.
00:23:06.799 --> 00:23:10.880
We we want to encourage people to start hosting more of their agents on App Service.
00:23:10.960 --> 00:23:14.400
So um hopefully a feature like this encourages that.
00:23:15.279 --> 00:23:15.599
Okay.
00:23:16.079 --> 00:23:24.720
Uh what's what was that session that you you say you did at bail so people can have a listen to if it's still recorded?
00:23:25.359 --> 00:23:27.039
So we did a session.
00:23:27.200 --> 00:23:28.960
Um it is recorded.
00:23:29.119 --> 00:23:32.880
I will share a link with you so that you can share in the details of the session.
00:23:32.960 --> 00:23:34.799
Um, highly recommend watching that.
00:23:34.880 --> 00:23:40.079
Um, it'll go into more details about um app service managed instance.
00:23:40.160 --> 00:23:42.079
That was the the primary focus of that session.
00:23:42.240 --> 00:23:49.920
But at the end, we did do a talk about uh this MCP feature, this agent feature, just to give people some some context um to what that was.
00:23:50.000 --> 00:23:53.680
So I will share more details with you and definitely check that one out.
00:23:54.079 --> 00:23:54.400
Okay.
00:23:54.799 --> 00:24:02.799
So are you typically seeing more customers using these features, MCP apps, like building agents within App Service?
00:24:03.119 --> 00:24:16.400
Yeah, we we've had a lot of customers expressed interest in you know building MCP servers, deploying MCP servers on App Service, as well as some of our partner platforms like um Azure Functions and Azure Container Apps and AKS.
00:24:16.640 --> 00:24:21.759
Um choosing between those services, you know, is always a challenge for customers.
00:24:21.839 --> 00:24:24.880
It's like, you know, what's the best place for my situation?
00:24:25.039 --> 00:24:49.039
Um so to give some, I know we have some docs around helping you choose, but for MCP servers specifically, to give you some more um, you know, to help you choose for for app service, you know, we are a platform that allows you to you know host your server and then scale that server to a very broad and large audience with minimal you know work.
00:24:49.279 --> 00:24:56.559
You know, you can enable our always-on feature and your MCP server will be ready to go for whenever it's needed.
00:24:56.640 --> 00:25:01.920
And then you know you can have auto-scaling configured so it can scale to as many customers they need it.
00:25:02.160 --> 00:25:06.640
Functions, on the other hand, you know, is another great place to host MCP servers.
00:25:06.720 --> 00:25:09.759
It has a bunch of new features that I highly recommend checking out for this.
00:25:09.839 --> 00:25:18.400
Um, but if you're looking for that large, broad audience to use your your single MCP server over like remote connections, it's not really the best choice.
00:25:18.559 --> 00:25:19.920
App service is probably better for that.
00:25:20.079 --> 00:25:24.400
If you're looking for like a single personal tool, functions is probably better.
00:25:24.480 --> 00:25:26.240
Um, and there's other use cases for functions as well.
00:25:26.319 --> 00:25:29.279
I'm not going into great detail about that.
00:25:29.359 --> 00:25:34.720
Um, but I highly recommend you know evaluating each of these services to determine which one is the best fit for you.
00:25:34.880 --> 00:25:47.440
Um if you're familiar with app service, if you know about app services, you know, auto-scaling capabilities, security functions, uh functionality, um, MCP servers on app service inherit all of that.
00:25:47.519 --> 00:25:59.759
So you're getting the same you know SLA performance, um durability that you would be getting with you know your traditional app service now on MCP servers with you know no significant changes.
00:26:00.079 --> 00:26:00.720
Yeah.
00:26:01.359 --> 00:26:07.440
Uh I'm curious to know, you know, that module you just deployed for GPT 4.0 that's on the screen.
00:26:07.599 --> 00:26:10.000
Is that open AI module or foundry module?
00:26:10.079 --> 00:26:11.519
How is that token used?
00:26:12.400 --> 00:26:16.160
So this app is connected to Azure OpenAI.
00:26:16.400 --> 00:26:24.559
Um the way Azure OpenAI works with with um Azure now is um it's basically just another Foundry resource.
00:26:24.720 --> 00:26:28.319
Foundry and um Azure OpenAI resources are basically the same thing.
00:26:28.480 --> 00:26:34.880
They're both they're basic, they're like um Microsoft.cognitive services resource provider, I think.
00:26:35.039 --> 00:26:36.640
So they're basically the same thing.
00:26:36.880 --> 00:26:40.880
Um I could have gone to Foundry and deployed any model I wanted.
00:26:40.960 --> 00:26:46.960
I picked a small cheap one just because I didn't need a you know an expensive one for for this kind of process.
00:26:47.519 --> 00:26:50.720
Um but you know that's all external from app service.
00:26:50.799 --> 00:26:54.400
Like you can use whatever model you want, you know, if you're willing to pay for it.
00:26:54.720 --> 00:27:01.279
So in terms of cost, does this go into app service charges or the open AI charges?
00:27:01.599 --> 00:27:04.640
Azure OpenAI when you posted.
00:27:05.519 --> 00:27:10.319
The tokens are gonna still cost your you know, hit your Azure OpenAI billing.
00:27:10.480 --> 00:27:22.559
Your app service billing really shouldn't it actually probably won't be impacted with an application like this because app service, you pay for your app service plan, and that's a flat fee for the month.
00:27:22.799 --> 00:27:32.000
So if you have you know our premium v4 tier, you know, our you know, what is that, like$70 a month for one of our for an instance or something, that's all it's gonna be for the month.
00:27:32.079 --> 00:27:35.920
Unless you audit you to scale it up, add more instances, then your price is gonna change.
00:27:36.000 --> 00:27:43.039
But but app service gives you very consistent billing, unlike something like functions, where you know that that can scale all the way to zero.
00:27:43.119 --> 00:27:44.319
You only pay when you're using it.
00:27:44.480 --> 00:27:50.000
So that pricing will be a little bit, you know, it'll be variable, so you won't be able to predict it as much.
00:27:50.079 --> 00:27:52.720
Um, but that could be a benefit if you don't have a lot of usage one month.
00:27:52.960 --> 00:27:57.519
App service, you're always gonna be paying an amount, and you know, you can always predict that.
00:27:58.000 --> 00:28:04.880
Uh yeah, you get most of it, most of the the cost from the app service because you pay for what you use for that money.
00:28:05.599 --> 00:28:06.240
Right.
00:28:06.720 --> 00:28:18.880
Okay, so in terms of like app service, uh mcp apps, is there any is there a lot going on in the roadmap towards like improving it, making it better?
00:28:19.599 --> 00:28:25.279
Yeah, so as I mentioned earlier, one of my focuses is this easy AI work stream.
00:28:25.440 --> 00:28:35.119
So we are working towards making features like this MCP feature, yeah, where you can enable AI functionality with minimal changes to your app.
00:28:35.279 --> 00:28:42.799
So app service as a platform is able to host really like any AI agentic workload already.
00:28:42.880 --> 00:28:46.319
There's really nothing special that we need to change on our side.
00:28:46.480 --> 00:28:54.880
Um, there is one thing uh regarding like our default timeouts that make long-running agents difficult to manage on App Service.
00:28:55.119 --> 00:29:03.519
So I am working with the team internally to extend our um our timeouts just so agents can run more long-running processes without timing out.
00:29:03.759 --> 00:29:10.000
Um that's one small fix that hopefully will help users that have faced that issue resolve that.
00:29:10.160 --> 00:29:14.720
But in terms of new features, um, you know, we want to make AI more accessible.
00:29:14.880 --> 00:29:26.720
We want to make rag um adding agents, um, you know, converting your web apps to markdown so agents consume less tokens when they read your app.
00:29:26.880 --> 00:29:32.880
Um features like this are what we are working on right now and hopefully can release a little later this year.
00:29:33.039 --> 00:29:41.039
Um all features that allow you to add AI to your apps without really changing the code or what you're what your apps do already.
00:29:41.200 --> 00:29:58.000
So um hopefully it that means that a lot of users will be able to enable these without you know excessive testing and you know process that may be involved when onboarding new um new services and new features, which I know a lot of our larger enterprise customers have to go through.
00:29:58.079 --> 00:29:59.599
And it's you know it's not an easy process.
00:29:59.680 --> 00:30:01.920
So we're trying to make this as easy as possible for everyone.
00:30:02.000 --> 00:30:08.319
Um with AI, things are changing so quickly and things are coming out so quickly that you need to adapt fast to keep up.
00:30:09.519 --> 00:30:10.480
Yeah, yeah.
00:30:10.799 --> 00:30:22.640
It's so I take it you you can where can people like is there any like any other like learning path or modules that people can go to learn more about the MCP apps?
00:30:22.799 --> 00:30:26.000
Like whether it's through the learn modules or tech?
00:30:27.680 --> 00:30:31.759
We have tutorials on our uh Microsoft Learn Docs.
00:30:32.079 --> 00:30:34.559
Um we are constantly updating those.
00:30:34.720 --> 00:30:41.599
Um we're now implementing some you know agentic processes to keep those updated just because things are moving so fast.
00:30:42.079 --> 00:30:50.720
Um also as a as I mentioned, keep an eye on the and keep updating the the Azure skills that you know your your agents are using.
00:30:50.799 --> 00:31:00.000
Um there's like a pre-packaged Azure Skill, you know, app service as well as all of the other services across Azure are constantly contributing to that and iterating.
00:31:00.079 --> 00:31:14.000
And that's gonna give you, you know, when you ask your agent in VS Code or wherever you're interacting with your coding agents, that's gonna give you the latest information about how to use certain features, how to apply certain features, and you know, recommendations for what features to use.
00:31:14.160 --> 00:31:21.119
So definitely keep that updated and you know, keep talking to that because that's one of your best sources that's coming directly from the product group.
00:31:21.440 --> 00:31:21.759
Okay.
00:31:22.160 --> 00:31:27.200
I think it currently this feature is only support agent framework.
00:31:27.359 --> 00:31:28.640
Is there any plan?
00:31:28.799 --> 00:31:36.240
Maybe you don't have to say to support like maybe uh Langchain or different framework, or is it just an agent framework?
00:31:36.720 --> 00:31:46.000
So this agents feature supports any framework as long as you enable OpenTelemetry and the gen AI semantics.
00:31:46.079 --> 00:31:54.880
So I also just released another doc about that, and that's another link that we can we can share um after this about how to enable this kind of functionality in your app.
00:31:55.039 --> 00:31:58.960
So it's it's framework agnostic and platform agnostic.
00:31:59.119 --> 00:32:01.039
Um great feature.
00:32:01.759 --> 00:32:02.960
Okay, brilliant.
00:32:03.200 --> 00:32:06.480
Uh so thanks a lot for joining this episode, Jordan.
00:32:06.799 --> 00:32:12.400
So hope everyone learned about how amazing apps the apps mcp apps is.
00:32:12.480 --> 00:32:13.920
So I've got to play around with it.
00:32:14.079 --> 00:32:17.279
It's a new tool, so I've got to tell my team colleagues.
00:32:17.519 --> 00:32:20.319
So it's amazing because I know there's two now.
00:32:20.480 --> 00:32:25.759
There is Cosmos DB, there is Function App, there might be there, there was something Logic Office.
00:32:26.000 --> 00:32:29.039
Everywhere they're building AI in Azure.
00:32:29.359 --> 00:32:31.359
So I think things are changing every day.
00:32:31.440 --> 00:32:32.559
It's it's it's hard to keep up.
00:32:32.640 --> 00:32:35.680
So AI and Kubernetes as well.
00:32:35.759 --> 00:32:38.480
So everything in the port is different.
00:32:38.720 --> 00:32:39.039
Okay.
00:32:39.519 --> 00:32:50.559
Uh any last minute advice that you want to give to someone that haven't used it before, a customer that wants to try on, or um, putting me on the spot.
00:32:50.799 --> 00:32:59.119
Um personally, for me, like it is overwhelming keeping up with all of the the news and the features that are coming out.
00:32:59.359 --> 00:33:05.119
So, like whenever you see something, think how you can apply that to your day-to-day.
00:33:05.200 --> 00:33:06.079
And that's what I do.
00:33:06.160 --> 00:33:09.039
I'm like, I see a new feature, someone posts on LinkedIn about something they're working on.
00:33:09.119 --> 00:33:12.160
I'm like, how can I leverage that for my work?
00:33:12.480 --> 00:33:17.359
Um always be curious about you know what's coming out, what's available.
00:33:17.519 --> 00:33:20.480
Um, you know, see what works for you.
00:33:20.799 --> 00:33:28.400
There's just so much out there, and you know, eventually this will all calm down and we'll figure out you know what's actually the best process, what's the best thing.
00:33:28.480 --> 00:33:33.039
Um for now, just you know, just explore, be open to trying things, learning things.
00:33:33.200 --> 00:33:46.160
Um, I spend many hours a week, you know, as my full-time job, just playing around with new features and seeing how how can they apply to app service, how can they apply to my day-to-day to make my day you know easier, faster, quicker.
00:33:46.319 --> 00:33:49.200
Um so yeah, just be open and explore.
00:33:49.519 --> 00:33:50.160
Brilliant.
00:33:50.319 --> 00:33:53.920
Uh so that's it's good to keep fit healthy.
00:33:54.160 --> 00:33:54.480
Okay.
00:33:54.720 --> 00:33:55.279
No, right.
00:33:55.519 --> 00:33:57.039
Thanks a lot for joining this episode.
00:33:57.839 --> 00:33:58.480
Thank you.