[Kevin 00:19]
Welcome to the Copilot Connection.
[Zoe 00:22]
We're here to share with you all the news, insights, and capabilities of the Microsoft Copilot ecosystem across the entire Microsoft stack. I'm Zoe Wilson and I'm an Executive at Avanade in our Modern Work Business, an MVP for M365, a Regional Director, and a Viva Explorer.
[Kevin 00:39]
I'm Kevin McDonnell. I'm an MVP, a Viva Explorer, and the Copilot Strategy and Modern Workplace AI Lead at Avanade. We'll be releasing these episodes as podcasts on YouTube, and we're bringing insights from experts, from the community, from Microsoft, from all the different areas of Copilot, the impact they can make to you and your organization, what you need to do to prepare for them or start implementing them, and even how you can extend them. And in fact, we're gonna make sure we bring you the real experience of what it means with Copilot.
[Zoe 01:14]
So this week, we've got a slightly different format. We've got a few announcements to share with you, and we're also joined today by a very special guest. Roo, welcome to the Copilot Connection. Would you like to introduce yourself?
[Ru 01:29]
Thank you for the invitation, first of all. So good to be with you guys. My name is Roo Campbell. I'm a Microsoft MVP in security, and I lead the Microsoft Security Consultancy Practice, which just rolls off the tongue at Threadscape, and we are a pure play cybersecurity firm. So I do all sorts of work on Intune, Defender, Entra, basically that falls under that big Microsoft security bracket. I'm also the author of a book called Mastering Microsoft 365 Defender.
[Kevin 02:01]
We must get the link to that to put in the show notes, so I'll grab that from you afterwards. That's a good shout. Good stuff.
[Zoe 02:07]
Brilliant.
[Kevin 02:09]
Security in Microsoft, so you just turn it on. It's easy, isn't it?
[Ru 02:13]
Yeah. It's like Copilot for Microsoft 365. You just turn it on and you're good to go.
[Zoe 02:20]
I did like the comment we were about it just rolling off the tongue. I think both Kevin and I definitely suffer from that same thing as well. There's a reason I don't give my full job title on this show.
[Ru 02:32]
It's funny. I was talking to a customer yesterday, and they were talking about, it's almost a meme at this stage, they were talking about how a product had been renamed, and they said, I'm licensed for ATP plan 2, I don't know what it's called now. I was like, it's called Microsoft Defender for Office 365 Plan 2. And I'm like, this is not normal human language that we talk in this industry.
[Kevin 02:59]
But that's why we are here, to try and convert that into real language. Look at that. I'm spending too much time talking marketing. On that, a few announcements before we get into what we're going to chat to with Roo today. Hopefully, you've seen, and we hope you've enjoyed them. We've put through 12 episodes from the MVP Summit of the interviews we did, so please go back and look at those. I haven't thought so about this, but Zoe, maybe in one episode we should have a chat about the kind of the thoughts and the common themes and also the bits that jumped out from those. I know certainly things like the slightly more cynical view of Mark Anderson I thought was fantastic to bring that balance on there. So we might come back and do a summary and recap of those. We've also got an announcement.
[Zoe 03:45]
Oh, sorry, Jo. Yeah, so on that, so first of all, I think that's a great idea, Kevin, but we do actually have a bonus 13th MVP Summit interview as well, which we will share with you. Federico, if you're listening, I'm really sorry that we haven't done this yet, but we had one of those incidents where the battery died. Yeah, so the battery died halfway through the interview. We need to stitch them both together. So bear with us, the 13th bonus MVP Summit mini episode will be with you shortly.
[Kevin 04:14]
I was going to mention that, but good idea, yes. We'll get that one out for Federico as well on that. So yes, look out for that one. And Zoe, we have a big announcement that we made this week.
[Zoe 04:26]
We do. So one of the things that we've talked about right from our very first episode is the reason that we set this podcast up in the first place. And Kevin and I, we've both been working around this Copilot space for the last year now. And what we saw was just this deluge of marketing terms and branding and products and licensing.
[Kevin 04:49]
Questions.
[Zoe 04:50]
And renaming and all of the myths that we've busted in one of our previous episodes. And what we really wanted was just a way to kind of drive that education and help cut through some of the noise and the marketing and help people understand what the actual reality is of the different Copilot products. So to continue in that vein, we're turning June into the month of Copilots.
[Kevin 05:15]
Absolutely, so we're getting a set of speakers from all around, from Microsoft, from the community on there and we're gonna ask them to share sessions. So we're gonna put those into sort of different playlists. We'll have episodes released every day throughout June, covering across all those different areas all the different Copilots that are available within there but also covering different aspects so we'll have people doing demos showing what they are but having people talking about why you would use them how's the best way to use them and as much real experience on those as well so keep an eye out for that if you are a speaker or want to talk about those i i'm going to be honest especially if it's not Copilot for microsoft 365 because we have a lot of people talking about that so please do submit if you want to talk about there but we want to see those wider areas as well we'd really love to to be bring that large range of things to on there so please check out the site i'll put in the show notes uh link to that as well and yeah.
[Zoe 06:17]
Just just to add as well i think um it's just worth saying you know we're particularly interested like kevin said in that real world experience so if you can come and talk about the the actual hands-on stuff that you've done with Copilot, regardless of which Copilot it is, if you can talk about the lessons learned, the things that you might do differently, the things that have worked, where you found value, how you help people use it. You know, we want to move from having a lot of marketing content and readiness into what the actual reality is of this Copilot world. So I'm.
[Kevin 06:56]
I'm chuckling there because content that's a much better word than BS that I was thinking. So yeah. So there are a few other bits of news that's come out. There's things like the AI hub that's going to be built in London by Microsoft. We've got Gemini, Google's Gemini and Zoom workplace of announcing things. But rather than talk about that here, we're going to put that into a newsletter. So look out for the LinkedIn newsletter and we'll put links to that into the show nuts as well. But as we finally got a bit of time, I think we should get on to the main topic for today, which Roo. I've loved your security stuff for a long time. You're still my favorite session at the Scottish Summit on there. But you tweeted recently about Copilot for security and that caught my eye. And the second I saw that, I was like, we definitely need to get Roo into the show. Before we get too far into that, maybe you could kind of intro for for people who don't know what Copilot for Security is and what it does. Maybe we should kick off talking a bit about that.
[Ru 07:58]
Sure, absolutely. It was released to general availability on 1st of April, having previously been in early access with a number of Microsoft security partners.
[Kevin 08:08]
And really what it is- Not a joke, not a joke.
[Ru 08:11]
No, yeah, that's a... Maybe as we get on in the podcast, we'll talk about the interesting timing of the release date. but it's essentially the same type of approach you would expect with Copilot across the rest of the Microsoft stack, so generative language over Microsoft's security portfolio. So you experience Copilot for security really in two different ways. Number one is what's called the standalone experience, where you go to securityCopilot.microsoft.com, and it's just a single webpage that's prompts, prompt, response, prompt, response, the kind of folks that is going to be quite intuitive to folks that have used any type of Microsoft Chat, things like that. The other type of way you would experience Copilot for Security is what's called the embedded experience, which is where if I go to any of the other portals, like security.Microsoft.com, intro.Microsoft.com or compliance.Microsoft.com, really just embedded on the right-hand side of the screen is that little dialogue window. It can do things such as summarize security incidents. For example, if you get Defender for Endpoint, Defender for Office 365, etc. If those generate an incident, when you go to that incident now, it will quite well actually summarize a nice natural language to say, hey, based on all the data that's produced this incident, here's a few paragraphs to explain what has happened and what you might want to do about it. In my experiments so far with it, that's really the main use case is incident response. For example, if you think about security as either being left of bang or right of bang, where the bang is your actual point of breach or compromise, for that right of bang, it's very good. Yeah, I use that quite a lot. It's very useful. That's from the NIST cybersecurity framework. And so that's what Security Copilot intends to do. And what I've been experimenting with since the 1st of April is really kind of putting it through its paces to see has the marketing caught up with the reality or vice versa to figure out how good is this thing.
[Kevin 10:31]
So just quickly, before we get into that, which I think will be the fun bit of the show, I think some of the elements, I loved what you talked about, sort of summarizing the incident. I think there's that power to be able to summarize that to other people. You know, when you're there going through an issue that's happening, one of the important things is communicating out. And Copilot brings you that kind of ease of putting natural language. You can kind of churn out, ah, something's going horribly wrong, and we've got a problem, and turn that into a slightly more managerial speak that you could send out to other people, which is particularly useful on there. And I think it's meant to help you. It's interesting, you talk about the right of bang. It's intended to be able to look at logs and things that have happened that may indicate something that is happening as well. And I suspect we'll come on to that in a sec, but there's that element too. I think one other thing and something dear to my heart, extensibility. You can extend it into other platforms as well. So you can bring in your kind of other security tools that it allows allows you to bring into that element of it as well.
[Ru 11:39]
Correct, yes, so it supports plugins which vendors can develop. So for example, if I'm a third-party security developer, Tanium is one that comes to mind. If I have a solution, I can integrate it with Copilot as well. So as part of my prompts, I could say, okay, well, here's this information, now query third-party data source as well. And really one of the other ways that it's become kind of extensible and customizable rather than just being like a chat engine, is this thing that are called prompt books, which may be, let's say, an incident happens. I'll make it really simple. Let's say user involved in some kind of suspicious sign-in. Well, rather than me having to remember or manually type out my prompts to find out more information about this incident, I can just click a button, and it will say, well, here are seven prompts that a security architect has designed in the past that will help you get the type of information you want. For example, it will automatically say, show me the risky sign-ins associated with this user, show me the activity associated with that user, and you get all that just a click of a button. That's really one of the compelling use cases for it, is that really fast response versus previously on a manual investigation I had to do.
[Kevin 12:54]
I was chatting to someone at the South Coast user group, and they were looking at that capability to look at a script that's been run to understand what it did, and they used it in a live incident, and they said the speeds with which that script could be broken down to understand the impact and what it was trying to do, even where quite often in these scripts that run, that try and hide that they're not named nicely within there. They said the power of that was really, really strong.
[Ru 13:24]
Yeah. It depends how quickly you guys want to get into the watch and all approach that I've taken to security for Copilot, We can absolutely talk about that. But yes, one of the described features of Copilot for Security is this capability where, let's say a suspicious PowerShell script runs on a device, and maybe it's hundreds, thousands of lines. Well, for me to look at that script and figure out what's going on, number 1, I have to be skilled in the language to understand it, and number 2, I have to have the time to do that. Idea behind Copilot for Security is I could upload that script, and it can perform that analysis far quicker than I can, and then just tell me in natural language what is actually going on here?
[Kevin 14:07]
Okay, so I think one more before we get on to the warts and all. In terms of licensing you, and this will definitely come into the warts and all from your tweet thread, you pay for, and I forgot the name, but basically the amount it's used. So you have different units that give you the ability to run prompts for certain volumes, and it's that that sort of builds up. and I probably talking about that is one of the first things to chat about. Actually before we get onto that, if anyone's thinking about security and maybe invaders happening, know that meowing in the background wasn't in your head. So he does have a couple of invaders going on for those watching on video behind.
[Zoe 14:49]
You. Yeah as the home invasion.
[Ru 14:52]
That's the that's the defender ninja cat in the background. That's great. I got I got a couple of cats but luckily they're asleep just now, so we'll see.
[Zoe 15:02]
They normally sleep at this time, but my other half's just gone. He's actually just gone to do a security exam, so I opened the door and all of the cats have decided that now is the best time to come and get involved in my life.
[Ru 15:16]
Love it, love it. Yeah, so the point there, Kevin, about pricing, and you guys can keep me honest here because you maybe know the other Copilots better than I do, specifically Copilot for M365. But my understanding is most of these other Copilots are licensed kind of per user per month in a typical M365 fashion, right? And I think there's things like minimum commitments and things like that. Yeah. Copilot for security, I have to constantly keep reminding myself it's not called security Copilot. Copilot for security, It is really licensed on the same type of mechanism you would license Azure resources. You think about Azure, you think of, well, the price I pay for an Azure Virtual Machine or Azure Storage is consumption-based. VMs, I'm going to pay for however long that VM is allocated and running within Azure. With Copilot for security, it gets a little bit abstract. They've developed a measure that is called the SEU, or the Security Compute Unit. And like I said, it's a little bit abstract. But if you roughly translate that, it really relates to, well, what is the compute power required to run these AI models? In my head, I can just think of, well, it's reserving compute power for me, be it in virtual machines, GPUs, et cetera, et cetera, that kind of stuff. And it's all abstracted away from you. But fundamentally, that's what you're paying for, is this thing called an SCU. And you pay for that per hour. Again, very similar to like an Azure virtual machine. The cost at the minute is four US dollars per hour. And you cannot kind of at the minute do it on demand. So for example, let's say I don't have a 24 by seven operation and I might only need Copilot for security from 8-5 or something like that. If I don't want to pay out with those hours, I have to go to the Azure portal and completely delete it.
[Kevin 17:27]
Don't worry, you won't get attacked during the evening, it's fine.
[Ru 17:31]
Yeah, exactly. You raise a good point because that really explains where Copilot for security's use cases are, which is in that 24 by 7 security operation center. If you follow through on Microsoft's recommendations, they recommend that you reserve three SEUs per hour at $4 per hour. If you leave that running 24 by seven, that comes in at just over $100,000 per year that Microsoft recommended. Where I approach this, I approach it in the way that I approach all things in security when folks ask me, well, is it good? Is Copilot for security good? Is Defender good? is entry good, whatever, the point is, compared to what? And we are saying, compared to over $100,000 per year, roughly speaking. And you have to go through your own assessment of, what else could I spend $100,000 per year that's, frankly, the cost of maybe an extra human in your team, right? So the bar is very, very high when we consider how good and how useful as Copilot for security.
[Kevin 18:44]
But, you know, I'm gonna, I'm taking the positive track on this. There could be that you could replace two or three people to some degree, sorry, you could give.
[Zoe 18:55]
The capability. We don't.
[Kevin 18:55]
Like, we don't like that. Yeah, yeah, as I said that, not replace people, but you could allow that growth for people there that you wouldn't, and when I said replace, rather than have them kind of monitoring and dealing with scenarios, they could be doing things which bring additional value to the kind of longer term map from that. And you could, in theory, if Copilot for Security did all it promised, have them focusing on higher value items while the Copilot sat with one person to kind of go through that. Yeah. And how was your experiences of it being able to replace, being able to support that person.
[Zoe 19:36]
Augment, augment.
[Ru 19:37]
Augment.
[Kevin 19:38]
Augment, that's a good one. I do genuinely mean this. This is my bad language coming through with things that it should be on there. So I'm.
[Ru 19:46]
Going to get you on. No, you raise a good point insofar as look, and it's become very apparent that at least not in the very short term, AI is all about improving your efficiencies of the folks that are already there rather than being full on replacement for hiring new folks. That's kind of what's become apparent to me anyway, at least in the security space. And when we talk about, well, what are the improved efficiencies that someone might see? Again, I lean on that kind of place where if I am an incident responder, and I'm sitting in Defender or Sentinel, and my shift in my job pattern as I'm looking at inbound alerts coming in, I'm triaging those to figure out, well, where should my priorities be? What do I need to know very, very quickly. These AI models do a very good job of very quickly telling me what is, translate all this data and give it to me in language I can just read, like I would read a memo, you know what I mean? So that part is very good for it. And again, it does boil down to that question of is the increase in efficiency worth the cost? And every organization has to make that that determination themselves, right? From my experience so far, it's a tough justification, but I can see some circumstances that folks could do.
[Kevin 21:07]
And it's a tricky one with things like this. And again, it's the same with all security. You're paying to avoid a risk happening. And if something happens to reduce the impact of that, it's gonna be a very hard calculation to make. And I was chatting on the, it depends that Stephen Moraine runs. And it's like, you've got to balance that productivity versus the risk. You could block everything happening, but what is the impact of that? And actually, the impact of a security event, 100 grand could be absolutely nothing to certain organizations if something happens there. So it's balancing not just the cost of what you would be not doing by having someone more focused on that, but also the cost of not being able to react quickly. When those incidents happen, reacting quickly is key if you can do it.
[Ru 22:00]
Well. Yeah, yeah, absolutely. And that kind of, there's a whole kind of field and geez, it almost gets into this kind of philosophical slash theological discussion about, well, how much risk are we willing to tolerate? And more importantly, how do we actually measure that risk? That's, in my experience, very hard to do in cybersecurity. And we're always gonna have businesses that have that spending power to be early adopters of things like the Copilot for security. And for those folks, I'm kind of like, go for it. But if we think of the mass market, particularly the smaller to medium-sized business where every penny is a prisoner and we have to be very careful with how we spend our money, my kind of advice, at least for now, is let the bigger businesses figure out this beast. And then when it's more mature, maybe that's when you start looking into it. That's very, very high level opinion of me at the minute.
[Kevin 22:58]
Yeah. So coming back to some of the things we talked about, the nice bits, and we said we'd come back to, maybe analyzing scripts. And I saw all sorts of that story I'd heard that it did very well. And you seemed a bit more cynical about that.
[Ru 23:13]
I can only speak from experience so far, right? That's all I kind of have. And again, everything I say is based on the fact that this is very, very expensive. In my testing, I took four different PowerShell scripts that had gone through some type of obfuscation. So it kind of muddies up what exactly is this script doing? And I asked Copilot for security to read it and tell me what was.
[Kevin 23:44]
Happening. These were scripts that you'd seen happen in a security incident or what sort of ones you wrote?
[Ru 23:52]
Great, so these aren't ones I wrote, are ones that are kind of available for testing and things like that. So if I give you an example, Defender antivirus can block scripts running that are obfuscated. I took some of the examples I use when I'm testing that, and I threw them into Copilot, and I said, hey, assess these scripts and tell me what's going on here. And regretfully, in my testing, it simply couldn't do it. It spun away, and it spun away, and then it would eventually error out. Now, where things got got very interesting was if I copy and pasted that script into the Microsoft Copilot, previously called Bing Enterprise Chat, it could do it. Like I said, I can only speak from my experience, but that was not the best first impression.
[Kevin 24:41]
That's why I want to check whether the scripts, these are fairly standard ones that are out there. That's a little surprising, interesting. Yeah, correct. I guess.
[Ru 24:51]
The point is, even if it wasn't standard, at the end of the day, the bar for security is so high, and to your point earlier, the time to respond is so critical that, what would you say? I shouldn't really have to just test this with standard scripts, I should be able to throw this thing very demanding tasks and get good results.
[Kevin 25:13]
Yeah, interesting. And so, other areas, because I think you talked about in your thread also the cost, and there's going to be a lot of people who want to try this out, not necessarily run it, especially, as you mentioned, those smaller orgs who want to turn it on for five, ten minutes, give it a go. And I think that was interesting and didn't necessarily reflect what you'd expect there as well.
[Ru 25:40]
Yeah. I mean, look, one of the really good bits of news is because it is billed as an Azure resource at the starting point being $4 per hour for one secure computer unit. If you want to kick the tires on this thing, give it a spin in your own environment, that really lowers the cost for entry, right, you know, for just a few bucks. You can play with this thing for a few hours, see how it goes. But again, my experience is you very quickly run into the limitations of that one secure unit per hour. So I did a thread on Twitter, and I maybe ran somewhere in the region of 10 to 15, maybe at most 20 different prompts, all within the space of one hour. And you can look in the dedicated portal at your usage, so it'll say, hey, you've used x number of secure compute units this hour. It actually lets you go over your allocation before it throttles you. In that timescale, just looking at the numbers now, I had used rounding up to 16 secure units per hour for roughly 15-20 prompts. That's going to cost me $4 in that hour. If you assume that same demand, 24-7 in a security operation center, that would cost $48,000 per month. So at that scale, if you are demanding a lot from this thing because of the consumption-based model, you just have to be very conscious of price. And again, it goes back to the point where you have to make that determination with you and your own team as to, I've got a finite budget, where's the best place for me to spend this?
[Kevin 27:23]
Yeah, and I think it's fair to say that when you're trialing things out, there's a very different pattern of usage and you would see in a real environment from there as well. So it's looking at those different elements that come through from that, that needs to be considered. Sorry, I sound like the Microsoft fanboy that's apologizing for them, but I think it's true. I've seen this where I've done this from experience. Cosmos DB is a great example where I first start to use that and go, we can't pay this much. but then actually the real usage that levels out over time is it works better within there. So it's certainly something worth considering, and I love your kind of analytics of these things, but this may be different over time as well. So don't let it 100% put people off.
[Ru 28:16]
Yeah. I mean, at the end of the day, we're recording this and the 1st of April, it really wasn't that long ago, right? We were in the very, very early stages of this thing. And I already know based on, it was on the Microsoft Security Insights podcast, which Rod Trent at Microsoft hosts, and he's already said that they are looking at not the cost as such, but the ability to provision things on demand, clarifying some of the minimum costing and things like that. So one of the, and really this is kind of where I try to, at least in my my own little way is keep Microsoft honest. I think that's one of the key things we have to do as MVPs, is if we see problems, we need to highlight them. It's really just explain, well, here's what folks are experiencing with this and to their credit, I've always found that Microsoft are very, very good at course-correcting very, very quickly because they're an incredibly successful business and you don't become an incredibly successful business without doing that.
[Kevin 29:17]
Yeah, and I noticed that I was just having a look through your thread and reminded myself some of the things I like that was intriguing was one of the benefits of Copilot is being able to have a natural language, being able to interrogate that data as you said, and you kind of got mixed results from some of that. Some worked really well, some not quite so well, and I think you talked about the KQL generative, which is the query language for querying those logs wasn't amazing at times on there. But I think if I go back to Copilot for Microsoft 365, and I know Zoe, you were one of the very early ones there, the prompts that happen initially there versus what you do now is very different. And those prompts as they evolve, I think, will be tailored better towards things that work. And people will find the things that work and share those and gear.
[Zoe 30:05]
With us on that. Yeah, I mean, I think that comment that you made, Ru, about how early this is in the journey is really important, because for Copilot for M365, if you think about when that kind of first got into the hands of more people at the start of the early access programme in July, it's almost a year, it's like nine months since now, which means we've had nine months of people testing it, feedback going to product group, improving, iterating, adding in new features, new capabilities, And even though from a marketing perspective, Microsoft have been talking about all of the Copilots for quite a long time now, in terms of what's actually been in the hands of people from a product perspective, for a lot of the Copilots, it's really early days. So I think it's important that people have an appreciation of that and recognize that there are things that you'll be able to do now. And if the journey that we've seen with Copilot for M365, it holds true for all of the others, these will mature and evolve, and by the end of this year, we'll see much richer capabilities and possibilities across all of them.
[Ru 31:13]
Yeah, 100 percent, and it's one of those things where I don't use Copilot for Microsoft 365 day-to-day, but purely anecdotally, I've got customers who I maybe spoke to them maybe a few months ago when they downsized the requirements for it, and they'd play about with it, and they're like, okay, first impressions is it's fine, but it's not great. And then in the coming months, when I spoke to them again, they're like, you know what, I'm using this thing more and more, because number one, I'm getting better at understanding how to get the best out of it. But number two, it seems to be improving as well. And by both of those things improving, the overall level improves. And I think with Copilot for Security, you're gonna see that too. The service will improve its ability to interpret your prompts and get you the data you want will improve, but you'll become better at understanding how to prompt it to get what you want. Over time, those two things will converge to improve the experience.
[Zoe 32:04]
Yeah. Then the third thing that we'll weave into that is that extensibility as well. As people build more connectors, as people find new ways to extend and integrate these Copilots into our business processes and other systems, we'll start to see that up-leveling, that value that people find from it.
[Kevin 32:30]
Yeah, and Ruth, just looking from your screenshots, the one things I love about Copilot for Microsoft 365 and quite a few of the other ones is you have that kind of thumbs up and thumbs down to kind of feedback.
[Ru 32:41]
Yes.
[Kevin 32:42]
Is that not existing Copilot for security or has that just been trimmed off?
[Ru 32:46]
Great question. No, you can give it feedback as you go. So for example, if I prompt for something and it gives me the results of that, I can either say, hey, great job, which presumably informs the back-end of it. Or alternatively, I could say, hey, you know what, this wasn't a good job, here's my feedback. Actually, you see that quite a lot. I know in the Defender portal, there's obviously not AI-based, but you can provide feedback. Again, that's one of the things I do quite enjoy about working with the Microsoft products is, I know that feedback does actually go to, in most cases, real human eyes. That's quite a good feature. If folks are early adopters of Security for Copilot, I would recommend they give as much feedback as possible.
[Kevin 33:27]
Yeah. I'm also getting distracted by my favorites. We mentioned, oh, vulnerability in gaming services in those on playing Halo.
[Ru 33:38]
Yeah. Actually, that brings me to a great point. This is one of the things that's going to be smuggled in there and missed in a lot of the public knowledge of what Copilot for Security brings. Microsoft have a separate service called Microsoft Defender Threat Intelligence, which is a very expensive service in and of itself, but it does what it says on the tenant as far as it will give you threat intelligence about, okay, well, who are the bad actors out there? What are their known attack patterns? The target audience for that is potentially a security operation center, so you know your adversaries and who you may be up against. That is included in the price of Copilot for security. If you were looking at that and thinking, well, I'm interested in that, then it's another incentive for you to test this out.
[Kevin 34:30]
Interesting. I've noticed that Microsoft's done a few packaging with things, and we talked about Copilot for sales and service, having Copilot for Microsoft 365 is a core part of those. That's really useful to know. I know we're going to have to wrap up soonish, and it'd be great to summarize this, because I feel like Zoe and I are doing a nice positive marketing spiel to kind of balance out your real world feedback from it. So I want to try and bring some of that together, because that's just my natural balance way of things. It feels to me the important things.
[Ru 35:06]
Is.
[Kevin 35:07]
Probably right now, if you're looking at Copilot for security, we're looking at larger organisations or certainly ones that can invest a good amount of money into using this and supporting the teams they have to allow that growth with the right people there. Your kind of SMBs are certainly not looking at this area right now. So we're looking at those larger enterprises as the focus for this. We're looking at understand, you know, one of the key things you want to do is understand what your usage patterns are going to be, you know, how many people are looking at using it, understanding what the cost of that and balancing the value that it can bring, which can be large in the right circumstances, along with the risk of not doing that, as well. It's looking at those and understanding what can be done, the sort of scenarios you'd be looking at, the benefits from there. Is it fair to say, and you talked about, I love that sort of left of bang and right of bang, you're looking at the core value being that right of bang at the moment, being able to analyze things which might happen isn't great there at the moment.
[Ru 36:17]
Yeah, that's a very good point. In the documentation, Microsoft do describe kind of the, I think it's four key scenarios that Copilot Security is useful for. Incident response, threat hunting, intelligence gathering, and posture management. The incident response and the threat hunting is pretty good at. The posture management, So that's everything before a breach, which has to do with architecture, things like that. Again, personal experience so far, we're not there yet. So for example, I asked Copilot for security to say, tell me all the users who have a security key registered. And it just kind of said, no, sorry, not for me. And I'm like, okay, cool, so we're not there yet. But the point is, if you're looking at this from a security architecture point of view to find your gaps, potentially not as useful. I think that where the real value will be is when it can get there because if I'm a security architect, looking at exclusions, looking at groups, you folks will have been in some of the biggest Microsoft 365 tenants in the world in your consultancy time, and these can get very messy very quickly. Finding out gaps, finding out users who exist, who don't need to exist anymore, permissions, sensitive data that exist that doesn't need to exist anymore, that's a real challenge. Where I'm very optimistic is that AI, geez, I'm going to start sounding like a marketing person now, is I think AI can hopefully help us on that because it can reason and it can translate the important stuff very, very quickly. Yet to see it, but that's where I'm optimistic.
[Kevin 37:56]
We talked about Copilot for Microsoft 365, that journey always feels like it is improving. I think it's that expectation that may have been sold some people of what's there right now is worth looking at and understanding and making sure you know what you're getting from it.
[Zoe 38:14]
I think one of the things that will be interesting to me is that if we get to a point where organizations have got a Copilot for M365 and Copilot for security, if I work in the security team and I can just use Copilot chat and have that as my interface into the Copilot for security functionality you know that's that's the type of like next level I want the products to work and then I want this next level integration.
[Kevin 38:40]
I like that idea but I can't help thinking security people just avoid teams and outlook and things like the play I imagine them on black screens and just doing sort of text.
[Zoe 38:50]
Chat and thinking that.
[Kevin 38:51]
Maybe this is my own uh stereotypical view but.
[Zoe 38:55]
Yeah you've been watching too many science fiction off uh science fiction or thriller films came in. One question I've got for you Ru or a comment I guess, we'd really love to see you submit for a month of Copilot because I think you've got exactly the.
[Ru 39:09]
Time. You.
[Zoe 39:10]
Don't need to tell us now what you might talk about but I think you know if we look at.
[Kevin 39:16]
You. As long as it's not Copilot for Microsoft 365 then.
[Zoe 39:19]
Yeah it's two months away so hopefully by then you've got like even more kind of real world experience and insights that will be useful for other people who are at the start of the journey?
[Ru 39:30]
Yep, nope, cool, okay, right, you put me on the spot, so I'll do it. I'll get something to submit to you guys and we can talk about it.
[Kevin 39:35]
And I think also what I'd love to is maybe get you back in six months or so time to see if you've been using it more and see how that experience has changed. It'd be really interesting to kind of see that journey, whether it's, yep, we stopped, or no, we found that extension with it works better, or actually it's improved in itself. So it'd be interesting to see that evolution as.
[Ru 39:56]
100%, I'd love to come back and talk about it as it grows, because the reality is it's not going anywhere. And therefore, I'm not going to ignore it. I'll keep dipping in and out and finding use cases. And I guess, look, we've got a wrap up. But if I'm thinking of my own conclusions to this, I'm going to articulate this much clearer in a blog I've got coming out soon, which I'm co-authoring with other MVPs who have other and better feedback from it, so that it's more general feedback. The main thing in my advice to customers so far and what I'm leaving on and where I think it'd be very useful to review maybe in six months time is all about it's the cost efficiency question. And so far as we only have so much spending power as defenders and we have to be very selective about where we put our time and our budget. And for me, it's a question of, well, for every buck and every minute that I spend in Copilot for security, that be better spent somewhere else? And that's where I'm currently wrestling with at the minute, and I'll get my conclusions out there at some point, and it'll be good to review that again in six months.
[Kevin 41:01]
Because I would have thought that time to start using it is a lot quicker. You know, you've got some useful things from playing with it within minutes, so understanding that versus a more complex system that you have to train people up. I know for a lot of the Copilots, as we talk about leveling the playing field. That's an interesting one. Anyway, that was a lovely wrap up and I've just ruined it by extending that. But no, that'd be fantastic to see. I'd also say, you mentioned Rod Trent earlier. He's actually got a LinkedIn group about security for Copilots. He does, yes. And I'll put the links in for that into the show notes. So a good place to ask questions and see what people are being shared and talking about within there as well.
[Zoe 41:42]
Brilliant.
[Ru 41:42]
Yeah, Rod has a lot of great content.
[Zoe 41:44]
Yeah, yeah 100%. So we will have a whole heap of links in the show notes for this session as well. It's been really great having you on Roo. I think you're actually the second in person, is that right Kevin, the second in-person guest that we've actually managed to schedule a session with.
[Kevin 41:59]
And.
[Zoe 42:00]
You remember to.
[Kevin 42:01]
Get the cold this time.
[Ru 42:05]
Oh.
[Kevin 42:05]
Dear, sorry.
[Ru 42:06]
Tricia.
[Zoe 42:10]
Brilliant. Well, thanks so much Riri, it's been an absolute pleasure. This was a really great discussion and for me I feel like I've learned a huge amount about Copilot in the security space. We've got lots of big things planned for future episodes, we'll have more experts joining us, we will get better at actually scheduling people as well as grabbing those in-person interviews, and we really hope that there's lots of you out there who have been getting hands-on with Copilot and we're going to fill in that call for speakers for month of Copilot.
[Kevin 42:40]
Absolutely, and if you do want to hear more, please let us know which areas of Copilot you'd love to hear on, who you'd like to hear from, and please, I know you hear us on all your podcasts, smash that subscribe button, go and follow us on YouTube, on all the social medias, we're out there, add us to your favourite podcast app, add your friends to, add your clients to, add your family, we're here to kind of share the knowledge to everyone and Copilot will affect everyone from there. So please encourage as many people as possible to. Otherwise, thank you very much. Oh, and if you fancy writing a review, that would be even better because apparently that helps and I keep forgetting to mention that. So add a review in your favorite apps as well. But otherwise, thank you so much for joining us, Ru, and we'll be back in a couple of weeks.
[Ru 43:49]
Thanks everyone. Cheers guys. Bye. Bye.
Welcome to the Copilot Connection.
[Zoe 00:22]
We're here to share with you all the news, insights, and capabilities of the Microsoft Copilot ecosystem across the entire Microsoft stack. I'm Zoe Wilson and I'm an Executive at Avanade in our Modern Work Business, an MVP for M365, a Regional Director, and a Viva Explorer.
[Kevin 00:39]
I'm Kevin McDonnell. I'm an MVP, a Viva Explorer, and the Copilot Strategy and Modern Workplace AI Lead at Avanade. We'll be releasing these episodes as podcasts on YouTube, and we're bringing insights from experts, from the community, from Microsoft, from all the different areas of Copilot, the impact they can make to you and your organization, what you need to do to prepare for them or start implementing them, and even how you can extend them. And in fact, we're gonna make sure we bring you the real experience of what it means with Copilot.
[Zoe 01:14]
So this week, we've got a slightly different format. We've got a few announcements to share with you, and we're also joined today by a very special guest. Roo, welcome to the Copilot Connection. Would you like to introduce yourself?
[Ru 01:29]
Thank you for the invitation, first of all. So good to be with you guys. My name is Roo Campbell. I'm a Microsoft MVP in security, and I lead the Microsoft Security Consultancy Practice, which just rolls off the tongue at Threadscape, and we are a pure play cybersecurity firm. So I do all sorts of work on Intune, Defender, Entra, basically that falls under that big Microsoft security bracket. I'm also the author of a book called Mastering Microsoft 365 Defender.
[Kevin 02:01]
We must get the link to that to put in the show notes, so I'll grab that from you afterwards. That's a good shout. Good stuff.
[Zoe 02:07]
Brilliant.
[Kevin 02:09]
Security in Microsoft, so you just turn it on. It's easy, isn't it?
[Ru 02:13]
Yeah. It's like Copilot for Microsoft 365. You just turn it on and you're good to go.
[Zoe 02:20]
I did like the comment we were about it just rolling off the tongue. I think both Kevin and I definitely suffer from that same thing as well. There's a reason I don't give my full job title on this show.
[Ru 02:32]
It's funny. I was talking to a customer yesterday, and they were talking about, it's almost a meme at this stage, they were talking about how a product had been renamed, and they said, I'm licensed for ATP plan 2, I don't know what it's called now. I was like, it's called Microsoft Defender for Office 365 Plan 2. And I'm like, this is not normal human language that we talk in this industry.
[Kevin 02:59]
But that's why we are here, to try and convert that into real language. Look at that. I'm spending too much time talking marketing. On that, a few announcements before we get into what we're going to chat to with Roo today. Hopefully, you've seen, and we hope you've enjoyed them. We've put through 12 episodes from the MVP Summit of the interviews we did, so please go back and look at those. I haven't thought so about this, but Zoe, maybe in one episode we should have a chat about the kind of the thoughts and the common themes and also the bits that jumped out from those. I know certainly things like the slightly more cynical view of Mark Anderson I thought was fantastic to bring that balance on there. So we might come back and do a summary and recap of those. We've also got an announcement.
[Zoe 03:45]
Oh, sorry, Jo. Yeah, so on that, so first of all, I think that's a great idea, Kevin, but we do actually have a bonus 13th MVP Summit interview as well, which we will share with you. Federico, if you're listening, I'm really sorry that we haven't done this yet, but we had one of those incidents where the battery died. Yeah, so the battery died halfway through the interview. We need to stitch them both together. So bear with us, the 13th bonus MVP Summit mini episode will be with you shortly.
[Kevin 04:14]
I was going to mention that, but good idea, yes. We'll get that one out for Federico as well on that. So yes, look out for that one. And Zoe, we have a big announcement that we made this week.
[Zoe 04:26]
We do. So one of the things that we've talked about right from our very first episode is the reason that we set this podcast up in the first place. And Kevin and I, we've both been working around this Copilot space for the last year now. And what we saw was just this deluge of marketing terms and branding and products and licensing.
[Kevin 04:49]
Questions.
[Zoe 04:50]
And renaming and all of the myths that we've busted in one of our previous episodes. And what we really wanted was just a way to kind of drive that education and help cut through some of the noise and the marketing and help people understand what the actual reality is of the different Copilot products. So to continue in that vein, we're turning June into the month of Copilots.
[Kevin 05:15]
Absolutely, so we're getting a set of speakers from all around, from Microsoft, from the community on there and we're gonna ask them to share sessions. So we're gonna put those into sort of different playlists. We'll have episodes released every day throughout June, covering across all those different areas all the different Copilots that are available within there but also covering different aspects so we'll have people doing demos showing what they are but having people talking about why you would use them how's the best way to use them and as much real experience on those as well so keep an eye out for that if you are a speaker or want to talk about those i i'm going to be honest especially if it's not Copilot for microsoft 365 because we have a lot of people talking about that so please do submit if you want to talk about there but we want to see those wider areas as well we'd really love to to be bring that large range of things to on there so please check out the site i'll put in the show notes uh link to that as well and yeah.
[Zoe 06:17]
Just just to add as well i think um it's just worth saying you know we're particularly interested like kevin said in that real world experience so if you can come and talk about the the actual hands-on stuff that you've done with Copilot, regardless of which Copilot it is, if you can talk about the lessons learned, the things that you might do differently, the things that have worked, where you found value, how you help people use it. You know, we want to move from having a lot of marketing content and readiness into what the actual reality is of this Copilot world. So I'm.
[Kevin 06:56]
I'm chuckling there because content that's a much better word than BS that I was thinking. So yeah. So there are a few other bits of news that's come out. There's things like the AI hub that's going to be built in London by Microsoft. We've got Gemini, Google's Gemini and Zoom workplace of announcing things. But rather than talk about that here, we're going to put that into a newsletter. So look out for the LinkedIn newsletter and we'll put links to that into the show nuts as well. But as we finally got a bit of time, I think we should get on to the main topic for today, which Roo. I've loved your security stuff for a long time. You're still my favorite session at the Scottish Summit on there. But you tweeted recently about Copilot for security and that caught my eye. And the second I saw that, I was like, we definitely need to get Roo into the show. Before we get too far into that, maybe you could kind of intro for for people who don't know what Copilot for Security is and what it does. Maybe we should kick off talking a bit about that.
[Ru 07:58]
Sure, absolutely. It was released to general availability on 1st of April, having previously been in early access with a number of Microsoft security partners.
[Kevin 08:08]
And really what it is- Not a joke, not a joke.
[Ru 08:11]
No, yeah, that's a... Maybe as we get on in the podcast, we'll talk about the interesting timing of the release date. but it's essentially the same type of approach you would expect with Copilot across the rest of the Microsoft stack, so generative language over Microsoft's security portfolio. So you experience Copilot for security really in two different ways. Number one is what's called the standalone experience, where you go to securityCopilot.microsoft.com, and it's just a single webpage that's prompts, prompt, response, prompt, response, the kind of folks that is going to be quite intuitive to folks that have used any type of Microsoft Chat, things like that. The other type of way you would experience Copilot for Security is what's called the embedded experience, which is where if I go to any of the other portals, like security.Microsoft.com, intro.Microsoft.com or compliance.Microsoft.com, really just embedded on the right-hand side of the screen is that little dialogue window. It can do things such as summarize security incidents. For example, if you get Defender for Endpoint, Defender for Office 365, etc. If those generate an incident, when you go to that incident now, it will quite well actually summarize a nice natural language to say, hey, based on all the data that's produced this incident, here's a few paragraphs to explain what has happened and what you might want to do about it. In my experiments so far with it, that's really the main use case is incident response. For example, if you think about security as either being left of bang or right of bang, where the bang is your actual point of breach or compromise, for that right of bang, it's very good. Yeah, I use that quite a lot. It's very useful. That's from the NIST cybersecurity framework. And so that's what Security Copilot intends to do. And what I've been experimenting with since the 1st of April is really kind of putting it through its paces to see has the marketing caught up with the reality or vice versa to figure out how good is this thing.
[Kevin 10:31]
So just quickly, before we get into that, which I think will be the fun bit of the show, I think some of the elements, I loved what you talked about, sort of summarizing the incident. I think there's that power to be able to summarize that to other people. You know, when you're there going through an issue that's happening, one of the important things is communicating out. And Copilot brings you that kind of ease of putting natural language. You can kind of churn out, ah, something's going horribly wrong, and we've got a problem, and turn that into a slightly more managerial speak that you could send out to other people, which is particularly useful on there. And I think it's meant to help you. It's interesting, you talk about the right of bang. It's intended to be able to look at logs and things that have happened that may indicate something that is happening as well. And I suspect we'll come on to that in a sec, but there's that element too. I think one other thing and something dear to my heart, extensibility. You can extend it into other platforms as well. So you can bring in your kind of other security tools that it allows allows you to bring into that element of it as well.
[Ru 11:39]
Correct, yes, so it supports plugins which vendors can develop. So for example, if I'm a third-party security developer, Tanium is one that comes to mind. If I have a solution, I can integrate it with Copilot as well. So as part of my prompts, I could say, okay, well, here's this information, now query third-party data source as well. And really one of the other ways that it's become kind of extensible and customizable rather than just being like a chat engine, is this thing that are called prompt books, which may be, let's say, an incident happens. I'll make it really simple. Let's say user involved in some kind of suspicious sign-in. Well, rather than me having to remember or manually type out my prompts to find out more information about this incident, I can just click a button, and it will say, well, here are seven prompts that a security architect has designed in the past that will help you get the type of information you want. For example, it will automatically say, show me the risky sign-ins associated with this user, show me the activity associated with that user, and you get all that just a click of a button. That's really one of the compelling use cases for it, is that really fast response versus previously on a manual investigation I had to do.
[Kevin 12:54]
I was chatting to someone at the South Coast user group, and they were looking at that capability to look at a script that's been run to understand what it did, and they used it in a live incident, and they said the speeds with which that script could be broken down to understand the impact and what it was trying to do, even where quite often in these scripts that run, that try and hide that they're not named nicely within there. They said the power of that was really, really strong.
[Ru 13:24]
Yeah. It depends how quickly you guys want to get into the watch and all approach that I've taken to security for Copilot, We can absolutely talk about that. But yes, one of the described features of Copilot for Security is this capability where, let's say a suspicious PowerShell script runs on a device, and maybe it's hundreds, thousands of lines. Well, for me to look at that script and figure out what's going on, number 1, I have to be skilled in the language to understand it, and number 2, I have to have the time to do that. Idea behind Copilot for Security is I could upload that script, and it can perform that analysis far quicker than I can, and then just tell me in natural language what is actually going on here?
[Kevin 14:07]
Okay, so I think one more before we get on to the warts and all. In terms of licensing you, and this will definitely come into the warts and all from your tweet thread, you pay for, and I forgot the name, but basically the amount it's used. So you have different units that give you the ability to run prompts for certain volumes, and it's that that sort of builds up. and I probably talking about that is one of the first things to chat about. Actually before we get onto that, if anyone's thinking about security and maybe invaders happening, know that meowing in the background wasn't in your head. So he does have a couple of invaders going on for those watching on video behind.
[Zoe 14:49]
You. Yeah as the home invasion.
[Ru 14:52]
That's the that's the defender ninja cat in the background. That's great. I got I got a couple of cats but luckily they're asleep just now, so we'll see.
[Zoe 15:02]
They normally sleep at this time, but my other half's just gone. He's actually just gone to do a security exam, so I opened the door and all of the cats have decided that now is the best time to come and get involved in my life.
[Ru 15:16]
Love it, love it. Yeah, so the point there, Kevin, about pricing, and you guys can keep me honest here because you maybe know the other Copilots better than I do, specifically Copilot for M365. But my understanding is most of these other Copilots are licensed kind of per user per month in a typical M365 fashion, right? And I think there's things like minimum commitments and things like that. Yeah. Copilot for security, I have to constantly keep reminding myself it's not called security Copilot. Copilot for security, It is really licensed on the same type of mechanism you would license Azure resources. You think about Azure, you think of, well, the price I pay for an Azure Virtual Machine or Azure Storage is consumption-based. VMs, I'm going to pay for however long that VM is allocated and running within Azure. With Copilot for security, it gets a little bit abstract. They've developed a measure that is called the SEU, or the Security Compute Unit. And like I said, it's a little bit abstract. But if you roughly translate that, it really relates to, well, what is the compute power required to run these AI models? In my head, I can just think of, well, it's reserving compute power for me, be it in virtual machines, GPUs, et cetera, et cetera, that kind of stuff. And it's all abstracted away from you. But fundamentally, that's what you're paying for, is this thing called an SCU. And you pay for that per hour. Again, very similar to like an Azure virtual machine. The cost at the minute is four US dollars per hour. And you cannot kind of at the minute do it on demand. So for example, let's say I don't have a 24 by seven operation and I might only need Copilot for security from 8-5 or something like that. If I don't want to pay out with those hours, I have to go to the Azure portal and completely delete it.
[Kevin 17:27]
Don't worry, you won't get attacked during the evening, it's fine.
[Ru 17:31]
Yeah, exactly. You raise a good point because that really explains where Copilot for security's use cases are, which is in that 24 by 7 security operation center. If you follow through on Microsoft's recommendations, they recommend that you reserve three SEUs per hour at $4 per hour. If you leave that running 24 by seven, that comes in at just over $100,000 per year that Microsoft recommended. Where I approach this, I approach it in the way that I approach all things in security when folks ask me, well, is it good? Is Copilot for security good? Is Defender good? is entry good, whatever, the point is, compared to what? And we are saying, compared to over $100,000 per year, roughly speaking. And you have to go through your own assessment of, what else could I spend $100,000 per year that's, frankly, the cost of maybe an extra human in your team, right? So the bar is very, very high when we consider how good and how useful as Copilot for security.
[Kevin 18:44]
But, you know, I'm gonna, I'm taking the positive track on this. There could be that you could replace two or three people to some degree, sorry, you could give.
[Zoe 18:55]
The capability. We don't.
[Kevin 18:55]
Like, we don't like that. Yeah, yeah, as I said that, not replace people, but you could allow that growth for people there that you wouldn't, and when I said replace, rather than have them kind of monitoring and dealing with scenarios, they could be doing things which bring additional value to the kind of longer term map from that. And you could, in theory, if Copilot for Security did all it promised, have them focusing on higher value items while the Copilot sat with one person to kind of go through that. Yeah. And how was your experiences of it being able to replace, being able to support that person.
[Zoe 19:36]
Augment, augment.
[Ru 19:37]
Augment.
[Kevin 19:38]
Augment, that's a good one. I do genuinely mean this. This is my bad language coming through with things that it should be on there. So I'm.
[Ru 19:46]
Going to get you on. No, you raise a good point insofar as look, and it's become very apparent that at least not in the very short term, AI is all about improving your efficiencies of the folks that are already there rather than being full on replacement for hiring new folks. That's kind of what's become apparent to me anyway, at least in the security space. And when we talk about, well, what are the improved efficiencies that someone might see? Again, I lean on that kind of place where if I am an incident responder, and I'm sitting in Defender or Sentinel, and my shift in my job pattern as I'm looking at inbound alerts coming in, I'm triaging those to figure out, well, where should my priorities be? What do I need to know very, very quickly. These AI models do a very good job of very quickly telling me what is, translate all this data and give it to me in language I can just read, like I would read a memo, you know what I mean? So that part is very good for it. And again, it does boil down to that question of is the increase in efficiency worth the cost? And every organization has to make that that determination themselves, right? From my experience so far, it's a tough justification, but I can see some circumstances that folks could do.
[Kevin 21:07]
And it's a tricky one with things like this. And again, it's the same with all security. You're paying to avoid a risk happening. And if something happens to reduce the impact of that, it's gonna be a very hard calculation to make. And I was chatting on the, it depends that Stephen Moraine runs. And it's like, you've got to balance that productivity versus the risk. You could block everything happening, but what is the impact of that? And actually, the impact of a security event, 100 grand could be absolutely nothing to certain organizations if something happens there. So it's balancing not just the cost of what you would be not doing by having someone more focused on that, but also the cost of not being able to react quickly. When those incidents happen, reacting quickly is key if you can do it.
[Ru 22:00]
Well. Yeah, yeah, absolutely. And that kind of, there's a whole kind of field and geez, it almost gets into this kind of philosophical slash theological discussion about, well, how much risk are we willing to tolerate? And more importantly, how do we actually measure that risk? That's, in my experience, very hard to do in cybersecurity. And we're always gonna have businesses that have that spending power to be early adopters of things like the Copilot for security. And for those folks, I'm kind of like, go for it. But if we think of the mass market, particularly the smaller to medium-sized business where every penny is a prisoner and we have to be very careful with how we spend our money, my kind of advice, at least for now, is let the bigger businesses figure out this beast. And then when it's more mature, maybe that's when you start looking into it. That's very, very high level opinion of me at the minute.
[Kevin 22:58]
Yeah. So coming back to some of the things we talked about, the nice bits, and we said we'd come back to, maybe analyzing scripts. And I saw all sorts of that story I'd heard that it did very well. And you seemed a bit more cynical about that.
[Ru 23:13]
I can only speak from experience so far, right? That's all I kind of have. And again, everything I say is based on the fact that this is very, very expensive. In my testing, I took four different PowerShell scripts that had gone through some type of obfuscation. So it kind of muddies up what exactly is this script doing? And I asked Copilot for security to read it and tell me what was.
[Kevin 23:44]
Happening. These were scripts that you'd seen happen in a security incident or what sort of ones you wrote?
[Ru 23:52]
Great, so these aren't ones I wrote, are ones that are kind of available for testing and things like that. So if I give you an example, Defender antivirus can block scripts running that are obfuscated. I took some of the examples I use when I'm testing that, and I threw them into Copilot, and I said, hey, assess these scripts and tell me what's going on here. And regretfully, in my testing, it simply couldn't do it. It spun away, and it spun away, and then it would eventually error out. Now, where things got got very interesting was if I copy and pasted that script into the Microsoft Copilot, previously called Bing Enterprise Chat, it could do it. Like I said, I can only speak from my experience, but that was not the best first impression.
[Kevin 24:41]
That's why I want to check whether the scripts, these are fairly standard ones that are out there. That's a little surprising, interesting. Yeah, correct. I guess.
[Ru 24:51]
The point is, even if it wasn't standard, at the end of the day, the bar for security is so high, and to your point earlier, the time to respond is so critical that, what would you say? I shouldn't really have to just test this with standard scripts, I should be able to throw this thing very demanding tasks and get good results.
[Kevin 25:13]
Yeah, interesting. And so, other areas, because I think you talked about in your thread also the cost, and there's going to be a lot of people who want to try this out, not necessarily run it, especially, as you mentioned, those smaller orgs who want to turn it on for five, ten minutes, give it a go. And I think that was interesting and didn't necessarily reflect what you'd expect there as well.
[Ru 25:40]
Yeah. I mean, look, one of the really good bits of news is because it is billed as an Azure resource at the starting point being $4 per hour for one secure computer unit. If you want to kick the tires on this thing, give it a spin in your own environment, that really lowers the cost for entry, right, you know, for just a few bucks. You can play with this thing for a few hours, see how it goes. But again, my experience is you very quickly run into the limitations of that one secure unit per hour. So I did a thread on Twitter, and I maybe ran somewhere in the region of 10 to 15, maybe at most 20 different prompts, all within the space of one hour. And you can look in the dedicated portal at your usage, so it'll say, hey, you've used x number of secure compute units this hour. It actually lets you go over your allocation before it throttles you. In that timescale, just looking at the numbers now, I had used rounding up to 16 secure units per hour for roughly 15-20 prompts. That's going to cost me $4 in that hour. If you assume that same demand, 24-7 in a security operation center, that would cost $48,000 per month. So at that scale, if you are demanding a lot from this thing because of the consumption-based model, you just have to be very conscious of price. And again, it goes back to the point where you have to make that determination with you and your own team as to, I've got a finite budget, where's the best place for me to spend this?
[Kevin 27:23]
Yeah, and I think it's fair to say that when you're trialing things out, there's a very different pattern of usage and you would see in a real environment from there as well. So it's looking at those different elements that come through from that, that needs to be considered. Sorry, I sound like the Microsoft fanboy that's apologizing for them, but I think it's true. I've seen this where I've done this from experience. Cosmos DB is a great example where I first start to use that and go, we can't pay this much. but then actually the real usage that levels out over time is it works better within there. So it's certainly something worth considering, and I love your kind of analytics of these things, but this may be different over time as well. So don't let it 100% put people off.
[Ru 28:16]
Yeah. I mean, at the end of the day, we're recording this and the 1st of April, it really wasn't that long ago, right? We were in the very, very early stages of this thing. And I already know based on, it was on the Microsoft Security Insights podcast, which Rod Trent at Microsoft hosts, and he's already said that they are looking at not the cost as such, but the ability to provision things on demand, clarifying some of the minimum costing and things like that. So one of the, and really this is kind of where I try to, at least in my my own little way is keep Microsoft honest. I think that's one of the key things we have to do as MVPs, is if we see problems, we need to highlight them. It's really just explain, well, here's what folks are experiencing with this and to their credit, I've always found that Microsoft are very, very good at course-correcting very, very quickly because they're an incredibly successful business and you don't become an incredibly successful business without doing that.
[Kevin 29:17]
Yeah, and I noticed that I was just having a look through your thread and reminded myself some of the things I like that was intriguing was one of the benefits of Copilot is being able to have a natural language, being able to interrogate that data as you said, and you kind of got mixed results from some of that. Some worked really well, some not quite so well, and I think you talked about the KQL generative, which is the query language for querying those logs wasn't amazing at times on there. But I think if I go back to Copilot for Microsoft 365, and I know Zoe, you were one of the very early ones there, the prompts that happen initially there versus what you do now is very different. And those prompts as they evolve, I think, will be tailored better towards things that work. And people will find the things that work and share those and gear.
[Zoe 30:05]
With us on that. Yeah, I mean, I think that comment that you made, Ru, about how early this is in the journey is really important, because for Copilot for M365, if you think about when that kind of first got into the hands of more people at the start of the early access programme in July, it's almost a year, it's like nine months since now, which means we've had nine months of people testing it, feedback going to product group, improving, iterating, adding in new features, new capabilities, And even though from a marketing perspective, Microsoft have been talking about all of the Copilots for quite a long time now, in terms of what's actually been in the hands of people from a product perspective, for a lot of the Copilots, it's really early days. So I think it's important that people have an appreciation of that and recognize that there are things that you'll be able to do now. And if the journey that we've seen with Copilot for M365, it holds true for all of the others, these will mature and evolve, and by the end of this year, we'll see much richer capabilities and possibilities across all of them.
[Ru 31:13]
Yeah, 100 percent, and it's one of those things where I don't use Copilot for Microsoft 365 day-to-day, but purely anecdotally, I've got customers who I maybe spoke to them maybe a few months ago when they downsized the requirements for it, and they'd play about with it, and they're like, okay, first impressions is it's fine, but it's not great. And then in the coming months, when I spoke to them again, they're like, you know what, I'm using this thing more and more, because number one, I'm getting better at understanding how to get the best out of it. But number two, it seems to be improving as well. And by both of those things improving, the overall level improves. And I think with Copilot for Security, you're gonna see that too. The service will improve its ability to interpret your prompts and get you the data you want will improve, but you'll become better at understanding how to prompt it to get what you want. Over time, those two things will converge to improve the experience.
[Zoe 32:04]
Yeah. Then the third thing that we'll weave into that is that extensibility as well. As people build more connectors, as people find new ways to extend and integrate these Copilots into our business processes and other systems, we'll start to see that up-leveling, that value that people find from it.
[Kevin 32:30]
Yeah, and Ruth, just looking from your screenshots, the one things I love about Copilot for Microsoft 365 and quite a few of the other ones is you have that kind of thumbs up and thumbs down to kind of feedback.
[Ru 32:41]
Yes.
[Kevin 32:42]
Is that not existing Copilot for security or has that just been trimmed off?
[Ru 32:46]
Great question. No, you can give it feedback as you go. So for example, if I prompt for something and it gives me the results of that, I can either say, hey, great job, which presumably informs the back-end of it. Or alternatively, I could say, hey, you know what, this wasn't a good job, here's my feedback. Actually, you see that quite a lot. I know in the Defender portal, there's obviously not AI-based, but you can provide feedback. Again, that's one of the things I do quite enjoy about working with the Microsoft products is, I know that feedback does actually go to, in most cases, real human eyes. That's quite a good feature. If folks are early adopters of Security for Copilot, I would recommend they give as much feedback as possible.
[Kevin 33:27]
Yeah. I'm also getting distracted by my favorites. We mentioned, oh, vulnerability in gaming services in those on playing Halo.
[Ru 33:38]
Yeah. Actually, that brings me to a great point. This is one of the things that's going to be smuggled in there and missed in a lot of the public knowledge of what Copilot for Security brings. Microsoft have a separate service called Microsoft Defender Threat Intelligence, which is a very expensive service in and of itself, but it does what it says on the tenant as far as it will give you threat intelligence about, okay, well, who are the bad actors out there? What are their known attack patterns? The target audience for that is potentially a security operation center, so you know your adversaries and who you may be up against. That is included in the price of Copilot for security. If you were looking at that and thinking, well, I'm interested in that, then it's another incentive for you to test this out.
[Kevin 34:30]
Interesting. I've noticed that Microsoft's done a few packaging with things, and we talked about Copilot for sales and service, having Copilot for Microsoft 365 is a core part of those. That's really useful to know. I know we're going to have to wrap up soonish, and it'd be great to summarize this, because I feel like Zoe and I are doing a nice positive marketing spiel to kind of balance out your real world feedback from it. So I want to try and bring some of that together, because that's just my natural balance way of things. It feels to me the important things.
[Ru 35:06]
Is.
[Kevin 35:07]
Probably right now, if you're looking at Copilot for security, we're looking at larger organisations or certainly ones that can invest a good amount of money into using this and supporting the teams they have to allow that growth with the right people there. Your kind of SMBs are certainly not looking at this area right now. So we're looking at those larger enterprises as the focus for this. We're looking at understand, you know, one of the key things you want to do is understand what your usage patterns are going to be, you know, how many people are looking at using it, understanding what the cost of that and balancing the value that it can bring, which can be large in the right circumstances, along with the risk of not doing that, as well. It's looking at those and understanding what can be done, the sort of scenarios you'd be looking at, the benefits from there. Is it fair to say, and you talked about, I love that sort of left of bang and right of bang, you're looking at the core value being that right of bang at the moment, being able to analyze things which might happen isn't great there at the moment.
[Ru 36:17]
Yeah, that's a very good point. In the documentation, Microsoft do describe kind of the, I think it's four key scenarios that Copilot Security is useful for. Incident response, threat hunting, intelligence gathering, and posture management. The incident response and the threat hunting is pretty good at. The posture management, So that's everything before a breach, which has to do with architecture, things like that. Again, personal experience so far, we're not there yet. So for example, I asked Copilot for security to say, tell me all the users who have a security key registered. And it just kind of said, no, sorry, not for me. And I'm like, okay, cool, so we're not there yet. But the point is, if you're looking at this from a security architecture point of view to find your gaps, potentially not as useful. I think that where the real value will be is when it can get there because if I'm a security architect, looking at exclusions, looking at groups, you folks will have been in some of the biggest Microsoft 365 tenants in the world in your consultancy time, and these can get very messy very quickly. Finding out gaps, finding out users who exist, who don't need to exist anymore, permissions, sensitive data that exist that doesn't need to exist anymore, that's a real challenge. Where I'm very optimistic is that AI, geez, I'm going to start sounding like a marketing person now, is I think AI can hopefully help us on that because it can reason and it can translate the important stuff very, very quickly. Yet to see it, but that's where I'm optimistic.
[Kevin 37:56]
We talked about Copilot for Microsoft 365, that journey always feels like it is improving. I think it's that expectation that may have been sold some people of what's there right now is worth looking at and understanding and making sure you know what you're getting from it.
[Zoe 38:14]
I think one of the things that will be interesting to me is that if we get to a point where organizations have got a Copilot for M365 and Copilot for security, if I work in the security team and I can just use Copilot chat and have that as my interface into the Copilot for security functionality you know that's that's the type of like next level I want the products to work and then I want this next level integration.
[Kevin 38:40]
I like that idea but I can't help thinking security people just avoid teams and outlook and things like the play I imagine them on black screens and just doing sort of text.
[Zoe 38:50]
Chat and thinking that.
[Kevin 38:51]
Maybe this is my own uh stereotypical view but.
[Zoe 38:55]
Yeah you've been watching too many science fiction off uh science fiction or thriller films came in. One question I've got for you Ru or a comment I guess, we'd really love to see you submit for a month of Copilot because I think you've got exactly the.
[Ru 39:09]
Time. You.
[Zoe 39:10]
Don't need to tell us now what you might talk about but I think you know if we look at.
[Kevin 39:16]
You. As long as it's not Copilot for Microsoft 365 then.
[Zoe 39:19]
Yeah it's two months away so hopefully by then you've got like even more kind of real world experience and insights that will be useful for other people who are at the start of the journey?
[Ru 39:30]
Yep, nope, cool, okay, right, you put me on the spot, so I'll do it. I'll get something to submit to you guys and we can talk about it.
[Kevin 39:35]
And I think also what I'd love to is maybe get you back in six months or so time to see if you've been using it more and see how that experience has changed. It'd be really interesting to kind of see that journey, whether it's, yep, we stopped, or no, we found that extension with it works better, or actually it's improved in itself. So it'd be interesting to see that evolution as.
[Ru 39:56]
100%, I'd love to come back and talk about it as it grows, because the reality is it's not going anywhere. And therefore, I'm not going to ignore it. I'll keep dipping in and out and finding use cases. And I guess, look, we've got a wrap up. But if I'm thinking of my own conclusions to this, I'm going to articulate this much clearer in a blog I've got coming out soon, which I'm co-authoring with other MVPs who have other and better feedback from it, so that it's more general feedback. The main thing in my advice to customers so far and what I'm leaving on and where I think it'd be very useful to review maybe in six months time is all about it's the cost efficiency question. And so far as we only have so much spending power as defenders and we have to be very selective about where we put our time and our budget. And for me, it's a question of, well, for every buck and every minute that I spend in Copilot for security, that be better spent somewhere else? And that's where I'm currently wrestling with at the minute, and I'll get my conclusions out there at some point, and it'll be good to review that again in six months.
[Kevin 41:01]
Because I would have thought that time to start using it is a lot quicker. You know, you've got some useful things from playing with it within minutes, so understanding that versus a more complex system that you have to train people up. I know for a lot of the Copilots, as we talk about leveling the playing field. That's an interesting one. Anyway, that was a lovely wrap up and I've just ruined it by extending that. But no, that'd be fantastic to see. I'd also say, you mentioned Rod Trent earlier. He's actually got a LinkedIn group about security for Copilots. He does, yes. And I'll put the links in for that into the show notes. So a good place to ask questions and see what people are being shared and talking about within there as well.
[Zoe 41:42]
Brilliant.
[Ru 41:42]
Yeah, Rod has a lot of great content.
[Zoe 41:44]
Yeah, yeah 100%. So we will have a whole heap of links in the show notes for this session as well. It's been really great having you on Roo. I think you're actually the second in person, is that right Kevin, the second in-person guest that we've actually managed to schedule a session with.
[Kevin 41:59]
And.
[Zoe 42:00]
You remember to.
[Kevin 42:01]
Get the cold this time.
[Ru 42:05]
Oh.
[Kevin 42:05]
Dear, sorry.
[Ru 42:06]
Tricia.
[Zoe 42:10]
Brilliant. Well, thanks so much Riri, it's been an absolute pleasure. This was a really great discussion and for me I feel like I've learned a huge amount about Copilot in the security space. We've got lots of big things planned for future episodes, we'll have more experts joining us, we will get better at actually scheduling people as well as grabbing those in-person interviews, and we really hope that there's lots of you out there who have been getting hands-on with Copilot and we're going to fill in that call for speakers for month of Copilot.
[Kevin 42:40]
Absolutely, and if you do want to hear more, please let us know which areas of Copilot you'd love to hear on, who you'd like to hear from, and please, I know you hear us on all your podcasts, smash that subscribe button, go and follow us on YouTube, on all the social medias, we're out there, add us to your favourite podcast app, add your friends to, add your clients to, add your family, we're here to kind of share the knowledge to everyone and Copilot will affect everyone from there. So please encourage as many people as possible to. Otherwise, thank you very much. Oh, and if you fancy writing a review, that would be even better because apparently that helps and I keep forgetting to mention that. So add a review in your favorite apps as well. But otherwise, thank you so much for joining us, Ru, and we'll be back in a couple of weeks.
[Ru 43:49]
Thanks everyone. Cheers guys. Bye. Bye.
