Microsoft Secure Score: Improving Security Posture Without the Checkbox Trap

M365 FM - Microsoft Secure Score - Improving Security Posture Without the Checkbox Trap
00:00:00 Speaker: Walks into a board meeting. One slide, one number, seventy two percent heads nod. Someone says, good progress. Nobody asks what the number is made of. If you want clarity instead of checklists, subscribe to the M365 FM podcast. Here's the floor. The score isn't wrong. The model around it is, so we'll cover what it measures, how to prioritize, what actions cost your users, how to track progress honestly, and how to pair it with threat thinking. But first, one short word from the sponsor sponsor segment. Educational pause. Here's something that gets missed. A migration changes your security posture. The moment mailboxes move different tenant, different licenses, different policies applying to different people, your secure score inputs shift overnight and nobody touched a single setting on purpose. That's why tenant consolidation and mailbox moves deserve a security review, not just a cutover plan. If you're planning an exchange, migration tools like stellar Migrator for exchange help you move mailboxes in a controlled way and controlled matters because you want to know exactly what lands, where and what protections it lands under. Think of it as an input to your posture, not an afterthought. Okay, back to the score and the assumption. Hiding inside it. The dashboard. Everyone reads wrong. Open the Microsoft Defender portal. Go to secure score. The first thing you see is a percentage. Big, clean, easy to screenshot. And that's where it starts to go wrong. Because leadership doesn't read it as a measurement. They read it as a grade seventy two percent. That's a C plus, right? Not great, not terrible. Let's get it to a B by next quarter. It sounds reasonable. It's actually the first broken assumption. A grade implies a finish line. You study, you improve, you hit ninety, you're done. The exam is over. But security has no finish line. Attackers don't grade on a curve. They don't care that you passed. They care whether one door is open today. So what is the number actually telling you? Here's the structural flaw. The score measures configuration. It doesn't measure outcomes. It tells you which settings are turned on. It does not tell you how many attacks failed, how fast you'd spot a breach, or whether your most exposed account would survive. A phishing email on a Tuesday afternoon. Configuration is an input. Outcome is a result. The dashboard shows the input and lets you feel like you're looking at the result. And that's not Microsoft's fault. The portal is quite clear about what it counts. The problem is what happens after the number leaves the portal. What typically happens is this someone puts the score in a monthly report. It sits next to other numbers, ticket counts, patch rates, uptime. Then someone asks for a target because that's what you do with a number on a report. You set a target eighty percent by year end, and now the score is a KPI. But here's what happens to every KPI. Sooner or later, it becomes a target. And once a measure becomes a target, people stop asking what it was supposed to tell them. They start asking how to move it. Nobody planned this. Nobody sat down and decided to optimize the wrong thing. It just follows the incentives, the dashboard rewards points. The report rewards progress. The team delivers points so the team picks the actions that add the most points for the least effort quietly, reasonably, one ticket at a time, and three months later, the number is higher and the exposure might not have moved at all. That's not a security failure. That's a model failure. The percentage was never meant to carry that much weight. It was meant to point at work. Instead, it got promoted to a verdict. And you can't fix that by trying harder to hit the number. You fix it by understanding what the number is built from. Because once you see how the points are actually calculated, the grade stops looking like a grade. It starts looking like a list. What's a cure? Score actually, measures start with the unit. The unit is not a percentage. It's a point. Every recommendation insecure score is called an improvement action. Turn on MFA for admins. Block legacy authentication. Enable a Safe links policy. Each action carries a set number of points, and you earn those points when your configuration matches what the action asks for. That's the whole mechanism. Notice what's missing from it. No attack was stopped. No threat was detected. No incident was closed. The portal looks at the state of your settings and checks them against a list. So the score is a count of controls, not a count of protection. Now look at the two numbers sitting side by side, your current score and your max score. Most people read the max score as a fixed ceiling one hundred percent and everyone shares the same one hundred. But the max isn't fixed. It's the total points available to your tenant based on the products you license and the workloads you actually run, which means the denominator moves at a license, the max goes up, Microsoft releases a new action, the max goes up, retire a workload. It goes down. Your percentage can shift while your configuration stays exactly the same. We'll come back to that later because it causes real arguments in real meetings. For now, hold one idea. The percentage is a ratio and both sides of the ratio can change without you. So what goes into it? The scope covers the main layers of a Microsoft three hundred sixty five environment identity, which is your account's authentication and admin roles devices, which is endpoint protection, and Device Settings apps, which covers email and collaboration protection, and data, which touches information protection and sharing. Each category contributes its own pool of points, and the pools are not equal in size. That matters more than it sounds. If one category simply has more actions available, it will dominate the total. A big pile of small device settings can outweigh a handful of identity actions, even when the identity actions are the ones attackers actually go after. Points reflect how many things Microsoft can check. They don't reflect how often each thing gets abused, and that's a very different question. Here's the cleanest way to say it. Secure score is a posture snapshot of controls. It answers one thing. How many of the recommended settings are in place right now? It does not answer how likely you are to be breached. It does not answer which of your accounts an attacker would pick first, and it can't tell you whether a setting that's turned on is covering the people who need it. Take MFA. The score can tell you a policy exists. It can't tell you. The three executive assistants with mailbox delegation were left out of it or take a device policy. The score sees the policy. It doesn't see the laptops that never enrolled. Posture is the state of your controls. Risk is the chance that something bad happens and how bad it gets. Those are cousins. They are not the same person. When you treat them as the same thing, you get the grade problem from earlier. A tidy number standing in for a messy truth. But here's the useful part. Once you stop reading the scores risk, it becomes something much better. It becomes a map of configuration gaps drawn by someone who knows the platform well. Microsoft built that list from what they see across millions of tenants. That's real knowledge. It's just knowledge about what's missing, not knowledge about what matters most to you. That judgment is your job. And before you can do that job, there's another layer to understand because the waypoints get awarded has a few quirks. Some actions pay out in full, some pay out in part, and some can be marked complete without you changing a single setting. That's where the number starts to surprise people. How points are earned full, partial, and third party open any improvement action in the portal. Look for the line that says Score Impact. It shows a number of points, and that number is the most you can earn from that action, not what you'll earn the most. Some actions pay out in full the moment one condition is met. Turn on the setting. Get the points. Others pay out in part. Microsoft checks how much of your environment the control actually covers and gives you a share of the points to match. Take MFA registration. If seventy percent of your users are registered, you don't get zero and you don't get full. You get something in between. So partial points are actually the more honest design. They reward coverage, not just existence, but even that has limits coverage of what seventy percent of users doesn't tell you which seventy percent the admins could be in the missing group, and the score would still look healthy. Now look at the status field on each action. This is where it gets interesting. You can mark an action as to address planned risk accepted resolved through third party or completed. Most of those are labels for your own tracking. Planned means you intend to do it. Risk accepted means you decided not to. Neither one changes your configuration, so neither one should change your points in a way that implies protection, but two of them behave differently. Resolve through third party means you're covering that risk with a tool. Microsoft can't see maybe a separate email security gateway, maybe a different identity provider for part of the estate. You tell the portal and the action is credited as done. And here's the quiet problem. Microsoft isn't verifying that claim. It can't. It has no view into the other product. So the score is now trusting you, which sounds fine until you ask who exactly did the trusting? In a lot of organizations, the person clicking that button is under pressure to move a number. They have a vendor. They have a rough idea. The vendor covers it. Nobody has tested whether it actually does and the credit goes in anyway. That's where the trust breaks. Not because anyone lied because the action closed on an assumption instead of a result. There's a related case. A setting can be configured correctly and still protect nobody. You enable a policy, but scope it to a pilot group and forget to widen it. You turn on a feature in a license tier that half your staff don't hold. The checkbox is ticked, the coverage is thin. So here's what that means. A number can go up without a single real risk going down. Think through how that happens in practice, someone marks three actions as covered by a third party. Another person enables two settings on a small test group, a policy gets switched from audit to enabled but excludes a long list of accounts. Nobody reviews. The score, climbs the report looks better, and the open door is exactly where it was. None of this requires bad intent. It requires only a dashboard that rewards status changes and a team that's busy. So how do you protect yourself from your own number? Ask one question of every action that earned points. What would I see if this control failed? If the answer is nothing, you don't have evidence. You have a status for a third party claim. That might mean a test. Send a known phishing sample through and watch what the other tool does for a partial points action. It means looking at who sits in the uncovered group, because the points tell you Microsoft saw something, they don't tell you it works. And there's one more detail that matters before we move on. Points aren't weighted evenly across actions. Some are worth a lot, some are worth almost nothing. A tiny setting and a major identity control can sit side by side on the same list, and the weights don't always match how much each one matters to an attacker. That uneven weighting is what makes the next behavior so predictable. When a dashboard hands out different amounts of reward for different amounts of effort, people don't have to be told what to do. They find the cheap points on their own. The checkbox trap defined. Here's the trap. In one sentence, you start optimizing the number instead of the exposure. Those two things sound close. They aren't. The exposure is the real problem and the count that can be fished. A path that leads to an admin, a mailbox nobody protects. The number is just a count of how many boxes got ticked on the way, and the two can drift apart for months without anyone noticing. Watch how it starts. Nobody walks in on day one and says, let's chase points. It begins with a perfectly sensible request open the list, sort by score impact and see what we can get done. This sprint. That's not a bad instinct. That's how most teams work. You look for the quick win, you clear the easy stuff first, but a list sorted by points is not a list sorted by risk points. Tell you what the dashboard will reward. They don't tell you what an attacker will use, so the easy items go first. The settings that take ten minutes. The ones with no user impact, no meeting, no awkward conversation with a department head. They're fast, they're safe and they move the number and the hard ones. Wait. The ones that need a pilot. A policy that touches every sign in a change that might annoy the finance team on a Monday morning. Those carry the real weight and they sit at the bottom because they cost the most to do. That's the natural behavior. When a dashboard rewards volume, you get a lot of small things done, you feel productive. The chart goes up and to the right. Then there's the quieter version of the same problem gaming without malice. A ticket gets closed because it was small, not because it mattered. A setting gets enabled because it was sitting there, easy to reach. An action gets marked as planned, then as covered, then quietly dropped from the review. Because nobody wants to explain why it's still open. Nobody is cheating. Everybody is busy and busy. People take the path with the least friction, which in this case happens to be the path that flatters the score. It's worth saying that out loud, because the usual reaction is blame. Who closed those tickets? Why didn't they prioritize better? But that misses the point. If a system pays out for the cheap work, you'll get the cheap work. The behavior is a symptom. The incentive is the cause. There's a second force pushing the same way, and it's the compliance reflex. In a lot of organizations, a security list arrives and the first question is what do we have to do? Which of these are required? What does the auditor want to see? That's a fair question for compliance. It's the wrong question for security. Compliance asks whether a control exists. Security asks whether the control would stop the thing that's actually coming for you. Those can both be true at once. Often they aren't. You can satisfy a framework and still leave a path open. You can complete every item on a list and still have no idea which three items an attacker would try first. Compliance reasoning ends when the box is ticked. Security reasoning only starts there because the next question is ticked. For whom, covering what and what happens if it fails. So the trap has three moving parts a dashboard that rewards volume, teams that take the cheapest points first, and a reflex that treats completion as the same thing as protection. Put them together and you get a tenant that looks better every month and might not be safer in any way that counts. And here's what makes it hard to catch. Every single step along the way looked reasonable. Nobody made a bad decision. The trap isn't a mistake someone made. It's a shape the system settles into when nobody is asking the question underneath, which means you can't solve it with discipline alone. Telling people to prioritize better won't hold, because the next report will still show a number, and the next target will still ask for it to move. This is a model problem. It sits in how the score gets used, who reads it, and what question it's expected to answer. So that's where we need to look, not at the dashboard at the model wrapped around it. The old model score as scoreboard. In most organizations, the score has a calendar slot. Once a month, someone exports the number. It goes into a deck. Next to it sits a line showing last month and a line showing the benchmark up two points. Good. Down one point. Why? That's the whole ritual. The score becomes a scoreboard and the meeting becomes a game review. Nobody is looking at the place, they're looking at the total. And underneath that ritual sits one assumption. Higher means safer. It feels obvious. More points, more controls, more protection. If you asked a room to vote on it, nobody would object. That's exactly why it survives. It never gets questioned because it sounds like common sense. But look at what we've covered so far. Points count settings, not attacks stopped. Coverage can be partial. Credit can be claimed without proof. The weights don't follow what attackers actually use, so higher can mean safer, and it can also mean nothing at all. The number doesn't say which. Then comes the benchmark line. And this is where the comfort gets dangerous. The portal compares you to organizations like yours. Similar size, similar industry. You're at seventy two, they average sixty one. You're ahead. That feels like evidence. It isn't. Think about what the comparison actually tells you. It tells you that you ticked more boxes than a group of other organizations. It doesn't tell you whether that group is secure. For all you know, the whole peer group is leaving the same door open and you're just leaving it open with better paperwork. Being above average on a flawed measure is still a flawed result. And there's a quieter effect. A benchmark gives leadership permission to stop asking if you're ahead of the pack. Why spend more if you're behind? Why panic? Either way, the question shifts from what could hurt us to where do we rank? Those are different conversations, and only one of them makes you safer. So the old model holds together nicely a number, a target, a comparison, a monthly review. Everyone knows their part until it breaks. The moment this breaks is an incident and it usually isn't a dramatic one. Someone in finance gets a convincing email. They approve a sign in prompt. They shouldn't have a token gets stolen, a mailbox rule appears, and a few days later an invoice goes out with the wrong bank details. Then the incident review starts and someone opens the dashboard eighty percent above benchmark green arrows. And now the room has a problem. It can't explain. How does a tenant at eighty percent get breached through a mailbox? The answer is uncomfortable, but simple. The score never measured that path. The account that got phished sat outside the MFA coverage, or it was covered by a method an attacker can steal around. The number was accurate, every point was earned, and the door was still open. That's the real cost of the scoreboard model. It doesn't just fail to prevent the incident. It makes the incident harder to understand because the evidence on the screen says you were doing well. Trust takes the next hit. Leadership asked for a number, got a number and watched it fail to mean anything when it counted. So the next time security asks for budget, the reply is but we were at eighty. That's the damage. Not a bad score. A good score that taught everyone the wrong lesson. And you can't patch that by chasing a higher number. A ninety wouldn't have changed that incident. It would only have made the surprise bigger. What has to change is the question the score is asked to answer. The scoreboard asks how are we doing? That question invites a grade. The better question is what should we work on next and why? That one has an owner, a reason and a direction. It doesn't end in a number, it ends in a decision. And that's the new model. The new model score as backlog signal.
00:15:26 Speaker: Start with a different job description for the number. The score isn't a result. It's a list. A prioritized list of possible work drawn up by people who've seen a lot of tenants and know where the gaps usually sit. That's a smaller claim than how secure are we? But it's a far more useful one. A list can be sorted, it can be argued with. You can look at item fourteen and say not for us and give a reason. A grade can't do any of that. So the model flips. In the old version, the score is the output. You do work, the number moves, the number is the verdict. In the new version, the score is an input. You read it alongside other things you know, and the decision comes out the other side input to risk decisions, not the output of them. And what are those other things? That's where context comes in. Before you open the list, ask what attackers actually do to tenants like yours. Not in theory. In practice, what do they go after first? Which accounts? Which paths? What have they already tried against you? If you've seen any of it, because the list is generic, your situation is not. A recommendation. Can look minor in the portal and sit right on the root an attacker would take into your environment. Another can carry a lot of points and cover something that was never a realistic way, in only context tells you which is which. The portal can't because it doesn't know your business, your people, or your history. Take a small firm where three people can approve payments. Their mailboxes matter more than almost anything else in the tenant. The score treats those mailboxes like any other. You shouldn't. That's the shift. The score tells you what's possible. You decide what's worth doing. Now a list needs somebody to look after it, and this is the part most organizations skip. Under the old model, nobody really owns the score. It belongs to the dashboard, which belongs to everyone, which means it belongs to no one. It gets reported. It doesn't get decided. The new model needs an owner, a name person, someone who's accountable for the decisions, not for the points. That sounds like a small distinction. It changes what the job is. A point owner asks, how do I raise the number? A decisions owner asks, which of these should we do? Which should we skip? And can I defend either choice in front of the board? One of those roles produces ticked boxes, the other produces judgment. You can explain. And that owner has permission to say no. This matters more than it sounds. If skipping an action drags the score down and the owner is judged by the score, they'll never skip anything. They'll just tick it. But if they're judged on whether the decisions were sound, they can leave an item open on purpose. Write down why and move on to the work that counts. That's what turns a dashboard into a working tool. Look at what the backlog framing does to the monthly meeting. To the slide stops saying, we're at seventy two. It starts saying, here are the five things we're working on. Here's why they're first, and here's what we chose not to do. That's a better conversation. It's harder to have because it requires judgment instead of a number, but it's the only one that tells leadership something true. And notice what it leaves out. No grade, no finish line, no benchmark rank to feel good or bad about. Just a queue of work ordered by a reason someone can explain. Of course, that raises the obvious question. Ordered by what reason? If the portals point values aren't the right way to sort the list, what is? You need a lens of your own, one built before you touch a single setting. Building a risk lens. Before you touch a setting, start with a short list, not of settings of things you can't afford to lose your Crown jewels. For most tenants, that's four places identities, because every other asset sits behind one mailboxes. Because that's where money moves and decisions get made. Data stores the SharePoint sites and file shares that hold contracts, payroll, customer records and admin paths, the routes by which somebody becomes powerful, write them down. Name the actual ones. Which mailboxes can approve a payment? Which sites hold the files that would hurt most if they walked out the door? Which accounts can change a policy? It takes an hour. Most organizations have never done it, and without it, every recommendation in the portal looks equally urgent, which is another way of saying none of them do. Now add the two questions behind any risk. How likely is this to happen and how bad is it? If it does? That's it. Likelihood and impact. You don't need a color coded grid with twenty five cells and a workshop to fill it in. Matrix theater looks rigorous and mostly produces arguments about whether something is a three or a four. Use plain words instead. Likely or unlikely. Painful or survivable. If something is both likely and painful, it goes to the top. If it's unlikely and survivable, it can wait. And nobody needs to feel guilty about that. The honest difficulty is likelihood. How do you know what's likely? You don't have to guess blind because the pattern is well documented. Attackers aren't creative with Microsoft three hundred sixty five tenants. They're efficient. They go where the effort is lowest and look at where that is. Follow almost any real intrusion backwards and you'll find an identity at the start of it, a password that was guessed, a sign in prompt, somebody approved by mistake, a token lifted from a browser. A consent screen clicked through on a Friday afternoon. Devices matter. Data matters, but the way in is usually a person's account, and the way up is usually another account with more rights. That doesn't mean ignore everything else. It means that when you're weighing likelihood, identity starts with a head start. A control that protects sign ins begins life nearer the top of the queue than a setting that tidies a corner. Nobody is attacking. So you have your crown jewels and you have a plain language. Read on likelihood and impact. Now you need something you can apply quickly to every single item on the list without a committee. Here's a filter. Three questions. First, what does this protect? Name the asset. If the honest answer is nothing in particular, it's just on the list. You've learned something already. Second, how would an attacker actually get past it if it were missing? Walk the route in your head. Is there a realistic path from the outside to this gap, or would they need three other failures? First, a gap with a short, obvious route matters more than a gap that needs luck. Third, what would we see if this control failed? We touched on this earlier. If the answer is nothing, then you can't rely on it and you can't tell whether it's working. Control you can't observe, is a control you're trusting on faith. Run any recommendation through those three. It takes about two minutes. Say the portal suggests blocking a certain kind of attachment type. What does it protect? Every mailbox. How would an attacker get past it? Easily? By sending the file in another format. What would you see if it failed a user complaint? Probably long after the fact. That's a medium item, not an emergency. Now say the portal suggests requiring stronger sign in for administrators. What does it protect? The keys to everything. How would an attacker get past it if it were missing by phishing one admin and logging in? What would you see if it failed? Possibly nothing until the damage showed up. That one moves to the front and it doesn't need a debate. Notice what happened there. Neither item was sorted by points. Both were sorted by a reason. You can say out loud. That's the whole purpose of the lens. It doesn't replace the portals list. It puts your situation on top of it so the order reflects what you'd actually lose and not what the dashboard happens to pay for. And since identity keeps showing up at the top of that order, that's where the first real work belongs. Identity first. Why most points should start here. Open the list and filter for identity. Then look at what sits near the top. Phishing resistant authentication. MFA coverage blocking legacy protocols. Tightening admin roles these aren't the biggest items by point count, but run them through the three questions and they win almost every time they protect the keys. The route around them is short, and when they fail, you often see nothing. Start with MFA. Most tenants have it somewhere. The question is where it stops. A policy exists. Good. Now ask who's outside it. Service accounts, shared mailboxes. Someone turned into a login. The executive who complained once and got an exemption that nobody ever reviewed. Those gaps are where attackers go because they read the same list you do and they test the edges. So coverage matters more than the policy. A control that protects ninety percent of people and skips the CFO's assistant is a control with a hole exactly where the money sits. Then there's the method. Not all MFA stops the same attacks. A text message code can be fished. An approval prompt can be approved by a tired person. At the end of a long day. A session token can be stolen after the sign in is done. Phishing resistant methods like passkeys, fido2 keys, and certificate based sign in work differently. The credential is tied to the real site, so a fake page gets nothing useful. That's a different class of protection. It's also harder to roll out, which is why it tends to sit lower on the list than it should. Now legacy authentication. This one is easy to forget because nobody complains about it. Older protocols like Pop, iMap, and basic Smtp don't understand modern sign in prompts. They take a username and a password. And that's the whole conversation. So an attacker with a guest password can walk in through that side door. Even when your MFA policy looks perfect on paper. That's the quiet part. Your policy says everyone needs a second step. The old protocol never asks, and because the sign in succeeds, nothing looks wrong. Blocking it is usually a clean win. But check first. A scanner in the corner, an old line of business app. A mail client from a decade ago. Any of these may still depend on it. You want to know that before you flip the switch, not after admin accounts come next. And this is where small changes carry the most weight. Start with separation. An admin shouldn't read, email and browse the web on the same account. That can change policy. Give them a second account used only for admin work. If the everyday account gets phished, the attacker lands somewhere with no power. Then look at scope. How many people hold global administrator in many tenants. The honest answer is too many, and nobody can say why. Most of them need a narrower role. Someone who resets passwords doesn't need the right to delete the tenant. And then there's time standing. Access means the power is always on whether or not anyone is using it. Just in time access flips that the role is off. By default. You request it, you use it and it expires. An attacker who steals the account on a quiet Tuesday finds nothing to work with. Put those three together and you've changed the shape of the worst case. Not by adding more controls, by shrinking what any single account can do. Which brings us to the comparison that matters. Why can one identity action outweigh ten device settings? Because of what each one stops. A device setting hardens one machine against one kind of attack. An identity control sits in front of everything that account can reach mail, files, apps, other people's data. Close that one door and you've closed a route to all of it. And an attacker who has a valid sign and doesn't need to defeat your devices, they just log in. That's why most of your early effort belongs here, not because the other categories are unimportant, but because this is where a single decision covers the most ground. There's a catch, though. Every one of these actions touches people. Stronger. Sign in changes how someone starts their day. Blocking an old protocol can silence a tool. Somebody relies on the portal labels that cost with a single word low, moderate, or high. And that word hides more than it tells you. User impact the cost. Nobody puts in the score. Open any improvement action in the portal. Find the field called user impact. Three values low. Moderate. High. It's useful, but it's a very short answer to a very long question. That rating comes from Microsoft, and Microsoft doesn't know your users. It doesn't know your warehouse staff share tablets. It doesn't know finance runs a ten year old tool that sends mail through a protocol you're about to block. It knows what the change does in a typical tenant. Yours isn't typical, so treat that label as a starting estimate, not a promise. Now split the cost into two parts. They behave differently. The first is implementation cost. That's the work. To get the change live, someone has to plan it, test it, write the notice, and flip the setting. It's a one time bill. You can see it coming and put a date on it. The second is ongoing friction and this part never shows up in the portal. Stronger sign in adds a few seconds to every morning. Multiply that by two thousand people and two hundred working days. Nobody notices it on any single day. But it's real and it never stops. Ask people to do something awkward every time they sign in, and they'll find a way around it, or they'll start ignoring the prompt. Both outcomes undo the control. That's why friction is a security cost, not just a comfort cost. A control people fight against is a control that degrades. So how do you see it? You count something. Helpdesk volume is the best signal you have before you change anything. Note how many tickets come in per week and what they're about. Password resets, locked accounts. I can't open my mail on my phone. Then watch what happens after each change. A week later, if sign in tickets have doubled, you've learned the true impact, not the label, the result, and you learned it in a form you can show to a manager, which beats an opinion every time. Pick two or three numbers and keep them simple. Ticket count for the affected topic. Time to resolve and how many people hit the new prompt and gave up. Even rough counts will tell you more than the rating does. Now here's the trap. Hiding inside a low rating. Low impact means few users will notice the change. It doesn't mean the change is safe. Those are separate things. Picture a setting rated low. It affects almost nobody. Great. But the one team it does affect is the team that sends invoices to customers through an old automated mailbox. The change quietly stops that mail. Nobody complains for three days because nobody realizes it stopped. Then a customer calls asking where their payment request went. A small number of people. Big business problem. That's the pattern to watch the ratings count how many users are touched. They don't weigh what those users do. A change that hits five people in payroll on the last day of the month can matter more than one that hits five hundred people on an ordinary Tuesday. So add your own column to the list next to the portal's rating. Write down who is affected and what they'd stop being able to do if it went wrong. Two sentences are enough, you'll find most actions are fine. A few will make you pause. Those few are the ones worth a conversation before the change, not after it. And that conversation is different from asking permission. You're not looking for a veto. You're looking for the thing you don't know. Ask the team. Lead one question what do you use? That might still depend on the old way of doing this? People will often answer within a minute and save you a week of cleanup. There's one more cost worth naming, which is the cost of waiting. If friction is a real price, so is delay. Every month an admin account sits without stronger sign in is a month. It could be fished. Impact on users is real. So is impact on risk. The goal isn't to avoid friction, it's to spend it where it buys the most protection and to spend it on purpose. That means you don't pick between protecting people and protecting the tenant. You pick how to introduce the change so nobody is surprised by it. And there's a safe way to do exactly that. It lets you see who would be blocked before anyone actually is. Report only mode and staged rollouts. Conditional access has a setting most people walk past. It sits next to on and off. It's called report only. The policy runs. It evaluates every sign in against its rules. It records what would have happened, and then it lets everyone through anyway. Nobody gets blocked, nobody gets prompted. But you get a log of exactly who would have been. That's the safest first step you have. You're asking the tenant a question and getting the answer before it costs anyone anything. Before you turn anything on, though, set up the exits. Two things need to exist. First, break glass accounts. These are emergency admin accounts excluded from your policies with long passwords stored somewhere safe. If a policy goes wrong and locks everyone out, these are how you get back in. Without them. One bad rule can mean a call to Microsoft support and a very long afternoon. And a pilot group. A small set of real people who know they're the test, not a made up group. Nobody sits in. Now the part that gets skipped exclusions. Every policy ends up with a few. The break glass accounts, a service account that can't do a second step, a shared kiosk. That's normal, but each one is a whole you chose to leave open, so each one needs a line somewhere. Who is excluded, why? Who approved it, and when you'll look at it again, an exclusion with no note is how a temporary exception becomes the CFO's permanent bypass three years later. Now run the policy and report only for a couple of weeks, then go read what it found. Open the sign in logs. For each sign in, there's a tab showing report only results. It says whether the policy would have succeeded, failed, or not applied at all. Filter for the failures. Those are your future help desk tickets. Look at who they are. An old male client still using a legacy protocol. A scheduled script, signing in as a person, a contractor on a personal laptop. A sales team logging in from a country you forgot they travel to. Each one is a conversation you can have now, calmly instead of at nine o'clock on Monday morning with a queue forming and look at the not applied results too. They tell you where your policy doesn't reach if you expected it to cover everyone and the whole group shows as not applied, your scoping is wrong. Better to learn that from a log than from an incident. A useful habit here. Pick ten failures and trace each to a cause. Not all of them. Ten. You'll usually find three or four patterns repeating. Fix the patterns and most of the noise disappears. Once the log looks clean, you don't flip the switch for everyone. You move in. Rings ring one. Is it? They're technical, they're forgiving, and they'll tell you exactly what broke. Run there for a week. Ring two is your champions. A handful of people from different departments who are happy to give feedback. They're not technical, which is the point. They'll hit the friction. The way ordinary users will ring three is a full department. Pick, one with a manager who's been warned and a clear path to ask for help. Now you're testing at real volume. Ring four is everyone between each ring stop and look, ticket count, sign in failures. Anything strange in the logs? If the numbers hold, widen. If they don't pause, fix and try again. That pauses the whole value. A rollout with no pause points is just a big switch with a delay.
00:30:51 Speaker: Compare the two approaches, flip it on for everyone, and every problem arrives together from every corner on the same morning. Roll in rings and problems arrive in small batches, one group at a time when you still have room to respond. Same control, same end state. Very different week. Of course, policies are only one part of this. Plenty of what Secure Score recommends isn't a policy at all. Some of it is architecture. And architecture doesn't have a report only mode conditional access without self-inflicted outages. Here's the failure everyone hears about and a surprising number of people cause. Anyway, an admin builds a new policy. Require compliant devices. All users, all apps. It looks right. It feels like good security. They save it, switch it to on, and within a minute the tenant stops answering. Nobody can sign in, including the admin who just made the change. The reasons are boring, the scope said all users and nobody removed the admins own account. The device compliance data hasn't synced yet, so every laptop looked non-compliant or the policy applied to a sign in the admin portal itself depends on different cause each time. Same Monday, and the fix is rarely clever. It's whatever you set up before the problem started. Which brings us back to the emergency accounts. We mentioned them earlier as an exit. Now look at what makes them work. Two accounts, minimum cloud only, so they don't depend on your directory sync or a Federation server. That might be the thing that's down excluded from every conditional access policy. Long random passwords split between two people or kept in a physical safe. Ideally a phishing resistant method that doesn't rely on a single phone number. But here's the part people skip an account that never signs in is an account nobody is watching, and an attacker would love an admin account that nobody is watching, so monitor it. Set an alert on any sign in. By those accounts, any single one. If it fires on an ordinary afternoon, you either have a test you forgot about or you have an incident. Both are worth knowing within minutes. Then test the accounts twice a year on purpose so you know the passwords still work and the alert still fires. Now the policies themselves, where do they go wrong even when nothing locks you out? Named locations are one place you mark your office addresses as trusted, and a policy relaxes the sign in rules from those addresses. Sensible, but an IP range is not a person. Anyone on that network inherits the trust, including a guest on the Wi fi or a compromised machine sitting in the corner. And remote staff never get the benefit, so you end up with two rule sets that drift apart. Treat location as one signal among several. Never. As a pass device, compliance is another. A policy that requires a compliant device sounds air tight. Then you remember the unmanaged ones personal phones, a contractor's laptop, a browser on a hotel computer. Either those users get blocked, which creates a flood of exceptions, or they get excluded, which creates a hole. You need to decide which on purpose and write it down. The blind spot isn't the policy, it's the group of devices the policy never sees. And there are the apps. People forget. Policies that target all cloud apps sometimes carve out a few, and attackers read those carve outs to legacy clients. Device code sign in, and old service connections can slip past a rule that looks complete on screen. Check each policy's conditions for what it leaves out, not just what it covers. Then comes the slow problem, which isn't a single bad policy at all. Policy sprawl. Every incident, every request, every audit finding adds one more rule a policy for the sales team, one for contractors, a temporary one from two years ago that nobody removed. After a while, you have forty policies and nobody can say what happens when three of them apply to the same sign in. They stack, they conflict and everyone is another place an outage can hide. The better shape is small, a short set of policies, each with one clear purpose. A name that says what it does and an owner require stronger sign in for admins. Block legacy authentication. Require compliant devices for sensitive apps. Handle guests separately. If you can't explain a policy in one sentence, it's probably doing too much or two jobs that should be split. Fewer rules are easier to test and report, only easier to roll out in rings and easier to defend when someone asks why they were blocked. That's the discipline, not more policy. Better policy with an exit door and someone watching it. And all of that sits on top of identity. But identity only governs who signs in. It says very little about the machine they sign in from. That's a different layer, and it fails in different ways. Devices where score meets reality. Identity tells you who is signing in. Devices tell you what they're signing in from. And this is where the score and the real world start to disagree in a very specific way. Most device actions in the portal are about settings, but the bigger problem is almost never a setting. It's coverage. Take defender for endpoint. The portal will happily show you a tidy set of recommendations for the machines. It knows about. Tamper protection on real time protection on cloud delivered protection on fine. But ask a different question how many of your machines is it actually watching? Because a perfect configuration on eight hundred laptops says nothing about the two hundred that never onboarded a server. Nobody remembers a laptop that came back from a repair and never reconnected a new hires machine that skipped the setup step. Those machines don't lower your score. They don't show up as a failed action. They simply aren't there. And an attacker only needs one of them. So start with the inventory. Compare what you think you own against what's reporting in the gap between those two lists is your real device risk, and no percentage will show it to you next. Attack surface reduction rules. These block specific behaviors that malware loves. Office apps launching child processes, scripts that download and run executables. Credential theft from system memory. There are some of the strongest controls on the list. They're also the easiest to break something with. A rule that blocks an office macro from starting a process will stop a phishing payload. It will also stop the finance spreadsheet that has quietly run a macro every month since twenty fourteen, so don't turn them on in block mode. Turn them on in audit mode. First, the rule runs logs what it would have stopped and touches nobody. Then read the audit data. Look at what would have been blocked and by which app. Most of it will be noise you can ignore. A small slice will be a real business process, and you'll want to add an exclusion or fix the process before you enforce. Do that rule by rule, not all at once. Sixteen rules switched on together means sixteen possible causes when something breaks. Now Intune compliance policies. These are the bridge between the device layer and everything we covered on conditional access. A compliance policy defines what a healthy device looks like. Disk encryption on firewall on a minimum operating system version defender reporting no active threat on its own. A compliance policy is just a label. A machine is compliant or it isn't. The power comes when conditional access reads that label and decides what the sign in can reach. Non-Compliant device. No access to company mail. That's how a device setting becomes a real barrier. But it only works if the policy is realistic. Set the bar too high. On day one and half your fleet turns non-compliant overnight and you've built your own outage. Set it to low and the label means nothing. Start where most machines already sit, then raise the bar in steps and then the uncomfortable part. Unmanaged devices. Personal phones or contractors. Laptop, a partner's browser. A home PC that someone uses for a quick check of their email. The score can't count what isn't enrolled. It sees your managed estate and grades, that the unmanaged slice is the uncounted risk, and it's often where the weakest hygiene lives. You have choices here. App protection policies that wrap company data on a personal phone without managing the whole phone. Browser only access with downloads blocked, or a firm line that sensitive apps require a managed device full stop. Each has a cost in friction and a cost in coverage. What matters is that you choose one on purpose instead of discovering the gap after an incident. Devices are the layer where the dashboard looks most complete and is most often incomplete. But there's another surface carrying more daily attack traffic than any laptop. It arrives through the inbox all day, every day. Email and collaboration. Defender for office three hundred sixty five actions most attacks you'll ever see don't start with an exploit. They start with a message, a link, an attachment, a polite note from someone who looks like a supplier, and the control that sits in front of all of that is defender for office three hundred sixty five. Start with safe links. When a user clicks a link in an email, Safe Links checks the destination. At that moment, not just when the message arrived. That matters because attackers often send a clean link and flip it to something malicious. An hour later, the check at click time closes that gap. Turn it on for teams and office apps to not only for mail. Since people paste links everywhere, then Safe Attachments files get opened in a sandbox first and only released if nothing bad happens. The cost is time. A scan can add a minute or two before the file lands. For most people, that's invisible for the person waiting on a contract at four on a Friday, it isn't. So tell them it exists. A short note to the sales team saves a dozen tickets now. Anti-phishing policy. This is where tuning pays off. Impersonation protection lets you name the people attackers love to copy your CEO, your finance lead, the three people who can approve a payment, add them, and any message pretending to be them gets treated with suspicion. Mailbox intelligence builds on that by learning who each user normally talks to, so an odd sender stands out, but that list has to be maintained. People change roles. New finance staff join a protected list from two years ago. Protects the wrong people. So how do you set all this up without building it from scratch? Microsoft gives you two routes. Pre-Set security policies come as standard and strict. You assign users to one, and Microsoft maintains the settings as new protections arrive, they apply for most organizations. That's the right place to begin. It's fast, it's consistent, and it removes a whole class of drift. Custom policies give you control. You pick every threshold and every action that helps. When you have a specific need, like a department with unusual mail patterns. But it also means you own the upkeep. And over time, custom policies get stale conflict with each other, and nobody remembers why a setting was changed. A sensible pattern is simple put most people on standard. Put your highest risk users, executives and finance and admins on strict. Then add custom policy only when you can state the reason in a sentence. Now the part that makes people nervous. Male flow tighter filtering means some legitimate mail gets caught. That's a false positive and it will happen. A supplier's invoice lands in quarantine. A newsletter somebody wanted goes missing. The reaction is often to loosen everything and that's the wrong response. Handle it calmly. Instead, check the quarantine regularly. In the first weeks. Look at why each message was caught. Often it's one sender with broken authentication, and the fix is a narrow allow for that sender, not a weaker policy for everyone. Use the submission tools so Microsoft learns from your corrections and give users a clear way to report a missing message. So the problem reaches you in hours, not days. Track one number here. How many legitimate messages were quarantined per week and how many were released? If that count falls over time, your tuning is working. Email is only half of the picture, though the other half is where people work together. Teams and SharePoint carry a quieter exposure and it lives in external sharing out of the box. Many tenants let users share files with anyone who has a link. Guests join teams, channels and stay for years. Nobody reviews them. Look at three settings who can share outside the organization and with whom, whether links can be created for anyone or only for named people, and how long a guest account lives before someone confirms it still needs to exist. None of these makes a dramatic score jump, but a stale guest with access to a finance site is a real way in, and no email filter will see it. That leaves the thing attackers are actually after once they're inside the data itself. Data and apps. The parts. The score Underrepresents. Open the data side of the portal and you'll find actions. Create sensitivity labels. Turn on data loss prevention. Review your retention settings. There. Real controls. But look closely at what a score action can actually prove. Take sensitivity labels. The action asks whether a label policy exists. It doesn't ask whether anyone uses the labels. You can publish five labels, tick the box and watch the number move, while ninety percent of your files carry no label at all, the control exists, the coverage doesn't, and the score has no way to tell the difference. Data loss prevention works the same way. A policy that detects credit card numbers counts as a policy whether it covers the SharePoint site where finance actually keeps those numbers is a different question. So is whether anyone reads the alerts it creates a rule that fires into an empty mailbox, protects nothing, and it earns the same points as one that stops a leak. That's the pattern with data, the score can show that a mechanism was switched on. It can't show that the mechanism touches the things you can't afford to lose. For that, you go back to the Crown Jewels list from earlier and check each item by hand. Which sites hold payroll? Are those sites labeled? What happens when someone tries to share a file from there with a personal address? Now the apps and this is where the blind spot gets wider. Think about what happens when someone clicks a sign in with Microsoft button on a website they've never seen. A consent screen appears. It lists permissions. Read your mail, access your files. Stay connected when you're not using the app. Most people click accept because the screen looks official and the button is blue. That's OAuth consent and attackers have learned to use it. They build an app, send a link, and wait for someone to approve it. No password is stolen. No MFA prompt is bypassed, the user simply hands over access and it keeps working after they change their password. So look at how consent is set up in your tenant. Can any user approve any app? If so, every person in the company is a possible way in. Tighter options exist. Limit users to apps from verified publishers require admin approval for anything that asks for broad permissions, and then review what's already been approved because the list is usually longer than anyone expects. Then there's a quieter problem. App registrations and service principles. Every time a developer connects a script to your tenant, an identity gets created. It has a secret or a certificate. It has permissions and it has no person attached to it, so it never gets a password reset. A lever process, or a manager asking whether it's still needed. The developer moves teams. The project ends, the app registration stays its secret, sits in a config file somewhere. Still valid, still holding rights to read mail across the tenant. Nobody owns it, which means nobody reviews it. That's an identity that outlived its owner, and it's the kind of path an attacker will look for because nobody is watching it. So run an inventory list every app registration. Find the owner of each check when its secret was last used and what it can reach. Anything with no owner and no recent activity is a candidate for removal. Anything with broad permissions deserves a second look, even if someone vouches for it. Notice what these gaps have in common a sensitivity label with no coverage, a consent grant. Nobody remembers a service principle with no owner. None of them shows up as a failed action. None of them lowers the percentage. They sit outside what the score counts, so the number stays calm while the exposure grows. Which brings up the harder question if the percentage can't see these things, what can you use to tell whether you're actually getting better? Tracking improvements beyond the percentage. Start with the tool. The portal already gives you the history and trend view. It plots your score over time. A line goes up, a line goes down. It's tempting to stop reading there. Don't align without a cause is just decoration. So attach a name to every movement. Up four points in March. Write down why the admin MFA policy went live down two in May. Write that down. Two a new recommendation appeared or a setting got changed by someone who didn't know. Keep it in a plain log. A shared note or a simple table. Include a date, the change and the person responsible. Six months later, that log is worth more than the chart. The chart tells you something moved. The log tells you what you did about it. Now break the total apart. One number hides the shape of your tenant and the shape is what matters. The portal lets you view points by category and by individual action. Use both. Say identity sits at eighty percent and devices sit at forty. The total might look fine at sixty five, but now you know where the weak layer is and you know it before an attacker does. Or say the total hasn't moved in two months. While three identity actions closed and a pile of low value device settings reopened after a policy reset. The flatline would have hidden all of that. So watch categories for direction and actions for decisions. Which ones are done? Which are planned? Which are accepted? That's the working view. The total is just the summary on top, but even the best breakdown still measures configuration. It still can't tell you whether you're harder to attack. For that, you need numbers from outside the score. Pick a few that describe outcomes. MFA coverage is the obvious one, not whether a policy exists, but what share of active accounts actually completed a strong sign in last month? Break it out for admins and for the accounts that can approve payments. If admin coverage is ninety nine percent, ask about the missing one. That's your answer to where would they get in patch? Latency is another. How many days pass between a fix being released and being installed on your machines? A tenant that patches critical issues in five days sits in a different world from one that takes forty. The portals percentage will never show you that gap. Then meantime, to remediate when something is flagged, a risky sign in a vulnerable device, a malicious mailbox rule. How long until it's closed? Hours. Days. Weeks. This one measures your team's response, not your settings. It's often the best predictor of how an incident will go. Three numbers are plenty. Keep them stable so you can see direction over a year. Switching metrics every quarter just makes a story out of noise. Put these beside the score and the picture gets honest. If the percentage climbs while MFA coverage stalls, you're collecting easy points. If the percentage stays flat while remediation time drops from ten days to two, you're getting safer and the dashboard doesn't know it.
00:46:39 Speaker: That second case matters. It tells you the score can sit still while real progress happens underneath. One more habit makes the whole thing trustworthy. Annotate every time the score moves. Record what caused it. Sort the cause into one of a few buckets. Something you changed, a license you added, or Microsoft altering the recommendations themselves. That last bucket is the one people miss. You can do nothing for a month and watch the number shift, and when it does, someone will ask why? That's the next problem. Why the score changes without you doing anything. You open the dashboard on a Monday. Last week it said seventy four. Now it says seventy one. Nobody touched a policy. Nobody disabled anything. And yet the number moved. This isn't a bug. It's how the score is built. The list of improvement actions isn't fixed. Microsoft adds new ones, retires old ones, and changes how much some of them are worth. Your tenant stays the same. The ruler changes. Start with additions. A new threat shows up, or a new control ships, and a matching recommendation lands in your list. You don't have it configured yet, so it counts against you. The points you could earn go up your earned points don't. The percentage drops. Retirements work the other way. An action that no longer makes sense gets removed. Sometimes it was something you'd already completed, so you lose points you had banked. Other times it was something you'd skipped and your score rises for no reason you can claim. Then reweighting. Microsoft can decide an action matters more than it used to and give it more points. If you haven't done it, the gap widens. If you have, you gain. Either way, you weren't consulted. Now the other source of drift scope. The score only counts what applies to the products you're licensed for and using. Add a new license or say, defender for endpoint or defender for identity, and a whole set of actions appears that wasn't there before. Turn on a new workload and the same thing happens. The maximum score grows because there's more to be secured. That's the denominator we talked about earlier. When it grows, your percentage can fall even though you did nothing wrong. In fact, you may have done something right. You expanded protection and the dashboard treated that as new homework. Here's what that looks like in practice. A tenant buys the endpoint product in April, two hundred machines now report in a new group of device recommendations appears and almost none are done yet. The score drops eight points in a week. Read that drop at face value and it looks like regression. Read it properly and it's the opposite. The organization can now see a part of its estate it couldn't see before. The exposure was always there. The number just caught up with it. So you need a way to tell a real regression from scope growth. The annotation log we discussed does most of the work. Check the date of the drop against three things. Did anyone change a policy that day? Did a license or workload get added? Did the list of recommendations itself change? Usually one of those answers is yes, and if it's none of those, then you may have a real problem. A setting got reverted. Someone turned off a policy by accident. That case is worth catching quickly, which is another reason to look at action level history and not only the total. Now the harder part. Telling executives the instinct when a number falls is to explain it defensively. It's not really a drop. It's because of the license change said in that tone, it sounds like an excuse even when it's true. So change the framing. Don't lead with the score. Lead with what changed in your coverage and let the score follow something like this. We added endpoint protection to two hundred machines, which gave us visibility into a set of gaps we couldn't measure before the percentage dipped because the measured area grew. Here are the three gaps we're closing first. Notice what that does. It turns a movement you didn't cause into a decision you did make, and it keeps the conversation on exposure, which is where it belongs. Better still, warn people ahead of time. Tell leadership early before any license change that the number may fall and why. A drop you predicted reads is planning a drop. You explain afterward reads as damage control. That conversation, though, is only one of several. Someone senior is waiting for the next update and they'll want a slide. What goes on? It matters more than most people think. Reporting to executives without creating a target picture the slide. Most boards have seen one big number a green arrow, maybe a small line underneath that says industry average sixty eight. It looks clear it isn't. And the moment you put it up, you've handed the room a target because the number invites a question. And the question is always the same. Why aren't we at ninety? Now you're answering for points, not for exposure, and the team spends the next quarter chasing the percentage. That's exactly the trap we've been walking around. So change what goes on the slide instead of we are at seventy four percent. Say something closer to this. Three exposures are closed, two remain. Here's what each one means. Look at the difference. The first sentence is a grade. The second is a status report on things that could hurt you. One invites a debate about arithmetic. The other invites a decision. Make it concrete. Admin accounts now require phishing resistant sign in. Legacy protocols are blocked after we move the two apps that depended on them. Guest access is still open on the finance sites, and we'll close it by the end of next month. An executive can repeat that sentence in the hallway. Nobody can repeat up four points. That's the real shift. You show risk reduced, not points earned. A good way to hold yourself to it is a one page view. Just one. If it needs two pages, it's a report and reports don't get read for things. Go on it first. The top risks not twenty three to five written in plain words, each tied to something the business cares about. A stolen sign in on a payments mailbox, not MFA gap category identity. Second, an owner for each a name, not a team. A person, someone who can say yes or no and be held to it. Third, a date when it closes or when you'll next review it. A risk without a date is a wish. Fourth, accepted risks the things you've decided not to fix with the reason. So nobody discovers them later and calls them an oversight. We'll come back to why that matters. Notice what's missing. There's no percentage on the page. You can put the score in a corner if leadership likes seeing it. But it's context. Small and quiet, not the headline. Then there's the benchmark. That line comparing you to organizations like yours. It's tempting to include it feels like evidence, but think about what it actually says. Your tenants scored higher than the average of a group you didn't choose. Measured by a ruler you don't control on a list that includes settings your business may not need. That tells you almost nothing about whether an attacker can get in. Worse, it cuts both ways. Score above average and leadership relaxes. Score below and someone asks for a crash program that produces easy points and little else. Either reaction pulls attention away from your real exposure, so use comparison for what it's good for context. A rough sense of where you sit. Say it once, then move on. Never set it as a goal because the moment it becomes a goal, it stops being a useful measure. That's an old problem, and it's older than any dashboard. One more habit helps give leadership a question to answer, not just a status to read. We can close the guest access gap in two weeks, but it will break file sharing for three partner teams. Do you want that now or after the quarter closes? That's a real decision with a real cost, and it pulls the executive into the model instead of leaving them watching a number. When people see the trade offs, they stop asking for a higher score. They start asking better questions. And sometimes the right answer to one of those questions is no, not now, not this way. Maybe not at all. That answer needs a place to live. Risk acceptance is a valid answer. Open an improvement action and you'll see the status options planned, completed, resolved through a third party and one that makes people uneasy. Risk accepted. It feels like admitting defeat. It isn't. It's a decision. And decisions are the whole point of this model. The score hands you a list of possible work. Some of it you'll do, some of it. You'll look at way against your own tenant and choose not to do. But the status can be used two ways. Used. Well, it's a record. Someone looked at the action, understood what it protects against, and decided the cost or the disruption wasn't worth it. Right now, used badly. It's a hiding place. The action is hard. Nobody wants to own it. So it gets marked, accepted and the item leaves the list. The number improves, the exposure stays. You can tell the difference by what's written next to it. A real acceptance has three things. The reason in plain words, an owner by name and a review date. Without all three. It's not an acceptance, it's a shrug with a status attached. Take the reason first. Too hard isn't a reason. Blocking this protocol would stop the plant's label printers, which can't be replaced until the January refresh and the printers sit on an isolated network, is a reason someone reading that a year later can judge whether it still holds the owner matters because acceptance transfers the risk to a person. If something goes wrong through that gap. There's a name on the decision. That sounds harsh, but it works in your favor. People think harder before signing their name to a gap and they push to close it sooner. The review date is what keeps an acceptance honest. Circumstances change, the printers get replaced. A new attack makes the gap more dangerous. Without a date, an accepted risk quietly becomes a permanent one. Set it for three months, six at the most and put it in a calendar someone actually opens. Now, sometimes you don't need to accept a risk at all because something else already covers it. That's a compensating control. Say a recommendation asks you to restrict a certain kind of sign in, but you've already blocked those sign ins at the network edge and monitor for any attempt. The action is redundant. You've reached the same protection by a different road. Be careful here, though. A compensating control has to genuinely do the same job we have. A firewall doesn't compensate for an identity gap. Ask whether an attacker who tried the exact path the action closes would hit your alternative, and whether you'd see it. If you can't answer yes to both, you have an acceptance, not a replacement, right? The compensating control down to name it, name who maintains it, note what would make it stop working? If the control disappears, the original risk is back and someone needs to know. Which brings us to the comparison that tends to settle arguments. Picture two tenants. One sits at eighty percent and nobody can explain which gaps remain or why the other sits at seventy five. And every open item has a reason and owner and a date. The second tenant is safer not because of the five points, because someone understands the tenant. When an incident happens, that team knows which doors were left open on purpose and which ones nobody noticed. They spend the first hour investigating. Not discovering a lower score with documented reasoning is a map. A higher score without it is just a number. Somebody got to. And that's why leadership should be comfortable seeing accepted risks on the page. They aren't failures. They are evidence that the team is making choices instead of chasing a percentage. But there's a limit to how good those choices can be. You can only decide a risk is acceptable if you know how likely it is to be used against you. That means knowing what attackers actually do. Threat lead thinking starting from the attacker. There are two ways to read that list of improvement actions. The first is control lead. You look at the list and ask what's missing? Every open action is a gap. Every gap is a failure and the job is to close them all. That's the natural reading because the dashboard was built to produce it. The second is threat LED. You ask a different question what's likely not what's missing in general? What would someone actually try against a tenant like mine this month with the tools they already have? Those two questions sound close. They produce very different backlogs, control lead thinking thoughts, work by category, identity devices, apps, data threat lead thinking sorts. Work by what an attacker does one step after another. And once you see the steps, a lot of the list rearranges itself. Start with a path that shows up again and again. Phishing to token theft. An employee gets a message that looks like a shared document. The link leads to a fake sign in page that sits between the user and the real one. The user types a password, approves the MFA prompt, and the page quietly copies the session token that comes back. Now, the attacker doesn't need the password. They hold a valid session and it already passed MFA. Read your own actions with that path in mind. Which ones break it? Phishing resistant sign in breaks it because the credential can't be relayed. Requiring a compliant device breaks it because the stolen token won't work from the attacker's machine. Shorter session lifetimes. Shrink it. Click time link checking makes the first step harder. Notice that those actions come from four different categories. A category view would never have put them in one group. Second path password spray. No cleverness. Here. The attacker takes a short list of very common passwords and tries each one against thousands of accounts slowly, so no single account locks. Somebody always picked spring twenty twenty five. Then they sign in through whatever door doesn't ask for a second step, which is often a legacy protocol. This is why blocking legacy authentication and covering every account with MFA come before nearly everything else. They cut the path in half. Third path consent abuse. We touched on it with apps. The attacker never steals a credential at all. They send a link, a user approves an app, and the app keeps access to mail and files long after any password change. The actions that matter here are about consent settings and app review. They're easy to overlook because they hold few points and look minor on the list. That's the quiet benefit of this approach. Some actions that look small become obvious once you see which path they cut, and some that look big turn out to protect against something nobody's doing to you right now. So here's the practice. Pick three or four attack paths relevant to your tenant. For each one, write the steps in order. Then place your open actions against each step. Where a step has no action covering it, you found a real gap where a step has three, you probably have enough. Do it on one page with arrows. It takes an hour and it beats a month of sorting by points. Now how do you choose the paths? You don't have to invent them. Microsoft publishes a great deal about what it sees. Threat intelligence reports describe which groups are active, which techniques they favor, and which industries they target. Read them with one question in mind does this apply to me? A ransomware crew going after hospitals tells a retailer less than a fraud group going after payment teams. Pick what matches your size, your sector and your exposure. Your own data is even better. What has hit you before? Which accounts get password spray every week? Which phishing messages reached an inbox? Which sign ins from odd countries did the system flag? That's evidence about your tenant and no benchmark can offer it. Then there's the discipline of treating incidents as input. When something goes wrong, even something small, ask which step the attacker took and which action would have stopped it. That answer should move an item up your list the same week. This changes what the score is for. It stops being a to do list to finish and becomes a catalog to search. You come with a question about an attacker and you look inside it for the controls that answer. And the portal already has a bridge for exactly this. It doesn't live inside Secure score itself, but it sits right next to it. Connecting secure score to defender signals. The bridge is exposure management. It lives in the same defender portal a few clicks from the score, and most teams never open it. Secure score tells you which controls are missing. Exposure management adds the part the score can't how those gaps connect. Here's what that means. In practice. The portal builds attack paths. It maps how an attacker could move from one point in your environment to something that matters step by step. A device with a known weakness assigned an admin session on that device. A route from there to a critical server. Each piece looks minor alone, chained together. There a way in and it marks which assets are critical. Domain controllers. Admin accounts systems tied to sensitive data. So a week setting on a laptop nobody uses. And the same week setting on a machine holding admin tokens no longer look equal. They stop sharing a line on the list. That's the context. The percentage was missing. Instead of asking which action is worth the most points, you ask which gap sits on a path to something you can't lose. Those two lists rarely match. Now look at what's already happened in your tenant. Incidents and alerts are evidence and they're sitting in the portal right now. Take a month of them. Which techniques showed up? Maybe a run of sign ins from odd locations, a mailbox rule that forwards everything outside, a malicious attachment that reached an inbox before it was caught. Each one tells you something the benchmark never could. This is what people actually try against you. Then ask the follow up which control would have stopped it earlier if a stolen session was used from an unmanaged machine? The answer points at device compliance. If a forwarding rule survived for days, it points at mailbox auditing and alert tuning. You're reading your own incidents as a vote on which actions matter. That vote should change the backlog. An action you'd ranked fortieth becomes third because an attacker just walked through the gap it covers. No dashboard would have told you that, which leads to something worth building into the routine. After any real incident, run a short post-mortem with one purpose. Rerank the list. Not a long report. Just three questions. What did the attacker do first? Which open action sat on that step? And what else sits on the same path? Move those items up and write down why do that a few times a year and your backlog stops, reflecting what Microsoft thinks is generically important. It starts reflecting what's been tried against you. The same logic applies to devices through vulnerability management. This part of defender scans your machines for known weaknesses. Unpatched software, risky configurations expose services, and it scores each one by how easy it is to exploit and whether attackers are using it right now. That second part matters most. A device sitting in secure score might be worth a few points, whether or not anyone is attacking it. A vulnerability that's being used in live campaigns is a different thing. It jumps the queue. Think of it this way the score gives you a standing list and vulnerability. Data tells you which item is on fire today.
01:02:11 Speaker: So feed one into the other. Look at your device recommendations, then check which machines can be actively exploited. Weaknesses. Fix the intersection first. A machine with both a missing setting and a live vulnerability is where your hours should go before anything else. Notice how this changes the daily work. You still open the score, but you read it next to attack paths, alerts, and vulnerability data, and each one corrects the others. The score says what's undone. The signals say what's dangerous. That's easier to see than to describe. So let's take two tenants with the exact same percentage and watch them split apart. A worked example two tenants, same score, two tenants, two security teams. Both open the dashboard on the same morning. Both see seventy percent. Same number, same green arrow. Put them on one slide and a board would say they're in the same shape. They aren't. Not even close. Start with tenant, a mid-sized services firm, about nine hundred people. The score came from lots of small wins. Safe links is on. Anti-phishing policies are tuned. Sensitivity labels are published. Audit logging is enabled. Plenty of items sit in the completed column, but look at the admin accounts. Four of the eleven global administrators have no MFA registered, not weak MFA none. And legacy authentication is still open because an old scanning app depends on it and nobody wanted to touch it now. Tenant be a manufacturer about the same size, same seventy percent. Here. The identity layer is strong. Every admin signs in with a phishing resistant method. Legacy protocols are blocked. Conditional access is small and clean with an owner on each policy, but devices are thin. Defender for endpoint covers maybe half the laptops. Nobody has inventoried the rest. Attack surface reduction rules don't exist yet, and compliance policies are loose enough that most machines pass without much of a check. Same score. Opposite problems. So how does that happen? Because the points came from different places. Tenant A collected a large number of low effort actions. Tenant B put its effort into a few heavy ones and skipped a whole category. The percentage adds everything together and throws away the shape. Now run the attacker test from earlier. Take password spray first against tenant. A spray as a gift for admin accounts. Have a single password standing between the attacker and the whole tenant, and the legacy door means no second step is ever requested. One weak password, one successful sign in, and the attacker holds a global admin session. That's a short path to a very bad day against tenant B. The same attack mostly bounces, legacy protocols are shut and admins use credentials that can't be guessed or relayed. The attacker moves on. Now flip it. Take a phishing email carrying a malicious attachment aimed at a laptop tenant B is the soft target. Now if the machine isn't onboarded, defender never sees the payload run. Nothing blocks the child process. No alert fires the attacker lands on a device that's invisible to the security team, and from there they look for tokens and move sideways. Tenant A handles this better. It's device coverage is decent, and even if one laptop falls, the damage has a ceiling. Since the attacker still has to find a way to those admin accounts, which admittedly they might. So neither tenant is safe, but they're exposed in different places and the number can't tell you which place. What should each one do first? Tenant A has a short urgent list. Register strong MFA on those four admin accounts this week, then separate admin roles from daily accounts so nobody browses email as a global admin, then deal with legacy authentication properly. Find the scanning app, move it to a modern method or isolated, and block the protocol. All of this sits in identity and it would likely move the score only a few points. That's the point. The biggest drop in real risk is worth the least on the dashboard. Tenant B starts somewhere else. Build the device inventory first. Compare what the directory says exists against what defender sees and find the missing laptops on board them. Then turn on attack surface reduction rules in audit mode. Read the logs for a few weeks and enforce rule by rule. Raise compliance thresholds and steps so conditional access has something real to read. Notice what's common. Neither plan starts with the highest point action. Both start with the biggest hole on the shortest attacker path. That's what the percentage was hiding. Two identical totals, two completely different priorities, and a dashboard that would have told both teams to go and fix whatever happened to be worth the most points. A single number can't carry that much information, but a team that knows its paths can. Of course, knowing the right order of work is only half the problem. Priorities decay without a routine for revisiting them. Even a good plan drifts back into chasing whatever the portal puts on top. A review rhythm that actually holds most plans don't fail on day one. They fail around week six, when the urgent work of the month pushes the review off the calendar and nobody notices it's gone. So the fix isn't a better plan. It's a rhythm small enough that it survives a busy quarter. Start with the monthly triage. One hour. Same time every month. Same few people in the room. The agenda never changes. And that's the point. First. What's new? Check the improvement actions for anything added since last time and for any action whose points changed. Second, what moved on its own. Look at the annotation log and confirm every score change has a cause written next to it. Third what closed? Mark the finished work and check that it stayed finished because a reverted setting rarely announces itself. That's it. You're not fixing anything in this meeting. You're sorting. Each new item goes into one of three places. Do soon schedule or consider for acceptance. Then you leave and the work happens through your normal change process. The monthly hour keeps the list from piling up, but it can't answer a bigger question, which is whether you're working on the right things at all. That's the quarterly threat review, and it's a different kind of meeting. Give it two hours and bring someone from the incident side. The question is simple what did attackers actually do this quarter? Pull the incidents, the alert trends. The spray attempts, the phishing that reached inboxes, then run the exercise from earlier in the script. Which step did they take and which open action sat on that step. Out of that comes a reranked backlog. Maybe consent abuse showed up twice, so the app consent actions move up. Maybe nothing touched devices so that work can wait a little longer. You also read the new threat intelligence from Microsoft here and ask one honest question. Does any of this describe us? If yes, adjust. If no, say so and move on. Then there's the yearly Rebaseline once a year, step back from the individual actions and check the frame itself. Which licenses do you hold now that you didn't hold twelve months ago? Which workloads are live? Scope drifts quietly and the list of actions drifts with it. This is also when you check ownership, people leave, teams merge, and an action that belong to someone who left in March belongs to nobody in October, and you revisit every accepted risk with a review date in the coming year. Some will still hold, some won't. Now, running through all three rhythms is one rule that does most of the work. Every action has an owner or a documented acceptance, one or the other. Nothing sits in a third state called. Somebody should look at this, That third state is where backlogs go to die. An action without an owner has no deadline, no questions asked, and no one to ask them of. It just ages. A year later, it's still there and everyone assumes someone else has it. So in the monthly meeting, sweep for orphans. Any action with no name gets one before you leave the room or it moves to acceptance with a reason and a date. Both outcomes are fine. Staying unowned isn't a few small habits keep the whole thing alive. Put the meetings in calendars as recurring events, not reminders you'll set later. Keep the agenda on one page. Write the outcomes down even briefly, since the notes from March are what make September's meeting short and rotate who runs it so the routine doesn't depend on one person staying employed. Notice what this rhythm is protecting you from, not attackers directly. It's protecting you from drift. The slow slide back to whatever the dashboard puts at the top. But a calendar only gets meetings into a room. It doesn't decide who in that room is allowed to say yes. And that question has sunk more good programs than any technical problem. Ownership and governance. Who decides? Look at who actually touches the secure score action. It's never one person identity. Actions belong to the identity team. Device settings sit with whoever runs Intune and Defender for endpoint mail filtering belongs to the messaging admins. Data labels often belong to compliance or to nobody. One list on the dashboard four or five owners behind it. That's the first structural fact the score pretends to be a single backlog, but the work is split across teams that don't share a manager, a sprint, or a budget. So the review rhythm only works if each slice has a named owner, and the security lead holds the whole picture without owning every task. Think of that lead as an editor, not an author. They track the full list, push on the stuck items and bring the threat view. The slice owners do the changes. When those roles blur, you get one overloaded person who becomes the bottleneck for every setting in the tenant. Now the harder question who signs off on a change that users will feel? Take enforcing MFA for a department or blocking a protocol. An old app depends on. The security team can argue it's right, but they can't decide alone because they don't carry the cost. The help desk takes the calls. The sales manager takes the complaint. Someone has to say yes on behalf of the people who live with it. So split changes into two kinds before the meeting starts. Changes with no visible user effect like turning on audit, logging or tightening a back end setting need only the owner's approval, and a note in the log changes users will notice need a second name, a business or IT service owner who can weigh the disruption and accept it. Write that rule down once, then nobody has to argue it case by case. Next problem change advisory boards. Plenty of organizations already have one. It meets weekly or every two weeks. And for a security change, it can add three weeks of waiting. By then, the threat has moved or the person who raised the change has forgotten the details. You don't have to dodge the process. You have to shape it. Pre-approve a class of low risk changes so standard items skip the queue, run user impacting changes through the board, but arrive with the pilot results and the rollback plan already attached. A board that sees report only data and a tested exit says yes quickly. A board that sees a bare request says come back next month, then give emergencies their own door. If an actively exploited gap appears, an owner and one senior approver can act the same day and the board reviews it afterward, that path should exist before you need it, which leaves the business. When must it be in the loop? Not for every setting. Leadership doesn't need an opinion on Safelink policy, but they do need to hear about three things. Anything that changes how people sign in or share files across the whole company. Any risk acceptance on a path to your crown jewels since the business owns the loss. If that gap is used, and any trade off where security and a revenue process collide like the partner file sharing example from earlier. For those bring a choice, not a lecture. Here's the exposure. Here's the fix. Here's what it breaks. And here's the date we'd need to decide by. Notice what governance really is here. It's not a committee. It's three clear answers. Who does the work, who accepts the disruption and who owns the leftover risk. If a team can say those out loud for every open action the process holds, if they can't, the same gaps will keep resurfacing in the monthly meeting. And when governance is missing, the failures look familiar. Teams repeat the same mistakes in the same order across tenants of every size, common failure patterns and how to spot them for mistakes. Show up again and again. They look different on the surface. Underneath, they share one root. Someone stopped asking what the work was for. The first is chasing the percentage as a quarterly objective. It usually starts with a good intention. Leadership wants a measurable goal. The team picks the only number the portal gives them, and the target becomes eighty five by June. From that day on, the backlog gets sorted by points per hour. The cheap actions go first. The hard ones, the ones that cut real paths, slide to next quarter. You can spot it in how people talk. Listen for we need four more points. Or what's the quickest way to get over eighty. Nobody mentions an attacker or a user. The meeting is about arithmetic, and if the team hits the target in May and spends June idle, that's the proof. The goal was the number. So when the number arrived, the work stopped. The second is enforcing without piloting, then rolling back in panic, someone reads a recommendation on a Friday it says require MFA for all users. It looks clear it's worth a lot of points and the policy goes live for everyone. By Monday. By ten o'clock, the help desk queue is long. A warehouse can't scan stock because it's shared accounts can't complete a prompt, and a director is on the phone, so the policy gets switched off quickly without a review. And now there's a second cost. The business has watched security break something, and the next request gets pushed back before anyone's read it. The control didn't fail. The rollout did. But the damage lands on trust, and trust takes longer to rebuild than a policy. Takes to rewrite. The tell here is a changelog with a gap in it a policy created, enabled, then disabled within a day or two, with no pilot group in between and no report, only data behind it. If you see that pattern once, ask who approved it. If you see it twice, you found a process problem, not a one off. The third is treating third party credit as proof of protection. We covered how a third party tool can earn an action. Its points. The failure is what happens next. The action shows as resolved. The item leaves the list and nobody opens the tool again. Was it configured for every mailbox or only the pilot group? Is it still licensed? Does it still run after the last reorganization? Credit is a claim. It isn't a test and claims age. A product bought two years ago, deployed by someone who has since left, can sit in the score as a finished action while covering a fraction of the estate. To catch it, pick one credited action each quarter and verify it from scratch. Don't read the portal. Go to the tool, check the coverage and try to trigger the thing. It should stop. If it fails, you've learned something. The percentage never would. The fourth is ignoring the actions with high impact and high effort. They're easy to see and hard to start retiring. A legacy protocol that three business apps still use. Redesigning admin roles across a decade of accumulated permissions. Replacing shared accounts. Each one needs weeks of preparation, a few uncomfortable conversations, and a team willing to be unpopular for a while. So they sit at the bottom quarter after quarter because the low effort items always look more reasonable. But that bottom pile often holds the actions that sit on the shortest attacker paths. The effort is the price of the risk reduction, and avoiding it doesn't make the risk smaller. Check for it by sorting your open list by age. If the oldest items are also the ones tied to your crown jewels, you're protecting your comfort. Instead of your tenant. Break one of them into stages. Give it an owner and put a date on the first stage. Spot these four early and they cost you a conversation. Spot them late and they cost you an incident. Which is why the last piece is a plan that puts all of it in order. A ninety day plan that respects users and risk ninety days can change a tenant. It can't fix one. So the plan runs on order, not volume. Days one to thirty are about ground truth. Start with the baseline, export the current score, the category breakdown, and every open action. Freeze that copy. It's your before picture and you'll need it later. Then give every open action a name or a documented acceptance. Anything sitting unowned gets sorted in the first monthly meeting, not the third. Now handle the quiet safety work. Create two emergency access accounts. Exclude them from every conditional access policy. Store the credentials offline. Set an alert that fires the moment either one signs in. Do this before you write a single new policy. Skip it and one bad rule can lock you out of the tenant you're trying to protect. Then the first real moves. Pick the identity actions on your shortest attacker paths. Strong MFA for admins blocking legacy authentication. Separate admin accounts. Build those policies but switch them on in report only mode. Nobody feels anything yet. You're collecting evidence. Days thirty one to sixty are where enforcement starts in rings. Read the sign in logs from the first month. Who would have been blocked? Which apps still use old protocols? Which shared accounts can't complete a prompt? Fix those cases first, one by one, with the people involved then in force for the IT ring. A week later the champions. Then one department at a time. And finally everyone watch helpdesk volume at each step because it's your cheapest early warning. If tickets spike, pause the ring and find the cause before moving on while identity rolls out. Start the quieter layers. Check defender for endpoint coverage against your device inventory and onboard the gaps. Turn on attack surface reduction rules in audit mode, review the email policies and tune them where false positives show up. None of this needs a big announcement. It needs patience and a log. Day sixty one to ninety are for checking whether any of it worked. Return to the outcome numbers. What share of active accounts now complete. Strong sign in. What's the admin figure? How long do flagged items take to close compared with day one? Put those beside the score and annotate every movement with its cause. Then review the sign in data again, this time for the enforced policies. Anyone still blocked repeatedly needs a fix or an exception written down. Run one attack path exercise against what you've built and note which steps still have no control behind them. Finally, document the acceptances. Every action you decided not to take gets a reason, an owner and a review date. Put them on the one page view, then hold the first quarterly threat review and rerank what's left. So how do you know it worked? Not by the percentage. It will probably rise and that's fine, but it can't be the verdict. Success means exposure closed admin accounts can't be relayed. The old protocol door is shut. Devices that matter are visible, and when something does fire, your team closes it in hours instead of weeks. That last part is the real test. The first incident after day ninety should feel different. Fewer surprises, faster answers, and a team that knows which gaps were left open on purpose. Keep the pace after the plan ends, the rhythm carries it. Monthly triage. Quarterly threats. Yearly. Rebaseline. Conclusion from checkbox to context. So the score is a signal. It isn't a verdict. The flaw was never the number. It was the model around it, the one that treated a list of possible work as a grade. If this changed how you think about posture, leave a review. It helps more people find it. And connect with me on LinkedIn and tell me which topic to take apart next. Because a tenant can sit at ninety percent with one admin account that has no MFA and the dashboard will still be green.