1
00:00:00,040 --> 00:00:03,600
Welcome to the M365 FM podcast,
where we explore the people,
2
00:00:03,600 --> 00:00:06,559
ideas and technologies shaping
the future of work.
3
00:00:06,559 --> 00:00:08,119
What log analytics actually is?
4
00:00:08,119 --> 00:00:10,279
Log analytics is a filing
cabinet for everything your
5
00:00:10,279 --> 00:00:13,759
systems whisper to each other,
every sign in every file open,
6
00:00:13,759 --> 00:00:17,079
and every error becomes a record
in that cabinet and on its own.
7
00:00:17,079 --> 00:00:18,920
One record is nearly useless.
8
00:00:18,920 --> 00:00:21,879
Stack millions of them together,
and patterns appear like an
9
00:00:21,879 --> 00:00:24,440
account signing in from two
countries five minutes apart.
10
00:00:24,440 --> 00:00:25,600
That's the part that stays
11
00:00:25,600 --> 00:00:27,399
hidden, and it's why security
12
00:00:27,399 --> 00:00:28,879
teams and developers both end up
13
00:00:28,879 --> 00:00:29,320
here.
14
00:00:29,320 --> 00:00:30,399
You don't need to be a developer
15
00:00:30,399 --> 00:00:31,519
to follow this, because there's
16
00:00:31,519 --> 00:00:32,600
a point and click mode for
17
00:00:32,600 --> 00:00:33,799
people who have never written a
18
00:00:33,799 --> 00:00:34,439
query.
19
00:00:34,439 --> 00:00:37,759
Welcome to the M365 FM podcast
where we explore the people,
20
00:00:37,759 --> 00:00:40,359
ideas, and technologies shaping
the future of work.
21
00:00:40,359 --> 00:00:41,719
A record is one line that says
22
00:00:41,719 --> 00:00:42,560
something happened at a
23
00:00:42,560 --> 00:00:44,079
particular moment and nothing
24
00:00:44,079 --> 00:00:44,560
more.
25
00:00:44,560 --> 00:00:45,880
Someone signed in, someone
26
00:00:45,880 --> 00:00:47,640
opened a file and a page, took
27
00:00:47,640 --> 00:00:49,520
four seconds to load a service,
28
00:00:49,520 --> 00:00:50,960
restarted itself at three in the
29
00:00:50,960 --> 00:00:51,560
morning.
30
00:00:51,560 --> 00:00:54,320
Each line is boring on its own,
which is exactly why nobody
31
00:00:54,320 --> 00:00:55,479
reads them one at a time.
32
00:00:55,479 --> 00:00:57,679
There's a distinction that trips
people up, and it's worth
33
00:00:57,679 --> 00:00:58,960
getting straight early.
34
00:00:58,960 --> 00:01:01,240
Azure monitor watches your
systems in two ways.
35
00:01:01,240 --> 00:01:02,509
Metrics are the numbers on a
36
00:01:02,509 --> 00:01:04,109
dashboard, like a server sitting
37
00:01:04,109 --> 00:01:05,390
at eighty percent memory right
38
00:01:05,390 --> 00:01:05,790
now.
39
00:01:05,790 --> 00:01:08,750
Logs are the detailed story of
what happened, like who signed
40
00:01:08,750 --> 00:01:11,189
in, what they opened, and what
broke right after.
41
00:01:11,189 --> 00:01:14,349
Metrics tell you something is
wrong, and logs tell you why the
42
00:01:14,349 --> 00:01:15,670
scale is worth pausing on.
43
00:01:15,670 --> 00:01:17,870
Those records arrive from
everywhere, including Azure
44
00:01:17,870 --> 00:01:20,989
Resources, other clouds, your
own applications and servers
45
00:01:20,989 --> 00:01:22,469
sitting in an office closet.
46
00:01:22,469 --> 00:01:25,590
A single busy service can write
thousands of lines an hour, and
47
00:01:25,590 --> 00:01:27,709
a whole company can produce
millions a day.
48
00:01:27,709 --> 00:01:29,129
One failed sign in is a
49
00:01:29,129 --> 00:01:30,069
non-event that you'd never
50
00:01:30,069 --> 00:01:30,670
notice.
51
00:01:30,670 --> 00:01:32,950
Ten thousand failed sign ins in
a single hour.
52
00:01:32,950 --> 00:01:35,829
Sorted by country is a story,
and it's a story you can only
53
00:01:35,829 --> 00:01:37,829
read if those records were kept
in one place.
54
00:01:37,829 --> 00:01:39,310
That's the whole job of log
analytics.
55
00:01:39,310 --> 00:01:40,670
It gathers scattered records
56
00:01:40,670 --> 00:01:41,909
from everywhere and puts them
57
00:01:41,909 --> 00:01:42,870
somewhere you can actually
58
00:01:42,870 --> 00:01:43,549
search.
59
00:01:43,549 --> 00:01:45,349
The honest part is this a log is
60
00:01:45,349 --> 00:01:46,629
only as useful as your ability
61
00:01:46,629 --> 00:01:48,230
to search it, and a cabinet full
62
00:01:48,230 --> 00:01:49,390
of records you never open is
63
00:01:49,390 --> 00:01:49,989
just storage.
64
00:01:49,989 --> 00:01:51,109
You pay for every month.
65
00:01:51,109 --> 00:01:53,069
That cost is real and we'll get
to it later.
66
00:01:53,069 --> 00:01:55,909
Because how much data you send
in and how long you hold on to
67
00:01:55,909 --> 00:01:58,870
it, is where this whole thing
gets interesting, where all
68
00:01:58,870 --> 00:01:59,989
these logs come from.
69
00:01:59,989 --> 00:02:01,629
If records are the raw material,
70
00:02:01,629 --> 00:02:02,829
something has to go and collect
71
00:02:02,829 --> 00:02:04,349
them because nothing arrives on
72
00:02:04,349 --> 00:02:04,909
its own.
73
00:02:04,909 --> 00:02:06,069
On the other side, that's
74
00:02:06,069 --> 00:02:07,590
something is usually the Azure
75
00:02:07,590 --> 00:02:08,270
Monitor agent.
76
00:02:08,270 --> 00:02:09,389
You install it on a virtual
77
00:02:09,389 --> 00:02:10,469
machine or a cluster of
78
00:02:10,469 --> 00:02:12,030
containers, and it connects back
79
00:02:12,030 --> 00:02:13,509
to Azure to ask what it should
80
00:02:13,509 --> 00:02:14,229
be watching.
81
00:02:14,229 --> 00:02:16,590
The answer comes from a data
collection rule, which is really
82
00:02:16,590 --> 00:02:17,590
just a set of instructions.
83
00:02:17,590 --> 00:02:20,270
You write once and reuse across
many machines.
84
00:02:20,270 --> 00:02:21,909
It says which events to gather,
85
00:02:21,909 --> 00:02:22,990
what shape they should be in,
86
00:02:22,990 --> 00:02:24,310
and which table they should land
87
00:02:24,310 --> 00:02:24,669
in.
88
00:02:24,669 --> 00:02:27,389
If you've ever set up a rule for
sorting your email into folders,
89
00:02:27,389 --> 00:02:30,629
the idea is the same just aimed
at servers instead of messages.
90
00:02:30,629 --> 00:02:32,990
Those rules can also clean up
data on the way in.
91
00:02:32,990 --> 00:02:34,110
They can drop the noise you
92
00:02:34,110 --> 00:02:35,729
never look at and reshape what's
93
00:02:35,729 --> 00:02:37,189
left, so it fits neatly into a
94
00:02:37,189 --> 00:02:37,949
table.
95
00:02:37,949 --> 00:02:40,509
The reshaping itself is written
in the same query language.
96
00:02:40,509 --> 00:02:41,750
You'll meet in a few minutes.
97
00:02:41,750 --> 00:02:42,889
Hold on to that thought, because
98
00:02:42,889 --> 00:02:44,030
filtering early is the single
99
00:02:44,030 --> 00:02:45,229
biggest lever you have over the
100
00:02:45,229 --> 00:02:45,710
bill later.
101
00:02:45,710 --> 00:02:48,669
Not everything lives in Azure,
and that's fine if you build
102
00:02:48,669 --> 00:02:49,669
your own application.
103
00:02:49,669 --> 00:02:53,189
An interface called the Logs
Ingestion API lets it send its
104
00:02:53,189 --> 00:02:55,789
records straight in and it
replaces an older method.
105
00:02:55,789 --> 00:02:58,389
Microsoft has since retired.
106
00:02:58,389 --> 00:03:00,750
The same agent reaches into
other clouds and on premises
107
00:03:00,750 --> 00:03:03,830
servers, so a mixed environment
isn't a problem if your world is
108
00:03:03,830 --> 00:03:06,300
Microsoft three hundred sixty
five rather than Azure.
109
00:03:06,300 --> 00:03:08,099
You're already generating this
data.
110
00:03:08,099 --> 00:03:10,699
Every team's message, every
SharePoint document you open,
111
00:03:10,699 --> 00:03:13,780
and every sign in writes an
audit record into what Microsoft
112
00:03:13,780 --> 00:03:15,500
calls the Unified Audit Log.
113
00:03:15,500 --> 00:03:16,860
Defender adds its own security
114
00:03:16,860 --> 00:03:18,379
events on top, so the records
115
00:03:18,379 --> 00:03:19,740
exist whether or not anyone is
116
00:03:19,740 --> 00:03:20,580
managing them.
117
00:03:20,580 --> 00:03:22,819
The real question is whether
they land somewhere you can use
118
00:03:22,819 --> 00:03:25,539
them, or somewhere they pile up
until someone asks a question
119
00:03:25,539 --> 00:03:27,180
you can no longer answer.
120
00:03:27,180 --> 00:03:29,180
Inside a workspace and its
tables.
121
00:03:29,180 --> 00:03:30,659
All those records need a home.
122
00:03:30,659 --> 00:03:32,620
And in Azure, that home is
called a workspace.
123
00:03:32,620 --> 00:03:33,979
Picture it as the container
124
00:03:33,979 --> 00:03:35,539
where everything lands and the
125
00:03:35,539 --> 00:03:37,099
tables inside it as the labeled
126
00:03:37,099 --> 00:03:38,520
drawers that sort records by
127
00:03:38,520 --> 00:03:39,139
type.
128
00:03:39,139 --> 00:03:41,780
Sign in records go in one
drawer, performance numbers in
129
00:03:41,780 --> 00:03:43,620
another, and errors in a third.
130
00:03:43,620 --> 00:03:44,819
Azure creates the drawers it
131
00:03:44,819 --> 00:03:46,099
needs automatically, and you can
132
00:03:46,099 --> 00:03:47,419
add your own for data it doesn't
133
00:03:47,419 --> 00:03:48,300
recognize yet.
134
00:03:48,300 --> 00:03:50,539
Custom tables are worth a
mention because they're.
135
00:03:50,539 --> 00:03:52,900
How you bring in data Microsoft
has never seen.
136
00:03:52,900 --> 00:03:56,139
You describe the shape of your
records once, and from then on,
137
00:03:56,139 --> 00:03:58,300
anything that matches can flow
in and be queried.
138
00:03:58,300 --> 00:04:01,300
Like everything else, every
record also carries a timestamp
139
00:04:01,300 --> 00:04:03,020
the exact moment it happened.
140
00:04:03,020 --> 00:04:04,340
That single detail is what makes
141
00:04:04,340 --> 00:04:05,340
a question like what changed
142
00:04:05,340 --> 00:04:05,810
last night?
143
00:04:05,810 --> 00:04:08,430
Answerable because you can
always narrow a search to a
144
00:04:08,430 --> 00:04:09,370
window of time.
145
00:04:09,370 --> 00:04:12,289
A small beginner decision here
turns into real money.
146
00:04:12,289 --> 00:04:15,150
Each table gets assigned a plan,
and the plan controls how
147
00:04:15,150 --> 00:04:18,569
expensive the data is to store,
how fast you can search it, and
148
00:04:18,569 --> 00:04:20,009
how long it sticks around.
149
00:04:20,009 --> 00:04:22,850
Analytics is for data you watch
closely and need back instantly.
150
00:04:22,850 --> 00:04:25,610
If you only dig into something
when it breaks, that's basic.
151
00:04:25,610 --> 00:04:27,810
Auxiliary is for the high volume
stuff you mostly keep.
152
00:04:27,810 --> 00:04:30,649
For the record, one workspace
can mix all three, which means
153
00:04:30,649 --> 00:04:33,449
you can treat a noisy log and a
critical one very differently in
154
00:04:33,449 --> 00:04:34,089
the same building.
155
00:04:34,089 --> 00:04:35,089
The price gap between those
156
00:04:35,089 --> 00:04:36,089
plans is bigger than you'd
157
00:04:36,089 --> 00:04:36,769
expect.
158
00:04:36,769 --> 00:04:38,209
The same gigabyte of data costs
159
00:04:38,209 --> 00:04:39,689
roughly two dollars thirty cents
160
00:04:39,689 --> 00:04:41,009
on analytics, fifty cents on
161
00:04:41,009 --> 00:04:42,050
basic, and five cents on
162
00:04:42,050 --> 00:04:42,689
auxiliary.
163
00:04:42,689 --> 00:04:44,850
That's about forty six times
cheaper at the bottom end.
164
00:04:44,850 --> 00:04:47,129
And it's the kind of gap that
decides whether a project is
165
00:04:47,129 --> 00:04:48,329
affordable at all.
166
00:04:48,329 --> 00:04:49,449
The catch is that you can't just
167
00:04:49,449 --> 00:04:50,410
dump everything into the
168
00:04:50,410 --> 00:04:51,250
cheapest drawer.
169
00:04:51,250 --> 00:04:53,810
The cheap plans are slower to
search and aren't built for the
170
00:04:53,810 --> 00:04:56,370
real time detection that
security teams rely on.
171
00:04:56,370 --> 00:04:58,810
Every table is a trade off
between speed, cost, and how
172
00:04:58,810 --> 00:05:00,529
often you actually look at
what's inside it.
173
00:05:00,529 --> 00:05:01,730
Get that plan wrong, and you
174
00:05:01,730 --> 00:05:02,930
either overpay for data you
175
00:05:02,930 --> 00:05:04,449
rarely touch, or slow down a
176
00:05:04,449 --> 00:05:04,730
query.
177
00:05:04,730 --> 00:05:06,410
You run every morning asking
178
00:05:06,410 --> 00:05:07,860
questions with a few lines of
179
00:05:07,860 --> 00:05:08,720
KCL.
180
00:05:08,720 --> 00:05:11,519
A container full of records is
only useful if you can ask it
181
00:05:11,519 --> 00:05:15,360
questions, and the language you
ask in is called k q l, short
182
00:05:15,360 --> 00:05:17,079
for Kusto Query Language.
183
00:05:17,079 --> 00:05:18,519
Don't let the name scare you off
184
00:05:18,519 --> 00:05:19,560
because it doesn't read like
185
00:05:19,560 --> 00:05:20,199
programming.
186
00:05:20,199 --> 00:05:21,439
It reads like a short list of
187
00:05:21,439 --> 00:05:22,920
steps you'd hand to a very
188
00:05:22,920 --> 00:05:24,120
patient assistant.
189
00:05:24,120 --> 00:05:26,600
One of those steps in plain
English looks like this.
190
00:05:26,600 --> 00:05:28,160
Start with the table of sign
ins.
191
00:05:28,160 --> 00:05:29,800
Keep only the ones that failed
192
00:05:29,800 --> 00:05:31,759
and show me the newest ten in
193
00:05:31,759 --> 00:05:32,160
KCL.
194
00:05:32,160 --> 00:05:34,759
That's three or four lines, each
one separated by a pipe.
195
00:05:34,759 --> 00:05:38,040
And the pipe simply means take
what you had and pass it along.
196
00:05:38,040 --> 00:05:39,160
The verbs you'll use most are
197
00:05:39,160 --> 00:05:40,399
easy to recognize once you see
198
00:05:40,399 --> 00:05:40,639
them.
199
00:05:40,639 --> 00:05:42,019
There's one to filter, one to
200
00:05:42,019 --> 00:05:43,339
sort, one to count, one to
201
00:05:43,339 --> 00:05:44,800
group, and one to limit how much
202
00:05:44,800 --> 00:05:45,399
comes back.
203
00:05:45,399 --> 00:05:46,639
Combine a few of them and you go
204
00:05:46,639 --> 00:05:48,019
from a million records to the
205
00:05:48,019 --> 00:05:49,800
handful that actually matter if
206
00:05:49,800 --> 00:05:50,959
writing anything at all sounds
207
00:05:50,959 --> 00:05:51,759
miserable.
208
00:05:51,759 --> 00:05:55,399
Microsoft built a second way in
simple mode, lets you point and
209
00:05:55,399 --> 00:05:57,899
click your way through filters,
sorting and grouping without
210
00:05:57,899 --> 00:06:01,240
typing a single line, and the
results update as you go.
211
00:06:01,240 --> 00:06:03,959
The advanced editor sits right
next to it so you can graduate
212
00:06:03,959 --> 00:06:05,199
whenever you feel ready.
213
00:06:05,199 --> 00:06:06,759
Two small things are worth
knowing.
214
00:06:06,759 --> 00:06:09,800
Kql cares about capital letters
and table and column names, so a
215
00:06:09,800 --> 00:06:12,920
lowercase letter can break a
query for no obvious reason, and
216
00:06:12,920 --> 00:06:16,000
a query that searches everything
at once runs slowly by design.
217
00:06:16,000 --> 00:06:18,639
So the habit that pays off is
starting from the right table
218
00:06:18,639 --> 00:06:19,560
and narrowing down.
219
00:06:19,560 --> 00:06:21,879
The reassuring part is that this
skill travels.
220
00:06:21,879 --> 00:06:23,220
The same language shows up in
221
00:06:23,220 --> 00:06:24,779
Microsoft Defender's Advanced
222
00:06:24,779 --> 00:06:25,399
hunting.
223
00:06:25,399 --> 00:06:26,560
So once you can ask one
224
00:06:26,560 --> 00:06:28,360
Microsoft tool a question, you
225
00:06:28,360 --> 00:06:29,879
can ask several of them the same
226
00:06:29,879 --> 00:06:30,319
way.
227
00:06:30,319 --> 00:06:32,939
It's the same engine underneath,
which is why the muscle memory
228
00:06:32,939 --> 00:06:34,600
carries over so cleanly.
229
00:06:34,600 --> 00:06:36,800
When a query finally gives you
something useful, you don't have
230
00:06:36,800 --> 00:06:37,920
to keep it to yourself.
231
00:06:37,920 --> 00:06:39,000
You can pin the result to a
232
00:06:39,000 --> 00:06:40,759
dashboard, save it for later, or
233
00:06:40,759 --> 00:06:41,720
hand it to a colleague as a
234
00:06:41,720 --> 00:06:42,759
starting point.
235
00:06:42,759 --> 00:06:44,560
What teams actually do with it?
236
00:06:44,560 --> 00:06:46,759
In practice, people use it to
answer questions that are
237
00:06:46,759 --> 00:06:48,360
already costing them sleep.
238
00:06:48,360 --> 00:06:50,680
Say a website gets slow every
night around two in the morning
239
00:06:50,680 --> 00:06:51,600
and nobody knows why.
240
00:06:51,600 --> 00:06:54,120
Instead of guessing, you pull
the records from that hour,
241
00:06:54,120 --> 00:06:56,439
group them by the thing that
keeps failing and the culprit
242
00:06:56,439 --> 00:06:59,879
usually falls out in minutes, or
an important file disappears
243
00:06:59,879 --> 00:07:02,279
from a shared site and someone
needs to know who touched it.
244
00:07:02,279 --> 00:07:04,959
And when the audit records hold
the answer along with the
245
00:07:04,959 --> 00:07:08,800
account, the time and the device
security teams use the same data
246
00:07:08,800 --> 00:07:10,149
for a different question.
247
00:07:10,149 --> 00:07:12,670
They watch for a sign in from a
country nobody on the team has
248
00:07:12,670 --> 00:07:15,589
ever visited, or an account that
suddenly starts downloading far
249
00:07:15,589 --> 00:07:16,509
more than usual.
250
00:07:16,509 --> 00:07:19,350
Microsoft Sentinel, the security
product built on top of this
251
00:07:19,350 --> 00:07:22,069
same workspace, watches for
those patterns around the clock
252
00:07:22,069 --> 00:07:24,389
and raises an alert when
something looks wrong.
253
00:07:24,389 --> 00:07:25,629
Analysts can then hunt through
254
00:07:25,629 --> 00:07:27,269
the raw records themselves using
255
00:07:27,269 --> 00:07:28,589
the same kql we just talked
256
00:07:28,589 --> 00:07:29,149
about.
257
00:07:29,149 --> 00:07:30,870
Operations teams lean on it,
too.
258
00:07:30,870 --> 00:07:33,069
They want to know whether a
service keeps restarting,
259
00:07:33,069 --> 00:07:35,670
whether memory is creeping up
week after week, or whether a
260
00:07:35,670 --> 00:07:38,110
deployment broke something at
four in the morning.
261
00:07:38,110 --> 00:07:41,470
Microsoft ships ready made views
called insights that answer a
262
00:07:41,470 --> 00:07:43,949
lot of those questions before
you write a single query.
263
00:07:43,949 --> 00:07:45,350
There's also a smarter layer for
264
00:07:45,350 --> 00:07:46,730
the cases nobody thought to
265
00:07:46,730 --> 00:07:46,910
write.
266
00:07:46,910 --> 00:07:49,550
A rule for built in anomaly
detection can flag a pattern
267
00:07:49,550 --> 00:07:52,230
that simply looks wrong compared
to the usual rhythm, which is
268
00:07:52,230 --> 00:07:54,750
how you catch the problem you
didn't know to look for.
269
00:07:54,750 --> 00:07:57,350
If your business lives in
dashboards, query results can be
270
00:07:57,350 --> 00:08:00,509
exported straight into power BI,
so people who never open Azure
271
00:08:00,509 --> 00:08:01,990
can still see what's going on.
272
00:08:01,990 --> 00:08:04,990
There are also alerts, which are
just saved, queries that check
273
00:08:04,990 --> 00:08:07,750
themselves on a schedule and
shout when a pattern shows up.
274
00:08:07,750 --> 00:08:09,189
That's the payoff, and it's
real.
275
00:08:09,189 --> 00:08:11,540
Every one of those questions
depends on records that were
276
00:08:11,540 --> 00:08:12,300
collected and kept.
277
00:08:12,300 --> 00:08:15,100
Which brings us to the part
nobody enjoys thinking about
278
00:08:15,100 --> 00:08:16,860
where the money actually goes.
279
00:08:16,860 --> 00:08:18,540
Log analytics isn't free, and
280
00:08:18,540 --> 00:08:19,740
pretending otherwise would waste
281
00:08:19,740 --> 00:08:20,500
your time.
282
00:08:20,500 --> 00:08:21,740
The model is simple, though.
283
00:08:21,740 --> 00:08:22,819
You pay for the volume of data
284
00:08:22,819 --> 00:08:23,920
you collect and for how long you
285
00:08:23,920 --> 00:08:24,379
keep it.
286
00:08:24,379 --> 00:08:26,300
And creating a workspace costs
nothing.
287
00:08:26,300 --> 00:08:28,899
Every gigabyte you send in shows
up on that bill.
288
00:08:28,899 --> 00:08:30,060
Whether or not anyone ever looks
289
00:08:30,060 --> 00:08:31,459
at it, that volume is where the
290
00:08:31,459 --> 00:08:32,539
surprises live.
291
00:08:32,539 --> 00:08:35,059
A chatty source like a verbose
application log, or an agent
292
00:08:35,059 --> 00:08:37,500
that reports every heartbeat can
produce an enormous amount of
293
00:08:37,500 --> 00:08:39,019
data without anyone noticing.
294
00:08:39,019 --> 00:08:40,080
Multiply that across a few
295
00:08:40,080 --> 00:08:41,539
hundred machines and the bill
296
00:08:41,539 --> 00:08:42,899
can grow faster than the systems
297
00:08:42,899 --> 00:08:43,620
it's watching.
298
00:08:43,620 --> 00:08:45,419
Two facts here run against
instinct.
299
00:08:45,419 --> 00:08:47,659
The size you're billed for is
usually smaller than the raw
300
00:08:47,659 --> 00:08:50,580
event that produced it by
roughly a quarter on average,
301
00:08:50,580 --> 00:08:53,659
because Microsoft trims the
overhead before charging you.
302
00:08:53,659 --> 00:08:56,500
A handful of tables, including
Heartbeat and Azure activity,
303
00:08:56,500 --> 00:08:59,500
are also free to send data into,
so not every gigabyte carries
304
00:08:59,500 --> 00:09:00,620
the same price tag.
305
00:09:00,620 --> 00:09:02,299
Prices are set by region two, so
306
00:09:02,299 --> 00:09:03,500
the same data can cost different
307
00:09:03,500 --> 00:09:04,620
amounts depending on where your
308
00:09:04,620 --> 00:09:05,700
workspace lives.
309
00:09:05,700 --> 00:09:07,539
That's a detail worth checking
310
00:09:07,539 --> 00:09:08,659
before you commit to a big
311
00:09:08,659 --> 00:09:09,220
rollout.
312
00:09:09,220 --> 00:09:10,700
The trap is a natural instinct.
313
00:09:10,700 --> 00:09:12,059
When something feels uncertain,
314
00:09:12,059 --> 00:09:13,299
Collecting more data feels
315
00:09:13,299 --> 00:09:14,019
safer.
316
00:09:14,019 --> 00:09:16,759
But most of what you collect is
never looked at, and you pay to
317
00:09:16,759 --> 00:09:18,340
store every last bit of it.
318
00:09:18,340 --> 00:09:20,820
Teams that treat the workspace
like a junk drawer end up paying
319
00:09:20,820 --> 00:09:21,659
for a junk drawer.
320
00:09:21,659 --> 00:09:22,779
The right fix is the one we
321
00:09:22,779 --> 00:09:24,000
mentioned earlier, which is to
322
00:09:24,000 --> 00:09:25,340
filter and reshape data before
323
00:09:25,340 --> 00:09:26,139
it's stored.
324
00:09:26,139 --> 00:09:29,100
If a log line is noise, you'll
never query, drop it at the door
325
00:09:29,100 --> 00:09:30,500
and never pay for it.
326
00:09:30,500 --> 00:09:32,080
Microsoft also gives you a daily
327
00:09:32,080 --> 00:09:33,779
cap, a hard limit that stops
328
00:09:33,779 --> 00:09:35,100
collection once you hit a
329
00:09:35,100 --> 00:09:35,580
threshold.
330
00:09:35,580 --> 00:09:37,259
It sounds like a safety net, and
it is.
331
00:09:37,259 --> 00:09:40,100
But Microsoft's own guidance
warns against leaning on it.
332
00:09:40,100 --> 00:09:43,460
When the cap Trips collection
stops completely and you lose
333
00:09:43,460 --> 00:09:45,860
visibility into your systems for
the rest of the day.
334
00:09:45,860 --> 00:09:47,100
That's a worse outcome than a
335
00:09:47,100 --> 00:09:48,259
bill if something breaks
336
00:09:48,259 --> 00:09:49,259
overnight.
337
00:09:49,259 --> 00:09:52,179
How long your data sticks around
cost and time are tied together,
338
00:09:52,179 --> 00:09:55,179
so it's worth understanding how
long a record actually lives.
339
00:09:55,179 --> 00:09:58,460
Every table gives you a window
of fast interactive retention
340
00:09:58,460 --> 00:10:01,419
where you can query data
instantly on analytics data.
341
00:10:01,419 --> 00:10:04,419
That window is thirty one days
out of the box, and it stretches
342
00:10:04,419 --> 00:10:07,059
to ninety days if Microsoft
Sentinel is enabled on the
343
00:10:07,059 --> 00:10:10,460
workspace after that window
closes, the data doesn't vanish.
344
00:10:10,460 --> 00:10:11,980
It moves into long term storage
345
00:10:11,980 --> 00:10:13,139
where it sits cheaply, and you
346
00:10:13,139 --> 00:10:14,490
pull it back only when you need
347
00:10:14,490 --> 00:10:14,929
it.
348
00:10:14,929 --> 00:10:17,809
Keeping data queryable costs
about ten cents per gigabyte per
349
00:10:17,809 --> 00:10:20,350
month, and long term storage
runs about two cents per
350
00:10:20,350 --> 00:10:21,409
gigabyte per month.
351
00:10:21,409 --> 00:10:23,970
You can hold onto data for up to
twelve years, which is the kind
352
00:10:23,970 --> 00:10:26,049
of number that matters if you
answer to an auditor.
353
00:10:26,049 --> 00:10:27,370
Getting old data back isn't
354
00:10:27,370 --> 00:10:28,529
instant, and that's the trade
355
00:10:28,529 --> 00:10:29,289
you're making.
356
00:10:29,289 --> 00:10:30,610
You run a search job that
357
00:10:30,610 --> 00:10:31,889
reaches into the archive and
358
00:10:31,889 --> 00:10:33,269
pulls the records you asked for
359
00:10:33,269 --> 00:10:34,850
into a fresh table where they
360
00:10:34,850 --> 00:10:36,210
can be queried like anything
361
00:10:36,210 --> 00:10:37,490
else.
362
00:10:37,490 --> 00:10:38,210
The Microsoft three.
363
00:10:38,210 --> 00:10:39,690
Sixty five clock is different.
364
00:10:39,690 --> 00:10:40,370
It's unified.
365
00:10:40,370 --> 00:10:41,950
Audit log keeps records for one
366
00:10:41,950 --> 00:10:43,129
hundred and eighty days by
367
00:10:43,129 --> 00:10:43,529
default.
368
00:10:43,529 --> 00:10:44,690
That used to be ninety days and
369
00:10:44,690 --> 00:10:46,029
it changed in October of twenty
370
00:10:46,029 --> 00:10:47,330
twenty three.
371
00:10:47,330 --> 00:10:49,789
Defenders advanced hunting,
meanwhile, only reaches back
372
00:10:49,789 --> 00:10:52,570
about thirty days of raw
security data, so you can have
373
00:10:52,570 --> 00:10:55,289
three different clocks running
at once or ticking for records
374
00:10:55,289 --> 00:10:56,570
that feel like the same thing.
375
00:10:56,570 --> 00:10:59,690
The practical move is to decide
table by table how long each
376
00:10:59,690 --> 00:11:01,250
kind of record needs to survive.
377
00:11:01,250 --> 00:11:04,289
Compliance data might need
years, while a chatty debug log
378
00:11:04,289 --> 00:11:05,529
might only need a week.
379
00:11:05,529 --> 00:11:07,929
The honest limit is that a
default isn't forever, and
380
00:11:07,929 --> 00:11:10,490
finding something later only
works if you set the retention
381
00:11:10,490 --> 00:11:11,529
before you need it.
382
00:11:11,529 --> 00:11:14,090
By the time you're hunting for a
record from eight months ago.
383
00:11:14,090 --> 00:11:16,919
The decision that would have
kept it is long gone.
384
00:11:16,919 --> 00:11:19,559
How it fits into Microsoft three
hundred sixty five and Azure.
385
00:11:19,559 --> 00:11:21,120
Zoom out for a second, because
386
00:11:21,120 --> 00:11:22,200
the pieces fit together more
387
00:11:22,200 --> 00:11:23,399
neatly than they first appear,
388
00:11:23,399 --> 00:11:24,639
as your monitor is the umbrella
389
00:11:24,639 --> 00:11:25,679
that watches everything running
390
00:11:25,679 --> 00:11:27,399
in Azure, and it splits that job
391
00:11:27,399 --> 00:11:27,919
in two.
392
00:11:27,919 --> 00:11:28,879
One half is metrics.
393
00:11:28,879 --> 00:11:29,840
The fast numbers.
394
00:11:29,840 --> 00:11:32,600
The other half is logs, which is
log analytics.
395
00:11:32,600 --> 00:11:33,480
So when someone says log
396
00:11:33,480 --> 00:11:35,240
analytics, they're really naming
397
00:11:35,240 --> 00:11:36,360
the log half of a bigger
398
00:11:36,360 --> 00:11:37,320
monitoring system.
399
00:11:37,320 --> 00:11:39,600
The connection worth remembering
is Microsoft Sentinel.
400
00:11:39,600 --> 00:11:41,000
It looks like a separate product
401
00:11:41,000 --> 00:11:42,200
with its own name and its own
402
00:11:42,200 --> 00:11:43,840
screens, but its workspace is
403
00:11:43,840 --> 00:11:44,200
the same.
404
00:11:44,200 --> 00:11:47,639
Log analytics workspace just
switched on for security.
405
00:11:47,639 --> 00:11:49,279
The same place that holds your
application.
406
00:11:49,279 --> 00:11:52,759
Logs can also hold your security
signals, and analysts query both
407
00:11:52,759 --> 00:11:53,960
with the same language.
408
00:11:53,960 --> 00:11:55,039
Microsoft three hundred sixty
409
00:11:55,039 --> 00:11:56,159
five has its own version of
410
00:11:56,159 --> 00:11:57,519
this, called the Unified Audit
411
00:11:57,519 --> 00:11:57,840
Log.
412
00:11:57,840 --> 00:11:58,960
It's where your teams,
413
00:11:58,960 --> 00:12:00,720
SharePoint and sign in activity
414
00:12:00,720 --> 00:12:01,399
lives.
415
00:12:01,399 --> 00:12:02,600
When a team needs to keep that
416
00:12:02,600 --> 00:12:03,879
data longer than the default
417
00:12:03,879 --> 00:12:05,600
window, it can be piped into a
418
00:12:05,600 --> 00:12:06,960
security tool through something
419
00:12:06,960 --> 00:12:08,039
called the office three hundred
420
00:12:08,039 --> 00:12:09,299
sixty five Management Activity
421
00:12:09,299 --> 00:12:10,080
API.
422
00:12:10,080 --> 00:12:12,039
There's a trade off hiding in
all of this.
423
00:12:12,039 --> 00:12:13,279
Putting security data in the
424
00:12:13,279 --> 00:12:14,279
same workspace as everything
425
00:12:14,279 --> 00:12:15,559
else can save money because a
426
00:12:15,559 --> 00:12:17,029
bigger volume can qualify for a
427
00:12:17,029 --> 00:12:17,710
discount.
428
00:12:17,710 --> 00:12:20,070
That discount is called a
commitment tier, and it works
429
00:12:20,070 --> 00:12:21,149
like a bulk deal.
430
00:12:21,149 --> 00:12:23,429
Commit to a daily amount
starting at one hundred
431
00:12:23,429 --> 00:12:26,789
gigabytes, and Microsoft will
knock up to about thirty percent
432
00:12:26,789 --> 00:12:28,389
off the per gigabyte price.
433
00:12:28,389 --> 00:12:30,149
Sentinel enabled data is built
434
00:12:30,149 --> 00:12:31,149
differently, though, which is
435
00:12:31,149 --> 00:12:32,110
why a lot of teams end up
436
00:12:32,110 --> 00:12:33,529
splitting security and
437
00:12:33,529 --> 00:12:34,590
operations into separate
438
00:12:34,590 --> 00:12:35,509
workspaces.
439
00:12:35,509 --> 00:12:37,590
The reason all of this matters
is that the concept you learned
440
00:12:37,590 --> 00:12:40,549
today isn't confined to one
product, the same workspace, the
441
00:12:40,549 --> 00:12:43,750
same tables, and the same query
language keep showing up across
442
00:12:43,750 --> 00:12:46,389
Microsoft's tools, and learning
them once pays off everywhere.
443
00:12:46,389 --> 00:12:49,149
So the next time a Microsoft
tool asks you to point something
444
00:12:49,149 --> 00:12:51,830
at a workspace, you'll know
exactly what it's asking for.
445
00:12:51,830 --> 00:12:53,350
You're not learning four
separate products.
446
00:12:53,350 --> 00:12:54,509
You're learning one idea that
447
00:12:54,509 --> 00:12:56,470
keeps reappearing, the honest
448
00:12:56,470 --> 00:12:56,870
limits.
449
00:12:56,870 --> 00:13:00,029
And one question for you before
we finish the limits deserve
450
00:13:00,029 --> 00:13:02,629
their due, because understanding
a tool's limits is what keeps
451
00:13:02,629 --> 00:13:03,710
you out of trouble.
452
00:13:03,710 --> 00:13:06,909
Log analytics can only show you
what was collected if a system
453
00:13:06,909 --> 00:13:10,269
never sent its records in, no
query will invent them, and no
454
00:13:10,269 --> 00:13:13,070
amount of cleverness will find
what was never written down.
455
00:13:13,070 --> 00:13:14,230
It's also not a place to hoard
456
00:13:14,230 --> 00:13:15,629
everything for free, because
457
00:13:15,629 --> 00:13:17,129
both volume and time carry a
458
00:13:17,129 --> 00:13:17,669
price.
459
00:13:17,669 --> 00:13:18,669
It can even show you the same
460
00:13:18,669 --> 00:13:20,029
event twice because the pipeline
461
00:13:20,029 --> 00:13:21,070
retrieves to make sure data
462
00:13:21,070 --> 00:13:22,149
arrives safely.
463
00:13:22,149 --> 00:13:24,809
A duplicate is usually the
system being careful rather than
464
00:13:24,809 --> 00:13:26,110
your query being wrong.
465
00:13:26,110 --> 00:13:27,909
Two more limits are worth
holding on to.
466
00:13:27,909 --> 00:13:29,190
Hunting through every table at
467
00:13:29,190 --> 00:13:30,409
once is slow, which is why
468
00:13:30,409 --> 00:13:31,629
knowing where to start is a real
469
00:13:31,629 --> 00:13:32,110
skill.
470
00:13:32,110 --> 00:13:33,830
The Daily Cap, meanwhile, is a
471
00:13:33,830 --> 00:13:35,269
hard stop and not a gentle
472
00:13:35,269 --> 00:13:35,870
break.
473
00:13:35,870 --> 00:13:38,590
Hit it and collection stops for
the remainder of the day, which
474
00:13:38,590 --> 00:13:41,549
means an overnight problem could
go completely unseen.
475
00:13:41,549 --> 00:13:43,230
A good first step is to start
small.
476
00:13:43,230 --> 00:13:44,830
Pick one system that matters,
477
00:13:44,830 --> 00:13:46,509
turn on collection for it, and
478
00:13:46,509 --> 00:13:47,669
watch what actually shows up
479
00:13:47,669 --> 00:13:48,909
before you open the floodgates
480
00:13:48,909 --> 00:13:49,789
everywhere else.
481
00:13:49,789 --> 00:13:51,149
The short version is simple.
482
00:13:51,149 --> 00:13:52,990
A log is a record of something
that happened.
483
00:13:52,990 --> 00:13:55,610
The workspace is where those
records are kept and tables sort
484
00:13:55,610 --> 00:13:56,549
them by type.
485
00:13:56,549 --> 00:13:59,230
Kql is how you ask questions,
and you don't need to write any
486
00:13:59,230 --> 00:14:00,269
code to use it.
487
00:14:00,269 --> 00:14:03,049
The two decisions that actually
shape your bill and your safety
488
00:14:03,049 --> 00:14:05,669
net are what you choose to
collect and how long you choose
489
00:14:05,669 --> 00:14:06,389
to keep it.
490
00:14:06,389 --> 00:14:08,750
That leaves one question worth
sitting with somewhere in your
491
00:14:08,750 --> 00:14:11,190
world, a system is already
recording everything it does.
492
00:14:11,190 --> 00:14:12,509
So who's actually reading it?
493
00:14:12,509 --> 00:14:13,830
If this made the topic feel less
494
00:14:13,830 --> 00:14:16,389
intimidating, subscribe to M365
495
00:14:16,389 --> 00:14:17,909
FM to master the tools and the
496
00:14:17,909 --> 00:14:19,309
mindset shaping the future of
497
00:14:19,309 --> 00:14:19,950
work.
00:00:00,040 --> 00:00:03,600
Welcome to the M365 FM podcast,
where we explore the people,
2
00:00:03,600 --> 00:00:06,559
ideas and technologies shaping
the future of work.
3
00:00:06,559 --> 00:00:08,119
What log analytics actually is?
4
00:00:08,119 --> 00:00:10,279
Log analytics is a filing
cabinet for everything your
5
00:00:10,279 --> 00:00:13,759
systems whisper to each other,
every sign in every file open,
6
00:00:13,759 --> 00:00:17,079
and every error becomes a record
in that cabinet and on its own.
7
00:00:17,079 --> 00:00:18,920
One record is nearly useless.
8
00:00:18,920 --> 00:00:21,879
Stack millions of them together,
and patterns appear like an
9
00:00:21,879 --> 00:00:24,440
account signing in from two
countries five minutes apart.
10
00:00:24,440 --> 00:00:25,600
That's the part that stays
11
00:00:25,600 --> 00:00:27,399
hidden, and it's why security
12
00:00:27,399 --> 00:00:28,879
teams and developers both end up
13
00:00:28,879 --> 00:00:29,320
here.
14
00:00:29,320 --> 00:00:30,399
You don't need to be a developer
15
00:00:30,399 --> 00:00:31,519
to follow this, because there's
16
00:00:31,519 --> 00:00:32,600
a point and click mode for
17
00:00:32,600 --> 00:00:33,799
people who have never written a
18
00:00:33,799 --> 00:00:34,439
query.
19
00:00:34,439 --> 00:00:37,759
Welcome to the M365 FM podcast
where we explore the people,
20
00:00:37,759 --> 00:00:40,359
ideas, and technologies shaping
the future of work.
21
00:00:40,359 --> 00:00:41,719
A record is one line that says
22
00:00:41,719 --> 00:00:42,560
something happened at a
23
00:00:42,560 --> 00:00:44,079
particular moment and nothing
24
00:00:44,079 --> 00:00:44,560
more.
25
00:00:44,560 --> 00:00:45,880
Someone signed in, someone
26
00:00:45,880 --> 00:00:47,640
opened a file and a page, took
27
00:00:47,640 --> 00:00:49,520
four seconds to load a service,
28
00:00:49,520 --> 00:00:50,960
restarted itself at three in the
29
00:00:50,960 --> 00:00:51,560
morning.
30
00:00:51,560 --> 00:00:54,320
Each line is boring on its own,
which is exactly why nobody
31
00:00:54,320 --> 00:00:55,479
reads them one at a time.
32
00:00:55,479 --> 00:00:57,679
There's a distinction that trips
people up, and it's worth
33
00:00:57,679 --> 00:00:58,960
getting straight early.
34
00:00:58,960 --> 00:01:01,240
Azure monitor watches your
systems in two ways.
35
00:01:01,240 --> 00:01:02,509
Metrics are the numbers on a
36
00:01:02,509 --> 00:01:04,109
dashboard, like a server sitting
37
00:01:04,109 --> 00:01:05,390
at eighty percent memory right
38
00:01:05,390 --> 00:01:05,790
now.
39
00:01:05,790 --> 00:01:08,750
Logs are the detailed story of
what happened, like who signed
40
00:01:08,750 --> 00:01:11,189
in, what they opened, and what
broke right after.
41
00:01:11,189 --> 00:01:14,349
Metrics tell you something is
wrong, and logs tell you why the
42
00:01:14,349 --> 00:01:15,670
scale is worth pausing on.
43
00:01:15,670 --> 00:01:17,870
Those records arrive from
everywhere, including Azure
44
00:01:17,870 --> 00:01:20,989
Resources, other clouds, your
own applications and servers
45
00:01:20,989 --> 00:01:22,469
sitting in an office closet.
46
00:01:22,469 --> 00:01:25,590
A single busy service can write
thousands of lines an hour, and
47
00:01:25,590 --> 00:01:27,709
a whole company can produce
millions a day.
48
00:01:27,709 --> 00:01:29,129
One failed sign in is a
49
00:01:29,129 --> 00:01:30,069
non-event that you'd never
50
00:01:30,069 --> 00:01:30,670
notice.
51
00:01:30,670 --> 00:01:32,950
Ten thousand failed sign ins in
a single hour.
52
00:01:32,950 --> 00:01:35,829
Sorted by country is a story,
and it's a story you can only
53
00:01:35,829 --> 00:01:37,829
read if those records were kept
in one place.
54
00:01:37,829 --> 00:01:39,310
That's the whole job of log
analytics.
55
00:01:39,310 --> 00:01:40,670
It gathers scattered records
56
00:01:40,670 --> 00:01:41,909
from everywhere and puts them
57
00:01:41,909 --> 00:01:42,870
somewhere you can actually
58
00:01:42,870 --> 00:01:43,549
search.
59
00:01:43,549 --> 00:01:45,349
The honest part is this a log is
60
00:01:45,349 --> 00:01:46,629
only as useful as your ability
61
00:01:46,629 --> 00:01:48,230
to search it, and a cabinet full
62
00:01:48,230 --> 00:01:49,390
of records you never open is
63
00:01:49,390 --> 00:01:49,989
just storage.
64
00:01:49,989 --> 00:01:51,109
You pay for every month.
65
00:01:51,109 --> 00:01:53,069
That cost is real and we'll get
to it later.
66
00:01:53,069 --> 00:01:55,909
Because how much data you send
in and how long you hold on to
67
00:01:55,909 --> 00:01:58,870
it, is where this whole thing
gets interesting, where all
68
00:01:58,870 --> 00:01:59,989
these logs come from.
69
00:01:59,989 --> 00:02:01,629
If records are the raw material,
70
00:02:01,629 --> 00:02:02,829
something has to go and collect
71
00:02:02,829 --> 00:02:04,349
them because nothing arrives on
72
00:02:04,349 --> 00:02:04,909
its own.
73
00:02:04,909 --> 00:02:06,069
On the other side, that's
74
00:02:06,069 --> 00:02:07,590
something is usually the Azure
75
00:02:07,590 --> 00:02:08,270
Monitor agent.
76
00:02:08,270 --> 00:02:09,389
You install it on a virtual
77
00:02:09,389 --> 00:02:10,469
machine or a cluster of
78
00:02:10,469 --> 00:02:12,030
containers, and it connects back
79
00:02:12,030 --> 00:02:13,509
to Azure to ask what it should
80
00:02:13,509 --> 00:02:14,229
be watching.
81
00:02:14,229 --> 00:02:16,590
The answer comes from a data
collection rule, which is really
82
00:02:16,590 --> 00:02:17,590
just a set of instructions.
83
00:02:17,590 --> 00:02:20,270
You write once and reuse across
many machines.
84
00:02:20,270 --> 00:02:21,909
It says which events to gather,
85
00:02:21,909 --> 00:02:22,990
what shape they should be in,
86
00:02:22,990 --> 00:02:24,310
and which table they should land
87
00:02:24,310 --> 00:02:24,669
in.
88
00:02:24,669 --> 00:02:27,389
If you've ever set up a rule for
sorting your email into folders,
89
00:02:27,389 --> 00:02:30,629
the idea is the same just aimed
at servers instead of messages.
90
00:02:30,629 --> 00:02:32,990
Those rules can also clean up
data on the way in.
91
00:02:32,990 --> 00:02:34,110
They can drop the noise you
92
00:02:34,110 --> 00:02:35,729
never look at and reshape what's
93
00:02:35,729 --> 00:02:37,189
left, so it fits neatly into a
94
00:02:37,189 --> 00:02:37,949
table.
95
00:02:37,949 --> 00:02:40,509
The reshaping itself is written
in the same query language.
96
00:02:40,509 --> 00:02:41,750
You'll meet in a few minutes.
97
00:02:41,750 --> 00:02:42,889
Hold on to that thought, because
98
00:02:42,889 --> 00:02:44,030
filtering early is the single
99
00:02:44,030 --> 00:02:45,229
biggest lever you have over the
100
00:02:45,229 --> 00:02:45,710
bill later.
101
00:02:45,710 --> 00:02:48,669
Not everything lives in Azure,
and that's fine if you build
102
00:02:48,669 --> 00:02:49,669
your own application.
103
00:02:49,669 --> 00:02:53,189
An interface called the Logs
Ingestion API lets it send its
104
00:02:53,189 --> 00:02:55,789
records straight in and it
replaces an older method.
105
00:02:55,789 --> 00:02:58,389
Microsoft has since retired.
106
00:02:58,389 --> 00:03:00,750
The same agent reaches into
other clouds and on premises
107
00:03:00,750 --> 00:03:03,830
servers, so a mixed environment
isn't a problem if your world is
108
00:03:03,830 --> 00:03:06,300
Microsoft three hundred sixty
five rather than Azure.
109
00:03:06,300 --> 00:03:08,099
You're already generating this
data.
110
00:03:08,099 --> 00:03:10,699
Every team's message, every
SharePoint document you open,
111
00:03:10,699 --> 00:03:13,780
and every sign in writes an
audit record into what Microsoft
112
00:03:13,780 --> 00:03:15,500
calls the Unified Audit Log.
113
00:03:15,500 --> 00:03:16,860
Defender adds its own security
114
00:03:16,860 --> 00:03:18,379
events on top, so the records
115
00:03:18,379 --> 00:03:19,740
exist whether or not anyone is
116
00:03:19,740 --> 00:03:20,580
managing them.
117
00:03:20,580 --> 00:03:22,819
The real question is whether
they land somewhere you can use
118
00:03:22,819 --> 00:03:25,539
them, or somewhere they pile up
until someone asks a question
119
00:03:25,539 --> 00:03:27,180
you can no longer answer.
120
00:03:27,180 --> 00:03:29,180
Inside a workspace and its
tables.
121
00:03:29,180 --> 00:03:30,659
All those records need a home.
122
00:03:30,659 --> 00:03:32,620
And in Azure, that home is
called a workspace.
123
00:03:32,620 --> 00:03:33,979
Picture it as the container
124
00:03:33,979 --> 00:03:35,539
where everything lands and the
125
00:03:35,539 --> 00:03:37,099
tables inside it as the labeled
126
00:03:37,099 --> 00:03:38,520
drawers that sort records by
127
00:03:38,520 --> 00:03:39,139
type.
128
00:03:39,139 --> 00:03:41,780
Sign in records go in one
drawer, performance numbers in
129
00:03:41,780 --> 00:03:43,620
another, and errors in a third.
130
00:03:43,620 --> 00:03:44,819
Azure creates the drawers it
131
00:03:44,819 --> 00:03:46,099
needs automatically, and you can
132
00:03:46,099 --> 00:03:47,419
add your own for data it doesn't
133
00:03:47,419 --> 00:03:48,300
recognize yet.
134
00:03:48,300 --> 00:03:50,539
Custom tables are worth a
mention because they're.
135
00:03:50,539 --> 00:03:52,900
How you bring in data Microsoft
has never seen.
136
00:03:52,900 --> 00:03:56,139
You describe the shape of your
records once, and from then on,
137
00:03:56,139 --> 00:03:58,300
anything that matches can flow
in and be queried.
138
00:03:58,300 --> 00:04:01,300
Like everything else, every
record also carries a timestamp
139
00:04:01,300 --> 00:04:03,020
the exact moment it happened.
140
00:04:03,020 --> 00:04:04,340
That single detail is what makes
141
00:04:04,340 --> 00:04:05,340
a question like what changed
142
00:04:05,340 --> 00:04:05,810
last night?
143
00:04:05,810 --> 00:04:08,430
Answerable because you can
always narrow a search to a
144
00:04:08,430 --> 00:04:09,370
window of time.
145
00:04:09,370 --> 00:04:12,289
A small beginner decision here
turns into real money.
146
00:04:12,289 --> 00:04:15,150
Each table gets assigned a plan,
and the plan controls how
147
00:04:15,150 --> 00:04:18,569
expensive the data is to store,
how fast you can search it, and
148
00:04:18,569 --> 00:04:20,009
how long it sticks around.
149
00:04:20,009 --> 00:04:22,850
Analytics is for data you watch
closely and need back instantly.
150
00:04:22,850 --> 00:04:25,610
If you only dig into something
when it breaks, that's basic.
151
00:04:25,610 --> 00:04:27,810
Auxiliary is for the high volume
stuff you mostly keep.
152
00:04:27,810 --> 00:04:30,649
For the record, one workspace
can mix all three, which means
153
00:04:30,649 --> 00:04:33,449
you can treat a noisy log and a
critical one very differently in
154
00:04:33,449 --> 00:04:34,089
the same building.
155
00:04:34,089 --> 00:04:35,089
The price gap between those
156
00:04:35,089 --> 00:04:36,089
plans is bigger than you'd
157
00:04:36,089 --> 00:04:36,769
expect.
158
00:04:36,769 --> 00:04:38,209
The same gigabyte of data costs
159
00:04:38,209 --> 00:04:39,689
roughly two dollars thirty cents
160
00:04:39,689 --> 00:04:41,009
on analytics, fifty cents on
161
00:04:41,009 --> 00:04:42,050
basic, and five cents on
162
00:04:42,050 --> 00:04:42,689
auxiliary.
163
00:04:42,689 --> 00:04:44,850
That's about forty six times
cheaper at the bottom end.
164
00:04:44,850 --> 00:04:47,129
And it's the kind of gap that
decides whether a project is
165
00:04:47,129 --> 00:04:48,329
affordable at all.
166
00:04:48,329 --> 00:04:49,449
The catch is that you can't just
167
00:04:49,449 --> 00:04:50,410
dump everything into the
168
00:04:50,410 --> 00:04:51,250
cheapest drawer.
169
00:04:51,250 --> 00:04:53,810
The cheap plans are slower to
search and aren't built for the
170
00:04:53,810 --> 00:04:56,370
real time detection that
security teams rely on.
171
00:04:56,370 --> 00:04:58,810
Every table is a trade off
between speed, cost, and how
172
00:04:58,810 --> 00:05:00,529
often you actually look at
what's inside it.
173
00:05:00,529 --> 00:05:01,730
Get that plan wrong, and you
174
00:05:01,730 --> 00:05:02,930
either overpay for data you
175
00:05:02,930 --> 00:05:04,449
rarely touch, or slow down a
176
00:05:04,449 --> 00:05:04,730
query.
177
00:05:04,730 --> 00:05:06,410
You run every morning asking
178
00:05:06,410 --> 00:05:07,860
questions with a few lines of
179
00:05:07,860 --> 00:05:08,720
KCL.
180
00:05:08,720 --> 00:05:11,519
A container full of records is
only useful if you can ask it
181
00:05:11,519 --> 00:05:15,360
questions, and the language you
ask in is called k q l, short
182
00:05:15,360 --> 00:05:17,079
for Kusto Query Language.
183
00:05:17,079 --> 00:05:18,519
Don't let the name scare you off
184
00:05:18,519 --> 00:05:19,560
because it doesn't read like
185
00:05:19,560 --> 00:05:20,199
programming.
186
00:05:20,199 --> 00:05:21,439
It reads like a short list of
187
00:05:21,439 --> 00:05:22,920
steps you'd hand to a very
188
00:05:22,920 --> 00:05:24,120
patient assistant.
189
00:05:24,120 --> 00:05:26,600
One of those steps in plain
English looks like this.
190
00:05:26,600 --> 00:05:28,160
Start with the table of sign
ins.
191
00:05:28,160 --> 00:05:29,800
Keep only the ones that failed
192
00:05:29,800 --> 00:05:31,759
and show me the newest ten in
193
00:05:31,759 --> 00:05:32,160
KCL.
194
00:05:32,160 --> 00:05:34,759
That's three or four lines, each
one separated by a pipe.
195
00:05:34,759 --> 00:05:38,040
And the pipe simply means take
what you had and pass it along.
196
00:05:38,040 --> 00:05:39,160
The verbs you'll use most are
197
00:05:39,160 --> 00:05:40,399
easy to recognize once you see
198
00:05:40,399 --> 00:05:40,639
them.
199
00:05:40,639 --> 00:05:42,019
There's one to filter, one to
200
00:05:42,019 --> 00:05:43,339
sort, one to count, one to
201
00:05:43,339 --> 00:05:44,800
group, and one to limit how much
202
00:05:44,800 --> 00:05:45,399
comes back.
203
00:05:45,399 --> 00:05:46,639
Combine a few of them and you go
204
00:05:46,639 --> 00:05:48,019
from a million records to the
205
00:05:48,019 --> 00:05:49,800
handful that actually matter if
206
00:05:49,800 --> 00:05:50,959
writing anything at all sounds
207
00:05:50,959 --> 00:05:51,759
miserable.
208
00:05:51,759 --> 00:05:55,399
Microsoft built a second way in
simple mode, lets you point and
209
00:05:55,399 --> 00:05:57,899
click your way through filters,
sorting and grouping without
210
00:05:57,899 --> 00:06:01,240
typing a single line, and the
results update as you go.
211
00:06:01,240 --> 00:06:03,959
The advanced editor sits right
next to it so you can graduate
212
00:06:03,959 --> 00:06:05,199
whenever you feel ready.
213
00:06:05,199 --> 00:06:06,759
Two small things are worth
knowing.
214
00:06:06,759 --> 00:06:09,800
Kql cares about capital letters
and table and column names, so a
215
00:06:09,800 --> 00:06:12,920
lowercase letter can break a
query for no obvious reason, and
216
00:06:12,920 --> 00:06:16,000
a query that searches everything
at once runs slowly by design.
217
00:06:16,000 --> 00:06:18,639
So the habit that pays off is
starting from the right table
218
00:06:18,639 --> 00:06:19,560
and narrowing down.
219
00:06:19,560 --> 00:06:21,879
The reassuring part is that this
skill travels.
220
00:06:21,879 --> 00:06:23,220
The same language shows up in
221
00:06:23,220 --> 00:06:24,779
Microsoft Defender's Advanced
222
00:06:24,779 --> 00:06:25,399
hunting.
223
00:06:25,399 --> 00:06:26,560
So once you can ask one
224
00:06:26,560 --> 00:06:28,360
Microsoft tool a question, you
225
00:06:28,360 --> 00:06:29,879
can ask several of them the same
226
00:06:29,879 --> 00:06:30,319
way.
227
00:06:30,319 --> 00:06:32,939
It's the same engine underneath,
which is why the muscle memory
228
00:06:32,939 --> 00:06:34,600
carries over so cleanly.
229
00:06:34,600 --> 00:06:36,800
When a query finally gives you
something useful, you don't have
230
00:06:36,800 --> 00:06:37,920
to keep it to yourself.
231
00:06:37,920 --> 00:06:39,000
You can pin the result to a
232
00:06:39,000 --> 00:06:40,759
dashboard, save it for later, or
233
00:06:40,759 --> 00:06:41,720
hand it to a colleague as a
234
00:06:41,720 --> 00:06:42,759
starting point.
235
00:06:42,759 --> 00:06:44,560
What teams actually do with it?
236
00:06:44,560 --> 00:06:46,759
In practice, people use it to
answer questions that are
237
00:06:46,759 --> 00:06:48,360
already costing them sleep.
238
00:06:48,360 --> 00:06:50,680
Say a website gets slow every
night around two in the morning
239
00:06:50,680 --> 00:06:51,600
and nobody knows why.
240
00:06:51,600 --> 00:06:54,120
Instead of guessing, you pull
the records from that hour,
241
00:06:54,120 --> 00:06:56,439
group them by the thing that
keeps failing and the culprit
242
00:06:56,439 --> 00:06:59,879
usually falls out in minutes, or
an important file disappears
243
00:06:59,879 --> 00:07:02,279
from a shared site and someone
needs to know who touched it.
244
00:07:02,279 --> 00:07:04,959
And when the audit records hold
the answer along with the
245
00:07:04,959 --> 00:07:08,800
account, the time and the device
security teams use the same data
246
00:07:08,800 --> 00:07:10,149
for a different question.
247
00:07:10,149 --> 00:07:12,670
They watch for a sign in from a
country nobody on the team has
248
00:07:12,670 --> 00:07:15,589
ever visited, or an account that
suddenly starts downloading far
249
00:07:15,589 --> 00:07:16,509
more than usual.
250
00:07:16,509 --> 00:07:19,350
Microsoft Sentinel, the security
product built on top of this
251
00:07:19,350 --> 00:07:22,069
same workspace, watches for
those patterns around the clock
252
00:07:22,069 --> 00:07:24,389
and raises an alert when
something looks wrong.
253
00:07:24,389 --> 00:07:25,629
Analysts can then hunt through
254
00:07:25,629 --> 00:07:27,269
the raw records themselves using
255
00:07:27,269 --> 00:07:28,589
the same kql we just talked
256
00:07:28,589 --> 00:07:29,149
about.
257
00:07:29,149 --> 00:07:30,870
Operations teams lean on it,
too.
258
00:07:30,870 --> 00:07:33,069
They want to know whether a
service keeps restarting,
259
00:07:33,069 --> 00:07:35,670
whether memory is creeping up
week after week, or whether a
260
00:07:35,670 --> 00:07:38,110
deployment broke something at
four in the morning.
261
00:07:38,110 --> 00:07:41,470
Microsoft ships ready made views
called insights that answer a
262
00:07:41,470 --> 00:07:43,949
lot of those questions before
you write a single query.
263
00:07:43,949 --> 00:07:45,350
There's also a smarter layer for
264
00:07:45,350 --> 00:07:46,730
the cases nobody thought to
265
00:07:46,730 --> 00:07:46,910
write.
266
00:07:46,910 --> 00:07:49,550
A rule for built in anomaly
detection can flag a pattern
267
00:07:49,550 --> 00:07:52,230
that simply looks wrong compared
to the usual rhythm, which is
268
00:07:52,230 --> 00:07:54,750
how you catch the problem you
didn't know to look for.
269
00:07:54,750 --> 00:07:57,350
If your business lives in
dashboards, query results can be
270
00:07:57,350 --> 00:08:00,509
exported straight into power BI,
so people who never open Azure
271
00:08:00,509 --> 00:08:01,990
can still see what's going on.
272
00:08:01,990 --> 00:08:04,990
There are also alerts, which are
just saved, queries that check
273
00:08:04,990 --> 00:08:07,750
themselves on a schedule and
shout when a pattern shows up.
274
00:08:07,750 --> 00:08:09,189
That's the payoff, and it's
real.
275
00:08:09,189 --> 00:08:11,540
Every one of those questions
depends on records that were
276
00:08:11,540 --> 00:08:12,300
collected and kept.
277
00:08:12,300 --> 00:08:15,100
Which brings us to the part
nobody enjoys thinking about
278
00:08:15,100 --> 00:08:16,860
where the money actually goes.
279
00:08:16,860 --> 00:08:18,540
Log analytics isn't free, and
280
00:08:18,540 --> 00:08:19,740
pretending otherwise would waste
281
00:08:19,740 --> 00:08:20,500
your time.
282
00:08:20,500 --> 00:08:21,740
The model is simple, though.
283
00:08:21,740 --> 00:08:22,819
You pay for the volume of data
284
00:08:22,819 --> 00:08:23,920
you collect and for how long you
285
00:08:23,920 --> 00:08:24,379
keep it.
286
00:08:24,379 --> 00:08:26,300
And creating a workspace costs
nothing.
287
00:08:26,300 --> 00:08:28,899
Every gigabyte you send in shows
up on that bill.
288
00:08:28,899 --> 00:08:30,060
Whether or not anyone ever looks
289
00:08:30,060 --> 00:08:31,459
at it, that volume is where the
290
00:08:31,459 --> 00:08:32,539
surprises live.
291
00:08:32,539 --> 00:08:35,059
A chatty source like a verbose
application log, or an agent
292
00:08:35,059 --> 00:08:37,500
that reports every heartbeat can
produce an enormous amount of
293
00:08:37,500 --> 00:08:39,019
data without anyone noticing.
294
00:08:39,019 --> 00:08:40,080
Multiply that across a few
295
00:08:40,080 --> 00:08:41,539
hundred machines and the bill
296
00:08:41,539 --> 00:08:42,899
can grow faster than the systems
297
00:08:42,899 --> 00:08:43,620
it's watching.
298
00:08:43,620 --> 00:08:45,419
Two facts here run against
instinct.
299
00:08:45,419 --> 00:08:47,659
The size you're billed for is
usually smaller than the raw
300
00:08:47,659 --> 00:08:50,580
event that produced it by
roughly a quarter on average,
301
00:08:50,580 --> 00:08:53,659
because Microsoft trims the
overhead before charging you.
302
00:08:53,659 --> 00:08:56,500
A handful of tables, including
Heartbeat and Azure activity,
303
00:08:56,500 --> 00:08:59,500
are also free to send data into,
so not every gigabyte carries
304
00:08:59,500 --> 00:09:00,620
the same price tag.
305
00:09:00,620 --> 00:09:02,299
Prices are set by region two, so
306
00:09:02,299 --> 00:09:03,500
the same data can cost different
307
00:09:03,500 --> 00:09:04,620
amounts depending on where your
308
00:09:04,620 --> 00:09:05,700
workspace lives.
309
00:09:05,700 --> 00:09:07,539
That's a detail worth checking
310
00:09:07,539 --> 00:09:08,659
before you commit to a big
311
00:09:08,659 --> 00:09:09,220
rollout.
312
00:09:09,220 --> 00:09:10,700
The trap is a natural instinct.
313
00:09:10,700 --> 00:09:12,059
When something feels uncertain,
314
00:09:12,059 --> 00:09:13,299
Collecting more data feels
315
00:09:13,299 --> 00:09:14,019
safer.
316
00:09:14,019 --> 00:09:16,759
But most of what you collect is
never looked at, and you pay to
317
00:09:16,759 --> 00:09:18,340
store every last bit of it.
318
00:09:18,340 --> 00:09:20,820
Teams that treat the workspace
like a junk drawer end up paying
319
00:09:20,820 --> 00:09:21,659
for a junk drawer.
320
00:09:21,659 --> 00:09:22,779
The right fix is the one we
321
00:09:22,779 --> 00:09:24,000
mentioned earlier, which is to
322
00:09:24,000 --> 00:09:25,340
filter and reshape data before
323
00:09:25,340 --> 00:09:26,139
it's stored.
324
00:09:26,139 --> 00:09:29,100
If a log line is noise, you'll
never query, drop it at the door
325
00:09:29,100 --> 00:09:30,500
and never pay for it.
326
00:09:30,500 --> 00:09:32,080
Microsoft also gives you a daily
327
00:09:32,080 --> 00:09:33,779
cap, a hard limit that stops
328
00:09:33,779 --> 00:09:35,100
collection once you hit a
329
00:09:35,100 --> 00:09:35,580
threshold.
330
00:09:35,580 --> 00:09:37,259
It sounds like a safety net, and
it is.
331
00:09:37,259 --> 00:09:40,100
But Microsoft's own guidance
warns against leaning on it.
332
00:09:40,100 --> 00:09:43,460
When the cap Trips collection
stops completely and you lose
333
00:09:43,460 --> 00:09:45,860
visibility into your systems for
the rest of the day.
334
00:09:45,860 --> 00:09:47,100
That's a worse outcome than a
335
00:09:47,100 --> 00:09:48,259
bill if something breaks
336
00:09:48,259 --> 00:09:49,259
overnight.
337
00:09:49,259 --> 00:09:52,179
How long your data sticks around
cost and time are tied together,
338
00:09:52,179 --> 00:09:55,179
so it's worth understanding how
long a record actually lives.
339
00:09:55,179 --> 00:09:58,460
Every table gives you a window
of fast interactive retention
340
00:09:58,460 --> 00:10:01,419
where you can query data
instantly on analytics data.
341
00:10:01,419 --> 00:10:04,419
That window is thirty one days
out of the box, and it stretches
342
00:10:04,419 --> 00:10:07,059
to ninety days if Microsoft
Sentinel is enabled on the
343
00:10:07,059 --> 00:10:10,460
workspace after that window
closes, the data doesn't vanish.
344
00:10:10,460 --> 00:10:11,980
It moves into long term storage
345
00:10:11,980 --> 00:10:13,139
where it sits cheaply, and you
346
00:10:13,139 --> 00:10:14,490
pull it back only when you need
347
00:10:14,490 --> 00:10:14,929
it.
348
00:10:14,929 --> 00:10:17,809
Keeping data queryable costs
about ten cents per gigabyte per
349
00:10:17,809 --> 00:10:20,350
month, and long term storage
runs about two cents per
350
00:10:20,350 --> 00:10:21,409
gigabyte per month.
351
00:10:21,409 --> 00:10:23,970
You can hold onto data for up to
twelve years, which is the kind
352
00:10:23,970 --> 00:10:26,049
of number that matters if you
answer to an auditor.
353
00:10:26,049 --> 00:10:27,370
Getting old data back isn't
354
00:10:27,370 --> 00:10:28,529
instant, and that's the trade
355
00:10:28,529 --> 00:10:29,289
you're making.
356
00:10:29,289 --> 00:10:30,610
You run a search job that
357
00:10:30,610 --> 00:10:31,889
reaches into the archive and
358
00:10:31,889 --> 00:10:33,269
pulls the records you asked for
359
00:10:33,269 --> 00:10:34,850
into a fresh table where they
360
00:10:34,850 --> 00:10:36,210
can be queried like anything
361
00:10:36,210 --> 00:10:37,490
else.
362
00:10:37,490 --> 00:10:38,210
The Microsoft three.
363
00:10:38,210 --> 00:10:39,690
Sixty five clock is different.
364
00:10:39,690 --> 00:10:40,370
It's unified.
365
00:10:40,370 --> 00:10:41,950
Audit log keeps records for one
366
00:10:41,950 --> 00:10:43,129
hundred and eighty days by
367
00:10:43,129 --> 00:10:43,529
default.
368
00:10:43,529 --> 00:10:44,690
That used to be ninety days and
369
00:10:44,690 --> 00:10:46,029
it changed in October of twenty
370
00:10:46,029 --> 00:10:47,330
twenty three.
371
00:10:47,330 --> 00:10:49,789
Defenders advanced hunting,
meanwhile, only reaches back
372
00:10:49,789 --> 00:10:52,570
about thirty days of raw
security data, so you can have
373
00:10:52,570 --> 00:10:55,289
three different clocks running
at once or ticking for records
374
00:10:55,289 --> 00:10:56,570
that feel like the same thing.
375
00:10:56,570 --> 00:10:59,690
The practical move is to decide
table by table how long each
376
00:10:59,690 --> 00:11:01,250
kind of record needs to survive.
377
00:11:01,250 --> 00:11:04,289
Compliance data might need
years, while a chatty debug log
378
00:11:04,289 --> 00:11:05,529
might only need a week.
379
00:11:05,529 --> 00:11:07,929
The honest limit is that a
default isn't forever, and
380
00:11:07,929 --> 00:11:10,490
finding something later only
works if you set the retention
381
00:11:10,490 --> 00:11:11,529
before you need it.
382
00:11:11,529 --> 00:11:14,090
By the time you're hunting for a
record from eight months ago.
383
00:11:14,090 --> 00:11:16,919
The decision that would have
kept it is long gone.
384
00:11:16,919 --> 00:11:19,559
How it fits into Microsoft three
hundred sixty five and Azure.
385
00:11:19,559 --> 00:11:21,120
Zoom out for a second, because
386
00:11:21,120 --> 00:11:22,200
the pieces fit together more
387
00:11:22,200 --> 00:11:23,399
neatly than they first appear,
388
00:11:23,399 --> 00:11:24,639
as your monitor is the umbrella
389
00:11:24,639 --> 00:11:25,679
that watches everything running
390
00:11:25,679 --> 00:11:27,399
in Azure, and it splits that job
391
00:11:27,399 --> 00:11:27,919
in two.
392
00:11:27,919 --> 00:11:28,879
One half is metrics.
393
00:11:28,879 --> 00:11:29,840
The fast numbers.
394
00:11:29,840 --> 00:11:32,600
The other half is logs, which is
log analytics.
395
00:11:32,600 --> 00:11:33,480
So when someone says log
396
00:11:33,480 --> 00:11:35,240
analytics, they're really naming
397
00:11:35,240 --> 00:11:36,360
the log half of a bigger
398
00:11:36,360 --> 00:11:37,320
monitoring system.
399
00:11:37,320 --> 00:11:39,600
The connection worth remembering
is Microsoft Sentinel.
400
00:11:39,600 --> 00:11:41,000
It looks like a separate product
401
00:11:41,000 --> 00:11:42,200
with its own name and its own
402
00:11:42,200 --> 00:11:43,840
screens, but its workspace is
403
00:11:43,840 --> 00:11:44,200
the same.
404
00:11:44,200 --> 00:11:47,639
Log analytics workspace just
switched on for security.
405
00:11:47,639 --> 00:11:49,279
The same place that holds your
application.
406
00:11:49,279 --> 00:11:52,759
Logs can also hold your security
signals, and analysts query both
407
00:11:52,759 --> 00:11:53,960
with the same language.
408
00:11:53,960 --> 00:11:55,039
Microsoft three hundred sixty
409
00:11:55,039 --> 00:11:56,159
five has its own version of
410
00:11:56,159 --> 00:11:57,519
this, called the Unified Audit
411
00:11:57,519 --> 00:11:57,840
Log.
412
00:11:57,840 --> 00:11:58,960
It's where your teams,
413
00:11:58,960 --> 00:12:00,720
SharePoint and sign in activity
414
00:12:00,720 --> 00:12:01,399
lives.
415
00:12:01,399 --> 00:12:02,600
When a team needs to keep that
416
00:12:02,600 --> 00:12:03,879
data longer than the default
417
00:12:03,879 --> 00:12:05,600
window, it can be piped into a
418
00:12:05,600 --> 00:12:06,960
security tool through something
419
00:12:06,960 --> 00:12:08,039
called the office three hundred
420
00:12:08,039 --> 00:12:09,299
sixty five Management Activity
421
00:12:09,299 --> 00:12:10,080
API.
422
00:12:10,080 --> 00:12:12,039
There's a trade off hiding in
all of this.
423
00:12:12,039 --> 00:12:13,279
Putting security data in the
424
00:12:13,279 --> 00:12:14,279
same workspace as everything
425
00:12:14,279 --> 00:12:15,559
else can save money because a
426
00:12:15,559 --> 00:12:17,029
bigger volume can qualify for a
427
00:12:17,029 --> 00:12:17,710
discount.
428
00:12:17,710 --> 00:12:20,070
That discount is called a
commitment tier, and it works
429
00:12:20,070 --> 00:12:21,149
like a bulk deal.
430
00:12:21,149 --> 00:12:23,429
Commit to a daily amount
starting at one hundred
431
00:12:23,429 --> 00:12:26,789
gigabytes, and Microsoft will
knock up to about thirty percent
432
00:12:26,789 --> 00:12:28,389
off the per gigabyte price.
433
00:12:28,389 --> 00:12:30,149
Sentinel enabled data is built
434
00:12:30,149 --> 00:12:31,149
differently, though, which is
435
00:12:31,149 --> 00:12:32,110
why a lot of teams end up
436
00:12:32,110 --> 00:12:33,529
splitting security and
437
00:12:33,529 --> 00:12:34,590
operations into separate
438
00:12:34,590 --> 00:12:35,509
workspaces.
439
00:12:35,509 --> 00:12:37,590
The reason all of this matters
is that the concept you learned
440
00:12:37,590 --> 00:12:40,549
today isn't confined to one
product, the same workspace, the
441
00:12:40,549 --> 00:12:43,750
same tables, and the same query
language keep showing up across
442
00:12:43,750 --> 00:12:46,389
Microsoft's tools, and learning
them once pays off everywhere.
443
00:12:46,389 --> 00:12:49,149
So the next time a Microsoft
tool asks you to point something
444
00:12:49,149 --> 00:12:51,830
at a workspace, you'll know
exactly what it's asking for.
445
00:12:51,830 --> 00:12:53,350
You're not learning four
separate products.
446
00:12:53,350 --> 00:12:54,509
You're learning one idea that
447
00:12:54,509 --> 00:12:56,470
keeps reappearing, the honest
448
00:12:56,470 --> 00:12:56,870
limits.
449
00:12:56,870 --> 00:13:00,029
And one question for you before
we finish the limits deserve
450
00:13:00,029 --> 00:13:02,629
their due, because understanding
a tool's limits is what keeps
451
00:13:02,629 --> 00:13:03,710
you out of trouble.
452
00:13:03,710 --> 00:13:06,909
Log analytics can only show you
what was collected if a system
453
00:13:06,909 --> 00:13:10,269
never sent its records in, no
query will invent them, and no
454
00:13:10,269 --> 00:13:13,070
amount of cleverness will find
what was never written down.
455
00:13:13,070 --> 00:13:14,230
It's also not a place to hoard
456
00:13:14,230 --> 00:13:15,629
everything for free, because
457
00:13:15,629 --> 00:13:17,129
both volume and time carry a
458
00:13:17,129 --> 00:13:17,669
price.
459
00:13:17,669 --> 00:13:18,669
It can even show you the same
460
00:13:18,669 --> 00:13:20,029
event twice because the pipeline
461
00:13:20,029 --> 00:13:21,070
retrieves to make sure data
462
00:13:21,070 --> 00:13:22,149
arrives safely.
463
00:13:22,149 --> 00:13:24,809
A duplicate is usually the
system being careful rather than
464
00:13:24,809 --> 00:13:26,110
your query being wrong.
465
00:13:26,110 --> 00:13:27,909
Two more limits are worth
holding on to.
466
00:13:27,909 --> 00:13:29,190
Hunting through every table at
467
00:13:29,190 --> 00:13:30,409
once is slow, which is why
468
00:13:30,409 --> 00:13:31,629
knowing where to start is a real
469
00:13:31,629 --> 00:13:32,110
skill.
470
00:13:32,110 --> 00:13:33,830
The Daily Cap, meanwhile, is a
471
00:13:33,830 --> 00:13:35,269
hard stop and not a gentle
472
00:13:35,269 --> 00:13:35,870
break.
473
00:13:35,870 --> 00:13:38,590
Hit it and collection stops for
the remainder of the day, which
474
00:13:38,590 --> 00:13:41,549
means an overnight problem could
go completely unseen.
475
00:13:41,549 --> 00:13:43,230
A good first step is to start
small.
476
00:13:43,230 --> 00:13:44,830
Pick one system that matters,
477
00:13:44,830 --> 00:13:46,509
turn on collection for it, and
478
00:13:46,509 --> 00:13:47,669
watch what actually shows up
479
00:13:47,669 --> 00:13:48,909
before you open the floodgates
480
00:13:48,909 --> 00:13:49,789
everywhere else.
481
00:13:49,789 --> 00:13:51,149
The short version is simple.
482
00:13:51,149 --> 00:13:52,990
A log is a record of something
that happened.
483
00:13:52,990 --> 00:13:55,610
The workspace is where those
records are kept and tables sort
484
00:13:55,610 --> 00:13:56,549
them by type.
485
00:13:56,549 --> 00:13:59,230
Kql is how you ask questions,
and you don't need to write any
486
00:13:59,230 --> 00:14:00,269
code to use it.
487
00:14:00,269 --> 00:14:03,049
The two decisions that actually
shape your bill and your safety
488
00:14:03,049 --> 00:14:05,669
net are what you choose to
collect and how long you choose
489
00:14:05,669 --> 00:14:06,389
to keep it.
490
00:14:06,389 --> 00:14:08,750
That leaves one question worth
sitting with somewhere in your
491
00:14:08,750 --> 00:14:11,190
world, a system is already
recording everything it does.
492
00:14:11,190 --> 00:14:12,509
So who's actually reading it?
493
00:14:12,509 --> 00:14:13,830
If this made the topic feel less
494
00:14:13,830 --> 00:14:16,389
intimidating, subscribe to M365
495
00:14:16,389 --> 00:14:17,909
FM to master the tools and the
496
00:14:17,909 --> 00:14:19,309
mindset shaping the future of
497
00:14:19,309 --> 00:14:19,950
work.
