1
00:00:06,547 -->
00:00:08,527
Krishna Gade: Welcome and
thank you everyone for joining
2
00:00:08,527 -->
00:00:10,477
us today's AI Explained.
3
00:00:11,307 -->
00:00:17,087
Today's topic is AI Security and
Observability for Agentic Workflows.
4
00:00:17,467 -->
00:00:21,907
Everyone touts that this year is
going to be the year of AI agents.
5
00:00:23,387 -->
00:00:26,957
Let's see how we need
to address these issues.
6
00:00:27,677 -->
00:00:29,377
I am your host today.
7
00:00:29,607 -->
00:00:30,467
I'm Krishna Gade.
8
00:00:30,467 -->
00:00:32,727
I'm one of the Founders
and CEO of Fiddler AI.
9
00:00:34,322 -->
00:00:37,412
Again, please put your questions
in your Q&A box at any time
10
00:00:37,412 -->
00:00:38,552
during the fireside chat.
11
00:00:39,102 -->
00:00:41,812
Today's session will also be
recorded and sent to all the
12
00:00:41,812 -->
00:00:42,912
attendees after the session.
13
00:00:44,062 -->
00:00:44,372
Okay.
14
00:00:44,402 -->
00:00:50,202
So with, without further ado, um, I
want to welcome Karthik Bharti, um,
15
00:00:50,342 -->
00:00:54,942
General Manager for AI Ops and Governance
for Amazon SageMaker AI at AWS.
16
00:00:57,752 -->
00:01:00,962
Karthik, um, if you could
turn on your camera, um,
17
00:01:02,222 -->
00:01:02,712
Karthik Bharathy: Hey Krishna
18
00:01:02,732 -->
00:01:04,382
Krishna Gade: to a thank you.
19
00:01:04,392 -->
00:01:05,352
Welcome to AI Explain.
20
00:01:05,912 -->
00:01:07,522
So it's just a brief bio of Karthik.
21
00:01:07,522 -->
00:01:10,322
Karthik is a leader with over
20 years of experience driving
22
00:01:10,322 -->
00:01:11,712
innovation in AI and ML.
23
00:01:11,712 -->
00:01:16,672
Um, as a General Manager for AI Ops and
Governance for SageMaker AI, Karthik leads
24
00:01:16,672 -->
00:01:21,672
the development of cutting edge generative
AI capabilities in Amazon SageMaker AI.
25
00:01:23,032 -->
00:01:25,542
Karthik, uh, you know, thank
you so much for joining us.
26
00:01:25,602 -->
00:01:28,432
Uh, maybe let's start you
know, with your background.
27
00:01:28,472 -->
00:01:33,462
You know, how has your role in AI
Ops and Governance at AWS shaped your
28
00:01:33,462 -->
00:01:38,047
perspective on you know, monitoring and
securing AI workflows in the enterprises.
29
00:01:39,367 -->
00:01:41,267
Karthik Bharathy: Yeah, that's
a great question, Krishna.
30
00:01:41,327 -->
00:01:48,827
And, um, if I think about how AI Ops and
governance has evolved over the years,
31
00:01:49,227 -->
00:01:56,887
um, in fact, a lot of the changes has
been in tandem with innovations that we've
32
00:01:56,887 -->
00:02:02,067
seen in AI ML over the last few years,
you know, starting with traditional ML
33
00:02:02,067 -->
00:02:08,762
systems to more recently GenAI and Uh,
agentic workflows as as you aptly put,
34
00:02:09,232 -->
00:02:14,332
um, and throughout these years from what
I've seen, um, there are three things
35
00:02:14,332 -->
00:02:21,812
that that stand out really one is,
um, security and governance are built
36
00:02:21,892 -->
00:02:24,032
into ML workflows from the ground up.
37
00:02:24,542 -->
00:02:26,972
It's, it's not an afterthought anymore.
38
00:02:27,452 -->
00:02:32,512
Um, what essentially that means is,
uh, enterprises are thinking about,
39
00:02:32,792 -->
00:02:38,802
uh, robust data governance techniques,
access controls, and, and how they, uh,
40
00:02:38,802 -->
00:02:41,022
incorporate audit trails from day one.
41
00:02:42,092 -->
00:02:46,412
Um, and, and effective security
isn't about just, uh, you
42
00:02:46,412 -->
00:02:47,762
know, protecting your models.
43
00:02:47,762 -->
00:02:52,952
It's about creating a comprehensive system
that includes, uh, uh, looking at your,
44
00:02:53,002 -->
00:02:55,432
uh, monitoring in an automated manner.
45
00:02:55,797 -->
00:02:59,397
Uh, doing version control and,
and also having audit trails.
46
00:03:00,672 -->
00:03:08,092
The, um, second one I'll call out is, uh,
the need for end-to-end observability,
47
00:03:08,642 -->
00:03:14,052
um, and this is across both the data
and the ML workflows, um, right from,
48
00:03:14,052 -->
00:03:18,212
you know, how data is ingested, how you
can have lineage starting from all the
49
00:03:18,212 -->
00:03:24,172
way to data to ML, um, and all the way
to, uh, observability during, um, model
50
00:03:24,202 -->
00:03:26,352
deployment to look for drift and so on.
51
00:03:27,147 -->
00:03:32,347
Um, and, and finally, I would call out
the third thing is, uh, while all these
52
00:03:32,357 -->
00:03:37,927
sophisticated tooling is in place, um,
you want to have the necessary, um,
53
00:03:38,007 -->
00:03:43,877
human element, um, to sort of oversee
the process, uh, while it's automated,
54
00:03:43,877 -->
00:03:47,997
there are critical junctures where
human, uh, oversight is needed, and that
55
00:03:48,017 -->
00:03:49,777
helps in the decision making process.
56
00:03:51,292 -->
00:03:51,682
Krishna Gade: Awesome.
57
00:03:52,162 -->
00:03:56,152
So, uh, you know, being at, you know, at
the helm of SageMaker, you're probably
58
00:03:56,152 -->
00:04:00,472
seeing the current state of AI in the
enterprise, you know, it's adoption.
59
00:04:00,532 -->
00:04:02,412
Um, you know, how would you describe it?
60
00:04:02,412 -->
00:04:04,902
You know, did you shed some
light for our audience?
61
00:04:05,382 -->
00:04:06,282
Karthik Bharathy: Yeah, yeah.
62
00:04:06,732 -->
00:04:11,432
Um, I, I think if you look at it again,
the last four or five years, right, the,
63
00:04:11,812 -->
00:04:14,032
the enterprise landscape is evolving.
64
00:04:14,482 -->
00:04:14,632
Uh.
65
00:04:15,072 -->
00:04:16,042
pretty rapidly, right?
66
00:04:16,042 -->
00:04:19,982
And you can notice, um, several
distinct patterns, right?
67
00:04:20,452 -->
00:04:24,032
Um, and for what's worth,
like, we are in the third year
68
00:04:24,032 -->
00:04:25,582
of generative AI was, right?
69
00:04:25,732 -->
00:04:29,082
Uh, I think the first year was more
around, hey, there's this cool thing,
70
00:04:29,082 -->
00:04:31,112
like, what can GenAI do, right?
71
00:04:31,522 -->
00:04:37,497
But last year, Based on customer
conversations, we saw that, um, customer
72
00:04:37,497 -->
00:04:43,147
conversation moving from, "Hey, what is
GenAI" to, "Hey, is this right for me?"
73
00:04:43,147 -->
00:04:48,877
And how can I adapt this, um, into
having a real impact for my business?
74
00:04:49,457 -->
00:04:53,267
Um, and this year, um, we
are hearing customers want
75
00:04:53,267 -->
00:04:55,337
to go big with generative AI.
76
00:04:55,337 -->
00:04:59,237
You know, both in terms of going
wide and going deep and, you know,
77
00:04:59,237 -->
00:05:00,797
deploying these systems at scale.
78
00:05:01,687 -->
00:05:06,787
and also leverage the promise
of agentic AI that can create
79
00:05:06,857 -->
00:05:08,667
tangible business value, right?
80
00:05:09,407 -->
00:05:15,457
And as, as we see more of these systems
being developed, like AI systems,
81
00:05:16,767 -->
00:05:22,522
we, there is a need to integrate
these different AI systems so you can
82
00:05:23,162 -->
00:05:29,332
orchestrate more complex workflows and
while at the same time you want to keep in
83
00:05:29,332 -->
00:05:32,062
mind aspects of security and reliability.
84
00:05:32,082 -->
00:05:37,412
So that's definitely one trend and
the other one I would call out is as
85
00:05:37,412 -->
00:05:40,952
you bring in these systems and want
to make complex decision making, you
86
00:05:40,952 -->
00:05:42,982
want to do so in an automated manner.
87
00:05:44,467 -->
00:05:48,317
While keeping in mind, hey, there is
transparency and accountability, right?
88
00:05:48,317 -->
00:05:53,677
So there is increasing, uh, increasingly
customers are looking for ways
89
00:05:54,077 -->
00:05:58,127
to have human oversight and they
want to scale their AI operations.
90
00:05:58,627 -->
00:05:59,037
That's right.
91
00:05:59,317 -->
00:06:00,467
Yeah, especially in the.
92
00:06:01,017 -->
00:06:05,647
Regulated industry, which we play in,
uh, there is some, um, cautious approach
93
00:06:05,677 -->
00:06:09,297
behind, you know, you know, with respect
to the usage of generative AI or AI
94
00:06:09,297 -->
00:06:10,837
agents, like the whole human in the loop.
95
00:06:10,837 -->
00:06:14,767
Krishna Gade: Um, so I guess that's
that begs the question, right?
96
00:06:14,767 -->
00:06:17,547
What potential are you seeing
for these agentic AI systems?
97
00:06:17,547 -->
00:06:20,277
You know, how are they going to
transform the business operations?
98
00:06:20,687 -->
00:06:23,407
Any real life examples
that would be amazing.
99
00:06:23,417 -->
00:06:26,777
Karthik Bharathy: Yeah, yeah, I
think there are quite a few, right?
100
00:06:26,787 -->
00:06:31,527
And, um, let me first break it down into
sort of the different patterns we see,
101
00:06:31,957 -->
00:06:37,237
um, based on the customer conversations
in AWS, and then sort of look at, um,
102
00:06:37,277 -->
00:06:39,117
examples for each of those, right?
103
00:06:39,637 -->
00:06:46,327
Um, so with agentic AI, um, And the
business value that provides it fall
104
00:06:46,457 -->
00:06:49,427
largely in three different categories.
105
00:06:49,987 -->
00:06:56,327
Um, the first one, um, would be
using agentic AI to accelerate,
106
00:06:56,437 -->
00:06:58,577
um, workplace productivity, right?
107
00:06:58,897 -->
00:07:04,777
So think of these as, um, day to
day repetitive tasks that employees
108
00:07:04,827 -->
00:07:09,437
are doing, and they want to automate
this and, and gain the advantage,
109
00:07:09,507 -->
00:07:11,647
uh, of using such an agentic system.
110
00:07:12,017 -->
00:07:12,247
Right.
111
00:07:12,667 -->
00:07:15,327
A good example is NFL media.
112
00:07:16,007 -->
00:07:21,697
They use business agents today to
help their producers and editors to
113
00:07:21,757 -->
00:07:23,447
accelerate their content production.
114
00:07:24,317 -->
00:07:29,867
They have a research tool that
allows them to gather insights from
115
00:07:29,867 -->
00:07:33,187
video footage from a specific place.
116
00:07:33,762 -->
00:07:37,922
And, um, what essentially that
provides is, uh, when, when you're
117
00:07:38,362 -->
00:07:42,592
onboarding a new hire, it reduces
the training time, um, by up to 67%.
118
00:07:43,262 -->
00:07:49,872
And, um, um, and when employees, uh, their
employees ask questions, um, about what,
119
00:07:49,912 -->
00:07:54,632
what's going on, that can be surfaced
in less than 10 minutes, uh, of what
120
00:07:54,632 -->
00:07:56,862
used to take, um, close to 24 hours.
121
00:07:57,242 -->
00:07:59,392
So that's one such example.
122
00:07:59,452 -->
00:08:02,642
And, uh, more closer to
the to the software world.
123
00:08:02,932 -->
00:08:05,542
We're all familiar with coding assistance.
124
00:08:06,072 -->
00:08:08,772
And many of you may have
already used coding assistance
125
00:08:08,772 -->
00:08:09,902
in one shape or the other.
126
00:08:10,262 -->
00:08:16,802
Um, and and largely, well, they help with,
um, building better code or providing.
127
00:08:17,167 -->
00:08:21,287
Um, documentation or explaining, um,
existing code, it's, it's not just
128
00:08:21,287 -->
00:08:25,687
about the code itself, but it's,
it's more about automating the entire
129
00:08:25,847 -->
00:08:30,347
software development lifecycle, um,
including, you know, upgrading software.
130
00:08:30,682 -->
00:08:33,942
Um, or, um, modernizing
a legacy application for
131
00:08:34,002 -->
00:08:35,332
Krishna Gade: Migrating to new languages.
132
00:08:35,552 -->
00:08:36,132
Karthik Bharathy: Absolutely.
133
00:08:36,342 -->
00:08:36,842
Absolutely.
134
00:08:37,092 -->
00:08:43,632
So case in point, um, within
Amazon, we had, um, these agents for
135
00:08:43,752 -->
00:08:47,482
transforming our code base from an older
version of Java to a newer version.
136
00:08:48,012 -->
00:08:53,437
And, uh, there was savings of,
you know, a mammoth, like 4, 4,500
137
00:08:53,457 -->
00:08:55,387
developer years worth of effort, right?
138
00:08:55,627 -->
00:09:00,967
Roughly translate to, um, you know,
260 million annual CapEx savings.
139
00:09:01,487 -->
00:09:06,107
Um, so that's that's the first trend
I would think in terms of using it
140
00:09:06,117 -->
00:09:08,067
to accelerate workplace productivity.
141
00:09:08,967 -->
00:09:13,557
The second one would be in
transforming business workflows
142
00:09:14,387 -->
00:09:16,387
and uncovering new insights, right?
143
00:09:17,017 -->
00:09:22,547
What I mean by that is, uh, as
enterprises are adopting agents, they
144
00:09:22,547 -->
00:09:27,647
want to streamline their operations
and gain insights on their data, right?
145
00:09:27,987 -->
00:09:31,162
Um, and the example that
comes to mind is, is.
146
00:09:31,582 -->
00:09:32,512
Of cognizant.
147
00:09:33,172 -->
00:09:38,582
They're using business agents to automate
mortgage compliance workflows, and they've
148
00:09:38,582 -->
00:09:44,402
seen improvements of more than 50 percent
that reduces, um, uh, errors and rework.
149
00:09:44,892 -->
00:09:46,492
Um, similarly.
150
00:09:46,842 -->
00:09:48,862
Um, Moody's is another great example.
151
00:09:48,862 -->
00:09:53,152
They've used a multi agentic
system, um, that looks at
152
00:09:53,542 -->
00:09:55,382
generating credit risk reports.
153
00:09:55,752 -->
00:10:02,062
Um, and again, the benefit is, uh,
what used to, uh, take humans about one
154
00:10:02,062 -->
00:10:06,112
week to generate a specific report is
now cut down to just one hour, right?
155
00:10:06,112 -->
00:10:10,312
So that's the magnitude of, um, impact
that, that customers are seeing.
156
00:10:11,242 -->
00:10:15,702
Finally, the third one I would call
out is more in the research area
157
00:10:15,962 -->
00:10:20,092
that's sort of, uh, fueling, you know,
industry transformation and innovation.
158
00:10:20,532 -->
00:10:24,062
Um, a good example
there is from Genentech.
159
00:10:24,482 -->
00:10:28,822
Um, they've deployed a genetic
solution running on AWS, um,
160
00:10:29,072 -->
00:10:32,262
and, and they're improving
their, uh, drug research process.
161
00:10:32,632 -->
00:10:38,157
So what they've done is, uh, Their
solution roughly automates, um, you
162
00:10:38,157 -->
00:10:42,577
know, huge, about five years worth
of research, right, across different
163
00:10:42,587 -->
00:10:46,687
therapeutic areas, uh, right, and then
it, what it does, it helps them speed
164
00:10:46,697 -->
00:10:52,007
up the, uh, drug target identification
and also improve their research
165
00:10:52,007 -->
00:10:55,937
efficiency, um, ultimately leading
to, you know, faster drug development.
166
00:10:55,947 -->
00:10:59,957
So, um, net net, we're seeing
systems, agentic systems deployed
167
00:11:00,207 -->
00:11:01,967
broadly in these three categories.
168
00:11:02,562 -->
00:11:03,032
Krishna Gade: Absolutely.
169
00:11:03,032 -->
00:11:06,342
So it's like workplace productivity,
business transformations, and
170
00:11:06,342 -->
00:11:09,382
then, you know, new, you know,
new, new product innovations.
171
00:11:09,472 -->
00:11:13,352
Um, so one thing that you mentioned
and business transformations, you
172
00:11:13,352 -->
00:11:17,002
know, you mentioned a few examples,
especially like generating credit
173
00:11:17,062 -->
00:11:18,892
reports and claims processing, right?
174
00:11:19,192 -->
00:11:21,502
These are, you know,
high stakes AI use cases.
175
00:11:21,762 -->
00:11:25,752
So there is a need for, you
know, security, transparency into
176
00:11:25,752 -->
00:11:27,312
how, you know, AI is working.
177
00:11:27,632 -->
00:11:29,367
You know, what are some of the
challenges that we're trying to address.
178
00:11:29,617 -->
00:11:33,427
Yeah, you think our organizations
are facing when they're implementing
179
00:11:33,427 -->
00:11:35,897
agentic workflows for these, you
know, for these use cases or in
180
00:11:35,897 -->
00:11:37,157
general, other use cases too?
181
00:11:37,607 -->
00:11:41,227
Karthik Bharathy: Yeah, yeah, I think
that's a, that's a great call out, right?
182
00:11:41,227 -->
00:11:45,727
So, um, while you're looking at
these, uh, systems and I think
183
00:11:45,727 -->
00:11:50,352
they're definitely Um, you know,
security and visibility challenges
184
00:11:50,352 -->
00:11:52,512
that organization need to look into.
185
00:11:53,012 -->
00:11:57,072
Um, I'll, I'll call out a few that,
that we have seen and, uh, by no means
186
00:11:57,082 -->
00:12:02,672
this is comprehensive, but it, it sort
of comes down to, um, the stage of
187
00:12:02,672 -->
00:12:04,682
the ML workflow, if you will, right?
188
00:12:04,722 -->
00:12:08,302
And, uh, If you think about it at the
very beginning, when you're trying to
189
00:12:08,302 -->
00:12:14,312
use a specific model, um, it's, it's
quite possible that the data that's
190
00:12:14,312 -->
00:12:19,412
you being used, either to train a model
or, you know, fine tune model, use rag,
191
00:12:19,432 -->
00:12:23,572
whatever technique that you use, um, to
use a data that's, that's not authentic.
192
00:12:23,592 -->
00:12:26,662
And this might just compromise
the performance of the model.
193
00:12:26,672 -->
00:12:27,732
That's definitely.
194
00:12:28,147 -->
00:12:31,647
Um, you know, it's concerning at
the same time, harder to detect, but
195
00:12:32,187 -->
00:12:35,787
until the model is being used and
the interactions that are going on.
196
00:12:35,937 -->
00:12:37,077
So that's one category.
197
00:12:37,717 -->
00:12:43,037
Um, the second would be when, um,
you know, the model is being used,
198
00:12:43,207 -->
00:12:44,557
and again, it depends on the model.
199
00:12:44,987 -->
00:12:50,507
And in the case of a proprietary model,
where the model weights are not exposed.
200
00:12:50,837 -->
00:12:55,197
Um, it might be an actor trying to attempt
to reverse engineer saying, what is the
201
00:12:55,197 -->
00:12:59,687
specific, um, uh, weights that were used
at what level and so on and so forth.
202
00:12:59,917 -->
00:13:03,907
And that essentially, um,
um, you know, exposes.
203
00:13:05,027 -->
00:13:09,467
The how of the model, if you will,
um, and the third one, I would think
204
00:13:09,467 -->
00:13:14,727
is when the model is actually being
used, um, and there can, you know,
205
00:13:14,737 -->
00:13:19,897
actors can attempt to, uh, extract
information, which otherwise the model
206
00:13:19,927 -->
00:13:23,537
would not emit, uh, it might be sensitive
information about the training data,
207
00:13:23,737 -->
00:13:27,807
it might be information that may not
be what the model is intended for, or
208
00:13:27,807 -->
00:13:29,257
the use case that's being deployed.
209
00:13:29,677 -->
00:13:30,407
Um, so.
210
00:13:30,682 -->
00:13:35,402
Um, net net, I think, um, uh,
organizations would need to protect
211
00:13:35,482 -->
00:13:37,532
against these model weights.
212
00:13:37,842 -->
00:13:42,052
Um, you know, how necessary
controls around, um, uh, access,
213
00:13:42,352 -->
00:13:45,012
um, you know, ensure that
there's data privacy and so on.
214
00:13:45,202 -->
00:13:48,142
And more importantly, uh,
ensure that there's this
215
00:13:48,142 -->
00:13:49,722
observability that's end-to-end.
216
00:13:49,742 -->
00:13:54,692
So you can, you are having necessary
checks to see how the model is performing.
217
00:13:54,982 -->
00:13:58,882
Um, and more often than not, you
probably have a sandbox environment
218
00:13:58,892 -->
00:14:02,492
where you're testing it, have tooling,
you know, there are a few tools like
219
00:14:02,502 -->
00:14:04,122
Bedrock Guardrails is an excellent tool.
220
00:14:04,302 -->
00:14:07,912
So you sort of incorporate that, you know,
Fiddler has an observability tool as well.
221
00:14:08,192 -->
00:14:13,122
So these provide sufficient insights into
what is going on in the system, being
222
00:14:13,122 -->
00:14:16,912
it agentic or an automated workflow,
and you sort of take actions based on
223
00:14:17,907 -->
00:14:18,437
Krishna Gade: Absolutely.
224
00:14:18,687 -->
00:14:21,027
So I think you touched upon a
few things like, you know, uh,
225
00:14:21,127 -->
00:14:22,597
adversarial attacks on models.
226
00:14:22,927 -->
00:14:28,177
And now there's this whole, um, field of
AI security and model security coming up.
227
00:14:28,247 -->
00:14:31,777
Um, you know, I remember like, I think
a few, few weeks ago when DeepSeek
228
00:14:31,797 -->
00:14:35,707
launched, everyone was producing
benchmarks about how accurate it is
229
00:14:35,707 -->
00:14:39,307
or how close it is to accuracy when
it comes to closed source models.
230
00:14:39,557 -->
00:14:41,907
But it was pretty vulnerable
for security attacks.
231
00:14:41,907 -->
00:14:45,927
People were able to easily, you
know, make it, uh, leak PII content
232
00:14:45,927 -->
00:14:47,407
and whatnot in RAG workflows.
233
00:14:47,747 -->
00:14:51,697
So how, how, how do you think about,
you know, uh, what, what are some of
234
00:14:51,697 -->
00:14:55,607
the, uh, you know, best practices that
organizations should think about AI
235
00:14:55,607 -->
00:14:59,277
security and what, what, what are some
of the, you know, how do you think about
236
00:14:59,287 -->
00:15:03,207
that versus application level security in
general that has been around for a while?
237
00:15:04,202 -->
00:15:09,822
Karthik Bharathy: Yeah, I think, um,
at the end of the day, you need a
238
00:15:09,882 -->
00:15:12,252
comprehensive security approach, right?
239
00:15:12,512 -->
00:15:15,252
You want to operate at
the different levels.
240
00:15:15,742 -->
00:15:19,692
Um, you mentioned about
model level security, right?
241
00:15:19,712 -->
00:15:21,332
So let's start from there.
242
00:15:21,752 -->
00:15:25,892
Um, so when you're thinking about the
model, um, like I mentioned, you want
243
00:15:25,892 -->
00:15:27,542
to protect the model weights, right?
244
00:15:27,872 -->
00:15:32,042
Um, and in addition to model
weights, you want to protect
245
00:15:32,072 -->
00:15:34,522
the, um, access to the data.
246
00:15:34,922 -->
00:15:38,902
Um, you know, ensuring that the
data is, is, is authentic and so on.
247
00:15:39,412 -->
00:15:43,222
Um, and to address these, you would, you
would use, like, you would encrypt where
248
00:15:43,232 -->
00:15:48,472
the model is being stored, the actual file
um, or, uh, to your point on adversarial
249
00:15:48,762 -->
00:15:52,512
examples, you would, you would have a
test environment where you would exercise
250
00:15:52,522 -->
00:15:56,852
the model, monitor its output for, for
some of these adversarial examples.
251
00:15:57,502 -->
00:16:01,027
And um, at the end of the day, you
need continuous monitoring, right?
252
00:16:01,137 -->
00:16:04,647
Um, just to look at the input and
output patterns, but also look for
253
00:16:04,697 -->
00:16:09,637
drifts, drifts in the model, drifts
in the data, and have necessarily, uh,
254
00:16:09,667 -->
00:16:14,057
necessary, um, uh, alerts, so you can
trigger, like, a retraining, for example.
255
00:16:14,507 -->
00:16:15,707
So that's at the model level.
256
00:16:16,257 -->
00:16:21,537
Um, at, at the application level, I
think, um, there, there are the well
257
00:16:21,537 -->
00:16:25,507
known security practices, like, you know,
you enforce access controls, you have
258
00:16:25,527 -->
00:16:31,132
encryption in place, um, You have logging
off the interaction patterns and so on.
259
00:16:31,572 -->
00:16:34,422
Um, but in addition to that,
tooling is often needed.
260
00:16:34,662 -->
00:16:38,832
Uh, like I mentioned, the Bedrock example,
uh, Bedrock Guardrails example earlier.
261
00:16:39,102 -->
00:16:44,822
You, you want to think about how you
audit certain topics, um, be it at
262
00:16:44,822 -->
00:16:46,352
an input level or the output level.
263
00:16:46,382 -->
00:16:48,742
What is relevant to your use case?
264
00:16:48,772 -->
00:16:50,152
What should not be emitted?
265
00:16:50,682 -->
00:16:54,162
Or if there's certain information
that's being emitted like a
266
00:16:54,182 -->
00:16:57,292
PII data, how do you redact the
information and so on and so forth.
267
00:16:57,682 -->
00:17:03,932
So I think net net, the two layers of
model security and application level
268
00:17:03,942 -->
00:17:06,562
security need to integrate seamlessly.
269
00:17:06,562 -->
00:17:11,062
So in many ways, uh, uh, these
are complementary than treating
270
00:17:11,062 -->
00:17:11,962
it as separate constants.
271
00:17:13,692 -->
00:17:14,012
Krishna Gade: Awesome.
272
00:17:14,422 -->
00:17:14,822
That's great.
273
00:17:15,002 -->
00:17:19,082
So I guess, uh, you know, we talked
about, uh, a little bit about, uh,
274
00:17:19,092 -->
00:17:20,552
some high stakes use cases, right?
275
00:17:20,552 -->
00:17:25,092
So when it comes to, uh, you know,
transparency of AI decisions for these,
276
00:17:25,122 -->
00:17:28,842
you know, for regulators or business
stakeholders, how do you think, you
277
00:17:28,872 -->
00:17:33,932
know, this is going to change when,
you know, agents come about and, um,
278
00:17:33,942 -->
00:17:37,822
you know, and organizations employ
agentic workflows and what happens
279
00:17:37,822 -->
00:17:40,022
to the transparency behind AI?
280
00:17:42,087 -->
00:17:48,597
Karthik Bharathy: Yeah, I think
fundamentally, um, enterprise would
281
00:17:48,607 -->
00:17:52,097
benefit from having a governance model.
282
00:17:53,257 -->
00:17:55,657
That's, that's more federated, right?
283
00:17:55,697 -->
00:18:02,917
Meaning you have standards, policies
in place, that sort of dictate
284
00:18:02,917 -->
00:18:08,127
how these systems need to be
developed across the organization.
285
00:18:08,987 -->
00:18:14,217
But at the same time, you want to
provide enough flexibility, uh, where
286
00:18:14,287 -->
00:18:20,527
each team or business unit can adapt
these standards in a way that they can
287
00:18:20,527 -->
00:18:22,537
implement for their specific use cases.
288
00:18:22,807 -->
00:18:24,497
So that's sort of the trade off.
289
00:18:24,497 -->
00:18:28,077
And it's, it's a good one,
uh, in the sense that you want
290
00:18:28,077 -->
00:18:29,207
to provide the flexibility.
291
00:18:29,802 -->
00:18:33,492
Um, of developing these different
systems across different units.
292
00:18:33,872 -->
00:18:38,442
Um, and there are, again, tools, like,
for example, uh, just purely taking
293
00:18:38,442 -->
00:18:41,992
the example of SageMaker here, you
have SageMaker projects where you can
294
00:18:42,052 -->
00:18:46,902
automate, um, the ML workflow, say,
how should it be standardized, what
295
00:18:46,912 -->
00:18:50,542
pipelines do you need to use, what
models, and what quality, and so on.
296
00:18:51,722 -->
00:18:54,082
Krishna Gade: So the governance
is like both a tools problem as
297
00:18:54,082 -->
00:18:55,402
well as a people's problem, right?
298
00:18:55,402 -->
00:18:58,462
Like, you know, essentially, you
know, many companies do not have,
299
00:18:58,637 -->
00:19:01,787
The governance structures today,
you know, to sort of ensure that
300
00:19:01,807 -->
00:19:05,447
you know, AI is tested, monitored
and secured and securely operated.
301
00:19:05,817 -->
00:19:08,437
You know, what, what are some of the
best practices that you have seen
302
00:19:08,437 -->
00:19:12,197
in terms of, you know, customers
employing AI governance today across,
303
00:19:12,197 -->
00:19:13,497
you know, different business units?
304
00:19:14,947 -->
00:19:19,657
Karthik Bharathy: I think fundamentally,
um, at the highest level of abstraction,
305
00:19:19,657 -->
00:19:23,757
you have, um, you know, business
stakeholders, like the so called risk
306
00:19:23,757 -->
00:19:29,227
officers, if you will, who understand
the domain of what is being developed,
307
00:19:29,277 -->
00:19:35,322
and they would enforce certain standards
on what needs to be Um, at her too.
308
00:19:36,102 -->
00:19:39,692
And it's important that they work
in tandem with the technical team
309
00:19:39,702 -->
00:19:43,522
who are well versed with what's
being done with the model, right?
310
00:19:43,542 -->
00:19:48,752
For example, a model may have
a toxic score of like 0.1.
311
00:19:48,772 -->
00:19:53,102
But what does that mean is from a
use case perspective, whether this
312
00:19:53,102 -->
00:19:57,332
model can be approved and deployed
an organization is very specific
313
00:19:57,332 -->
00:19:58,862
to the domain they're operating.
314
00:19:59,252 -->
00:20:07,112
Um, I think successful organizations have
a good mix of both where, um, you have the
315
00:20:07,112 -->
00:20:12,182
necessary tooling, um, where these, uh,
different levels, for example, toxicity
316
00:20:12,182 -->
00:20:17,492
is being, uh, uh, monitored for, they're
being documented, uh, either through a
317
00:20:17,492 -->
00:20:21,972
model card or you have enough properties,
maybe in a model registry, for example.
318
00:20:22,442 -->
00:20:28,272
And this translates into visibility from
the risk officer who can effectively
319
00:20:28,272 -->
00:20:32,482
say whether this model or the system
is approved for deployment or not.
320
00:20:32,482 -->
00:20:37,062
So the two systems working together, I
think, definitely is a recipe for success.
321
00:20:37,812 -->
00:20:38,092
Krishna Gade: Got it.
322
00:20:38,372 -->
00:20:42,632
So are there any specific metrics that
you recommend that organizations need to
323
00:20:42,962 -->
00:20:47,082
track, like whether it's about security
or, you know, governance of AI, you
324
00:20:47,082 -->
00:20:50,442
know, when they're testing it or, you
know, when deploying to production?
325
00:20:51,417 -->
00:20:57,537
Karthik Bharathy: Yeah, so if you look
at the metrics again at the technical
326
00:20:57,537 -->
00:21:01,827
level, you have a set of metrics
right at the most foundation level.
327
00:21:02,227 -->
00:21:05,467
Um, you know, if you have to
document it, document what the
328
00:21:05,467 -->
00:21:07,247
model is doing as a model card.
329
00:21:07,582 -->
00:21:11,852
You would look at, um, the purpose of
the model, what data it's trained on,
330
00:21:12,182 -->
00:21:15,802
uh, what is the validation rules, what
is the quality of the model, and so on.
331
00:21:16,342 -->
00:21:21,082
Um, and going a little bit beyond
that, um, you may want to document
332
00:21:21,152 -->
00:21:26,452
how the model is, um, uh, emitting
or predicting a response, right?
333
00:21:26,682 -->
00:21:30,842
So, for example, with, with, with, you
may want to look at explainability.
334
00:21:31,337 -->
00:21:34,627
Approaches like you may look at a SHAP
score, for example, or you may look at
335
00:21:34,627 -->
00:21:38,967
a LIME score, for example, and these
may be documented with the model that
336
00:21:38,967 -->
00:21:40,307
those are good metrics to look at.
337
00:21:40,917 -->
00:21:43,997
And again, with GenAI, you can
look at additional metrics around
338
00:21:44,487 -->
00:21:46,297
toxicity, fairness, and so on.
339
00:21:46,357 -->
00:21:48,277
You can test these models.
340
00:21:49,117 -->
00:21:52,457
You can have periodic
evaluations on the level.
341
00:21:52,777 -->
00:21:56,877
of these metrics and test it
against, um, standardized data
342
00:21:56,877 -->
00:21:58,317
sets that are available today.
343
00:21:58,567 -->
00:22:02,927
Or you can use custom data sets that
are very specific to your, um, use case.
344
00:22:03,487 -->
00:22:06,717
Um, and then again, at the business
level, you want to interpret these
345
00:22:06,717 -->
00:22:11,997
as saying, uh, with a combination
of these, uh, objective metrics.
346
00:22:12,497 -->
00:22:16,257
How does the subjective standards
and policies play in and what does
347
00:22:16,317 -->
00:22:18,227
that mean from a risk perspective?
348
00:22:19,657 -->
00:22:22,317
Krishna Gade: So there is always
this tension, uh, within the
349
00:22:22,317 -->
00:22:24,517
organizations to adopt AI faster.
350
00:22:24,967 -->
00:22:26,537
Versus doing it right, right?
351
00:22:26,547 -->
00:22:29,577
So there's this like, you know, how
do you make sure you do it properly
352
00:22:29,577 -->
00:22:30,867
so that you don't get into trouble?
353
00:22:31,357 -->
00:22:35,247
Like what, how should like, you
know, organizations think about
354
00:22:35,267 -->
00:22:36,717
like, you know, this balance?
355
00:22:38,197 -->
00:22:41,327
Karthik Bharathy: Yeah, I think
um, that, that's a key one, right?
356
00:22:41,327 -->
00:22:43,997
I think there's no one easy
answer, if you will, right?
357
00:22:43,997 -->
00:22:48,917
And the, and the key to balance,
uh, robustness in, in having those
358
00:22:48,917 -->
00:22:54,357
security controls with the operational,
uh, efficiency lies in having some,
359
00:22:54,537 -->
00:22:56,187
having the right guardrails, right?
360
00:22:56,457 -->
00:22:59,937
Instead of, uh, creating or looking
at the problem as saying, "Hey, here's
361
00:22:59,937 -->
00:23:04,407
one way to do it," or one set of,
"Hey, this is risky versus non-risky."
362
00:23:04,587 -->
00:23:08,307
You're probably looking at,
uh, a, uh, a set of, um.
363
00:23:09,062 -->
00:23:12,672
A range of values, if you will, right
in terms of how to look at risk.
364
00:23:13,062 -->
00:23:18,012
Um, a good example would be, uh, let's say
you have the model or the system deployed.
365
00:23:18,372 -->
00:23:23,412
Um, and you notice that certain
changes introduce a higher risk.
366
00:23:23,822 -->
00:23:30,412
Um, it's better to trigger additional
approval workflows, um, rather than,
367
00:23:30,822 -->
00:23:34,052
um, you know, just waiting on it and
saying, here's a single way to do it.
368
00:23:34,652 -->
00:23:40,142
In contrast, if the same set of changes
result in a relatively lower risk,
369
00:23:40,637 -->
00:23:44,327
Um, you may want to proceed through
standardized approvals instead of, you
370
00:23:44,327 -->
00:23:46,167
know, requiring additional approvals.
371
00:23:46,497 -->
00:23:50,537
Um, a good example again would be,
let's say, there's a drift in the model,
372
00:23:50,637 -->
00:23:54,777
right, which is fairly common and you
have an observability solution in place.
373
00:23:55,067 -->
00:24:00,337
If the drift is, um, not significant
from the current state of the model,
374
00:24:00,637 -->
00:24:05,687
you may be okay with treating that as
an alert and being in the know how of
375
00:24:05,687 -->
00:24:09,447
what is being happened and you may just
trigger a retraining of the workflow.
376
00:24:09,822 -->
00:24:13,622
But on the other hand, if the drift
is significant and it exceeds what
377
00:24:13,622 -->
00:24:17,892
is the threshold that you've defined,
um, you may trigger additional
378
00:24:17,892 -->
00:24:21,422
approvals or in, in, in some extreme
cases, you might even consider
379
00:24:21,722 -->
00:24:23,922
rolling back to the previous version.
380
00:24:24,082 -->
00:24:28,272
Uh, so those are, uh, different options
that you can consider and, and, and the
381
00:24:28,272 -->
00:24:30,202
key is to maintain that configurable.
382
00:24:30,222 -->
00:24:34,252
So you can trade off between the, the
rigor and the robustness of security
383
00:24:34,252 -->
00:24:35,972
control with the efficiency that it
384
00:24:36,827 -->
00:24:37,127
Krishna Gade: Right.
385
00:24:37,537 -->
00:24:39,487
So when it comes to
evaluation of AI, right?
386
00:24:39,487 -->
00:24:42,747
So in the past for classical machine
learning, you could do things
387
00:24:42,747 -->
00:24:46,417
like ROC curve, AUC scores, you
know, precision recall, and maybe
388
00:24:46,467 -->
00:24:50,417
even do like SHAP, SHAP plots and
understand the feature importance.
389
00:24:50,667 -->
00:24:54,657
But now with in a generative AI and
agentic workflows, evaluating the
390
00:24:54,657 -->
00:24:56,407
performance is not straightforward, right?
391
00:24:56,407 -->
00:24:57,787
You know, there's no ground truth.
392
00:24:58,137 -->
00:25:01,447
So how do you, you know, can you
shed some light on like, you know,
393
00:25:01,457 -->
00:25:05,617
how, uh, customers are going about
this, you know, in, in, in sort
394
00:25:05,617 -->
00:25:08,797
of in the, in, in, in sectors that
you have been exposed to so far and
395
00:25:08,797 -->
00:25:10,137
what are some of the best practices?
396
00:25:11,207 -->
00:25:15,377
Karthik Bharathy: I think the ones
that I've seen are the areas where
397
00:25:15,417 -->
00:25:20,867
customers are exploring are, um,
evaluating the system end-to-end, right?
398
00:25:21,307 -->
00:25:24,247
There's no one unique metric
like going back to the example
399
00:25:24,247 -->
00:25:25,577
that mentioned earlier.
400
00:25:26,072 -->
00:25:33,692
Um, concretely, you can think off having
a pipeline, um, that that triggers, um,
401
00:25:33,842 -->
00:25:36,482
on either manually or on a periodic way.
402
00:25:36,542 -->
00:25:40,242
And that evaluates the model
on certain dimensions, right?
403
00:25:40,612 -->
00:25:43,852
Um, and, and evaluation
is sort of a broad topic.
404
00:25:43,852 -->
00:25:48,382
But, um, if, if there are certain
aspects of the model that you want,
405
00:25:48,382 -->
00:25:49,352
let, let's say, be it fairness.
406
00:25:50,077 -->
00:25:55,727
Or, um, uh, robust, um, toxicity,
for example, like you can look at
407
00:25:55,757 -->
00:25:59,997
evaluating, for example, a model against
a toxigen model and seeing, hey, if
408
00:26:00,017 -->
00:26:02,587
these inputs were send to the model.
409
00:26:02,587 -->
00:26:03,577
What is the output?
410
00:26:03,857 -->
00:26:07,607
And once you know the expected
output and the actual output, you
411
00:26:07,607 -->
00:26:08,967
can actually see the difference.
412
00:26:08,967 -->
00:26:12,277
Okay, the model is
working on expected lines.
413
00:26:12,517 -->
00:26:14,887
Therefore, this is the score
that you want to assign for
414
00:26:14,887 -->
00:26:16,297
that particular category, right?
415
00:26:16,597 -->
00:26:21,957
So developing that comprehensive
pipelining workflow and making sure you
416
00:26:21,967 -->
00:26:26,937
have observability and each of the places
and saying as a system, you do it first
417
00:26:26,937 -->
00:26:29,607
at the model level, and then you do it at
the system level when there are multiple
418
00:26:29,657 -->
00:26:31,547
models interacting with each other.
419
00:26:31,897 -->
00:26:36,397
And then saying, given the behavior
of the system, what what is the sort
420
00:26:36,397 -->
00:26:41,547
of the score that you want to, in some
cases, you know, you can be creative
421
00:26:41,547 -->
00:26:42,967
and creating a composite score.
422
00:26:42,967 -->
00:26:47,872
It purely depends on how how much
weight that you assign to each of
423
00:26:47,892 -->
00:26:51,872
these individuals score to create this
composite score and how you gauge that
424
00:26:51,872 -->
00:26:53,422
composite score with respect to use case.
425
00:26:54,852 -->
00:26:58,092
Krishna Gade: Especially for agentic
workflows when in some cases when they
426
00:26:58,092 -->
00:27:02,182
are automating the decision process
right in the enterprise space, there
427
00:27:02,182 -->
00:27:05,192
is a need to measure like whether
the decisions are optimal or not.
428
00:27:05,672 -->
00:27:08,042
You know, uh, it's a pretty hard problem.
429
00:27:08,092 -->
00:27:10,752
Uh, any, any, any thoughts on that?
430
00:27:10,762 -->
00:27:13,162
Like, you know, for
example, you know, this.
431
00:27:13,417 -->
00:27:15,727
So take, take the example that
you mentioned, like the claims
432
00:27:15,757 -->
00:27:16,847
processing workflow, right?
433
00:27:16,877 -->
00:27:20,547
Like which, which was probably
like much more manual in the past.
434
00:27:20,557 -->
00:27:22,567
Now it's like, you know, automated.
435
00:27:22,827 -->
00:27:26,277
How, how can, how can, you know,
customers measure like, you know, if it's
436
00:27:26,297 -->
00:27:30,167
working properly and if it's actually
working optimally for the business?
437
00:27:30,772 -->
00:27:35,462
Karthik Bharathy: Yeah, while while
you can have, um, you know, objective
438
00:27:35,462 -->
00:27:38,532
metrics at the end of the day,
it's the business use case, right?
439
00:27:38,542 -->
00:27:46,552
And I think, um, it would involve human,
um, in, um, human processes or seeing
440
00:27:46,882 -->
00:27:48,972
the sort of outputs from the system.
441
00:27:49,462 -->
00:27:56,047
Um, and I think that the key is to have
the necessary hooks in place, right?
442
00:27:56,527 -->
00:28:01,447
For example, while on one end you
want to enforce controls on like
443
00:28:01,617 -->
00:28:05,567
what data is being accessed or what
output is being generated or like
444
00:28:05,967 -->
00:28:10,637
what toxicity is the scoring or the
evaluation model being done, you want
445
00:28:10,637 -->
00:28:12,407
to make sure there's human insight.
446
00:28:12,787 -->
00:28:17,357
Um, and every decision, especially in
the early phases of when the system is
447
00:28:17,357 -->
00:28:21,437
deployed, you want to have these human
evaluation on on the system output.
448
00:28:21,887 -->
00:28:26,847
Um, more importantly, you also want to
have some sort of a pause switch, if you
449
00:28:26,847 -->
00:28:32,797
will, to say that if the model deviates
from the known patterns, what is the
450
00:28:32,797 -->
00:28:37,772
way to quickly have the humans come in
and have this pause switch or even a
451
00:28:37,972 -->
00:28:42,582
kill switch for that matter to make sure
that corrective actions can be taken.
452
00:28:43,802 -->
00:28:43,962
Krishna Gade: Yeah.
453
00:28:44,012 -->
00:28:46,852
And so, so I think basically,
you know, this might change from
454
00:28:46,872 -->
00:28:48,132
industry to industry, right?
455
00:28:48,132 -->
00:28:51,942
So, you know, like for example, you
know, what do you want to measure or
456
00:28:51,942 -->
00:28:54,382
what do you want to control around AI?
457
00:28:55,907 -->
00:28:59,257
Can can be different for different
domains, you know, have you seen any,
458
00:28:59,307 -->
00:29:03,487
any sort of, uh, insights, like, for
example, finance versus healthcare
459
00:29:03,587 -->
00:29:06,877
versus like, you know, you know, some
other industries, like what do what they
460
00:29:07,247 -->
00:29:12,417
make care about in terms of, uh, the
measuring and putting security controls.
461
00:29:13,147 -->
00:29:16,897
Karthik Bharathy: Yeah, it's,
it's, um, more than the industry.
462
00:29:17,327 -->
00:29:22,537
I think, um, like you call out, it
also depends on what set of policies
463
00:29:22,537 -->
00:29:24,457
and standards they're hearing to.
464
00:29:25,167 -->
00:29:29,637
And then, yes, it goes by also the
regions in which they are like the EU
465
00:29:29,637 -->
00:29:33,807
AI Act or of ISO 42001, the different
regulations that that come in.
466
00:29:35,227 -->
00:29:41,607
So there's no one size fits all, but
the more effective use cases that
467
00:29:41,607 -->
00:29:45,977
I've seen, or the ones that have been
deployed successfully, factor in both
468
00:29:45,987 -->
00:29:51,312
the subjectiveness of The standards
that require you, uh, to adhere to
469
00:29:51,312 -->
00:29:55,072
certain things like, hey, where the
data are stored, um, and sort of answer
470
00:29:55,112 -->
00:29:58,902
the different questions related to the
standard along with the objectiveness
471
00:29:59,012 -->
00:30:00,742
of the metrics that's being tracked.
472
00:30:01,092 -->
00:30:06,412
Um, so the more successful use cases,
um, uh, and they do vary across like
473
00:30:06,412 -->
00:30:08,372
the healthcare and financial services.
474
00:30:08,722 -->
00:30:12,592
Um, and, you know, even in the case
of retail, there are examples where
475
00:30:12,802 -->
00:30:14,652
a combination of the two is needed.
476
00:30:16,692 -->
00:30:19,712
Krishna Gade: So what are some of
the warning signs that, you know, one
477
00:30:19,712 -->
00:30:25,132
can, one can like actually see that
an agentic system may have security
478
00:30:25,182 -->
00:30:26,702
vulnerabilities or monitoring gaps?
479
00:30:26,702 -->
00:30:29,082
Like how can an organization
be aware of that?
480
00:30:30,287 -->
00:30:36,687
Karthik Bharathy: Yeah, I think
the first one to look for, um, um,
481
00:30:37,027 -->
00:30:38,647
is, is the data quality, right?
482
00:30:38,697 -->
00:30:45,327
You want to make sure, um, you know,
the, the model, um, data input and
483
00:30:45,337 -->
00:30:48,827
the model, what, what it's trained
on is, uh, secure and robust.
484
00:30:48,837 -->
00:30:51,137
That's, that's, uh, that's important.
485
00:30:51,592 -->
00:30:57,462
And once you have those in place, um,
I think you want to have an effective
486
00:30:57,542 -->
00:31:02,792
testing strategy, um, to ensure that you,
you defend against adversarial attacks.
487
00:31:03,282 -->
00:31:08,012
Um, so there's even, even if, for example,
there's a manipulation in the input, you
488
00:31:08,012 -->
00:31:11,992
want to make sure that the security of
the model and the system is taken care of.
489
00:31:12,612 -->
00:31:19,212
Um, and then the one that we talked
about about on model drift and looking
490
00:31:19,212 -->
00:31:21,322
for any degradations and performance.
491
00:31:21,332 -->
00:31:25,672
So continuously monitoring and looking
for those key parameters is important.
492
00:31:26,262 -->
00:31:28,002
Um, and from, uh.
493
00:31:28,872 -->
00:31:33,142
The system application standpoint,
um, you want to ensure that,
494
00:31:33,282 -->
00:31:35,112
uh, the API endpoints are.
495
00:31:35,507 -->
00:31:39,047
Are, uh, secured, um, again,
data transmission is secure
496
00:31:39,127 -->
00:31:40,087
and so on and so forth.
497
00:31:40,367 -->
00:31:44,037
And you have robust, um, controls
for both the authentication
498
00:31:44,067 -->
00:31:45,307
and authorization piece.
499
00:31:45,597 -->
00:31:50,637
Um, at the end of the day, I would think
of it as, uh, as an employee, right?
500
00:31:50,657 -->
00:31:55,037
An employee badges in and the
employee in many organization
501
00:31:55,057 -->
00:31:56,307
badges out of the building as well.
502
00:31:56,617 -->
00:31:58,447
And the next time you come
in, you badge in again.
503
00:31:58,457 -->
00:32:00,977
So you sort of re
authenticate and make sure.
504
00:32:01,202 -->
00:32:05,552
That, you know, you are aware of,
like, this person who is authorized
505
00:32:05,562 -->
00:32:06,832
to do this particular job.
506
00:32:07,062 -->
00:32:08,942
It's very similar to an agentic system.
507
00:32:09,392 -->
00:32:15,672
Um, so you want to ensure that, um,
another one that comes to mind is, uh, um,
508
00:32:15,702 -->
00:32:17,682
the principle of least privilege, right?
509
00:32:17,682 -->
00:32:21,132
You, you provide access only
when that's needed, right?
510
00:32:21,132 -->
00:32:25,152
And very similar, again, to the
employee example that I called out.
511
00:32:25,462 -->
00:32:29,317
Um, an employee may not have access to all
data, but when it's needed, You sort of
512
00:32:29,427 -->
00:32:33,007
ensure that, hey, the person who really
needs that information has access to it.
513
00:32:33,437 -->
00:32:36,777
So those would be some signs to look
for when you're designing situations.
514
00:32:37,242 -->
00:32:37,582
Krishna Gade: Got it.
515
00:32:37,942 -->
00:32:39,232
So there's an audience question here.
516
00:32:39,232 -->
00:32:42,742
So any specific frameworks, tools
you're using for agentic workflows
517
00:32:42,752 -->
00:32:44,412
to evaluate robustness and accuracy?
518
00:32:44,712 -->
00:32:46,832
This is probably a good time
to talk about our partnership
519
00:32:46,842 -->
00:32:48,192
between SageMaker and Fiddler.
520
00:32:48,542 -->
00:32:50,102
You know, can you share
your thoughts on that?
521
00:32:51,072 -->
00:32:52,382
Karthik Bharathy: Yeah, no, absolutely.
522
00:32:52,382 -->
00:32:56,472
Like, we're thrilled to
be working with Fiddler.
523
00:32:57,622 -->
00:33:01,302
And at the outset, you know, partnership
is something that's absolutely critical
524
00:33:02,292 -->
00:33:05,372
for AWS and SageMaker specifically.
525
00:33:05,662 -->
00:33:11,512
Um, as we look at extending the
core AI/ML capabilities, um, and
526
00:33:11,522 -->
00:33:14,732
to provide specialized solutions
for different industry needs.
527
00:33:14,962 -->
00:33:19,182
I think, uh, partnering with a company
like Fiddler is absolutely paramount.
528
00:33:19,732 -->
00:33:22,542
Um, and what the intent
is really simple, right?
529
00:33:22,572 -->
00:33:25,292
We want to make sure the best
of class solutions are available
530
00:33:25,342 -->
00:33:26,302
to our customers, right?
531
00:33:26,582 -->
00:33:31,177
So with Fiddler, we've combined the power
of SageMaker AI, where you can train
532
00:33:31,177 -->
00:33:36,017
and deploy your models with Fiddler AI,
Fiddler AI, which brings in observability
533
00:33:36,017 -->
00:33:37,977
to monitor and improve the ML models.
534
00:33:37,987 -->
00:33:44,787
So, so net net customers have a one click
way to do observability with SageMaker AI.
535
00:33:45,187 -->
00:33:49,317
Um, and this experience is available
in SageMaker Unified Studio.
536
00:33:49,607 -->
00:33:54,392
It provides a seamless experience
and, and I'm pretty excited about
537
00:33:54,392 -->
00:33:57,522
how customers can use these two
capabilities in a seamless manner.
538
00:33:58,602 -->
00:33:59,112
Krishna Gade: Absolutely.
539
00:33:59,132 -->
00:34:00,612
Yeah, we share the same excitement.
540
00:34:00,612 -->
00:34:04,972
And for those of you who are on AWS
SageMaker today on this call, feel
541
00:34:04,972 -->
00:34:08,932
free to, you know, use the one click
experience and one click integration
542
00:34:08,932 -->
00:34:13,262
that we built together with working
with AWS with Fiddler for monitoring
543
00:34:13,262 -->
00:34:15,062
and evaluation of your AI models.
544
00:34:15,762 -->
00:34:18,902
So let's actually, you know, maybe take
a few more audience questions here.
545
00:34:18,922 -->
00:34:20,992
There are some questions
around different industries.
546
00:34:21,287 -->
00:34:23,387
There's a question actually
about code migration.
547
00:34:23,387 -->
00:34:25,677
We touched, touched upon
it early in the call.
548
00:34:26,167 -->
00:34:30,157
What are some of the best practices
for verifying large code changes or
549
00:34:30,187 -->
00:34:31,977
migrating from one language to another?
550
00:34:32,807 -->
00:34:35,077
This is, I think, using
AI based code migration.
551
00:34:35,837 -->
00:34:36,117
Karthik Bharathy: Yeah.
552
00:34:37,297 -->
00:34:41,457
Um, I think the specifics actually
depend on the language itself, right?
553
00:34:41,457 -->
00:34:46,627
Depending on whether you're looking at a
more modern language or like a traditional
554
00:34:46,637 -->
00:34:48,427
language, like COBOL, for example, right?
555
00:34:48,767 -->
00:34:51,347
Um, so I think given that
556
00:34:53,562 -->
00:34:59,812
While the migration is being assisted,
you want to look for, um, patterns of
557
00:34:59,862 -->
00:35:02,352
like translation between the two systems.
558
00:35:02,362 -->
00:35:06,842
Sometimes the logic may be inherently
complex, so there's human in the loop,
559
00:35:07,182 -->
00:35:09,382
there's assisted AI that comes into play.
560
00:35:09,742 -->
00:35:13,232
Um, you should definitely try out some
of the tooling that's already available.
561
00:35:13,657 -->
00:35:19,067
Um, with Amazon queue, um, we recently
launched a train went the ability to
562
00:35:19,067 -->
00:35:21,467
look at the system workflow end-to-end.
563
00:35:21,767 -->
00:35:26,907
Um, and there are obviously pieces
around security that that's very
564
00:35:26,907 -->
00:35:28,917
specific to the organization as well.
565
00:35:29,247 -->
00:35:33,407
Um, so in terms of best practices,
I believe there's also a,
566
00:35:33,827 -->
00:35:35,667
um, detailed documentation.
567
00:35:35,727 -->
00:35:38,397
Um, we can find a way
to share that with you.
568
00:35:38,827 -->
00:35:42,912
Um, on, on what does need to
be looked at as, as you do
569
00:35:45,342 -->
00:35:48,472
Krishna Gade: And so, uh, there's another
question on like specific industry.
570
00:35:48,752 -->
00:35:52,922
Could you shed some lights on, on business
use cases within financial services
571
00:35:52,922 -->
00:35:56,682
or FinOps and plus, uh, you know, uh,
where AI observability makes sense.
572
00:35:57,522 -->
00:35:59,622
Karthik Bharathy: Yeah, I
think there are quite a few.
573
00:36:00,642 -->
00:36:05,852
Um, you know, the top two or three
that come to mind are the automated
574
00:36:05,852 -->
00:36:07,722
financial reporting that I called out.
575
00:36:08,282 -->
00:36:13,102
Um, you know, I mentioned about, uh,
uh, Moody's use case about generating
576
00:36:13,102 -->
00:36:16,882
credit reports or cognizance use case
about mortgage compliance workflows.
577
00:36:18,072 -->
00:36:23,272
Um, demand forecasting, uh, is
another one, um, that's, it's sort
578
00:36:23,272 -->
00:36:26,812
of relevant in the context of,
uh, financial services as well.
579
00:36:27,312 -->
00:36:31,852
Um, and more generally, I would say
incident management that applies
580
00:36:31,912 -->
00:36:35,972
across different industries is, is
also relevant as you look at more data.
581
00:36:36,422 -->
00:36:39,142
And you want to uncover
insights from that data.
582
00:36:40,732 -->
00:36:43,142
Krishna Gade: And then another question
from the insurance industry, you know,
583
00:36:43,152 -->
00:36:46,872
beyond models, what recommendation
of metrics would you have, for
584
00:36:46,872 -->
00:36:48,412
instance, for claims processing?
585
00:36:48,682 -->
00:36:52,002
Can you explain specific measures you
suggested to clients and share what
586
00:36:52,002 -->
00:36:55,432
your assessment is on the quality
improvements of business outcomes.
587
00:36:56,362 -->
00:36:59,162
Karthik Bharathy: To be honest, I'm
not from the insurance industry,
588
00:36:59,162 -->
00:37:01,372
so I'm commenting on that.
589
00:37:01,632 -->
00:37:05,302
Um, that said, I'm happy to take
that question back and come back
590
00:37:05,332 -->
00:37:07,272
if we have the contact information.
591
00:37:07,322 -->
00:37:09,882
I don't, I don't represent the
insurance industry, so I just don't
592
00:37:09,882 -->
00:37:11,082
want to give out the wrong answer.
593
00:37:11,462 -->
00:37:16,582
Krishna Gade: So Priya, feel free to reach
out to Uh, Karthik on further information.
594
00:37:17,092 -->
00:37:17,452
Awesome.
595
00:37:17,642 -->
00:37:21,852
So I guess, uh, you know, finally,
as we sort of get into the the last
596
00:37:21,862 -->
00:37:23,322
few minutes of the podcast, right?
597
00:37:23,322 -->
00:37:28,202
So, you know, what, what are some of
the things like, you know, maybe like,
598
00:37:28,202 -->
00:37:33,737
uh, sort of a life cycle workflow when,
you know, You know, organizations that
599
00:37:33,747 -->
00:37:37,547
thinking about, because life is moving
very fast in the last few years, you
600
00:37:37,547 -->
00:37:40,467
know, you were talking about ML and
all of a sudden there's generative AI.
601
00:37:40,737 -->
00:37:44,367
Now there's AI agents, like, you
know, when an organization thinking
602
00:37:44,517 -->
00:37:45,937
about it, how do they go about it?
603
00:37:45,937 -->
00:37:49,207
You know, you know, implementing these
things, what should be the priority?
604
00:37:49,207 -->
00:37:50,877
What should they be the best practices?
605
00:37:51,957 -->
00:37:55,017
Karthik Bharathy: I think the
playbook, if you will, right.
606
00:37:55,737 -->
00:38:00,387
It's certainly, there are a few common
things across these different systems,
607
00:38:00,387 -->
00:38:03,027
and I'm sure there will be a lot
more coming in the next few years.
608
00:38:03,417 -->
00:38:07,677
Um, but fundamentally, I think what has
not changed is starting with data, right?
609
00:38:07,957 -->
00:38:12,677
Um, I, I can't emphasize this enough
that the, the better your data is,
610
00:38:12,677 -->
00:38:16,277
you know, the better pretty much your
AI model, the genetic system, all of
611
00:38:16,277 -->
00:38:17,877
the goodness that's, that's out there.
612
00:38:18,237 -->
00:38:23,677
Um, so have a robust data infrastructure,
quality data, um, that feeds into
613
00:38:23,697 -->
00:38:24,832
your machine learning processes.
614
00:38:25,652 -->
00:38:31,752
Um, and if you're starting off with, um,
you know, GenAI and agentic system, uh,
615
00:38:32,372 -->
00:38:37,762
I would start with one high value use
case, uh, prototype it to your business
616
00:38:37,772 -->
00:38:43,547
problem, demonstrate the value quickly,
um, and taking it to the next level,
617
00:38:43,767 -->
00:38:49,067
you want to establish the, um, the
necessary MLOps foundations saying, how
618
00:38:49,107 -->
00:38:51,067
does monitoring play into the system?
619
00:38:51,167 -->
00:38:52,487
What does versioning mean?
620
00:38:52,877 -->
00:38:55,417
Um, how can I go from
one version to the other?
621
00:38:55,417 -->
00:38:58,697
These are fundamental as you think
about taking a system from just a
622
00:38:58,697 -->
00:39:03,377
POC to a production, these play in
and just building on that and very
623
00:39:03,427 -->
00:39:07,227
relevant to the topic of today is, uh,
looking at the governance frameworks.
624
00:39:07,547 -->
00:39:09,597
Um, what does it mean to have a simple.
625
00:39:10,242 -->
00:39:14,192
Approval workflow that needs to be set
up as you're scaling through the system.
626
00:39:15,102 -->
00:39:18,872
And a lot of this, um, also
requires that you invest in
627
00:39:18,872 -->
00:39:20,722
your own team and train them.
628
00:39:20,732 -->
00:39:24,982
So they are aware of the different
elements of taking or going live with
629
00:39:24,982 -->
00:39:26,362
these different systems in place.
630
00:39:26,682 -->
00:39:30,132
Um, just with AWS, there are
enough training and certification.
631
00:39:30,132 -->
00:39:33,122
I'm sure Fiddler has their training
and certification available.
632
00:39:33,322 -->
00:39:36,072
So those help you build
your internal expertise.
633
00:39:36,602 -->
00:39:37,722
And, and finally.
634
00:39:38,072 -->
00:39:39,822
Um, plan for scale, right?
635
00:39:39,832 -->
00:39:44,052
What worked for you when you started
off with a small system may not be
636
00:39:44,252 -->
00:39:49,622
applicable when you go for like a 10x
or 100x of what you intend to build.
637
00:39:49,892 -->
00:39:54,092
But there are, the goodness is there
are enough enterprise features at AWS,
638
00:39:54,202 -->
00:39:58,622
SageMaker, in Fiddler, that help you
scale as you go through this journey.
639
00:39:59,877 -->
00:40:05,377
In converse, what you want to avoid
is rushing through a system quickly to
640
00:40:05,377 -->
00:40:12,947
demonstrate value, not having a good
data or like a data quality approach, um,
641
00:40:12,967 -->
00:40:15,477
not in engaging a lot of stakeholders.
642
00:40:15,887 -->
00:40:19,687
Um, and, and then you, you have very
little insight into how you would do
643
00:40:19,887 -->
00:40:21,897
maintenance or upgrade or deployment.
644
00:40:21,907 -->
00:40:24,837
So the, that, that is a
recipe, recipe for failure.
645
00:40:25,027 -->
00:40:27,462
So as long as you avoid
that so on the fundamentals.
646
00:40:28,132 -->
00:40:29,732
Krishna Gade: No more colloquially
don't do vibe checking, you know,
647
00:40:29,732 -->
00:40:33,602
vibe checking and vibe testing of
your models, you know, actually,
648
00:40:33,682 -->
00:40:35,292
and actually know what you're doing.
649
00:40:35,642 -->
00:40:36,462
That's a great point.
650
00:40:37,282 -->
00:40:39,222
So I guess actually it's
a very related question.
651
00:40:39,232 -->
00:40:41,652
Someone is asking, you know, things
are moving really fast, even for
652
00:40:41,652 -->
00:40:42,922
us in the technology area, right?
653
00:40:42,952 -->
00:40:46,912
Like, you know, what type of problems
in two to three years within AI agents
654
00:40:46,912 -->
00:40:48,912
that you, that will keep you up at night?
655
00:40:48,922 -->
00:40:50,412
You know, what do you foresee?
656
00:40:51,297 -->
00:40:54,407
Karthik Bharathy: Yeah, so that's,
that's, um, there's so much I could
657
00:40:54,407 -->
00:40:58,947
predict, but see, that's a question,
you know, I ask myself every day, right?
658
00:40:58,947 -->
00:41:03,507
Fundamentally, you know, I go
back to, um, you know, back when I
659
00:41:03,507 -->
00:41:08,537
joined AWS many, many years ago, um,
there was this interesting quote by
660
00:41:08,557 -->
00:41:10,557
Jeff that still resonates with me.
661
00:41:10,857 -->
00:41:14,357
It's something around, "hey, what, what
will not change" as opposed to saying,
662
00:41:14,357 -->
00:41:15,977
"hey, what will change that part?"
663
00:41:15,977 -->
00:41:19,347
The second part, what will change
is sort of, you know, each of us can
664
00:41:19,347 -->
00:41:22,407
debate like for hours or days together.
665
00:41:22,747 -->
00:41:27,357
Um, but what does not change is
fundamentally customers asking for better
666
00:41:27,397 -->
00:41:31,287
value and what it translates to something
that's more performance, something
667
00:41:31,287 -->
00:41:32,877
that's robust, something that's secure.
668
00:41:33,282 -->
00:41:36,312
Those fundamentals are not going to
change or something that's cheaper, right?
669
00:41:36,612 -->
00:41:41,122
Uh, uh, like the way Bezos put it, no
one's going to come to you and say,
670
00:41:41,122 -->
00:41:44,472
hey, give me something that's more
expensive or slower to perform, right?
671
00:41:44,832 -->
00:41:49,202
So fundamentally looking at the
system, um, and seeing what value it
672
00:41:49,262 -->
00:41:53,002
adds to your business use case, what
does it translate to your customers?
673
00:41:53,212 -->
00:41:56,932
I think those would be paramount as
you look at these, uh, innovations
674
00:41:56,932 -->
00:41:58,652
that are happening in GenAI industries.
675
00:41:58,752 -->
00:42:00,922
Krishna Gade: Yeah, and there's
an innovation happening across
676
00:42:00,942 -->
00:42:02,462
like small and big players, right?
677
00:42:02,462 -->
00:42:05,892
So there's, you know, there's a question
around how, you know, small, you know,
678
00:42:05,902 -->
00:42:09,862
there's a lot of, you know, new AI
agentic applications that are coming up.
679
00:42:09,862 -->
00:42:12,672
And, you know, how do you think
like they're playing within the,
680
00:42:12,762 -->
00:42:15,702
you know, within the, you know,
the big, you know, big players in
681
00:42:15,702 -->
00:42:19,382
ecosystem, you know, you know, building
also building agentic workflows.
682
00:42:19,757 -->
00:42:20,657
Any thoughts on that?
683
00:42:21,307 -->
00:42:23,867
How AWS might be encouraging
on the ecosystem side as well?
684
00:42:24,347 -->
00:42:25,257
Karthik Bharathy: Yeah, absolutely.
685
00:42:25,267 -->
00:42:28,857
And I think, um, one is
definitely through the partners.
686
00:42:28,857 -->
00:42:31,827
We work closely with, um,
companies like Fiddler.
687
00:42:32,157 -->
00:42:37,547
I think the second dimension to
that question is um, AWS providing
688
00:42:37,547 -->
00:42:39,037
the choice to the customers, right?
689
00:42:39,037 -->
00:42:42,697
So there is not a single model that we
say, hey, this is what you need to do.
690
00:42:42,707 -->
00:42:45,727
That's something that um,
you as a customer can decide,
691
00:42:45,967 -->
00:42:47,817
um, right from DeepSeek.
692
00:42:48,132 -->
00:42:53,192
to, um, the latest Llama models to
our own in house Amazon Nova models.
693
00:42:53,382 -->
00:42:57,602
You have all of those available to
experiment and try for your use case.
694
00:42:57,602 -->
00:43:01,672
I'm sure a lot of it will be applicable
even in the world tomorrow, where
695
00:43:01,692 -->
00:43:05,607
you have the choice in choosing the
best of what's applicable for you.
696
00:43:05,807 -->
00:43:06,107
Krishna Gade: Awesome.
697
00:43:06,637 -->
00:43:06,957
Great.
698
00:43:07,007 -->
00:43:10,347
I think, uh, with that, we are
coming to the end of the podcast.
699
00:43:10,347 -->
00:43:13,167
You know, thank you so much, Karthik
for spending time with us today.
700
00:43:13,167 -->
00:43:16,827
Um, you know, I think one of the
things that I took away is that quote
701
00:43:16,827 -->
00:43:20,537
that you mentioned that Jeff said,
like, what is not going to change?
702
00:43:20,607 -->
00:43:23,147
And I, I believe what is not
going to change with AI is going
703
00:43:23,147 -->
00:43:26,747
to be, whether it's your simple
statistical model or deep learning
704
00:43:26,747 -->
00:43:29,467
model or generative AI or AI agents.
705
00:43:30,047 -->
00:43:31,607
You need to test it properly.
706
00:43:31,607 -->
00:43:34,777
You need to monitor it properly
and you need to make sure it's, you
707
00:43:34,777 -->
00:43:38,007
know, it's secure and, and, and it's
working for your, for your business.
708
00:43:38,017 -->
00:43:40,357
So, and so I think that's,
that's not going to change.
709
00:43:40,417 -->
00:43:44,417
Um, I think I, and so I think, you
know, that's, that's kind of where
710
00:43:44,417 -->
00:43:46,287
our partnership with Amazon comes in.
711
00:43:46,297 -->
00:43:49,887
And so, you know, thank you so much
for being on the show today and, um,
712
00:43:50,027 -->
00:43:53,617
you know, look forward to, you know,
further more conversations in the future.
713
00:43:54,247 -->
00:43:55,327
Karthik Bharathy: Thank
you for having me, Krishna.
714
00:43:55,327 -->
00:43:56,367
This was great chatting with you.
715
00:43:56,857 -->
00:43:57,137
Krishna Gade: Awesome.
716
00:43:57,237 -->
00:43:57,567
Thank you.
717
00:43:58,137 -->
00:43:58,807
Thanks everyone.
