Listen now | Recording of AURA - Anna Bergevin & Shane Gibson on 13th July 2026

WEBVTT
Kind: captions
Language: en
00:00:00.400 --> 00:00:01.990
Hey, Yina. Hey, thanks for coming on to
00:00:02.000 --> 00:00:04.230
Aura. You're the uh third of the Awesome
00:00:04.240 --> 00:00:06.270
Foursome. So, uh great to have you on
00:00:06.280 --> 00:00:07.310
here and I'm particularly looking
00:00:07.320 --> 00:00:09.310
forward to having conversations and
00:00:09.320 --> 00:00:11.230
things using questions around uh the
00:00:11.240 --> 00:00:13.070
kind of product management and product
00:00:13.080 --> 00:00:16.270
lens to the the data world. So, what's
00:00:16.280 --> 00:00:17.470
what's happening? What's top of your
00:00:17.480 --> 00:00:19.750
mind at the moment?
00:00:19.760 --> 00:00:22.150
>> Well, what is top of mind at the moment?
00:00:22.160 --> 00:00:24.350
I mean, it's it's an annual planning
00:00:24.360 --> 00:00:26.510
time uh at my company. So, top of mind
00:00:26.520 --> 00:00:29.150
is is road map and just uh commitment
00:00:29.160 --> 00:00:31.830
cycles. Um been doing a lot of work
00:00:31.840 --> 00:00:33.395
right now around
00:00:33.405 --> 00:00:34.030
>> [snorts]
00:00:34.040 --> 00:00:35.470
>> um
00:00:35.480 --> 00:00:38.030
you know, it's interesting because your
00:00:38.040 --> 00:00:40.830
your uh branding is about agility and I
00:00:40.840 --> 00:00:42.150
feel like
00:00:42.160 --> 00:00:43.910
the hospital is really struggling with
00:00:43.920 --> 00:00:45.470
this idea of like how to be agile and
00:00:45.480 --> 00:00:47.470
how to also know when you can make a
00:00:47.480 --> 00:00:48.790
commitment so other teams know what they
00:00:48.800 --> 00:00:50.070
can count on.
00:00:50.080 --> 00:00:51.550
Um which internally, you know, when
00:00:51.560 --> 00:00:54.110
you're an internal data team. So,
00:00:54.120 --> 00:00:56.710
explain to my product team of like
00:00:56.720 --> 00:00:58.950
what it means to like
00:00:58.960 --> 00:01:00.910
kind of lock in the quarter because our
00:01:00.920 --> 00:01:03.230
executives need to know what we're
00:01:03.240 --> 00:01:04.950
actually going to deliver. And by that,
00:01:04.960 --> 00:01:06.390
I don't mean every tiny report. I mean
00:01:06.400 --> 00:01:09.190
the big the big investments. And so,
00:01:09.200 --> 00:01:10.950
that doesn't feel agile. That feels
00:01:10.960 --> 00:01:13.110
anti-product in some ways to people.
00:01:13.120 --> 00:01:14.470
But, I'm like
00:01:14.480 --> 00:01:17.350
um here's how I handle it, which is when
00:01:17.360 --> 00:01:18.870
things come up mid-quarter and we're
00:01:18.880 --> 00:01:21.830
expected to pivot is if the executives
00:01:21.840 --> 00:01:23.710
accept that and tell us yes, but what we
00:01:23.720 --> 00:01:26.390
do is is try to explain
00:01:26.400 --> 00:01:28.750
this this thing that uh we committed to,
00:01:28.760 --> 00:01:30.390
this is what's at risk if I accept this
00:01:30.400 --> 00:01:33.150
new thing. So, we um we leave that call
00:01:33.160 --> 00:01:34.790
to those people who have the authority
00:01:34.800 --> 00:01:36.390
to make that call is what we try to do
00:01:36.400 --> 00:01:38.710
so that we get basic we can be agile,
00:01:38.720 --> 00:01:40.670
but we can be committed. So, that's been
00:01:40.680 --> 00:01:42.310
dominating my mind for the last month
00:01:42.320 --> 00:01:44.510
internally and then I think broader
00:01:44.520 --> 00:01:46.190
industry-wide
00:01:46.200 --> 00:01:48.350
what I'm thinking the most about is like
00:01:48.360 --> 00:01:49.710
knowledge management, context
00:01:49.720 --> 00:01:52.710
management, and how the role of the data
00:01:52.720 --> 00:01:54.950
team changes in the world of AI. And of
00:01:54.960 --> 00:01:56.870
course, in an enterprise
00:01:56.880 --> 00:01:58.750
it's not changing today. We don't have
00:01:58.760 --> 00:02:00.630
like everyone's talking to robots,
00:02:00.640 --> 00:02:01.830
everyone's talking to agents to get
00:02:01.840 --> 00:02:04.469
their analytics, but it's clear
00:02:04.479 --> 00:02:06.230
we need to build metadata pipelines and
00:02:06.240 --> 00:02:07.830
context pipelines in addition to
00:02:07.840 --> 00:02:09.430
traditional data pipelines. So, thinking
00:02:09.440 --> 00:02:10.630
about what's the system we're going to
00:02:10.640 --> 00:02:11.750
build that's going to enable that.
00:02:11.760 --> 00:02:14.310
That's my macro thing that I'm thinking
00:02:14.320 --> 00:02:15.630
about a lot.
00:02:15.640 --> 00:02:17.590
>> Yeah, the agility one's interesting,
00:02:17.600 --> 00:02:19.790
right? So, I kind of differentiate ad
00:02:19.800 --> 00:02:21.510
hoc to agility.
00:02:21.520 --> 00:02:24.949
Um and so, you know, for me uh
00:02:24.959 --> 00:02:26.830
it's not about, you know, moving fast
00:02:26.840 --> 00:02:28.710
and breaking things.
00:02:28.720 --> 00:02:31.110
Um it it's about being able to change at
00:02:31.120 --> 00:02:33.150
the lowest cost
00:02:33.160 --> 00:02:35.710
uh and with the greatest speed. And so,
00:02:35.720 --> 00:02:36.949
you know, when we talk about quarterly
00:02:36.959 --> 00:02:38.870
planning you know, it kind of reminds me
00:02:38.880 --> 00:02:42.150
of safe scaled agile. And yeah, I I'm
00:02:42.160 --> 00:02:43.949
not a great fan of those patterns
00:02:43.959 --> 00:02:46.030
because what I find is while the intent
00:02:46.040 --> 00:02:47.870
is the ability to give you a road map
00:02:47.880 --> 00:02:49.670
for 3 months, you actually end up
00:02:49.680 --> 00:02:51.310
locking one in for nine nine months,
00:02:51.320 --> 00:02:53.390
right? Cuz everybody's kind of
00:02:53.400 --> 00:02:55.350
three threes out. Um
00:02:55.360 --> 00:02:57.150
but what I always say to stakeholders is
00:02:57.160 --> 00:02:59.790
change is okay, just comes at a cost.
00:02:59.800 --> 00:03:01.990
It's not free. Um so,
00:03:02.000 --> 00:03:03.550
what would you like to do, right? Would
00:03:03.560 --> 00:03:06.229
you would you like to pay that cost
00:03:06.239 --> 00:03:08.150
uh in terms of work not done? Uh or
00:03:08.160 --> 00:03:09.509
would you not because, you know, if
00:03:09.519 --> 00:03:10.750
you're the people in charge of
00:03:10.760 --> 00:03:12.270
prioritization,
00:03:12.280 --> 00:03:13.550
um you get to make that call, right?
00:03:13.560 --> 00:03:14.830
What what's the next most important
00:03:14.840 --> 00:03:17.150
thing the team can build? Um I think in
00:03:17.160 --> 00:03:19.150
data lots of people treat it as free,
00:03:19.160 --> 00:03:21.910
right? You put a Jira ticket in and that
00:03:21.920 --> 00:03:23.430
request at the end you never have to pay
00:03:23.440 --> 00:03:25.430
money, you never have to pay any time,
00:03:25.440 --> 00:03:26.590
you just wait for somebody else to do
00:03:26.600 --> 00:03:28.630
all the work. So, it is that real
00:03:28.640 --> 00:03:30.550
challenge and and like you said,
00:03:30.560 --> 00:03:33.270
bringing product thinking into data it's
00:03:33.280 --> 00:03:35.110
not as easy as just treating it like a
00:03:35.120 --> 00:03:37.790
software project in my view, right? It's
00:03:37.800 --> 00:03:39.509
kind of the patterns are useful, but
00:03:39.519 --> 00:03:41.430
you've got to apply them in certain
00:03:41.440 --> 00:03:43.150
different ways.
00:03:43.160 --> 00:03:46.110
>> It's a lot of translation. So, like um
00:03:46.120 --> 00:03:47.910
the big reason I started writing when
00:03:47.920 --> 00:03:50.590
data met product was because
00:03:50.600 --> 00:03:53.110
when I started working in data platform
00:03:53.120 --> 00:03:55.990
as a product manager coming from
00:03:56.000 --> 00:03:57.830
like the software side,
00:03:57.840 --> 00:04:01.030
um it felt like every podcast I listened
00:04:01.040 --> 00:04:03.430
to, every book I read was like, "Well,
00:04:03.440 --> 00:04:05.510
yeah, but I don't have
00:04:05.520 --> 00:04:07.510
external customers. I don't have
00:04:07.520 --> 00:04:09.590
go-to-market. I don't have these like I
00:04:09.600 --> 00:04:10.949
these things people are talking about
00:04:10.959 --> 00:04:13.790
don't really apply in my context." And
00:04:13.800 --> 00:04:15.110
also data, the biggest difference
00:04:15.120 --> 00:04:16.752
between software and data is
00:04:16.762 --> 00:04:17.550
>> [snorts]
00:04:17.560 --> 00:04:19.509
>> the company I worked at um on the
00:04:19.519 --> 00:04:21.870
software side, they had bounded context,
00:04:21.880 --> 00:04:23.430
meaning every team kind of controlled
00:04:23.440 --> 00:04:24.510
their database, controlled their
00:04:24.520 --> 00:04:26.110
context, and they would share data
00:04:26.120 --> 00:04:27.870
streams between teams.
00:04:27.880 --> 00:04:30.190
And so you had kind of this very um
00:04:30.200 --> 00:04:31.710
ability to move fast because your team
00:04:31.720 --> 00:04:34.550
didn't was loosely coupled to other
00:04:34.560 --> 00:04:35.909
software teams.
00:04:35.919 --> 00:04:38.510
In data, it's like it's like your raw
00:04:38.520 --> 00:04:40.430
materials are just being thrown at you,
00:04:40.440 --> 00:04:41.790
and you have no control for how they're
00:04:41.800 --> 00:04:43.350
showing up, and you're supposed to
00:04:43.360 --> 00:04:46.750
create stability and and value out of
00:04:46.760 --> 00:04:48.510
that. And and yeah, we are trying to do
00:04:48.520 --> 00:04:51.030
data contracts and do a little bit more
00:04:51.040 --> 00:04:54.310
reliability upstream, but I think data
00:04:54.320 --> 00:04:55.990
is quite different than software, even
00:04:56.000 --> 00:04:57.470
though the patterns are useful. So, it's
00:04:57.480 --> 00:04:59.390
all translation, and hardly anyone's
00:04:59.400 --> 00:05:01.070
creating content about how they're doing
00:05:01.080 --> 00:05:02.150
it. And
00:05:02.160 --> 00:05:05.230
and there are people doing it. Um
00:05:05.240 --> 00:05:07.150
and there are leaders hearing about this
00:05:07.160 --> 00:05:08.590
and hearing it's valuable, and they're
00:05:08.600 --> 00:05:10.630
hiring. But even like
00:05:10.640 --> 00:05:13.110
Netflix's data blog just talked about
00:05:13.120 --> 00:05:14.950
data as a product like 6 months ago.
00:05:14.960 --> 00:05:16.550
Netflix, you know, Silicon Valley is
00:05:16.560 --> 00:05:18.510
supposed to be cutting edge. So, like if
00:05:18.520 --> 00:05:20.510
people are doing it, everyone is a
00:05:20.520 --> 00:05:22.750
pioneer, everyone is new, no one knows
00:05:22.760 --> 00:05:24.150
what they're doing.
00:05:24.160 --> 00:05:24.750
Um
00:05:24.760 --> 00:05:26.230
they're finding useful patterns, but
00:05:26.240 --> 00:05:27.670
nobody There's no best practice
00:05:27.680 --> 00:05:29.390
emerging. And we are seeing trying to
00:05:29.400 --> 00:05:32.350
see now, 3 years into it, 3 years into
00:05:32.360 --> 00:05:34.110
my 4 years actually into my journey of
00:05:34.120 --> 00:05:35.150
this,
00:05:35.160 --> 00:05:36.750
a couple books are being published, a
00:05:36.760 --> 00:05:39.030
couple more more than just data mesh's
00:05:39.040 --> 00:05:40.350
idea of a data product, more about
00:05:40.360 --> 00:05:42.310
product thinking applied to data. We're
00:05:42.320 --> 00:05:43.990
seeing a little bit more coming out, but
00:05:44.000 --> 00:05:46.270
it's still very much There's not enough
00:05:46.280 --> 00:05:47.670
content to guide new data product
00:05:47.680 --> 00:05:49.430
managers at all.
00:05:49.440 --> 00:05:50.710
>> Yeah, I've [snorts] seen seeing a lot of
00:05:50.720 --> 00:05:52.590
books coming out now or with the words
00:05:52.600 --> 00:05:54.550
product and data. And like you said,
00:05:54.560 --> 00:05:56.990
some of them are technology, how to
00:05:57.000 --> 00:05:59.110
build a product technically.
00:05:59.120 --> 00:06:01.470
Some are process.
00:06:01.480 --> 00:06:03.230
Some are trying to define it. Some are
00:06:03.240 --> 00:06:06.030
product thinking for data. Some are team
00:06:06.040 --> 00:06:08.190
design. So, yeah, it's all those lenses
00:06:08.200 --> 00:06:10.030
that actually there will never be one
00:06:10.040 --> 00:06:11.190
book to rule them all, I don't think,
00:06:11.200 --> 00:06:13.070
because it'll be too big a book. But
00:06:13.080 --> 00:06:14.470
they're taking those different lenses.
00:06:14.480 --> 00:06:16.710
But then how many product people or data
00:06:16.720 --> 00:06:19.430
people have time to and money to buy 10
00:06:19.440 --> 00:06:21.030
different books
00:06:21.040 --> 00:06:23.270
and figure out which ones are giving
00:06:23.280 --> 00:06:25.150
them the bits they need and then
00:06:25.160 --> 00:06:27.070
cobbling the LEGO blocks together
00:06:27.080 --> 00:06:28.270
themselves, right? That's the hard part
00:06:28.280 --> 00:06:29.830
because we're stealing from product,
00:06:29.840 --> 00:06:30.909
we're stealing from Agile, we're
00:06:30.919 --> 00:06:32.670
stealing from data when we try to put
00:06:32.680 --> 00:06:34.030
all those together into a new way of
00:06:34.040 --> 00:06:35.790
working. And that that's hard, right?
00:06:35.800 --> 00:06:39.590
That's pioneering work.
00:06:39.600 --> 00:06:42.430
>> Yeah, and this is why my favorite type
00:06:42.440 --> 00:06:44.670
of content is more podcast or
00:06:44.680 --> 00:06:47.270
conferences. I want to hear like
00:06:47.280 --> 00:06:49.390
we had to tackle like I want to talk to
00:06:49.400 --> 00:06:51.750
other hospitals about how they tackled a
00:06:51.760 --> 00:06:54.110
unified longitudinal model across EHR
00:06:54.120 --> 00:06:55.830
systems, right? So, electronic health
00:06:55.840 --> 00:06:57.750
records systems in the United States.
00:06:57.760 --> 00:06:59.950
Most hospitals have been around over 100
00:06:59.960 --> 00:07:02.390
years. Most hospitals have a long
00:07:02.400 --> 00:07:05.590
history. And if they have migrated EHRs,
00:07:05.600 --> 00:07:07.470
which many have,
00:07:07.480 --> 00:07:09.150
they didn't import all the data because
00:07:09.160 --> 00:07:12.350
it's costly and and difficult. And so,
00:07:12.360 --> 00:07:14.909
if doctors or data scientists want to
00:07:14.919 --> 00:07:17.590
look at patterns over time, if you want
00:07:17.600 --> 00:07:19.830
to create a seasonal forecast that says
00:07:19.840 --> 00:07:21.470
how big this year's flu year is going to
00:07:21.480 --> 00:07:23.470
be and how many extra nurses we need to
00:07:23.480 --> 00:07:24.990
handle the fact that like we're getting
00:07:25.000 --> 00:07:27.230
a surge in the ER, you need forecasting
00:07:27.240 --> 00:07:28.590
models, which requires longitudinal
00:07:28.600 --> 00:07:30.750
data. And that data is difficult to
00:07:30.760 --> 00:07:32.630
stitch together. So,
00:07:32.640 --> 00:07:33.830
every time you encounter difficult
00:07:33.840 --> 00:07:35.630
problems, what's kind of fun about
00:07:35.640 --> 00:07:37.030
working at a children's hospital is in
00:07:37.040 --> 00:07:38.750
the US you don't have competing children
00:07:38.760 --> 00:07:40.190
hospitals. Like we don't have a national
00:07:40.200 --> 00:07:41.830
health service like the UK or anything
00:07:41.840 --> 00:07:43.470
like that, but we have
00:07:43.480 --> 00:07:45.950
um in the adult healthcare system, you
00:07:45.960 --> 00:07:46.990
will have a little bit of market
00:07:47.000 --> 00:07:49.110
competition between similar hospitals in
00:07:49.120 --> 00:07:51.150
the same geographic area. That's not
00:07:51.160 --> 00:07:52.430
true of pediatric hospitals. There's
00:07:52.440 --> 00:07:53.710
usually like
00:07:53.720 --> 00:07:55.790
one for the state or one for many
00:07:55.800 --> 00:07:57.550
states. And so,
00:07:57.560 --> 00:07:58.870
pediatric hospitals will talk to each
00:07:58.880 --> 00:08:00.710
other and be like, "All right, how did
00:08:00.720 --> 00:08:02.150
you guys tackle this?" And there's kind
00:08:02.160 --> 00:08:03.470
of this informal peer network, and
00:08:03.480 --> 00:08:04.830
that's been really fun for me as a data
00:08:04.840 --> 00:08:06.990
product manager because um
00:08:07.000 --> 00:08:08.070
historically I've only gotten that
00:08:08.080 --> 00:08:10.270
through my friends uh like yourself or
00:08:10.280 --> 00:08:12.510
Nick Zerbeudas or other like data
00:08:12.520 --> 00:08:15.030
consultants or product managers, the few
00:08:15.040 --> 00:08:16.550
of us who are out there talking online
00:08:16.560 --> 00:08:18.190
have been my peer network. And so, now I
00:08:18.200 --> 00:08:20.110
have a little bit of this
00:08:20.120 --> 00:08:21.950
peer network non-competitive built in,
00:08:21.960 --> 00:08:23.270
which is kind of fun. But, that's what I
00:08:23.280 --> 00:08:25.070
want is I don't I don't want to hear in
00:08:25.080 --> 00:08:26.230
the abstract, I don't want to hear your
00:08:26.240 --> 00:08:27.470
theory, I don't want to read some giant
00:08:27.480 --> 00:08:29.710
manifesto. I want to hear
00:08:29.720 --> 00:08:32.070
"Last month we had to build this
00:08:32.080 --> 00:08:34.390
customer data platform for this specific
00:08:34.400 --> 00:08:36.110
campaign, and here's how we handled the
00:08:36.120 --> 00:08:39.070
ETL, augmentation, reverse ETL pattern,
00:08:39.080 --> 00:08:39.950
and these are the problems we
00:08:39.960 --> 00:08:41.149
encountered." I want to hear that
00:08:41.159 --> 00:08:42.430
because that's
00:08:42.440 --> 00:08:43.630
what I can take back to my team and go,
00:08:43.640 --> 00:08:45.070
"Hey, we're tackling a similar problem.
00:08:45.080 --> 00:08:47.190
This is how this company did it." And
00:08:47.200 --> 00:08:48.390
you don't get that in books. The books
00:08:48.400 --> 00:08:50.310
are too theoretical. And so, they're
00:08:50.320 --> 00:08:51.750
still a useful reference, I feel like,
00:08:51.760 --> 00:08:53.750
if you're a new product manager.
00:08:53.760 --> 00:08:55.430
I would love to have a handful of books
00:08:55.440 --> 00:08:57.270
to be like, "Read this, listen to this."
00:08:57.280 --> 00:08:58.950
Like, the Marty Cagan for software
00:08:58.960 --> 00:09:00.829
product management. I want like
00:09:00.839 --> 00:09:02.430
something like that.
00:09:02.440 --> 00:09:03.150
Um
00:09:03.160 --> 00:09:04.750
but, the content I personally am most
00:09:04.760 --> 00:09:07.550
interested in is podcast conferences or
00:09:07.560 --> 00:09:09.790
like Discord servers, talking to people
00:09:09.800 --> 00:09:11.950
about like the the nitty-gritty of the
00:09:11.960 --> 00:09:12.990
job.
00:09:13.000 --> 00:09:14.710
>> Yeah, I went to Data Mesh Live
00:09:14.720 --> 00:09:17.030
conference in Antwerp last month. Um
00:09:17.040 --> 00:09:18.670
just cuz I didn't been to Antwerp, so
00:09:18.680 --> 00:09:20.230
you know, pick a pick a conference in a
00:09:20.240 --> 00:09:22.070
city you've never been to and and treat
00:09:22.080 --> 00:09:24.590
it as a work holiday to a degree. What
00:09:24.600 --> 00:09:26.790
surprised me, uh so there were over 150
00:09:26.800 --> 00:09:28.470
people there. What surprised me was I
00:09:28.480 --> 00:09:31.070
kind of thought Data Mesh was dead. Um
00:09:31.080 --> 00:09:33.230
but, there were practitioners there, and
00:09:33.240 --> 00:09:35.590
they were talking about real use cases.
00:09:35.600 --> 00:09:37.390
Yeah, there were one or two fluffy
00:09:37.400 --> 00:09:39.150
corporate, yeah, here's our journey
00:09:39.160 --> 00:09:40.990
stuff. But the majority of them were
00:09:41.000 --> 00:09:43.270
hands-on, you know, talking about uh
00:09:43.280 --> 00:09:45.150
data contracts, you know, the standards
00:09:45.160 --> 00:09:46.350
around there and the data product
00:09:46.360 --> 00:09:47.950
standards and how they implemented them
00:09:47.960 --> 00:09:50.270
and how they layered their data products
00:09:50.280 --> 00:09:52.870
from source to, you know, aggregated to
00:09:52.880 --> 00:09:54.670
consume ones. And so they it was really,
00:09:54.680 --> 00:09:56.070
really hands-on, you know, not hands-on,
00:09:56.080 --> 00:09:57.950
but it was really, really um
00:09:57.960 --> 00:09:59.550
practitioner-level stuff, which I
00:09:59.560 --> 00:10:01.550
enjoyed because it it showed what people
00:10:01.560 --> 00:10:03.790
were doing and I learned from there. Um
00:10:03.800 --> 00:10:05.750
so, let's let's see what we get in terms
00:10:05.760 --> 00:10:08.310
of questions, right? Um
00:10:08.320 --> 00:10:10.430
So, yeah, Chris Chris Gamble's saying
00:10:10.440 --> 00:10:12.270
practical data community, that's a great
00:10:12.280 --> 00:10:14.350
if anybody wants to uh talk to data mesh
00:10:14.360 --> 00:10:16.190
practitioners, hop on there. If you
00:10:16.200 --> 00:10:18.270
Yeah, um there's a lot of us there. It
00:10:18.280 --> 00:10:20.910
is relatively unhinged. I will quantify
00:10:20.920 --> 00:10:21.592
that and uh
00:10:21.602 --> 00:10:21.710
>> [laughter]
00:10:21.720 --> 00:10:23.990
>> qualify that. You know, it is, yeah, it
00:10:24.000 --> 00:10:25.510
goes all over the place, so you have to
00:10:25.520 --> 00:10:27.390
be willing to uh to live with that kind
00:10:27.400 --> 00:10:29.230
of culture. Um but it is a a good
00:10:29.240 --> 00:10:31.230
culture in my view, so it's free, hop on
00:10:31.240 --> 00:10:34.230
and do it. Um Johnny Winter said, "Wait,
00:10:34.240 --> 00:10:36.002
data mesh still alive."
00:10:36.012 --> 00:10:36.590
>> [laughter]
00:10:36.600 --> 00:10:39.990
>> Uh yeah, so
00:10:40.000 --> 00:10:41.990
Yeah, the the key thing I got back from
00:10:42.000 --> 00:10:43.870
that conference is it is. Yeah, there
00:10:43.880 --> 00:10:45.990
are, especially in Europe, practitioners
00:10:46.000 --> 00:10:49.030
taking the four principles from the book
00:10:49.040 --> 00:10:51.670
uh and focusing on executing them and
00:10:51.680 --> 00:10:53.790
then sharing their knowledge. Uh how,
00:10:53.800 --> 00:10:55.430
you know, federated data governance was
00:10:55.440 --> 00:10:57.590
probably still the weakest of the four
00:10:57.600 --> 00:10:59.829
pillars um because it is the hardest,
00:10:59.839 --> 00:11:00.910
right? Uh
00:11:00.920 --> 00:11:03.670
they've kind of solved the the source
00:11:03.680 --> 00:11:06.110
aggregated consume layered problem. I
00:11:06.120 --> 00:11:07.470
mean, effectively they've built a
00:11:07.480 --> 00:11:09.710
layered architecture of data products.
00:11:09.720 --> 00:11:11.710
Um I found that interesting. Again,
00:11:11.720 --> 00:11:13.230
reuse of the pattern.
00:11:13.240 --> 00:11:14.430
Um
00:11:14.440 --> 00:11:16.750
They're they're using the the open data
00:11:16.760 --> 00:11:18.790
product standard uh
00:11:18.800 --> 00:11:20.790
in terms of input ports and output ports
00:11:20.800 --> 00:11:22.190
and that has given them a shared
00:11:22.200 --> 00:11:24.630
language. Um so they they can actually
00:11:24.640 --> 00:11:26.829
talk to each other and you know, deploy
00:11:26.839 --> 00:11:27.910
data products the same way. They're
00:11:27.920 --> 00:11:29.470
still treating them as
00:11:29.480 --> 00:11:31.430
containers.
00:11:31.440 --> 00:11:33.710
Uh so that was interesting, Although it
00:11:33.720 --> 00:11:35.550
was a mismatch of what lives in the
00:11:35.560 --> 00:11:37.110
container for a data product and what
00:11:37.120 --> 00:11:39.350
doesn't. That that can like I was
00:11:39.360 --> 00:11:41.350
Yeah, that kind of looks like a data
00:11:41.360 --> 00:11:43.430
warehouse pattern. One interesting thing
00:11:43.440 --> 00:11:45.110
is they were
00:11:45.120 --> 00:11:47.710
polite but negative about monolithic
00:11:47.720 --> 00:11:49.510
data warehouses.
00:11:49.520 --> 00:11:51.710
Um, so that was kind of interesting. Uh,
00:11:51.720 --> 00:11:54.230
but we will do that. Uh, and then
00:11:54.240 --> 00:11:55.750
What was the last one? Um,
00:11:55.760 --> 00:11:57.150
product thinking. Yeah, there was a lot
00:11:57.160 --> 00:11:59.510
of product thinking in there. So, yeah,
00:11:59.520 --> 00:12:00.870
yeah, I wouldn't say it's dominating the
00:12:00.880 --> 00:12:02.830
world anymore, but um, I think the core
00:12:02.840 --> 00:12:04.990
patterns um, that were in the book have
00:12:05.000 --> 00:12:07.270
come out and kind of got a life of their
00:12:07.280 --> 00:12:08.790
own. Yeah, what about you? Do you see it
00:12:08.800 --> 00:12:10.978
much in your side of the world?
00:12:10.988 --> 00:12:10.990
>> [snorts]
00:12:11.000 --> 00:12:12.510
>> I see it less in the US. I definitely
00:12:12.520 --> 00:12:15.030
hear it more from my European friends.
00:12:15.040 --> 00:12:17.510
Um, but I don't What I think is
00:12:17.520 --> 00:12:19.030
happening right now that is going to be
00:12:19.040 --> 00:12:21.510
very interesting for like the next 5 to
00:12:21.520 --> 00:12:23.590
10 years is
00:12:23.600 --> 00:12:26.070
AI is changing how we work, uh, not just
00:12:26.080 --> 00:12:29.470
in data, but across the or corporation.
00:12:29.480 --> 00:12:31.910
And what's clear is like org design is
00:12:31.920 --> 00:12:33.990
going to be impacted, the roles we hire
00:12:34.000 --> 00:12:35.910
for impacted, how we collaborate is
00:12:35.920 --> 00:12:37.910
going to be impacted. And so,
00:12:37.920 --> 00:12:39.430
um,
00:12:39.440 --> 00:12:41.430
whether and that will impact our data
00:12:41.440 --> 00:12:43.230
architecture and how we construct
00:12:43.240 --> 00:12:45.590
systems, right? So,
00:12:45.600 --> 00:12:47.310
the
00:12:47.320 --> 00:12:48.790
the people are going to shift and I
00:12:48.800 --> 00:12:49.950
think no one knows how this is going to
00:12:49.960 --> 00:12:52.190
go. So, I think it's going to be very
00:12:52.200 --> 00:12:54.910
interesting to be like, what emerges
00:12:54.920 --> 00:12:58.070
from this? Is it Does data mesh take a
00:12:58.080 --> 00:13:01.110
new flavor where it's like, uh,
00:13:01.120 --> 00:13:04.230
we have these micro domain warehouses,
00:13:04.240 --> 00:13:06.750
but a centralized metadata store that
00:13:06.760 --> 00:13:08.590
helps with like retrieval of the context
00:13:08.600 --> 00:13:09.790
and then you go find the data you need
00:13:09.800 --> 00:13:12.470
and construct it. Are we it will we lean
00:13:12.480 --> 00:13:14.790
even more into centralized because AI
00:13:14.800 --> 00:13:16.710
makes it easier to port the data and
00:13:16.720 --> 00:13:17.990
bring it together and mesh and do all
00:13:18.000 --> 00:13:19.230
the things you want to do? I don't know.
00:13:19.240 --> 00:13:20.390
I don't know how it's going to evolve,
00:13:20.400 --> 00:13:23.030
but what seems very clear is like
00:13:23.040 --> 00:13:25.070
I think what we'll figure out first is
00:13:25.080 --> 00:13:28.070
like new roles will start to emerge and
00:13:28.080 --> 00:13:29.670
then people thinking in these new roles
00:13:29.680 --> 00:13:31.430
and these new collaboration patterns, we
00:13:31.440 --> 00:13:34.470
will see architectures evolve. But I
00:13:34.480 --> 00:13:38.910
don't I still see people favoring
00:13:38.920 --> 00:13:40.070
or
00:13:40.080 --> 00:13:41.710
it's easier on the planning side to be
00:13:41.720 --> 00:13:42.830
like, we'll get all the data in one
00:13:42.840 --> 00:13:44.470
place and then we have lots of options,
00:13:44.480 --> 00:13:47.710
right? Of what to do. Where if you stay
00:13:47.720 --> 00:13:49.390
um
00:13:49.400 --> 00:13:51.310
if you stay separate, it's harder to
00:13:51.320 --> 00:13:53.230
think about the having ultimate
00:13:53.240 --> 00:13:55.510
flexibility or the ultimate it feels
00:13:55.520 --> 00:13:56.470
like you have to coordinate more with
00:13:56.480 --> 00:13:58.310
people. Uh people are always the hardest
00:13:58.320 --> 00:14:00.230
part of data, so
00:14:00.240 --> 00:14:01.870
centralized data teams don't love the
00:14:01.880 --> 00:14:03.830
idea of like, well now we have to trust
00:14:03.840 --> 00:14:05.470
those guys even more. Like that it's
00:14:05.480 --> 00:14:06.990
like all of it has to ship together or
00:14:07.000 --> 00:14:08.350
it doesn't quite work. And so I feel
00:14:08.360 --> 00:14:09.950
like the bigger the company is in some
00:14:09.960 --> 00:14:11.710
ways the harder it can be.
00:14:11.720 --> 00:14:12.630
But I
00:14:12.640 --> 00:14:14.830
uh I also think leadership has to buy
00:14:14.840 --> 00:14:16.710
into data mesh and be very opinionated.
00:14:16.720 --> 00:14:18.710
Like this is the way we're going. And I
00:14:18.720 --> 00:14:20.510
often see enterprise architectures quite
00:14:20.520 --> 00:14:23.030
weak uh in the company and so this not
00:14:23.040 --> 00:14:25.150
opinionated enough and so the gravity
00:14:25.160 --> 00:14:26.310
and the vendors pull you towards
00:14:26.320 --> 00:14:28.150
centralized warehouse. So
00:14:28.160 --> 00:14:30.310
um I think I have no idea where it's
00:14:30.320 --> 00:14:31.910
going to go. Um
00:14:31.920 --> 00:14:33.470
but I think it's really hard to fight
00:14:33.480 --> 00:14:35.110
for
00:14:35.120 --> 00:14:37.190
data contracts, decentralized data
00:14:37.200 --> 00:14:38.870
ownership. I'm trying to just get people
00:14:38.880 --> 00:14:40.790
to give me their column descriptions,
00:14:40.800 --> 00:14:42.285
you know? Like
00:14:42.295 --> 00:14:43.150
>> [laughter]
00:14:43.160 --> 00:14:45.390
>> just like, can you write it down
00:14:45.400 --> 00:14:48.310
somewhere in any format? In JSON, in in
00:14:48.320 --> 00:14:51.110
in no database, in a spreadsheet, just
00:14:51.120 --> 00:14:52.630
something you consistently update every
00:14:52.640 --> 00:14:54.230
time you add a new column so I can pull
00:14:54.240 --> 00:14:56.829
it. Um that's hard enough. So like the
00:14:56.839 --> 00:14:58.270
idea of going
00:14:58.280 --> 00:15:00.230
even more intensive decentralized
00:15:00.240 --> 00:15:03.950
governance feels like a big lift.
00:15:03.960 --> 00:15:05.590
>> Yeah, I mean don't you just get Claude
00:15:05.600 --> 00:15:07.750
to write the column descriptions for you
00:15:07.760 --> 00:15:09.630
so cuz it's easier, right? And then use
00:15:09.640 --> 00:15:11.470
Claude to read the column descriptions
00:15:11.480 --> 00:15:13.510
so that you know, it talks to itself and
00:15:13.520 --> 00:15:15.035
there's no human in the loop.
00:15:15.045 --> 00:15:15.430
>> [laughter]
00:15:15.440 --> 00:15:16.870
>> That that's a triggering comment, I
00:15:16.880 --> 00:15:18.310
know.
00:15:18.320 --> 00:15:19.870
>> My favorite
00:15:19.880 --> 00:15:21.630
uh thing is like I've used these
00:15:21.640 --> 00:15:24.190
automated column description like tools.
00:15:24.200 --> 00:15:26.350
I've tried this. And my my data
00:15:26.360 --> 00:15:28.030
governance analyst Cindy calls these
00:15:28.040 --> 00:15:29.710
cheeseburger definitions because they'll
00:15:29.720 --> 00:15:31.830
be like, "What's a hamburger? It's a I
00:15:31.840 --> 00:15:32.990
mean, a cheeseburger is a hamburger with
00:15:33.000 --> 00:15:34.950
cheese." And so, you're like, "Thanks,
00:15:34.960 --> 00:15:36.470
Claude. Thanks for writing that. Like, I
00:15:36.480 --> 00:15:37.990
could have inferred that in 2 seconds.
00:15:38.000 --> 00:15:39.230
Like, that doesn't tell me anything."
00:15:39.240 --> 00:15:41.830
And so, the real context people need is
00:15:41.840 --> 00:15:44.070
so much richer than that of like, you
00:15:44.080 --> 00:15:46.190
know, this column
00:15:46.200 --> 00:15:46.870
um
00:15:46.880 --> 00:15:49.350
contains the, you know, medication
00:15:49.360 --> 00:15:53.470
codes, but the codes shifted in 1992, so
00:15:53.480 --> 00:15:55.470
don't don't try to do time series across
00:15:55.480 --> 00:15:56.710
them, and you have to normalize them.
00:15:56.720 --> 00:15:57.630
But, if you're doing it at this time
00:15:57.640 --> 00:15:58.990
period, it's fine. Like, that richer
00:15:59.000 --> 00:16:00.230
context
00:16:00.240 --> 00:16:02.110
uh we are not to a place where the raw
00:16:02.120 --> 00:16:04.190
materials of that context are accessible
00:16:04.200 --> 00:16:05.510
to Claude yet. So, Claude can't write
00:16:05.520 --> 00:16:06.950
good descriptions yet. Claude's
00:16:06.960 --> 00:16:08.630
perfectly capable of summarizing and
00:16:08.640 --> 00:16:10.750
providing good syntax. The problem is
00:16:10.760 --> 00:16:13.190
still that context engineering problem.
00:16:13.200 --> 00:16:15.350
>> Yeah, and also storage. So,
00:16:15.360 --> 00:16:17.350
you know, natural reaction for me is to
00:16:17.360 --> 00:16:19.550
store that extended context somewhere
00:16:19.560 --> 00:16:21.550
else, right, outside of the physical
00:16:21.560 --> 00:16:23.230
physical structure of the data. But
00:16:23.240 --> 00:16:25.270
then, as I'm building things now, I've
00:16:25.280 --> 00:16:27.670
got to get the the gen AI tools to
00:16:27.680 --> 00:16:29.910
recognize, well, the structural context
00:16:29.920 --> 00:16:32.430
is here, and then the business context,
00:16:32.440 --> 00:16:34.390
that extended context is over there. And
00:16:34.400 --> 00:16:35.310
now, it's got to couple the two
00:16:35.320 --> 00:16:37.350
together. Whereas, if I take that
00:16:37.360 --> 00:16:39.230
extended context and I bound it to the
00:16:39.240 --> 00:16:41.470
physical stuff, it kind of sees it for
00:16:41.480 --> 00:16:44.190
free, right, to a degree. Um so, there's
00:16:44.200 --> 00:16:45.670
a whole lot of architectural decisions
00:16:45.680 --> 00:16:47.510
now about, well, and consequences. If I
00:16:47.520 --> 00:16:48.829
store it over there, it makes sense.
00:16:48.839 --> 00:16:50.910
I've got, you know, formal of a wiki or
00:16:50.920 --> 00:16:52.990
context store or whatever. But now,
00:16:53.000 --> 00:16:54.310
architecturally, I've got to make sure
00:16:54.320 --> 00:16:55.870
that whatever tool is using it is going
00:16:55.880 --> 00:16:58.270
to see both. What what I say to people
00:16:58.280 --> 00:17:00.870
now is um reduce the number of
00:17:00.880 --> 00:17:02.350
uncertainties.
00:17:02.360 --> 00:17:04.069
So, because we're all experimenting, cuz
00:17:04.079 --> 00:17:06.030
there's no good practice, and there's no
00:17:06.040 --> 00:17:07.550
proven architectures, everybody's
00:17:07.560 --> 00:17:08.990
pioneering, and we're all learning from
00:17:09.000 --> 00:17:11.030
each other, uh there's too many moving
00:17:11.040 --> 00:17:14.150
parts. And so, just pick some and make
00:17:14.160 --> 00:17:15.470
them certainties. Like, we're going to
00:17:15.480 --> 00:17:17.990
start off with a centralized data store,
00:17:18.000 --> 00:17:19.390
right? For now, we're not going to
00:17:19.400 --> 00:17:21.510
decentralize it um because that
00:17:21.520 --> 00:17:23.470
increases the complexity and the risk so
00:17:23.480 --> 00:17:25.030
much. And then we're going to we're
00:17:25.040 --> 00:17:26.910
going to do this. And therefore, the bit
00:17:26.920 --> 00:17:28.630
that we're not sure about is this space,
00:17:28.640 --> 00:17:29.910
and we have some hypotheses, and we're
00:17:29.920 --> 00:17:31.750
going to go test just that space, and
00:17:31.760 --> 00:17:32.670
we're going to try and make sure that
00:17:32.680 --> 00:17:34.430
nothing else moves while we're doing it.
00:17:34.440 --> 00:17:35.510
And then you kind of again, you just
00:17:35.520 --> 00:17:36.710
LEGO block it, right? You go, "That
00:17:36.720 --> 00:17:38.510
worked, right?" Uh the other thing I'd
00:17:38.520 --> 00:17:40.510
say is don't say data mesh anymore, say
00:17:40.520 --> 00:17:42.230
decentralized.
00:17:42.240 --> 00:17:44.110
Um you know, what do we want to
00:17:44.120 --> 00:17:45.630
decentralize and what do we want to
00:17:45.640 --> 00:17:47.830
centralize? Because that's the language
00:17:47.840 --> 00:17:50.430
uh that business leaders or stakeholders
00:17:50.440 --> 00:17:51.870
understand. And we'll kind of come back
00:17:51.880 --> 00:17:52.870
to language in a minute, but we've got a
00:17:52.880 --> 00:17:55.030
question here um from Raul, so I'll read
00:17:55.040 --> 00:17:56.510
it out. Uh I was thinking about data
00:17:56.520 --> 00:17:58.550
quality. I was thinking of using the
00:17:58.560 --> 00:18:01.270
test etc. 2.0.
00:18:01.280 --> 00:18:02.150
Uh
00:18:02.160 --> 00:18:04.790
and its application data to data. What
00:18:04.800 --> 00:18:07.190
are your thoughts? Uh I have absolutely
00:18:07.200 --> 00:18:10.630
no clue what that is.
00:18:10.640 --> 00:18:12.350
Do you?
00:18:12.360 --> 00:18:14.190
>> I'm looking.
00:18:14.200 --> 00:18:17.350
I do not, but um I think we can talk
00:18:17.360 --> 00:18:21.950
about data quality uh in general.
00:18:21.960 --> 00:18:24.590
Um I I'm on my mobile, so I can't pull
00:18:24.600 --> 00:18:27.750
up the link in a in a side um
00:18:27.760 --> 00:18:30.310
monitor. I would say
00:18:30.320 --> 00:18:33.310
um
00:18:33.320 --> 00:18:34.350
I would
00:18:34.360 --> 00:18:37.270
Here's what I'll say um where my my head
00:18:37.280 --> 00:18:40.310
is at with data quality right now is
00:18:40.320 --> 00:18:43.510
a lot thinking about um
00:18:43.520 --> 00:18:46.030
semantic data quality. Like, do we know
00:18:46.040 --> 00:18:48.230
what this word means? And do we agree as
00:18:48.240 --> 00:18:49.630
an enterprise what this word means? And
00:18:49.640 --> 00:18:51.070
if we finally have done the hard work of
00:18:51.080 --> 00:18:52.430
the enterprise of agreeing as humans,
00:18:52.440 --> 00:18:54.310
this is our common language.
00:18:54.320 --> 00:18:56.430
Is that version controlled storage? Are
00:18:56.440 --> 00:18:59.590
there systems to record that decision?
00:18:59.600 --> 00:19:01.270
So, we don't remake that decision in 18
00:19:01.280 --> 00:19:02.710
months when we all come back or new
00:19:02.720 --> 00:19:04.150
people come back and all this stuff. And
00:19:04.160 --> 00:19:06.270
so, I'm thinking about this a lot. I
00:19:06.280 --> 00:19:08.470
think and and part of it is the timing
00:19:08.480 --> 00:19:11.470
of of my new role. I joined this company
00:19:11.480 --> 00:19:13.910
uh in in September, and we got a new
00:19:13.920 --> 00:19:15.270
electronic health record system in
00:19:15.280 --> 00:19:18.070
March, which at a hospital means 80% of
00:19:18.080 --> 00:19:20.510
our data moved, the source system
00:19:20.520 --> 00:19:23.550
migrated. So, it's a massive, massive,
00:19:23.560 --> 00:19:26.150
uh, disruptive event. Um, but it also
00:19:26.160 --> 00:19:27.670
means it's an opportune moment to
00:19:27.680 --> 00:19:29.990
revisit like things like, okay, this new
00:19:30.000 --> 00:19:32.230
electronic health record says a visit, a
00:19:32.240 --> 00:19:34.990
hospital visit, is X or an encounter is
00:19:35.000 --> 00:19:36.390
X, but in the old system it was
00:19:36.400 --> 00:19:37.630
different. And then people have
00:19:37.640 --> 00:19:39.030
different conceptions in the hospital of
00:19:39.040 --> 00:19:40.390
those two things mean. And that affects
00:19:40.400 --> 00:19:41.670
a lot of metrics at the hospital because
00:19:41.680 --> 00:19:43.070
you care about
00:19:43.080 --> 00:19:44.630
visit volumes and admissions and
00:19:44.640 --> 00:19:45.910
readmissions and all these different
00:19:45.920 --> 00:19:47.350
things. And so there we're having these
00:19:47.360 --> 00:19:49.950
like workshops, many hours with very,
00:19:49.960 --> 00:19:51.750
uh, expensive senior people deciding
00:19:51.760 --> 00:19:54.510
what these words mean. And I'm thinking,
00:19:54.520 --> 00:19:57.390
now where do I record that so that
00:19:57.400 --> 00:19:58.910
two different analysts in the hospital
00:19:58.920 --> 00:20:00.670
don't, six months from now, build
00:20:00.680 --> 00:20:02.710
dashboards that
00:20:02.720 --> 00:20:04.230
don't agree with each other and restart
00:20:04.240 --> 00:20:05.670
this loop again where leaders are going,
00:20:05.680 --> 00:20:07.590
"Wait a second. What What are we saying
00:20:07.600 --> 00:20:09.830
a visit is again?" And so like that's
00:20:09.840 --> 00:20:11.070
where everything about data quality
00:20:11.080 --> 00:20:13.910
right now is like, uh, meaning quality.
00:20:13.920 --> 00:20:15.350
And then once we know what things mean
00:20:15.360 --> 00:20:17.030
or what our target state is, then we can
00:20:17.040 --> 00:20:19.230
look at just the data match our
00:20:19.240 --> 00:20:21.830
expectation of reality. And then if the
00:20:21.840 --> 00:20:24.230
data doesn't match our expectation, um,
00:20:24.240 --> 00:20:26.350
I'm also thinking a lot,
00:20:26.360 --> 00:20:28.070
um,
00:20:28.080 --> 00:20:31.870
about how the data team can ship
00:20:31.880 --> 00:20:33.550
data quality context back to source
00:20:33.560 --> 00:20:35.430
system to fix it at the source rather
00:20:35.440 --> 00:20:37.190
than trying to just do analytical fixes
00:20:37.200 --> 00:20:39.430
for data quality downstream. So, data
00:20:39.440 --> 00:20:43.070
quality has always kind of been, um,
00:20:43.080 --> 00:20:45.190
the idea of like automated data quality
00:20:45.200 --> 00:20:47.070
audits doesn't really appeal to me a
00:20:47.080 --> 00:20:49.230
whole lot because I feel like
00:20:49.240 --> 00:20:50.670
if no one's prepared to act on it, we're
00:20:50.680 --> 00:20:52.590
just spending money to discover problems
00:20:52.600 --> 00:20:54.510
in our data. Um,
00:20:54.520 --> 00:20:56.790
so I've been working more on the
00:20:56.800 --> 00:20:59.070
business problems and expectation
00:20:59.080 --> 00:21:00.870
setting so we can do targeted data
00:21:00.880 --> 00:21:02.950
quality audits and having a plan
00:21:02.960 --> 00:21:04.390
structurally or
00:21:04.400 --> 00:21:06.710
with, uh, between these two orgs because
00:21:06.720 --> 00:21:10.110
of the source system is owned by IT
00:21:10.120 --> 00:21:11.670
and the analytical system is owned by
00:21:11.680 --> 00:21:13.550
data. So, it's not just, can we surface
00:21:13.560 --> 00:21:15.470
a problem and pass it from this IC to
00:21:15.480 --> 00:21:17.910
this IC? We're saying, how do these two
00:21:17.920 --> 00:21:20.950
organizations now coordinate, uh, change
00:21:20.960 --> 00:21:24.510
across source and analytics? So, I'm
00:21:24.520 --> 00:21:26.110
looking at data quality more from the
00:21:26.120 --> 00:21:28.430
ownership, like architectural pattern
00:21:28.440 --> 00:21:29.670
problem of how we're going to work on
00:21:29.680 --> 00:21:31.790
it. And then once we get there, writing
00:21:31.800 --> 00:21:33.070
the test itself and uncovering the
00:21:33.080 --> 00:21:36.150
problems is, to me, the easy part.
00:21:36.160 --> 00:21:38.190
>> Yeah, cuz I'm doing a lot of work in,
00:21:38.200 --> 00:21:40.030
you know, we're building out our harness
00:21:40.040 --> 00:21:41.390
to help me do the data work a little bit
00:21:41.400 --> 00:21:43.510
quicker than I used to. And
00:21:43.520 --> 00:21:45.030
so, it's all running back into our
00:21:45.040 --> 00:21:46.590
product, but I'm using Cloud Code at the
00:21:46.600 --> 00:21:48.710
front end to to help me kind of do the
00:21:48.720 --> 00:21:51.070
work. Um, and what's interesting is the
00:21:51.080 --> 00:21:52.390
amount of stuff it does under the covers
00:21:52.400 --> 00:21:54.550
without telling me. So,
00:21:54.560 --> 00:21:56.550
one of the things, yeah, as as like I've
00:21:56.560 --> 00:21:58.590
got to keep telling it to It's not
00:21:58.600 --> 00:22:01.350
lying, it's just doing it. Um, so, some
00:22:01.360 --> 00:22:03.430
examples. So, whenever we bring data
00:22:03.440 --> 00:22:04.670
into our platform, we automatically
00:22:04.680 --> 00:22:06.190
profile all of it. We have a a set of
00:22:06.200 --> 00:22:08.350
profiling stats, and every row that
00:22:08.360 --> 00:22:10.310
comes in in every table, uh, as soon as
00:22:10.320 --> 00:22:12.630
there's a load, it gets recalculated.
00:22:12.640 --> 00:22:13.790
Um,
00:22:13.800 --> 00:22:15.790
but and there's an MCP access to it and
00:22:15.800 --> 00:22:17.470
a bunch of APIs, so Cloud knows that
00:22:17.480 --> 00:22:20.750
that profile stats exist. Yet, actually,
00:22:20.760 --> 00:22:22.390
whenever it tries to do the work, it'll
00:22:22.400 --> 00:22:24.870
go re-profile the data itself. And I'm
00:22:24.880 --> 00:22:25.990
like,
00:22:26.000 --> 00:22:28.750
why why are you doing that? Um, and then
00:22:28.760 --> 00:22:30.510
it does some really interesting things.
00:22:30.520 --> 00:22:32.430
Uh, so, with one of my demo data sets,
00:22:32.440 --> 00:22:35.510
it recognized that, uh, Auckland had,
00:22:35.520 --> 00:22:38.070
uh, various versions at different cases.
00:22:38.080 --> 00:22:40.110
Um, so, it right-cased it without
00:22:40.120 --> 00:22:41.350
telling me.
00:22:41.360 --> 00:22:44.230
Like, just did it. Uh, which is good and
00:22:44.240 --> 00:22:46.550
bad, right? Cuz it's Maybe I wanted the
00:22:46.560 --> 00:22:49.110
cases to to We're doing some work in the
00:22:49.120 --> 00:22:50.990
media space at the moment. And so,
00:22:51.000 --> 00:22:52.430
you'll have, uh, content title, you have
00:22:52.440 --> 00:22:54.350
a TV show, and then you'll have a rerun
00:22:54.360 --> 00:22:56.630
of it, where it's basically, um,
00:22:56.640 --> 00:22:57.710
uh,
00:22:57.720 --> 00:22:59.750
break it up. Um, and for our customer,
00:22:59.760 --> 00:23:01.110
it's really important that they get
00:23:01.120 --> 00:23:04.270
counts by uh, first thing, um, by
00:23:04.280 --> 00:23:07.470
re-runs and by both. But interestingly,
00:23:07.480 --> 00:23:09.070
cool thing first thing it did was it
00:23:09.080 --> 00:23:10.830
took off the odds.
00:23:10.840 --> 00:23:12.350
So it
00:23:12.360 --> 00:23:14.110
you want to and it didn't tell me,
00:23:14.120 --> 00:23:15.470
right? And so
00:23:15.480 --> 00:23:16.390
um
00:23:16.400 --> 00:23:18.590
I I think yeah, we got to be a little
00:23:18.600 --> 00:23:19.750
bit careful about what the tooling's
00:23:19.760 --> 00:23:22.670
doing without us seeing it. And so
00:23:22.680 --> 00:23:24.350
profiling's really good, data quality
00:23:24.360 --> 00:23:25.630
metrics are really good for it, but you
00:23:25.640 --> 00:23:27.110
have to make sure it uses it. And then I
00:23:27.120 --> 00:23:28.470
agree with you, actually the biggest
00:23:28.480 --> 00:23:30.590
data quality problem is
00:23:30.600 --> 00:23:31.990
uh what does it mean?
00:23:32.000 --> 00:23:33.550
You won't always use this. Rinke Hoekman
00:23:33.560 --> 00:23:35.070
did some work for
00:23:35.080 --> 00:23:37.750
a hospital in the Netherlands and
00:23:37.760 --> 00:23:40.710
he had to get definition of a bed.
00:23:40.720 --> 00:23:42.710
Um and
00:23:42.720 --> 00:23:44.230
what he said was you know every he
00:23:44.240 --> 00:23:45.230
thought this was going to be the easy
00:23:45.240 --> 00:23:46.430
one.
00:23:46.440 --> 00:23:48.350
But some of the organization thought it
00:23:48.360 --> 00:23:49.990
was a place a patient was, so it was
00:23:50.000 --> 00:23:52.230
effectively a room.
00:23:52.240 --> 00:23:53.270
And then the other part of the
00:23:53.280 --> 00:23:54.710
organization thought it was the metal
00:23:54.720 --> 00:23:56.110
thing that you lied on that they moved
00:23:56.120 --> 00:23:57.350
around. That was maintenance, right? You
00:23:57.360 --> 00:23:58.710
know, we're going to remove that bed and
00:23:58.720 --> 00:24:00.990
fix it versus how many beds do we have
00:24:01.000 --> 00:24:02.110
available?
00:24:02.120 --> 00:24:02.870
Um
00:24:02.880 --> 00:24:04.150
So yeah, I'm with you. It's the it's the
00:24:04.160 --> 00:24:05.790
definition. So
00:24:05.800 --> 00:24:08.190
I definitely do data quality work still.
00:24:08.200 --> 00:24:08.870
Um
00:24:08.880 --> 00:24:10.310
but I wouldn't do it as a stock take
00:24:10.320 --> 00:24:12.030
anymore. You know, that idea of going
00:24:12.040 --> 00:24:13.870
out and doing a data quality project, I
00:24:13.880 --> 00:24:15.310
would make it an And then the other
00:24:15.320 --> 00:24:17.510
thing that I would say is
00:24:17.520 --> 00:24:19.430
your data engineers
00:24:19.440 --> 00:24:21.350
are actually the people that often
00:24:21.360 --> 00:24:23.350
uncover the structural data quality
00:24:23.360 --> 00:24:26.190
issues every time. So they should be
00:24:26.200 --> 00:24:27.430
documenting those in a way that
00:24:27.440 --> 00:24:28.710
everybody can see. And then like you
00:24:28.720 --> 00:24:31.310
said, ideally pass it back to the system
00:24:31.320 --> 00:24:33.270
that owns the creation of that quality
00:24:33.280 --> 00:24:35.510
issue. But I really see that. I really
00:24:35.520 --> 00:24:37.070
see it going back into those source
00:24:37.080 --> 00:24:38.910
systems, especially in enterprise,
00:24:38.920 --> 00:24:41.350
right? Because it's just normally
00:24:41.360 --> 00:24:42.590
somebody else's software and it's
00:24:42.600 --> 00:24:45.230
normally too hard to configure.
00:24:45.240 --> 00:24:45.590
Yeah.
00:24:45.600 --> 00:24:47.110
>> Yeah.
00:24:47.120 --> 00:24:47.550
Absolutely.
00:24:47.560 --> 00:24:48.710
>> All right.
00:24:48.720 --> 00:24:50.430
Um got another question from Raul. We
00:24:50.440 --> 00:24:52.590
had a major price structure change from
00:24:52.600 --> 00:24:54.910
usage-based pricing to a mix of
00:24:54.920 --> 00:24:56.390
usage-based pricing and
00:24:56.400 --> 00:24:58.430
subscription-based pricing. So, the
00:24:58.440 --> 00:25:00.630
meaning uh confirmation with the leaders
00:25:00.640 --> 00:25:01.830
rings true.
00:25:01.840 --> 00:25:04.190
Yeah, um cuz like you said, it's not
00:25:04.200 --> 00:25:06.190
only about capturing the context of
00:25:06.200 --> 00:25:08.790
those terms and those definitions now,
00:25:08.800 --> 00:25:11.030
it's also capturing the context as it
00:25:11.040 --> 00:25:13.510
changes over time. And then
00:25:13.520 --> 00:25:15.270
yeah, we've almost now got time series
00:25:15.280 --> 00:25:16.550
data,
00:25:16.560 --> 00:25:19.030
uh time series business logic, and time
00:25:19.040 --> 00:25:21.230
series definitions, right? And they're
00:25:21.240 --> 00:25:22.830
all going to be moving at different
00:25:22.840 --> 00:25:24.310
times. And that that kind of makes it
00:25:24.320 --> 00:25:26.750
even more complex.
00:25:26.760 --> 00:25:30.310
>> Oh, yeah. It's very uh
00:25:30.320 --> 00:25:32.070
you know, the one I'm really interested
00:25:32.080 --> 00:25:33.870
in lately
00:25:33.880 --> 00:25:34.470
um
00:25:34.480 --> 00:25:39.070
is the finance use case, which is um
00:25:39.080 --> 00:25:41.630
finance cares what the data meant at a
00:25:41.640 --> 00:25:45.110
point in time. And so, finance doesn't
00:25:45.120 --> 00:25:47.910
doesn't just care what happened in June.
00:25:47.920 --> 00:25:50.070
They cared what hap- what it looked like
00:25:50.080 --> 00:25:52.030
happened in June as of July 1st. They
00:25:52.040 --> 00:25:53.710
don't want to know what it looks like
00:25:53.720 --> 00:25:55.270
today 6 months later. They want to know
00:25:55.280 --> 00:25:56.470
what it looked like on the day the books
00:25:56.480 --> 00:25:58.510
closed. They don't [snorts] want any
00:25:58.520 --> 00:26:00.590
after or they need to be able to look at
00:26:00.600 --> 00:26:02.190
both and see if there's a delta between
00:26:02.200 --> 00:26:03.990
the two. And so,
00:26:04.000 --> 00:26:05.270
I'm interested in that problem because I
00:26:05.280 --> 00:26:06.470
feel like it's in
00:26:06.480 --> 00:26:08.870
that reconstructing
00:26:08.880 --> 00:26:10.670
what this table looked like before any
00:26:10.680 --> 00:26:12.350
changes happened after a certain time
00:26:12.360 --> 00:26:14.590
point, right? Being able to reconstruct
00:26:14.600 --> 00:26:16.550
the state of a table
00:26:16.560 --> 00:26:19.030
over time and not just the time period
00:26:19.040 --> 00:26:22.150
itself is a problem we haven't really
00:26:22.160 --> 00:26:23.390
solved for in the companies I've been
00:26:23.400 --> 00:26:25.110
working at. And the reason that's
00:26:25.120 --> 00:26:27.990
interesting is because it leads finance
00:26:28.000 --> 00:26:29.310
to do things
00:26:29.320 --> 00:26:31.150
in their own analytical systems and in
00:26:31.160 --> 00:26:33.030
spreadsheets because they can pull a
00:26:33.040 --> 00:26:34.750
timestamp, and they know that data set
00:26:34.760 --> 00:26:36.830
they've pulled is frozen.
00:26:36.840 --> 00:26:38.390
Um and if we don't solve that problem
00:26:38.400 --> 00:26:39.550
for them in the warehouse, they're going
00:26:39.560 --> 00:26:42.750
to keep building fragile quasi data
00:26:42.760 --> 00:26:45.550
pipelines in spreadsheets or in FP&A
00:26:45.560 --> 00:26:47.550
tools. And sometimes those tools may be
00:26:47.560 --> 00:26:49.150
the right and appropriate way to do it,
00:26:49.160 --> 00:26:50.710
but the problem is
00:26:50.720 --> 00:26:53.110
it's often siloed and not documented and
00:26:53.120 --> 00:26:54.190
single point of failure and all the
00:26:54.200 --> 00:26:55.830
problems that we get in finance and I
00:26:55.840 --> 00:26:57.470
just had this aha moment one day where I
00:26:57.480 --> 00:26:58.950
went
00:26:58.960 --> 00:27:00.790
it isn't always just that they find the
00:27:00.800 --> 00:27:02.590
interface of the spreadsheet more
00:27:02.600 --> 00:27:04.710
pleasant to manipulate in cuz of course
00:27:04.720 --> 00:27:06.630
they do they they pivot tables are very
00:27:06.640 --> 00:27:08.390
intuitive formulas you've memorized from
00:27:08.400 --> 00:27:09.950
years and years ago
00:27:09.960 --> 00:27:11.830
but you could buy a tool like Sigma and
00:27:11.840 --> 00:27:13.670
have the spreadsheet interface the other
00:27:13.680 --> 00:27:16.310
reason they like it is not just the
00:27:16.320 --> 00:27:19.110
interface it's the ability to control
00:27:19.120 --> 00:27:21.830
and to know that data set they're using
00:27:21.840 --> 00:27:23.150
is frozen at this point in time that
00:27:23.160 --> 00:27:24.590
they care about so we either need to do
00:27:24.600 --> 00:27:26.550
snapshots or we need a way to ad hoc
00:27:26.560 --> 00:27:28.310
reconstruct the state of a table at a
00:27:28.320 --> 00:27:30.710
point in time so I'm very interested in
00:27:30.720 --> 00:27:32.310
that problem space because I feel like
00:27:32.320 --> 00:27:33.510
it's
00:27:33.520 --> 00:27:35.990
not talked about enough and finance is
00:27:36.000 --> 00:27:37.550
you know if there's
00:27:37.560 --> 00:27:39.310
well the domain we never want to touch
00:27:39.320 --> 00:27:41.230
as a data team it's finance and if there
00:27:41.240 --> 00:27:42.790
was a domain that's the most valuable to
00:27:42.800 --> 00:27:44.750
the company to be correct and reliable
00:27:44.760 --> 00:27:47.990
it would be finance and so I'm very
00:27:48.000 --> 00:27:49.750
intrigued by the finance use case I'm
00:27:49.760 --> 00:27:51.510
not going to say I've I've solved for
00:27:51.520 --> 00:27:53.110
this problem but having the awareness of
00:27:53.120 --> 00:27:55.550
like why is it so hard
00:27:55.560 --> 00:27:57.710
why do the finance team so resistant to
00:27:57.720 --> 00:27:59.110
using the data platform why are they
00:27:59.120 --> 00:28:01.110
still doing things in these what feel
00:28:01.120 --> 00:28:04.350
antiquated ways and then I realized
00:28:04.360 --> 00:28:05.870
it's cuz we're not solving one of the
00:28:05.880 --> 00:28:08.190
biggest problems for them
00:28:08.200 --> 00:28:10.150
>> yeah this this is dependent on by two in
00:28:10.160 --> 00:28:12.030
reality
00:28:12.040 --> 00:28:14.230
so you see a lot of that in the European
00:28:14.240 --> 00:28:16.190
data data platforms that have been built
00:28:16.200 --> 00:28:18.630
because they they go to that level with
00:28:18.640 --> 00:28:20.790
complexity basically and and so it
00:28:20.800 --> 00:28:23.870
allows you to say what did it look like
00:28:23.880 --> 00:28:25.910
at the data happened
00:28:25.920 --> 00:28:27.150
what did it look like with today's
00:28:27.160 --> 00:28:29.310
structures as at date and then what does
00:28:29.320 --> 00:28:30.990
it look like for the data that happened
00:28:31.000 --> 00:28:33.070
based on any other structure
00:28:33.080 --> 00:28:34.830
but that's just for data right it's
00:28:34.840 --> 00:28:36.950
quite complex to build and then it gets
00:28:36.960 --> 00:28:38.430
worse you know as you said you've done a
00:28:38.440 --> 00:28:41.110
data migration to a new platform so when
00:28:41.120 --> 00:28:43.190
that data comes in it's like is it
00:28:43.200 --> 00:28:46.070
the date that the record was created in
00:28:46.080 --> 00:28:48.150
the new system probably not because of
00:28:48.160 --> 00:28:49.630
their business effective date for that
00:28:49.640 --> 00:28:51.590
record was probably years ago because
00:28:51.600 --> 00:28:53.670
now it's a migration. So, that that's a
00:28:53.680 --> 00:28:56.350
really complex problem. Um, but then we
00:28:56.360 --> 00:28:58.830
overlay context to it. Okay, what was
00:28:58.840 --> 00:29:03.070
the definition of that, you know, margin
00:29:03.080 --> 00:29:04.190
uh on that date and what's the
00:29:04.200 --> 00:29:07.110
definition now? And, you know,
00:29:07.120 --> 00:29:08.510
do we want to be able to report the
00:29:08.520 --> 00:29:10.550
definition of margin as it now for that
00:29:10.560 --> 00:29:13.510
data and as the data today as it
00:29:13.520 --> 00:29:14.790
the definition back then, right? So,
00:29:14.800 --> 00:29:17.510
yeah, it's it's the kind of like a 3D
00:29:17.520 --> 00:29:20.510
matrix now um where, you know, it's more
00:29:20.520 --> 00:29:22.590
than just the data that's going to have
00:29:22.600 --> 00:29:25.230
uh temporality, yeah, or temporal, yeah.
00:29:25.240 --> 00:29:26.350
>> The other
00:29:26.360 --> 00:29:28.870
um thing that I'm very thinking a lot
00:29:28.880 --> 00:29:30.470
about lately um
00:29:30.480 --> 00:29:32.230
we just went through a data catalog
00:29:32.240 --> 00:29:34.590
vendor evaluation cuz we're up for
00:29:34.600 --> 00:29:37.630
renewal with our our current catalog.
00:29:37.640 --> 00:29:40.830
Is like um if you think of a catalog as
00:29:40.840 --> 00:29:43.150
the catalog's role is expanding in the
00:29:43.160 --> 00:29:45.510
state of AI, right? As catalog becomes a
00:29:45.520 --> 00:29:47.470
source system for context, not the
00:29:47.480 --> 00:29:49.070
source system, but a source system for
00:29:49.080 --> 00:29:51.710
context and how does that interoperate
00:29:51.720 --> 00:29:54.470
with the other places context is shared
00:29:54.480 --> 00:29:56.390
or created? Um
00:29:56.400 --> 00:29:58.030
So, we've been talking about, you know,
00:29:58.040 --> 00:29:59.390
how will this interact with the semantic
00:29:59.400 --> 00:30:01.990
layer downstream of it? Um
00:30:02.000 --> 00:30:03.790
you the cat the catalog becomes the
00:30:03.800 --> 00:30:05.670
source of truth for like this definition
00:30:05.680 --> 00:30:06.950
of visits and encounters we're working
00:30:06.960 --> 00:30:08.790
on at the hospital right now. Once we
00:30:08.800 --> 00:30:10.790
decide this, we will document this,
00:30:10.800 --> 00:30:13.190
right? And the definition and where it
00:30:13.200 --> 00:30:15.110
lives and then the canonical calculation
00:30:15.120 --> 00:30:16.830
downstream, which will live in YAML or,
00:30:16.840 --> 00:30:18.110
you know, all the things that we need to
00:30:18.120 --> 00:30:19.230
do.
00:30:19.240 --> 00:30:21.070
But, the interesting discussion I had
00:30:21.080 --> 00:30:22.950
with my boss maybe last week or the week
00:30:22.960 --> 00:30:25.070
before was
00:30:25.080 --> 00:30:27.030
you know, at the hospital, it's really
00:30:27.040 --> 00:30:28.790
interesting to have one source system
00:30:28.800 --> 00:30:30.150
that's so dominant because you can go
00:30:30.160 --> 00:30:32.230
really deep on like how does this system
00:30:32.240 --> 00:30:33.230
evolve?
00:30:33.240 --> 00:30:35.390
Where usually we're juggling so many
00:30:35.400 --> 00:30:37.950
systems in like my prior role, you know,
00:30:37.960 --> 00:30:40.230
like dozens of integrations that were
00:30:40.240 --> 00:30:41.510
like
00:30:41.520 --> 00:30:43.550
we never have the luxury of going deep
00:30:43.560 --> 00:30:45.510
on any of them. But at the hospital, 80%
00:30:45.520 --> 00:30:46.550
of the data, it's like how do we get
00:30:46.560 --> 00:30:48.390
this really, really right the way we
00:30:48.400 --> 00:30:50.510
bring in data from the system?
00:30:50.520 --> 00:30:52.750
Um and since the system is brand new, it
00:30:52.760 --> 00:30:54.630
becomes a really interesting moment of
00:30:54.640 --> 00:30:56.910
like we don't have
00:30:56.920 --> 00:30:58.670
a bunch of how things change over time
00:30:58.680 --> 00:31:00.110
in this system cuz the system only
00:31:00.120 --> 00:31:03.590
existed on March 1st to now. And so
00:31:03.600 --> 00:31:05.030
we have this opportunity to be like how
00:31:05.040 --> 00:31:06.950
are we going to manage this system
00:31:06.960 --> 00:31:08.590
effectively? And one thing I've been
00:31:08.600 --> 00:31:11.110
talking about with my boss is
00:31:11.120 --> 00:31:12.910
what if the catalog isn't just where the
00:31:12.920 --> 00:31:15.950
data team stores or the catalog in the
00:31:15.960 --> 00:31:18.390
broader context system, the catalog is
00:31:18.400 --> 00:31:20.350
one hub of it at least, and then the
00:31:20.360 --> 00:31:21.870
semantic layer of the YAML by the
00:31:21.880 --> 00:31:23.950
warehouse is another, and then the EHR's
00:31:23.960 --> 00:31:26.110
own context layer is its its own
00:31:26.120 --> 00:31:27.990
knowledge base as part of it.
00:31:28.000 --> 00:31:29.950
But what if the team that designed the
00:31:29.960 --> 00:31:31.870
workflows in the catalog, right? And
00:31:31.880 --> 00:31:34.990
sorry, in the EHR, when they design it,
00:31:35.000 --> 00:31:36.750
they're talking to the same stakeholders
00:31:36.760 --> 00:31:38.150
we're going to talk to for analytics
00:31:38.160 --> 00:31:39.230
being like
00:31:39.240 --> 00:31:40.870
how do you need this workflow to work
00:31:40.880 --> 00:31:42.910
when a patient comes in for surgery and
00:31:42.920 --> 00:31:44.190
you're going through these fields and
00:31:44.200 --> 00:31:45.990
you're doing all this checking?
00:31:46.000 --> 00:31:47.270
We are later going to go try to make
00:31:47.280 --> 00:31:50.230
sense of that data and need to know
00:31:50.240 --> 00:31:52.030
what the tradeoffs were, what the mental
00:31:52.040 --> 00:31:53.790
model of like the medical assistant
00:31:53.800 --> 00:31:54.910
doing the check-in is when they're
00:31:54.920 --> 00:31:56.550
filling in this field, where there's
00:31:56.560 --> 00:31:57.630
validation, where there's not
00:31:57.640 --> 00:31:59.390
validation. There's like all this
00:31:59.400 --> 00:32:03.110
context baked into the design stage. And
00:32:03.120 --> 00:32:05.310
that again, like I said, sits in IT and
00:32:05.320 --> 00:32:08.270
we sit in data. And I was like, what if
00:32:08.280 --> 00:32:10.870
what if we had a system where
00:32:10.880 --> 00:32:13.230
as they built it, they're, you know,
00:32:13.240 --> 00:32:14.790
recording these context gathering
00:32:14.800 --> 00:32:17.070
sessions and having AI summarize, you
00:32:17.080 --> 00:32:18.510
know, all this context that's learned at
00:32:18.520 --> 00:32:21.270
the time and contributing that to our
00:32:21.280 --> 00:32:22.950
our knowledge management layer, and then
00:32:22.960 --> 00:32:25.030
the actual design,
00:32:25.040 --> 00:32:26.790
how are they documenting what the system
00:32:26.800 --> 00:32:28.950
is? The screenshots, the fields, this is
00:32:28.960 --> 00:32:30.590
happening here, this is what you need to
00:32:30.600 --> 00:32:32.990
know. And then when we go to create
00:32:33.000 --> 00:32:35.430
analytics, before we go go to the
00:32:35.440 --> 00:32:37.590
stakeholders again, we can have our
00:32:37.600 --> 00:32:39.990
copilot or our Claude or whatever tool
00:32:40.000 --> 00:32:42.670
it is access all of that documentation
00:32:42.680 --> 00:32:45.110
from the source system team to say this
00:32:45.120 --> 00:32:47.190
is the meaning of this data, it lands in
00:32:47.200 --> 00:32:48.710
this table, this is where you can find
00:32:48.720 --> 00:32:49.870
it. And then we're going to the
00:32:49.880 --> 00:32:51.550
stakeholder being like
00:32:51.560 --> 00:32:52.870
all right, so I understand when someone
00:32:52.880 --> 00:32:54.150
comes in for surgery, the medical
00:32:54.160 --> 00:32:56.110
assistant is doing X.
00:32:56.120 --> 00:32:58.510
What I don't understand is how this
00:32:58.520 --> 00:33:00.230
works here. Or before this electronic
00:33:00.240 --> 00:33:01.390
health record, how did it work in the
00:33:01.400 --> 00:33:02.590
old system? And then we're going to have
00:33:02.600 --> 00:33:04.830
to figure out this historical problem.
00:33:04.840 --> 00:33:06.830
So I'm really thinking about
00:33:06.840 --> 00:33:09.270
how do we close the like tighten the
00:33:09.280 --> 00:33:10.910
feedback loop and the context sharing
00:33:10.920 --> 00:33:13.550
between source system maintainers and
00:33:13.560 --> 00:33:16.510
creators and the consumption because
00:33:16.520 --> 00:33:18.470
they both need the same context and it
00:33:18.480 --> 00:33:22.230
should be a shared connected system.
00:33:22.240 --> 00:33:23.990
Um, so that's what I'm really excited to
00:33:24.000 --> 00:33:25.990
like try to pick apart and solve this
00:33:26.000 --> 00:33:28.230
year at the hospital cuz I'm like
00:33:28.240 --> 00:33:30.510
why are we going back and having 6 hours
00:33:30.520 --> 00:33:31.990
of workshops again with these
00:33:32.000 --> 00:33:33.190
stakeholders to figure out their
00:33:33.200 --> 00:33:34.670
business context when someone else just
00:33:34.680 --> 00:33:36.950
talked to them about it 3 months ago to
00:33:36.960 --> 00:33:38.470
build the system? Like that doesn't make
00:33:38.480 --> 00:33:40.150
any sense.
00:33:40.160 --> 00:33:41.710
>> Yeah, this is something cuz you have and
00:33:41.720 --> 00:33:43.190
I are writing a book on modeling
00:33:43.200 --> 00:33:44.790
business concepts and one of the things
00:33:44.800 --> 00:33:47.190
we came to is when we're talking to
00:33:47.200 --> 00:33:49.150
subject matter experts,
00:33:49.160 --> 00:33:50.790
uh, we're having a conversation and
00:33:50.800 --> 00:33:52.270
we're summarizing it. We're we're
00:33:52.280 --> 00:33:53.750
effectively sketching out a conceptual
00:33:53.760 --> 00:33:55.590
model as we talk to them.
00:33:55.600 --> 00:33:55.670
>> Yes.
00:33:55.680 --> 00:33:56.710
>> What we're not doing is actually
00:33:56.720 --> 00:33:58.470
recording the conversation. So we can
00:33:58.480 --> 00:33:59.830
never go back and replay the
00:33:59.840 --> 00:34:01.670
conversation to see whether we got it
00:34:01.680 --> 00:34:03.510
right or not. And so when you said
00:34:03.520 --> 00:34:05.390
summarizing the transcripts, I'd say
00:34:05.400 --> 00:34:07.870
keep the raw transcripts because I think
00:34:07.880 --> 00:34:09.669
you're going to need to go and replay
00:34:09.679 --> 00:34:12.430
that raw data, that raw conversation
00:34:12.440 --> 00:34:13.270
>> Yeah.
00:34:13.280 --> 00:34:14.869
>> different ways when things change or
00:34:14.879 --> 00:34:16.430
when you don't understand them. Then the
00:34:16.440 --> 00:34:18.470
question is how many people are willing
00:34:18.480 --> 00:34:20.950
to have their their knowledge and their
00:34:20.960 --> 00:34:22.110
wisdom
00:34:22.120 --> 00:34:23.790
documented because basically a lot of
00:34:23.800 --> 00:34:25.110
the time that's their value to an
00:34:25.120 --> 00:34:27.430
organization as a subject matter expert.
00:34:27.440 --> 00:34:28.790
Um, but I want to come back to this idea
00:34:28.800 --> 00:34:30.830
of of catalogs because what I'm seeing
00:34:30.840 --> 00:34:31.990
is
00:34:32.000 --> 00:34:33.669
you know, it's all evolving and we're
00:34:33.679 --> 00:34:35.550
not quite sure who's going to own the
00:34:35.560 --> 00:34:37.310
context plane or whether it's going to
00:34:37.320 --> 00:34:39.389
be a certain type of vendor or a certain
00:34:39.399 --> 00:34:42.070
vendor or like it is a race.
00:34:42.080 --> 00:34:43.629
I definitely can see some patterns. So,
00:34:43.639 --> 00:34:46.350
I can see catalog centric patterns.
00:34:46.360 --> 00:34:49.310
Um I can see platform centric patterns.
00:34:49.320 --> 00:34:51.389
Uh I can see BI semantic metric layer
00:34:51.399 --> 00:34:54.590
centric patterns. Uh I can see wiki kind
00:34:54.600 --> 00:34:57.190
of centric patterns. Um
00:34:57.200 --> 00:34:59.270
so, yeah, what's your view? Like did you
00:34:59.280 --> 00:35:01.550
go catalog cuz you had one? You've got
00:35:01.560 --> 00:35:04.070
funding for it. It makes sense to carry
00:35:04.080 --> 00:35:06.350
on there.
00:35:06.360 --> 00:35:08.350
>> This is so interesting.
00:35:08.360 --> 00:35:09.190
Um
00:35:09.200 --> 00:35:10.510
I probably shouldn't name the company,
00:35:10.520 --> 00:35:12.910
but um my daughter, my 10-year-old
00:35:12.920 --> 00:35:14.310
daughter has a friend and I was at the
00:35:14.320 --> 00:35:16.710
pool with her mom and her mom works on a
00:35:16.720 --> 00:35:18.790
governance team at a
00:35:18.800 --> 00:35:20.110
uh tech vendor.
00:35:20.120 --> 00:35:22.870
Um tech not vendor, tech
00:35:22.880 --> 00:35:24.390
domain company.
00:35:24.400 --> 00:35:25.630
Um and I work at a hospital, so very
00:35:25.640 --> 00:35:26.950
different domains. We both work in data
00:35:26.960 --> 00:35:28.590
and governance. And so, it's really fun
00:35:28.600 --> 00:35:30.030
to be uh our kids played in the pool
00:35:30.040 --> 00:35:32.230
while we were like talking shop. And we
00:35:32.240 --> 00:35:34.390
were talking about catalogs because
00:35:34.400 --> 00:35:36.270
we're both currently on the same catalog
00:35:36.280 --> 00:35:39.190
who shall not be named. Um
00:35:39.200 --> 00:35:40.550
big legacy player, you can probably
00:35:40.560 --> 00:35:43.670
guess. Um and both of us are are
00:35:43.680 --> 00:35:47.950
likely leaving that. Um but I work at a
00:35:47.960 --> 00:35:50.590
hospital that has a very
00:35:50.600 --> 00:35:53.710
concentrated smaller team and she works
00:35:53.720 --> 00:35:54.710
at a
00:35:54.720 --> 00:35:58.590
um we'll call it a you know, an IPO'd
00:35:58.600 --> 00:36:01.350
tech company that's mature. And so, the
00:36:01.360 --> 00:36:03.830
talent density and you know, the
00:36:03.840 --> 00:36:06.190
bandwidth they have to handle things is
00:36:06.200 --> 00:36:08.190
different. And so, when we have we
00:36:08.200 --> 00:36:10.870
evaluated build versus buy, which is
00:36:10.880 --> 00:36:12.430
really interesting the catalog space cuz
00:36:12.440 --> 00:36:14.470
historically you may not even evaluate
00:36:14.480 --> 00:36:16.430
build, right? It was like which of these
00:36:16.440 --> 00:36:17.710
vendors we're going to go with, not
00:36:17.720 --> 00:36:19.310
should we even have one.
00:36:19.320 --> 00:36:20.470
And I would say this is the first time
00:36:20.480 --> 00:36:22.110
I've been in a company that was like are
00:36:22.120 --> 00:36:24.110
we ready to maybe just, you know, do
00:36:24.120 --> 00:36:26.550
YAML and code and like
00:36:26.560 --> 00:36:28.790
do these things? Um anyway, in her
00:36:28.800 --> 00:36:31.830
company decided they were ready. And so,
00:36:31.840 --> 00:36:34.230
they're building something custom.
00:36:34.240 --> 00:36:36.790
Her um platform engineers are building
00:36:36.800 --> 00:36:38.870
this semantic context layer, whatever
00:36:38.880 --> 00:36:39.710
they're building, they're ditching the
00:36:39.720 --> 00:36:41.470
catalog completely. And our discussion
00:36:41.480 --> 00:36:43.630
with the hospital was like,
00:36:43.640 --> 00:36:44.950
"We have too much volatility with this
00:36:44.960 --> 00:36:47.230
sort huge source system change, and our
00:36:47.240 --> 00:36:48.910
engineers need to focus on this core
00:36:48.920 --> 00:36:50.030
other stuff, and we do not have the
00:36:50.040 --> 00:36:51.950
bandwidth to build
00:36:51.960 --> 00:36:54.070
um this the functionality." And the way
00:36:54.080 --> 00:36:56.390
I broke that down is we
00:36:56.400 --> 00:36:58.630
I I guess this gets back to more like a
00:36:58.640 --> 00:37:00.830
how the product manager approaches this
00:37:00.840 --> 00:37:02.590
differently than maybe like an architect
00:37:02.600 --> 00:37:03.950
or an engineering leader would, which
00:37:03.960 --> 00:37:04.990
was
00:37:05.000 --> 00:37:06.230
"What are the jobs to be done that we
00:37:06.240 --> 00:37:07.910
want from a catalog? What's our catalog
00:37:07.920 --> 00:37:09.470
currently serving? What's our ideal
00:37:09.480 --> 00:37:10.910
state of what a catalog would serve?"
00:37:10.920 --> 00:37:12.630
And it's like it's a discovery plane,
00:37:12.640 --> 00:37:15.790
it's a system of record for meaning, it
00:37:15.800 --> 00:37:16.870
is
00:37:16.880 --> 00:37:18.870
um a governance platform for sign-offs
00:37:18.880 --> 00:37:20.470
and approvals. It like has these
00:37:20.480 --> 00:37:21.870
different functions. And then we looked
00:37:21.880 --> 00:37:23.510
at the various vendors.
00:37:23.520 --> 00:37:25.790
And which of those features were jobs
00:37:25.800 --> 00:37:28.390
that we absolutely had to have um
00:37:28.400 --> 00:37:29.910
done, and which were nice to have? And
00:37:29.920 --> 00:37:32.150
what we kind of decided was
00:37:32.160 --> 00:37:34.270
to build the minimum viable
00:37:34.280 --> 00:37:36.550
functionality was more of a build than
00:37:36.560 --> 00:37:37.750
our team could do right now. There were
00:37:37.760 --> 00:37:38.910
parts that we could do, right? We could
00:37:38.920 --> 00:37:40.550
build the semantics, we could use Unity
00:37:40.560 --> 00:37:42.590
Catalog, we could use
00:37:42.600 --> 00:37:44.230
YAML and Git repos. There's things we
00:37:44.240 --> 00:37:45.790
could do, but it wouldn't cover enough
00:37:45.800 --> 00:37:47.390
of the job. So, we decided to continue
00:37:47.400 --> 00:37:49.630
with a data catalog platform at this
00:37:49.640 --> 00:37:50.670
time.
00:37:50.680 --> 00:37:54.670
Um and what's interesting is
00:37:54.680 --> 00:37:56.470
um I don't know what that will look like
00:37:56.480 --> 00:37:58.950
in 3 years, right? But right now is the
00:37:58.960 --> 00:38:01.710
right choice. But like you said,
00:38:01.720 --> 00:38:03.070
when I thought about this, the other
00:38:03.080 --> 00:38:05.310
thing that I'm starting to see, you said
00:38:05.320 --> 00:38:06.750
you can build context at these other
00:38:06.760 --> 00:38:07.870
layers.
00:38:07.880 --> 00:38:09.870
The problem is with some of these
00:38:09.880 --> 00:38:11.350
layers, like building at the warehouse
00:38:11.360 --> 00:38:12.670
layer
00:38:12.680 --> 00:38:13.910
would not work for us because the
00:38:13.920 --> 00:38:16.470
warehouse layer only covers metadata for
00:38:16.480 --> 00:38:18.310
objects that exist in the warehouse. And
00:38:18.320 --> 00:38:19.990
we care about objects that exist outside
00:38:20.000 --> 00:38:22.110
the warehouse, and we care about
00:38:22.120 --> 00:38:24.070
um I mean in systems that aren't yet
00:38:24.080 --> 00:38:26.470
ingested as well as like objects like
00:38:26.480 --> 00:38:28.470
glossary terms and things and like Unity
00:38:28.480 --> 00:38:29.790
catalog is just going to provide
00:38:29.800 --> 00:38:31.550
metadata on the table to physical
00:38:31.560 --> 00:38:33.110
objects that are in the warehouse.
00:38:33.120 --> 00:38:34.990
Similarly, um
00:38:35.000 --> 00:38:38.190
semantic layers are BI layers. Um
00:38:38.200 --> 00:38:39.950
some semantic layers like DBT semantic
00:38:39.960 --> 00:38:42.270
layer, the analytics engineers codifying
00:38:42.280 --> 00:38:44.670
context in that semantic layer.
00:38:44.680 --> 00:38:46.270
But I don't want my analytics engineers
00:38:46.280 --> 00:38:47.750
in all those discovery calls for hours
00:38:47.760 --> 00:38:49.190
and hours building conceptual models
00:38:49.200 --> 00:38:51.430
because that's not an effective use of
00:38:51.440 --> 00:38:52.710
my limited engineering time. So my
00:38:52.720 --> 00:38:54.630
product managers are doing concept
00:38:54.640 --> 00:38:56.710
modeling and just artifacts and
00:38:56.720 --> 00:38:58.510
recording these calls and summarizing
00:38:58.520 --> 00:38:59.630
them and
00:38:59.640 --> 00:39:02.950
and so the moment the tooling supports
00:39:02.960 --> 00:39:04.590
only the personas who are further
00:39:04.600 --> 00:39:05.990
downstream and they don't support
00:39:06.000 --> 00:39:07.230
upstream enough. And so I'm like, well,
00:39:07.240 --> 00:39:09.350
that's a non-starter because
00:39:09.360 --> 00:39:11.310
I don't buy into the assumption that or
00:39:11.320 --> 00:39:12.550
they buy into the assumption that the
00:39:12.560 --> 00:39:14.750
decider of of meaning
00:39:14.760 --> 00:39:17.070
is an engineer and not a business user.
00:39:17.080 --> 00:39:19.310
Right? And so I'm like, no, my engineers
00:39:19.320 --> 00:39:21.390
should be surfacing their hypothesis of
00:39:21.400 --> 00:39:23.350
this is what I heard, is this correct?
00:39:23.360 --> 00:39:24.630
But a human
00:39:24.640 --> 00:39:25.950
and so they're too technical. They don't
00:39:25.960 --> 00:39:27.990
have enough of a UX. So
00:39:28.000 --> 00:39:29.710
to me the catalog is still provides a
00:39:29.720 --> 00:39:31.270
meaningful interface between those who
00:39:31.280 --> 00:39:33.710
decide what things mean and those who
00:39:33.720 --> 00:39:35.670
document meaning and version control it
00:39:35.680 --> 00:39:37.870
and iterate on it over time. And so I
00:39:37.880 --> 00:39:39.310
mean the catalog is still sticky for
00:39:39.320 --> 00:39:41.190
that reason and I feel like a lot of BI
00:39:41.200 --> 00:39:43.830
vendors and warehouse vendors and stuff
00:39:43.840 --> 00:39:45.550
are making some pretty big assumptions
00:39:45.560 --> 00:39:48.230
about what context needs to be gathered
00:39:48.240 --> 00:39:49.990
and which personas need to care. And so
00:39:50.000 --> 00:39:51.950
to me I have no interest in them because
00:39:51.960 --> 00:39:53.070
they've
00:39:53.080 --> 00:39:56.550
they've uh made too many decisions for
00:39:56.560 --> 00:39:58.670
me and it's not flexible enough.
00:39:58.680 --> 00:40:00.110
>> Yeah, and and that's an interesting just
00:40:00.120 --> 00:40:02.230
that journey you went on. So there was
00:40:02.240 --> 00:40:04.110
the pattern of team design, right? So
00:40:04.120 --> 00:40:05.670
how how's the team design for the data
00:40:05.680 --> 00:40:07.150
team and the organization? There was the
00:40:07.160 --> 00:40:08.990
pattern of the information value stream.
00:40:09.000 --> 00:40:10.630
How are we actually going from problem
00:40:10.640 --> 00:40:13.870
statement to build design and
00:40:13.880 --> 00:40:15.390
uh to value?
00:40:15.400 --> 00:40:16.870
Uh what are our personas? Yeah, know,
00:40:16.880 --> 00:40:19.270
are their skills? Where does job jobs to
00:40:19.280 --> 00:40:21.910
be done? Where should that job be done?
00:40:21.920 --> 00:40:24.070
Therefore, what technology or tooling
00:40:24.080 --> 00:40:26.630
helps those people get that job done?
00:40:26.640 --> 00:40:28.630
Um, and they're all decisions that you
00:40:28.640 --> 00:40:30.470
actually have to use before you then
00:40:30.480 --> 00:40:33.030
decide I need this technical thing over
00:40:33.040 --> 00:40:35.310
here to do this bit of work, right?
00:40:35.320 --> 00:40:36.910
Whereas if you look at a lot of
00:40:36.920 --> 00:40:38.510
organizations, they they just go
00:40:38.520 --> 00:40:39.670
straight and you know, it's it's a
00:40:39.680 --> 00:40:41.230
technical person going, "Oh, we need a
00:40:41.240 --> 00:40:43.390
data catalog or we need a BI semantic
00:40:43.400 --> 00:40:44.230
layer."
00:40:44.240 --> 00:40:46.670
Um, because they do a technical
00:40:46.680 --> 00:40:49.270
architecture up rather than, you know,
00:40:49.280 --> 00:40:50.710
all these different patterns. So, I'm
00:40:50.720 --> 00:40:52.230
I'm kind of always intrigued when I ask
00:40:52.240 --> 00:40:54.310
people how they make decisions, um,
00:40:54.320 --> 00:40:57.430
which answer I get back. Um, and we are
00:40:57.440 --> 00:40:58.870
in, you know, we are in a time of
00:40:58.880 --> 00:41:00.470
change. So,
00:41:00.480 --> 00:41:02.710
right now no choice is safe because, you
00:41:02.720 --> 00:41:04.110
know, like you said in three years time,
00:41:04.120 --> 00:41:05.390
who knows what it's going to look like.
00:41:05.400 --> 00:41:06.750
So, again, you've got to you've got to
00:41:06.760 --> 00:41:08.950
pick a a horse and then execute it
00:41:08.960 --> 00:41:10.510
because you can't sit there, uh,
00:41:10.520 --> 00:41:11.790
worrying about the change that's going
00:41:11.800 --> 00:41:13.790
to happen six months time.
00:41:13.800 --> 00:41:15.190
Um, all right.
00:41:15.200 --> 00:41:17.950
>> The thing is those vendors
00:41:17.960 --> 00:41:19.310
need to know that that calculus is
00:41:19.320 --> 00:41:21.710
changing. Like it's definitely overhyped
00:41:21.720 --> 00:41:23.950
the death of SaaS. I have no interest in
00:41:23.960 --> 00:41:25.830
building every tool my platform team
00:41:25.840 --> 00:41:27.390
uses from scratch and maintaining it.
00:41:27.400 --> 00:41:28.710
What a nightmare. That sounds awful. I
00:41:28.720 --> 00:41:30.150
don't want to do that.
00:41:30.160 --> 00:41:32.310
But it does mean their pricing has to be
00:41:32.320 --> 00:41:34.190
competitive or I'm going to get the
00:41:34.200 --> 00:41:36.110
question from my execs of like, "Oh my
00:41:36.120 --> 00:41:37.630
gosh, half a million dollars? Can't you
00:41:37.640 --> 00:41:39.830
guys build something for like half that
00:41:39.840 --> 00:41:40.830
at least?"
00:41:40.840 --> 00:41:41.990
You're like, "Yeah, I guess if you want
00:41:42.000 --> 00:41:43.750
to hire someone and it's their dedicated
00:41:43.760 --> 00:41:46.630
job to build this thing, we can." So, I
00:41:46.640 --> 00:41:47.950
think there's a lot of pressure coming
00:41:47.960 --> 00:41:50.950
on SaaS companies to make their products
00:41:50.960 --> 00:41:52.590
um,
00:41:52.600 --> 00:41:55.270
the ROI so obvious and so genuinely
00:41:55.280 --> 00:41:57.510
useful and it gone are the days when you
00:41:57.520 --> 00:41:59.750
can provide the software that like does
00:41:59.760 --> 00:42:01.670
one job to be done and it's like good
00:42:01.680 --> 00:42:02.870
enough and we're going to just keep
00:42:02.880 --> 00:42:05.190
renewing. It's like the ROI needs to be
00:42:05.200 --> 00:42:06.270
there, the stickiness, the
00:42:06.280 --> 00:42:07.750
user-friendliness, the things where it's
00:42:07.760 --> 00:42:09.150
like, "Oh my gosh, you create something
00:42:09.160 --> 00:42:10.990
this fully featured that solves all
00:42:11.000 --> 00:42:12.470
these problems for us?
00:42:12.480 --> 00:42:13.990
No way we could build the bare-bones
00:42:14.000 --> 00:42:16.150
solution. So I feel like
00:42:16.160 --> 00:42:17.830
uh it's going to be very interesting to
00:42:17.840 --> 00:42:19.710
see how SaaS vendors build their pricing
00:42:19.720 --> 00:42:21.950
side and their
00:42:21.960 --> 00:42:23.710
discipline around creating really,
00:42:23.720 --> 00:42:26.030
really good products changes under this
00:42:26.040 --> 00:42:27.350
pressure of like, "Well, we could just
00:42:27.360 --> 00:42:28.630
build it ourselves. If we want If we
00:42:28.640 --> 00:42:29.990
want a really crappy data catalog, we'll
00:42:30.000 --> 00:42:31.430
just go build our own crappy data
00:42:31.440 --> 00:42:32.630
catalog."
00:42:32.640 --> 00:42:34.270
>> Uh I Yeah, I think they're going to get
00:42:34.280 --> 00:42:35.470
squashed, right? Like they're not going
00:42:35.480 --> 00:42:37.790
to disappear, but yeah, the the platform
00:42:37.800 --> 00:42:40.150
players are going to eat a lot Yeah,
00:42:40.160 --> 00:42:42.190
like you said, Unity Catalog, those kind
00:42:42.200 --> 00:42:43.270
of things, they're they're getting
00:42:43.280 --> 00:42:44.270
bigger, right? They're starting to eat
00:42:44.280 --> 00:42:46.190
their space. I'm definitely seeing in
00:42:46.200 --> 00:42:48.070
smaller teams and certain types of
00:42:48.080 --> 00:42:49.830
organizations, depending on the culture
00:42:49.840 --> 00:42:51.870
of the team topology, their their value
00:42:51.880 --> 00:42:53.590
stream, uh they're building rather than
00:42:53.600 --> 00:42:55.510
buying now. Uh that's a cultural
00:42:55.520 --> 00:42:57.750
decision. Uh I'm with you, right? It
00:42:57.760 --> 00:43:00.110
sometimes costs more in people time, but
00:43:00.120 --> 00:43:02.550
that's seen as free. Um there's also the
00:43:02.560 --> 00:43:04.590
threat of, you know, a lot of leaders at
00:43:04.600 --> 00:43:06.230
a tech savvy now that just vibe coding
00:43:06.240 --> 00:43:08.550
and say, "Why can't use that?" But
00:43:08.560 --> 00:43:09.870
it comes back to like you said, you
00:43:09.880 --> 00:43:11.630
know, is it Is it valuable? Is it
00:43:11.640 --> 00:43:13.590
viable? Is it feasible? All those
00:43:13.600 --> 00:43:16.630
McKinsey lenses And you know, what's
00:43:16.640 --> 00:43:18.870
interesting is like I was just building
00:43:18.880 --> 00:43:20.830
out a cut of a data dictionary screen
00:43:20.840 --> 00:43:23.510
for us right now. Um just to iterate and
00:43:23.520 --> 00:43:24.750
and make it a little bit more usable for
00:43:24.760 --> 00:43:26.110
when I use it.
00:43:26.120 --> 00:43:27.710
And you know, the first cut of it was
00:43:27.720 --> 00:43:29.110
really quick. It did exactly what I
00:43:29.120 --> 00:43:30.390
wanted.
00:43:30.400 --> 00:43:32.990
And then I went into this deep window of
00:43:33.000 --> 00:43:36.150
time uh trying to make it beautiful cuz
00:43:36.160 --> 00:43:38.630
of cuz I care and and it just wasn't,
00:43:38.640 --> 00:43:40.950
right? I was like, "It works, but it's
00:43:40.960 --> 00:43:41.985
ugly."
00:43:41.995 --> 00:43:44.070
>> [laughter]
00:43:44.080 --> 00:43:45.310
>> And it doesn't matter what I'm doing to
00:43:45.320 --> 00:43:47.510
it, it's still ugly. Um
00:43:47.520 --> 00:43:49.150
and so I think you're right. I think the
00:43:49.160 --> 00:43:51.310
the companies that have that well
00:43:51.320 --> 00:43:54.190
moment, that they have that magic, uh
00:43:54.200 --> 00:43:57.190
they they will be the ones that survive
00:43:57.200 --> 00:43:59.470
better than than all the others.
00:43:59.480 --> 00:44:00.310
Um all right, we've got some more
00:44:00.320 --> 00:44:02.030
questions. Uh one that came in through
00:44:02.040 --> 00:44:03.910
LinkedIn was from Tim Frazier, who was
00:44:03.920 --> 00:44:06.030
the first awesome foursome on. Um he
00:44:06.040 --> 00:44:07.950
said, "What's the single most important
00:44:07.960 --> 00:44:09.870
skill to learn today in the data work
00:44:09.880 --> 00:44:13.150
that didn't exist 5 years ago?"
00:44:13.160 --> 00:44:15.790
What do you reckon?
00:44:15.800 --> 00:44:18.630
>> Uh, I think it's like the becoming meta
00:44:18.640 --> 00:44:22.670
skill of like um, being literate in
00:44:22.680 --> 00:44:26.190
what AI is capable of as of now, right?
00:44:26.200 --> 00:44:28.630
And what's required to get the results
00:44:28.640 --> 00:44:30.990
you want out of AI, right? Um, what's
00:44:31.000 --> 00:44:33.790
required from you as the user, what's
00:44:33.800 --> 00:44:35.670
required of a less skilled user, what's
00:44:35.680 --> 00:44:38.110
required of the system, how do you when
00:44:38.120 --> 00:44:39.430
do you need a deterministic, when is
00:44:39.440 --> 00:44:42.390
probabilistic okay, like the ways of
00:44:42.400 --> 00:44:44.590
like thinking about what it means to
00:44:44.600 --> 00:44:46.830
integrate these tools not as a personal
00:44:46.840 --> 00:44:48.310
productivity tool, but as part of a
00:44:48.320 --> 00:44:49.910
system
00:44:49.920 --> 00:44:52.070
um, is
00:44:52.080 --> 00:44:54.630
is like the thing I'm thinking about the
00:44:54.640 --> 00:44:56.790
most and the skill that like data people
00:44:56.800 --> 00:44:59.430
need. And so, I feel like
00:44:59.440 --> 00:45:01.270
it's interesting cuz you all the hype on
00:45:01.280 --> 00:45:04.590
LinkedIn is heavily skewed to
00:45:04.600 --> 00:45:07.110
solopreneurs, consultants, a certain
00:45:07.120 --> 00:45:08.910
type of person.
00:45:08.920 --> 00:45:11.270
Well, in an enterprise, I don't have a
00:45:11.280 --> 00:45:14.910
choice of like bringing any
00:45:14.920 --> 00:45:17.510
new startup cool tool. I don't have I
00:45:17.520 --> 00:45:19.350
live I'm a regulated hospital. I use
00:45:19.360 --> 00:45:21.470
what IT lets me use. So, I use Microsoft
00:45:21.480 --> 00:45:24.350
365 Copilot because that's what is
00:45:24.360 --> 00:45:26.830
um, authorized. Um, so, how do I get
00:45:26.840 --> 00:45:29.670
good results from that tool or or what
00:45:29.680 --> 00:45:31.270
are the limitations of that tool and
00:45:31.280 --> 00:45:32.790
when might we need to fight for
00:45:32.800 --> 00:45:35.310
something different? And like how to
00:45:35.320 --> 00:45:38.110
think about
00:45:38.120 --> 00:45:40.670
the limitations of AI and when to reach
00:45:40.680 --> 00:45:43.670
for AI and change your behavior and how
00:45:43.680 --> 00:45:45.550
do you change your team, how do you
00:45:45.560 --> 00:45:47.390
upskill your team's mental models, how
00:45:47.400 --> 00:45:48.910
do you upskill
00:45:48.920 --> 00:45:51.830
Okay, yes, you can have this written,
00:45:51.840 --> 00:45:53.590
but then we still need code review. You
00:45:53.600 --> 00:45:56.070
can't just auto push it or
00:45:56.080 --> 00:46:00.110
um, yes, you can use it for
00:46:00.120 --> 00:46:02.190
like you architecture review becomes
00:46:02.200 --> 00:46:03.830
even more important before you throw
00:46:03.840 --> 00:46:05.110
agents at it. So we need to make sure
00:46:05.120 --> 00:46:07.470
your instructions aren't just good for
00:46:07.480 --> 00:46:08.990
another developer. They're good enough
00:46:09.000 --> 00:46:11.310
that when you give them to a system,
00:46:11.320 --> 00:46:12.830
like the nature of the flavor of the
00:46:12.840 --> 00:46:14.310
code review is like the cloud's not
00:46:14.320 --> 00:46:15.350
going to understand this cuz cloud
00:46:15.360 --> 00:46:16.710
doesn't have this context. Let's improve
00:46:16.720 --> 00:46:18.030
this requirement or let's improve this
00:46:18.040 --> 00:46:19.590
technical design document. So I feel
00:46:19.600 --> 00:46:20.590
like
00:46:20.600 --> 00:46:23.030
how everything is changing when you make
00:46:23.040 --> 00:46:24.710
assumptions that
00:46:24.720 --> 00:46:27.750
um an AI collaborator will be part of
00:46:27.760 --> 00:46:29.830
the system. Like that's the that's the
00:46:29.840 --> 00:46:33.110
skill. Like that I don't
00:46:33.120 --> 00:46:34.790
Those of us who are early adopters or
00:46:34.800 --> 00:46:37.070
those who are working in a solo
00:46:37.080 --> 00:46:38.190
environment where they control the
00:46:38.200 --> 00:46:40.750
harness and they control their machine.
00:46:40.760 --> 00:46:43.190
Um or you control this whole stack that
00:46:43.200 --> 00:46:44.950
you've built, right? So you can make
00:46:44.960 --> 00:46:46.510
sure it all interoperates and works how
00:46:46.520 --> 00:46:47.950
you want to work it.
00:46:47.960 --> 00:46:49.790
Um
00:46:49.800 --> 00:46:52.030
That is great to and all of us should be
00:46:52.040 --> 00:46:53.430
trying to optimize our personal
00:46:53.440 --> 00:46:54.990
environments and all those things, but
00:46:55.000 --> 00:46:57.230
that's completely different from
00:46:57.240 --> 00:46:59.150
how is our whole organization going to
00:46:59.160 --> 00:47:01.910
evolve to use AI? Is it ready for us to
00:47:01.920 --> 00:47:03.950
use in this way at this time? And if it
00:47:03.960 --> 00:47:05.710
is ready,
00:47:05.720 --> 00:47:07.070
what needs to be true to do that
00:47:07.080 --> 00:47:09.110
responsibly and reliably and all those
00:47:09.120 --> 00:47:10.710
things. That's the meta skill to me that
00:47:10.720 --> 00:47:13.110
it's like
00:47:13.120 --> 00:47:15.310
the laggards, I worry about them. I
00:47:15.320 --> 00:47:16.590
worry about the people who are just
00:47:16.600 --> 00:47:17.790
still showing up and doing things the
00:47:17.800 --> 00:47:19.270
same way they were doing it 5 years ago.
00:47:19.280 --> 00:47:23.030
I'm like you're I do not want to be AI I
00:47:23.040 --> 00:47:24.550
don't want to see you lose your job
00:47:24.560 --> 00:47:25.990
because you're not staying on the edge
00:47:26.000 --> 00:47:28.590
and the edge now means
00:47:28.600 --> 00:47:31.590
rethinking the whole way we do things.
00:47:31.600 --> 00:47:32.670
>> Yeah, I
00:47:32.680 --> 00:47:33.830
thought about this one cuz I got to see
00:47:33.840 --> 00:47:36.510
it before you did. Um I'm not sure these
00:47:36.520 --> 00:47:38.790
new skills to learn, but I think these
00:47:38.800 --> 00:47:40.910
skills that we need to double down on
00:47:40.920 --> 00:47:44.110
and improve. Uh so I thought about, you
00:47:44.120 --> 00:47:46.550
know, as you were talking, uh
00:47:46.560 --> 00:47:49.030
the the skill of experimentation.
00:47:49.040 --> 00:47:49.910
>> Yeah.
00:47:49.920 --> 00:47:51.710
>> Because actually the only way right now
00:47:51.720 --> 00:47:53.710
if you want to stay in this game at this
00:47:53.720 --> 00:47:55.830
early stage, uh is you got to be up to
00:47:55.840 --> 00:47:57.310
play. You know, you you've I did it with
00:47:57.320 --> 00:47:59.350
a personal always. You went and did it
00:47:59.360 --> 00:48:03.590
with the app around your garden, but by
00:48:03.600 --> 00:48:04.870
by building something in your spare
00:48:04.880 --> 00:48:05.910
time, you're learning and you're
00:48:05.920 --> 00:48:07.310
experimenting.
00:48:07.320 --> 00:48:08.110
Um
00:48:08.120 --> 00:48:10.910
I think we we need to understand
00:48:10.920 --> 00:48:12.430
uh systems thinking.
00:48:12.440 --> 00:48:14.070
Like you said, I can change this little
00:48:14.080 --> 00:48:15.470
bit here, but it actually has no impact
00:48:15.480 --> 00:48:17.550
because of all these other things.
00:48:17.560 --> 00:48:18.870
Uh I think we have to understand how
00:48:18.880 --> 00:48:21.510
humans work. What I'm finding is in my
00:48:21.520 --> 00:48:23.710
harnesses, I'm more and more treating
00:48:23.720 --> 00:48:26.470
the LLMs like agents or employees that
00:48:26.480 --> 00:48:29.150
like humans and I'm giving them personas
00:48:29.160 --> 00:48:31.870
and policies and ways of working because
00:48:31.880 --> 00:48:34.550
when I lead teams, those things worked,
00:48:34.560 --> 00:48:36.590
right? We have to-do's and doings and
00:48:36.600 --> 00:48:38.550
don'ts and we keep track of them because
00:48:38.560 --> 00:48:40.630
as a team, it was easy to see what
00:48:40.640 --> 00:48:42.470
people were working on quickly by doing
00:48:42.480 --> 00:48:44.630
that. I'm getting the the LLMs or the
00:48:44.640 --> 00:48:47.030
GenAI tools to do that now. So,
00:48:47.040 --> 00:48:49.270
this idea of how do we scale
00:48:49.280 --> 00:48:51.510
how do we scale people and teams is now
00:48:51.520 --> 00:48:53.070
becoming more important as we scale the
00:48:53.080 --> 00:48:54.470
technology.
00:48:54.480 --> 00:48:56.830
Um and then recognizing it's bounded by
00:48:56.840 --> 00:48:58.310
context.
00:48:58.320 --> 00:49:00.150
You know, if I'm in if I'm working in a
00:49:00.160 --> 00:49:01.590
hospital
00:49:01.600 --> 00:49:03.030
and and I have a framework where I talk
00:49:03.040 --> 00:49:05.430
about ask AI, assisted AI, and automated
00:49:05.440 --> 00:49:08.590
AI. I would I would not put automated AI
00:49:08.600 --> 00:49:10.310
in there right now. You know, there is
00:49:10.320 --> 00:49:12.670
no way because the impact of getting
00:49:12.680 --> 00:49:14.990
that wrong is is a bad thing will happen
00:49:15.000 --> 00:49:16.990
to a human. Um
00:49:17.000 --> 00:49:18.150
>> Yes, exactly.
00:49:18.160 --> 00:49:20.110
>> if I'm building a demo system, then
00:49:20.120 --> 00:49:21.790
yeah, I can put it on to auto mode and
00:49:21.800 --> 00:49:23.750
see what happens because the blast
00:49:23.760 --> 00:49:26.150
impact that the the bad uh impact that I
00:49:26.160 --> 00:49:28.990
get it wrong is is minimal. Um
00:49:29.000 --> 00:49:30.350
And so, I think that context is really,
00:49:30.360 --> 00:49:31.990
really important. Context of the team
00:49:32.000 --> 00:49:33.950
and technology like you said, you know,
00:49:33.960 --> 00:49:35.790
uh we when we're independent, we get to
00:49:35.800 --> 00:49:37.070
play with the cool tools because it's
00:49:37.080 --> 00:49:38.510
our choice. When we work in large
00:49:38.520 --> 00:49:40.310
organizations, we follow the
00:49:40.320 --> 00:49:41.710
architecture and that architecture's
00:49:41.720 --> 00:49:43.630
there for a reason. Um
00:49:43.640 --> 00:49:43.910
>> Yeah.
00:49:43.920 --> 00:49:45.310
>> So, yeah, I think I think one of the key
00:49:45.320 --> 00:49:47.070
points you pulled out is if you're in
00:49:47.080 --> 00:49:48.630
the data space and you're watching all
00:49:48.640 --> 00:49:50.390
this cool stuff happen,
00:49:50.400 --> 00:49:51.590
um
00:49:51.600 --> 00:49:53.910
it's because there are people enough to
00:49:53.920 --> 00:49:55.950
play in these isolated environments and
00:49:55.960 --> 00:49:58.990
do these things and that's not the the
00:49:59.000 --> 00:50:01.870
mess of the people in the world. Um, but
00:50:01.880 --> 00:50:02.710
yeah, I think you still got to
00:50:02.720 --> 00:50:05.070
experiment. I think you should go and
00:50:05.080 --> 00:50:06.990
play with some things at home to
00:50:07.000 --> 00:50:08.590
whatever level you can because I think
00:50:08.600 --> 00:50:09.750
learning
00:50:09.760 --> 00:50:10.870
and keeping up is going to be really
00:50:10.880 --> 00:50:12.190
really important and I think we've seen
00:50:12.200 --> 00:50:13.590
that already.
00:50:13.600 --> 00:50:15.990
>> Yeah, and I think you can
00:50:16.000 --> 00:50:18.190
you can find like as a leader you also
00:50:18.200 --> 00:50:20.310
have to say how is our team going to
00:50:20.320 --> 00:50:23.270
start to experiment with this in safe
00:50:23.280 --> 00:50:25.430
ways. So,
00:50:25.440 --> 00:50:28.790
something I am not hot on until our
00:50:28.800 --> 00:50:31.350
context layer improves is natural
00:50:31.360 --> 00:50:33.470
language to sequel because we don't have
00:50:33.480 --> 00:50:35.190
enough structured context for me to know
00:50:35.200 --> 00:50:36.470
that that executive is going to get an
00:50:36.480 --> 00:50:40.070
accurate answer every time, right? Um,
00:50:40.080 --> 00:50:43.190
but I am very interested in text to
00:50:43.200 --> 00:50:45.310
sequel for the analyst on the central
00:50:45.320 --> 00:50:47.950
data team. So, our central data team at
00:50:47.960 --> 00:50:49.390
the hospital
00:50:49.400 --> 00:50:51.110
was working on business objects, right?
00:50:51.120 --> 00:50:52.750
Under a different paradigm and now under
00:50:52.760 --> 00:50:55.070
our new like lots of drag and drop
00:50:55.080 --> 00:50:56.830
configure, lots of I'm a subject matter
00:50:56.840 --> 00:50:58.430
expert and I know how to construct a
00:50:58.440 --> 00:51:00.750
report. And now we're moving to an era
00:51:00.760 --> 00:51:01.990
where they're going to need to know more
00:51:02.000 --> 00:51:04.830
sequel and how to work with analytics
00:51:04.840 --> 00:51:06.350
engineers and how to pull context
00:51:06.360 --> 00:51:08.070
anyway. So, it's just like their job has
00:51:08.080 --> 00:51:09.670
shifted a lot. Some [snorts] of them are
00:51:09.680 --> 00:51:11.590
already strong in sequel, some are still
00:51:11.600 --> 00:51:12.910
developing and but basically the
00:51:12.920 --> 00:51:14.230
requirements of the job have shifted a
00:51:14.240 --> 00:51:15.190
lot.
00:51:15.200 --> 00:51:16.670
So, I'm interested in the idea of like
00:51:16.680 --> 00:51:18.510
how does an like
00:51:18.520 --> 00:51:20.430
LLMs are amazing at writing syntax if
00:51:20.440 --> 00:51:23.350
you give them enough context and these
00:51:23.360 --> 00:51:25.070
these analysts have all the business
00:51:25.080 --> 00:51:27.590
context and even knowing what's in what
00:51:27.600 --> 00:51:28.950
table, right? That's where their
00:51:28.960 --> 00:51:31.070
expertise currently is. And so, it's
00:51:31.080 --> 00:51:32.990
like am I interested in building agent
00:51:33.000 --> 00:51:34.710
for only that persona, no one else in
00:51:34.720 --> 00:51:36.830
the hospital which is that behaves in a
00:51:36.840 --> 00:51:38.430
way that says
00:51:38.440 --> 00:51:40.870
"Okay, I found these five tables. I
00:51:40.880 --> 00:51:42.350
think this one looks promising." And the
00:51:42.360 --> 00:51:44.310
analyst can go
00:51:44.320 --> 00:51:47.470
"No, I don't know. That's wrong. You
00:51:47.480 --> 00:51:50.190
know, like where can their expertise
00:51:50.200 --> 00:51:52.350
show up? Um
00:51:52.360 --> 00:51:53.710
Sorry.
00:51:53.720 --> 00:51:56.790
>> That's right.
00:51:56.800 --> 00:51:58.150
>> Am I still live?
00:51:58.160 --> 00:51:59.430
Sorry.
00:51:59.440 --> 00:51:59.630
>> No.
00:51:59.640 --> 00:52:01.630
>> I had a little like notification pull up
00:52:01.640 --> 00:52:03.750
that was going to mess things
00:52:03.760 --> 00:52:04.710
>> That's right.
00:52:04.720 --> 00:52:05.630
>> But anyway
00:52:05.640 --> 00:52:06.510
>> Your camera's off now.
00:52:06.520 --> 00:52:07.590
>> I'm interested in that.
00:52:07.600 --> 00:52:09.430
>> Your camera's off now.
00:52:09.440 --> 00:52:11.150
>> Go ahead. Go ahead, Shane.
00:52:11.160 --> 00:52:12.430
Back to you.
00:52:12.440 --> 00:52:13.950
>> Ah, okay. Um
00:52:13.960 --> 00:52:14.870
>> Oh.
00:52:14.880 --> 00:52:17.030
>> Your camera's off but that's okay.
00:52:17.040 --> 00:52:20.790
>> I had some
00:52:20.800 --> 00:52:21.950
Yeah, I'll be back.
00:52:21.960 --> 00:52:22.630
>> Okay.
00:52:22.640 --> 00:52:24.510
>> I tried to like dismiss it. Something
00:52:24.520 --> 00:52:26.430
stupid. So, I'll be back in a second.
00:52:26.440 --> 00:52:29.350
>> All righty.
00:52:29.360 --> 00:52:30.950
>> Wait.
00:52:30.960 --> 00:52:31.990
Anyway, you go ahead and talk for a
00:52:32.000 --> 00:52:32.870
minute, Shane.
00:52:32.880 --> 00:52:33.670
>> Well, that's okay. [snorts] We've we've
00:52:33.680 --> 00:52:34.710
only got a couple minutes before we have
00:52:34.720 --> 00:52:36.310
to close it out. So, leave camera off.
00:52:36.320 --> 00:52:39.510
That'll be That'll be fine. Um
00:52:39.520 --> 00:52:41.750
I I think I think there are some patents
00:52:41.760 --> 00:52:43.950
out there that are hard to find but are
00:52:43.960 --> 00:52:45.590
highly valuable. So, that example we use
00:52:45.600 --> 00:52:47.310
of text to sequel. I agree, right? If
00:52:47.320 --> 00:52:49.230
you're in an organization where people
00:52:49.240 --> 00:52:51.070
need to trust the numbers, I don't think
00:52:51.080 --> 00:52:53.110
we've got the tooling yet that give us
00:52:53.120 --> 00:52:54.390
that trust.
00:52:54.400 --> 00:52:55.430
But
00:52:55.440 --> 00:52:56.790
one of the patents I've seen work
00:52:56.800 --> 00:52:59.350
really, really well is we can create a
00:52:59.360 --> 00:53:01.230
list of business questions that we know
00:53:01.240 --> 00:53:02.910
get asked all the time.
00:53:02.920 --> 00:53:05.950
We can create uh trusted queries or
00:53:05.960 --> 00:53:07.950
trusted dashboards or trusted reports
00:53:07.960 --> 00:53:10.230
that answer those questions and are
00:53:10.240 --> 00:53:12.270
curated. And then we can give an
00:53:12.280 --> 00:53:14.110
interface where people can type in a
00:53:14.120 --> 00:53:15.910
natural language question and it comes
00:53:15.920 --> 00:53:19.150
back with the curated questions
00:53:19.160 --> 00:53:20.990
that are close to what they're ans-
00:53:21.000 --> 00:53:23.390
asking. And then they go, "Oh, yeah,
00:53:23.400 --> 00:53:24.550
that one." And when they click on it,
00:53:24.560 --> 00:53:26.270
they then get the deterministic run,
00:53:26.280 --> 00:53:27.950
right? The sequel runs or the the
00:53:27.960 --> 00:53:30.110
dashboard comes up. Now, that Is that
00:53:30.120 --> 00:53:32.070
the magic, you know, ask any question of
00:53:32.080 --> 00:53:34.110
any data and get an answer? No. But is
00:53:34.120 --> 00:53:36.230
that a starting point? Is that a good
00:53:36.240 --> 00:53:38.070
way of experimenting how your your
00:53:38.080 --> 00:53:39.910
stakeholders and your users, your
00:53:39.920 --> 00:53:41.110
information consumers, are going to
00:53:41.120 --> 00:53:43.190
start engaging? Like, will they actually
00:53:43.200 --> 00:53:44.630
use that or will they go, "That was a
00:53:44.640 --> 00:53:46.030
great demo, now let me get back to my
00:53:46.040 --> 00:53:47.310
day job."
00:53:47.320 --> 00:53:47.870
Um
00:53:47.880 --> 00:53:49.430
We had that with self-service, you know,
00:53:49.440 --> 00:53:51.270
this idea that back when we had OLAP
00:53:51.280 --> 00:53:52.790
cubes and we had business objects, it
00:53:52.800 --> 00:53:53.670
was like, "Everybody's going to write
00:53:53.680 --> 00:53:54.750
their own report." And the number of
00:53:54.760 --> 00:53:57.070
times I worked with frontline staff, and
00:53:57.080 --> 00:53:58.750
they said to me, "I don't want to do
00:53:58.760 --> 00:54:00.950
that data right? It's not my job
00:54:00.960 --> 00:54:02.310
and I'm not interested in it, it's too
00:54:02.320 --> 00:54:03.390
slow.
00:54:03.400 --> 00:54:05.190
I just want to know the answer to this
00:54:05.200 --> 00:54:06.550
question because I've got an angry
00:54:06.560 --> 00:54:09.030
customer on the phone, right? Like,
00:54:09.040 --> 00:54:10.390
"What is the delivery date of the
00:54:10.400 --> 00:54:12.070
order?" I don't want to have to drag and
00:54:12.080 --> 00:54:13.950
drop something. I just want to go, "What
00:54:13.960 --> 00:54:15.230
is the delivery date of the order?" and
00:54:15.240 --> 00:54:16.910
then carry on with my job of making that
00:54:16.920 --> 00:54:18.830
customer happy. So, I think again, this
00:54:18.840 --> 00:54:22.390
is this ecosystem um of people like us
00:54:22.400 --> 00:54:24.950
in the data world that get to explore,
00:54:24.960 --> 00:54:26.990
and then the ecosystem of data people
00:54:27.000 --> 00:54:28.630
who think everybody wants to do the job
00:54:28.640 --> 00:54:31.430
that we do, uh and then the people out
00:54:31.440 --> 00:54:33.590
there that actually just want data to
00:54:33.600 --> 00:54:35.670
help them take an action and achieve an
00:54:35.680 --> 00:54:37.430
outcome. Uh so, I think we got to be
00:54:37.440 --> 00:54:39.390
really clear about where we sit in that
00:54:39.400 --> 00:54:42.150
continuum as an organization.
00:54:42.160 --> 00:54:44.630
>> Yeah, that's what makes me really um
00:54:44.640 --> 00:54:46.790
I think self-service is really
00:54:46.800 --> 00:54:50.110
interesting in the future of
00:54:50.120 --> 00:54:52.390
of a data products universe. Like, you
00:54:52.400 --> 00:54:54.190
take that customer service example, if
00:54:54.200 --> 00:54:57.030
we know based on
00:54:57.040 --> 00:54:59.510
you know, 10 years of data service
00:54:59.520 --> 00:55:01.350
ticket, like Zendesk tickets or
00:55:01.360 --> 00:55:03.070
whatever, the typical questions that are
00:55:03.080 --> 00:55:05.470
coming in and the data assets that those
00:55:05.480 --> 00:55:07.590
customer service agents need at hand to
00:55:07.600 --> 00:55:09.670
be able to answer questions like,
00:55:09.680 --> 00:55:11.470
"Has it shipped yet? When is the ship
00:55:11.480 --> 00:55:14.470
date?" You can build a really good data
00:55:14.480 --> 00:55:17.430
product that has authoritative answers
00:55:17.440 --> 00:55:19.870
to those questions and all the semantic
00:55:19.880 --> 00:55:21.790
context and everything the agent needs,
00:55:21.800 --> 00:55:23.030
and then you can put a natural language
00:55:23.040 --> 00:55:25.270
interface on top of it that is literally
00:55:25.280 --> 00:55:26.670
just
00:55:26.680 --> 00:55:28.430
their [snorts] ability to ask and even
00:55:28.440 --> 00:55:30.030
have shortcut questions. Here's a
00:55:30.040 --> 00:55:31.230
customer number, what do you want to
00:55:31.240 --> 00:55:32.590
know about that customer, you know, that
00:55:32.600 --> 00:55:34.750
order number? It or here's the answer I
00:55:34.760 --> 00:55:35.870
already know you're going to ask when it
00:55:35.880 --> 00:55:37.830
has it shipped yet. It hasn't shipped.
00:55:37.840 --> 00:55:40.230
It's due to ship tomorrow and like
00:55:40.240 --> 00:55:43.110
so is self-service really promising in a
00:55:43.120 --> 00:55:44.550
data product context where you're like,
00:55:44.560 --> 00:55:47.110
yes, this self-service point works for
00:55:47.120 --> 00:55:49.350
this user persona who has very frequent
00:55:49.360 --> 00:55:51.990
needs, very predictable and I'm going to
00:55:52.000 --> 00:55:53.750
build a data product behind it that has
00:55:53.760 --> 00:55:55.550
enough documentation. Yes, I'm very hot
00:55:55.560 --> 00:55:57.670
on that idea of self-service. I'm not
00:55:57.680 --> 00:56:00.030
hot on the idea of like
00:56:00.040 --> 00:56:01.710
I'll have all of the data products and
00:56:01.720 --> 00:56:03.390
my users are going to want to go in and
00:56:03.400 --> 00:56:05.230
pick the table and slice and dice it and
00:56:05.240 --> 00:56:06.990
come up with their own answers. I'm not
00:56:07.000 --> 00:56:10.270
that is still quite far away until what
00:56:10.280 --> 00:56:11.390
data bricks wants you to think data
00:56:11.400 --> 00:56:13.310
bricks genie or snowflake I can't
00:56:13.320 --> 00:56:14.150
remember what they call theirs right
00:56:14.160 --> 00:56:16.190
now. This idea that
00:56:16.200 --> 00:56:17.990
user can show up and ask any question
00:56:18.000 --> 00:56:20.190
and we can find an authoritative answer.
00:56:20.200 --> 00:56:22.710
That's a ways. That's ways away in my
00:56:22.720 --> 00:56:23.550
mind.
00:56:23.560 --> 00:56:24.710
>> Yeah, [snorts] and it comes back to what
00:56:24.720 --> 00:56:26.110
action and outcome does the the
00:56:26.120 --> 00:56:27.630
information consumer want to use? So in
00:56:27.640 --> 00:56:30.710
that context into like we should know
00:56:30.720 --> 00:56:32.950
that the person who's ringing has an
00:56:32.960 --> 00:56:34.230
open order.
00:56:34.240 --> 00:56:35.310
Yeah, and like you see we should know
00:56:35.320 --> 00:56:37.150
historically the majority of the people
00:56:37.160 --> 00:56:38.630
that have an open order and ring up at
00:56:38.640 --> 00:56:39.870
this time
00:56:39.880 --> 00:56:41.630
asking for delivery information and we
00:56:41.640 --> 00:56:44.230
should make that information usable and
00:56:44.240 --> 00:56:46.430
wherever the the context into person is
00:56:46.440 --> 00:56:47.670
it should be in front of them, right?
00:56:47.680 --> 00:56:49.110
They shouldn't have to go click five
00:56:49.120 --> 00:56:50.510
times to get there. It should just pop
00:56:50.520 --> 00:56:53.030
up and go, by the way delivery date was
00:56:53.040 --> 00:56:54.670
last week.
00:56:54.680 --> 00:56:56.390
You know, that's that's enough to know
00:56:56.400 --> 00:56:58.550
that the kind of call you're getting.
00:56:58.560 --> 00:56:59.190
Um
00:56:59.200 --> 00:57:01.390
so yeah, I think you know, that product
00:57:01.400 --> 00:57:03.710
thinking is still um
00:57:03.720 --> 00:57:04.790
uh
00:57:04.800 --> 00:57:07.110
a long long way away from what a lot of
00:57:07.120 --> 00:57:08.950
data people are doing
00:57:08.960 --> 00:57:11.310
and yet then we try and just double down
00:57:11.320 --> 00:57:13.190
on the new cool tech. So I think it's
00:57:13.200 --> 00:57:16.230
valuable, right? Um but but we're early
00:57:16.240 --> 00:57:17.910
we're early in there.
00:57:17.920 --> 00:57:20.350
Excellent. All right. Well, we're almost
00:57:20.360 --> 00:57:22.470
out of time. Um
00:57:22.480 --> 00:57:24.390
well
00:57:24.400 --> 00:57:26.150
>> We are this is hopefully this was useful
00:57:26.160 --> 00:57:28.110
to somebody who's going to watch it.
00:57:28.120 --> 00:57:30.950
Shane I I thought it was interesting to
00:57:30.960 --> 00:57:33.350
talk about. We we a lot of things. We
00:57:33.360 --> 00:57:35.470
hit, you know, context, knowledge
00:57:35.480 --> 00:57:37.390
engineering, data products, product
00:57:37.400 --> 00:57:38.710
practice,
00:57:38.720 --> 00:57:40.710
self-service,
00:57:40.720 --> 00:57:42.790
and I think like the takeaway from today
00:57:42.800 --> 00:57:44.950
is just like
00:57:44.960 --> 00:57:47.390
a lot is changing systemically, and I
00:57:47.400 --> 00:57:49.270
think data practitioners need to think
00:57:49.280 --> 00:57:50.830
in systems, they need to experiment, and
00:57:50.840 --> 00:57:52.470
they need to
00:57:52.480 --> 00:57:55.070
um
00:57:55.080 --> 00:57:56.390
they need to talk to each other. We have
00:57:56.400 --> 00:57:57.670
to learn from each other. That I think
00:57:57.680 --> 00:57:59.630
it makes things like content like this
00:57:59.640 --> 00:58:01.190
or
00:58:01.200 --> 00:58:02.790
podcasts or videos or going to
00:58:02.800 --> 00:58:04.710
conferences, like you've got to talk to
00:58:04.720 --> 00:58:05.990
other data people who aren't at your
00:58:06.000 --> 00:58:08.270
company. Um um
00:58:08.280 --> 00:58:09.870
because
00:58:09.880 --> 00:58:11.230
hopefully you're also talking to the
00:58:11.240 --> 00:58:12.710
data people at your company, but not at
00:58:12.720 --> 00:58:14.630
your company and figuring out like, how
00:58:14.640 --> 00:58:15.990
are you tackling this chain? How are you
00:58:16.000 --> 00:58:17.350
tackling this context engineering
00:58:17.360 --> 00:58:19.390
problem? Or like, are are you guys
00:58:19.400 --> 00:58:20.510
building your own catalog? Like I said,
00:58:20.520 --> 00:58:21.790
I was at the pool with another mom, and
00:58:21.800 --> 00:58:23.190
she's like, "Oh, we're building our own
00:58:23.200 --> 00:58:24.550
catalog." And I'm like, "Tell me all
00:58:24.560 --> 00:58:26.430
about this." Like we we have to talk to
00:58:26.440 --> 00:58:29.230
each other because
00:58:29.240 --> 00:58:30.630
there's too much experimentation to be
00:58:30.640 --> 00:58:31.830
done, and we should be learning across
00:58:31.840 --> 00:58:33.910
companies and not only within companies.
00:58:33.920 --> 00:58:35.230
>> Yeah, and the good thing with the office
00:58:35.240 --> 00:58:37.270
sessions is we get to catch up on the
00:58:37.280 --> 00:58:39.870
for you and me, it's the third Friday of
00:58:39.880 --> 00:58:41.470
the month-ish, depending on how our
00:58:41.480 --> 00:58:42.870
schedules look.
00:58:42.880 --> 00:58:45.070
So, if anybody's got a question just and
00:58:45.080 --> 00:58:46.350
they're watching this, just type the
00:58:46.360 --> 00:58:48.710
question in the Substack article.
00:58:48.720 --> 00:58:50.510
Send us a note on LinkedIn, right? DM
00:58:50.520 --> 00:58:52.750
us. And then we we talked to enough
00:58:52.760 --> 00:58:54.470
people that we can actually go and talk
00:58:54.480 --> 00:58:56.510
to some others between you asking that
00:58:56.520 --> 00:58:58.990
question now and us coming back on.
00:58:59.000 --> 00:59:01.070
And maybe come back with an answer from
00:59:01.080 --> 00:59:02.750
a couple of people and say you know, we
00:59:02.760 --> 00:59:03.750
talked to some people and they were
00:59:03.760 --> 00:59:05.750
doing this, we found it interesting. Or
00:59:05.760 --> 00:59:07.670
as always, hop on the show itself and
00:59:07.680 --> 00:59:10.910
ask us a question live, and if we can,
00:59:10.920 --> 00:59:12.150
we will.
00:59:12.160 --> 00:59:15.230
If if like Raul, you've got a link to a
00:59:15.240 --> 00:59:17.270
product or piece of technology or
00:59:17.280 --> 00:59:20.190
something, maybe yeah, give us a bit of
00:59:20.200 --> 00:59:21.470
a heads-up so that we [laughter] can
00:59:21.480 --> 00:59:23.430
both go and do a little bit of reading
00:59:23.440 --> 00:59:25.270
before we're there. We may have have of
00:59:25.280 --> 00:59:26.110
it,
00:59:26.120 --> 00:59:28.110
and we may not have, but I'm definitely
00:59:28.120 --> 00:59:29.430
going to go and look up that one now cuz
00:59:29.440 --> 00:59:31.270
I'm kind of intrigued. So,
00:59:31.280 --> 00:59:33.230
yeah, that was great. But thank you and
00:59:33.240 --> 00:59:36.510
we'll see you in a month's time.
00:59:36.520 --> 00:59:38.550
>> Great. This was fun, Shane. Have a good
00:59:38.560 --> 00:59:39.430
afternoon.
00:59:39.440 --> 00:59:40.310
>> Bye.
00:59:40.320 --> 00:59:42.440
>> Okay.