WEBVTT
Kind: captions
Language: en
00:00:01.920 --> 00:00:03.590
Yeah. So, what's what's been happening?
00:00:03.600 --> 00:00:04.870
What's top of mind? I know what's top of
00:00:04.880 --> 00:00:06.230
mind for me at the moment, but yeah.
00:00:06.240 --> 00:00:07.190
What do you got?
00:00:07.200 --> 00:00:10.310
>> I mean, for me it's Duck Labs and that
00:00:10.320 --> 00:00:11.669
change of the ecosystem. That's been
00:00:11.679 --> 00:00:13.110
really interesting and see how things
00:00:13.120 --> 00:00:15.669
are developing there. Um, it's
00:00:15.679 --> 00:00:18.070
interesting like Snowflake and Data
00:00:18.080 --> 00:00:19.590
Bricks kind of releasing numbers around
00:00:19.600 --> 00:00:21.750
DBT and like how many like how much it's
00:00:21.760 --> 00:00:24.470
moving into into their systems at a
00:00:24.480 --> 00:00:26.310
pretty breaknet
00:00:26.320 --> 00:00:29.349
pace. Um you know there's a lot of
00:00:29.359 --> 00:00:31.429
development happening in the park space
00:00:31.439 --> 00:00:33.110
um for data scans which I was like
00:00:33.120 --> 00:00:34.790
parquet gets upgrades like when is this
00:00:34.800 --> 00:00:36.709
happening um [laughter]
00:00:36.719 --> 00:00:38.630
>> isn't isn't park dead right it's got
00:00:38.640 --> 00:00:40.869
iceberg right it's it's like right what
00:00:40.879 --> 00:00:43.590
was parro [laughter]
00:00:43.600 --> 00:00:45.830
didn't you know it um it's a database
00:00:45.840 --> 00:00:48.549
and it's just like yeah um
00:00:48.559 --> 00:00:49.990
>> yeah those are some of the things on on
00:00:50.000 --> 00:00:54.389
top of mind I mean obviously AI AI AI um
00:00:54.399 --> 00:00:56.470
because we all got to talk about it
00:00:56.480 --> 00:00:58.790
>> uh but I would say
00:00:58.800 --> 00:01:00.790
uh it you know the other thing that's
00:01:00.800 --> 00:01:02.790
also top of mind now is like how fast
00:01:02.800 --> 00:01:04.630
all these like frontier models are
00:01:04.640 --> 00:01:09.590
converging like so fast and also I the
00:01:09.600 --> 00:01:13.109
cost is spiking so fast
00:01:13.119 --> 00:01:15.270
like it's I I was like experimenting
00:01:15.280 --> 00:01:20.550
with like um Mark Freeman's um Petri I
00:01:20.560 --> 00:01:23.190
had to shut it down for a minute tokens
00:01:23.200 --> 00:01:25.109
were blowing up in cost and I was like
00:01:25.119 --> 00:01:27.590
oh my god this is this is like real real
00:01:27.600 --> 00:01:30.710
cash is gonna cost me now.
00:01:30.720 --> 00:01:32.870
>> Yeah, it's it's moving fast, man. Um
00:01:32.880 --> 00:01:34.870
yeah, shout out to Jeff Long and Nick Z
00:01:34.880 --> 00:01:36.069
joined us. So, if you got anything you
00:01:36.079 --> 00:01:37.990
want to ask or want us to chat about,
00:01:38.000 --> 00:01:39.590
let us know in the chat. If you don't,
00:01:39.600 --> 00:01:42.630
we'll just carry on. Um yeah, it's it's
00:01:42.640 --> 00:01:44.630
moving fast and I've been kind of spent
00:01:44.640 --> 00:01:46.230
the whole month in build mode, right?
00:01:46.240 --> 00:01:48.069
So, as I said on LinkedIn, I've spent
00:01:48.079 --> 00:01:49.590
bugger all time researching and I've
00:01:49.600 --> 00:01:51.109
seen these these things come out, you
00:01:51.119 --> 00:01:54.230
know, like the orchestra uh getting
00:01:54.240 --> 00:01:56.310
funding, right? So, you know, congrats
00:01:56.320 --> 00:01:58.469
to that team. But I've got big questions
00:01:58.479 --> 00:02:01.109
about where do orchestration live these
00:02:01.119 --> 00:02:03.670
days and then data bricks with their
00:02:03.680 --> 00:02:05.749
accelerators, right? And I'm like, oh,
00:02:05.759 --> 00:02:07.749
are we cookie cuttering organizations
00:02:07.759 --> 00:02:11.510
again or we not? Um, yeah. And then DBT
00:02:11.520 --> 00:02:14.470
Labs, right? Like, oh my god, you know,
00:02:14.480 --> 00:02:17.430
and then Patchy Aussie, right? The OSI
00:02:17.440 --> 00:02:19.830
stuff's now actually under Apache, so in
00:02:19.840 --> 00:02:22.309
theory it can become a standard, but
00:02:22.319 --> 00:02:24.790
will it? And then uh Google just came
00:02:24.800 --> 00:02:26.390
out with BigQuery graph, right, which
00:02:26.400 --> 00:02:28.229
I've been playing around for measurement
00:02:28.239 --> 00:02:31.270
calculations. So it's a quasi metric
00:02:31.280 --> 00:02:33.190
layer and I'm like, oh, what does that
00:02:33.200 --> 00:02:35.670
actually mean to me and our platform? So
00:02:35.680 --> 00:02:38.229
yeah, you but you tend to keep up a lot
00:02:38.239 --> 00:02:41.509
more uh sometimes what's going on in the
00:02:41.519 --> 00:02:43.190
market. So let's go through some of
00:02:43.200 --> 00:02:45.750
those because I'm kind of it's kind of
00:02:45.760 --> 00:02:47.030
free research for me, right? I don't
00:02:47.040 --> 00:02:48.630
even need to perplexity. I just asked
00:02:48.640 --> 00:02:49.910
Tim on on
00:02:49.920 --> 00:02:51.190
>> Yeah, I mean that's fair. [laughter]
00:02:51.200 --> 00:02:53.350
This month the kids out of school and
00:02:53.360 --> 00:02:55.670
you know I have a as small fle I'm not
00:02:55.680 --> 00:02:58.550
as as on it as I would like typically. I
00:02:58.560 --> 00:03:00.470
know what are you going to do? Uh I do
00:03:00.480 --> 00:03:02.390
you know and I typically do write like a
00:03:02.400 --> 00:03:03.910
like a weekly but I took a two week
00:03:03.920 --> 00:03:05.670
pause just like chaos [clears throat]
00:03:05.680 --> 00:03:07.910
sessions but you know I still feel
00:03:07.920 --> 00:03:09.430
pretty close to everything that's going
00:03:09.440 --> 00:03:12.309
on right now. Um yeah I mean a lot of
00:03:12.319 --> 00:03:13.830
fundraising. I mean shout out to
00:03:13.840 --> 00:03:15.830
orchestra. I'll be honest I am like
00:03:15.840 --> 00:03:17.270
pretty biased towards them. I am an
00:03:17.280 --> 00:03:19.430
investor just to publicly state that so
00:03:19.440 --> 00:03:21.589
nobody thinks I'm inflating my bias
00:03:21.599 --> 00:03:24.149
towards them Hugo and team I very much
00:03:24.159 --> 00:03:26.710
strongly believe in their kind of play I
00:03:26.720 --> 00:03:30.070
love tools like um Airflow and there's
00:03:30.080 --> 00:03:32.789
these quasi open source ones like Kestra
00:03:32.799 --> 00:03:35.270
you know it's like
00:03:35.280 --> 00:03:36.710
that are all like really interesting in
00:03:36.720 --> 00:03:39.190
different plays um yeah I mean I've been
00:03:39.200 --> 00:03:42.550
using that their platform well over over
00:03:42.560 --> 00:03:44.789
a year and a bit now and yeah I'm pretty
00:03:44.799 --> 00:03:47.030
I'm very pleased with kind of like the
00:03:47.040 --> 00:03:50.630
velocity these tools are showing. Um
00:03:50.640 --> 00:03:52.149
it's also interesting in the play like
00:03:52.159 --> 00:03:54.149
orchestration inside of the platforms is
00:03:54.159 --> 00:03:56.949
starting to show up like like DBT is
00:03:56.959 --> 00:03:58.869
kind of starting to lean towards it but
00:03:58.879 --> 00:04:01.030
not not it's like one foot in the pool
00:04:01.040 --> 00:04:03.270
not in kind of thing. So, it's kind of
00:04:03.280 --> 00:04:05.509
like always been this like super awkward
00:04:05.519 --> 00:04:08.070
DBT cloud play, but I'm starting to see
00:04:08.080 --> 00:04:11.750
it like like morph into Fiverr
00:04:11.760 --> 00:04:14.550
essentially, right? Um, and where Fiverr
00:04:14.560 --> 00:04:16.789
and DBT cloud like I would not be
00:04:16.799 --> 00:04:18.310
surprised in in the next like couple
00:04:18.320 --> 00:04:20.310
weeks at DBT like that you start to see
00:04:20.320 --> 00:04:24.150
it more or less become like one surface.
00:04:24.160 --> 00:04:27.350
>> Oh, well, got to be the goal, right? Um,
00:04:27.360 --> 00:04:29.430
and and maybe one billing plan. No,
00:04:29.440 --> 00:04:30.310
probably not, right?
00:04:30.320 --> 00:04:32.469
>> Yeah. I mean it it's it's going to get
00:04:32.479 --> 00:04:34.310
close, right? Like and and you know with
00:04:34.320 --> 00:04:36.469
Census and Fiverr and all these like
00:04:36.479 --> 00:04:38.390
they're all kind of like conglomerating
00:04:38.400 --> 00:04:39.749
one. I mean I'm pretty big fan of
00:04:39.759 --> 00:04:42.070
Fiverr. I think they've done a a pretty
00:04:42.080 --> 00:04:45.110
pretty good product over time. Um you
00:04:45.120 --> 00:04:48.790
know I use competitors like Estuary. Um
00:04:48.800 --> 00:04:50.390
you know they all have pros and cons and
00:04:50.400 --> 00:04:51.909
changes right it's the same thing as
00:04:51.919 --> 00:04:54.310
like using open source tools like DT you
00:04:54.320 --> 00:04:56.790
know Python. You and I are old enough
00:04:56.800 --> 00:04:58.550
like have been using like the Singer
00:04:58.560 --> 00:05:00.230
framework. I actually know one of my
00:05:00.240 --> 00:05:02.070
friends is one of the original engineers
00:05:02.080 --> 00:05:03.510
on that on that standard
00:05:03.520 --> 00:05:05.350
>> on on Singer with Singer Tech.
00:05:05.360 --> 00:05:06.870
>> Yeah. Yeah. He worked he worked for
00:05:06.880 --> 00:05:07.909
them.
00:05:07.919 --> 00:05:10.790
>> We um Yeah. We we started we we kind of
00:05:10.800 --> 00:05:13.189
started off using Milano, right? Uh
00:05:13.199 --> 00:05:15.430
because Singer had been bought, so we
00:05:15.440 --> 00:05:17.110
kind of thought it was dead. And then
00:05:17.120 --> 00:05:18.390
right at the beginning, I think was it
00:05:18.400 --> 00:05:20.550
Milano versus Airflow back then?
00:05:20.560 --> 00:05:22.230
>> We went down the we went down the Milano
00:05:22.240 --> 00:05:23.590
line because we didn't Well, in
00:05:23.600 --> 00:05:25.029
hindsight, it was okay because we didn't
00:05:25.039 --> 00:05:27.749
need orchestration. Um, but you know,
00:05:27.759 --> 00:05:29.670
kind of probably wasn't the best choice
00:05:29.680 --> 00:05:32.550
given Milano kind of died a bit. I mean,
00:05:32.560 --> 00:05:35.430
I know Aaron's bought it and and uh is
00:05:35.440 --> 00:05:37.350
reinventing it. So, shout out to Aaron
00:05:37.360 --> 00:05:40.469
here in in uh the UK. Um, and he's a
00:05:40.479 --> 00:05:43.029
Kiwi, so good on him. Um,
00:05:43.039 --> 00:05:44.390
but yeah, I I don't know. I've always
00:05:44.400 --> 00:05:46.230
had a thing with uh open source and
00:05:46.240 --> 00:05:49.350
things like WT Labs uh and DBT, right?
00:05:49.360 --> 00:05:51.350
I'm like, ah. Uh but yeah know we we use
00:05:51.360 --> 00:05:54.070
Milano and we still do like uh we had to
00:05:54.080 --> 00:05:55.749
rewrite it a little bit because we want
00:05:55.759 --> 00:05:57.350
to run it serverless and back then you
00:05:57.360 --> 00:05:59.430
had to have a persistent server. Yeah.
00:05:59.440 --> 00:06:01.590
>> So Nigel did his magic but no we you
00:06:01.600 --> 00:06:03.590
know we actually uh we just did a whole
00:06:03.600 --> 00:06:06.150
lot of work on using it to do uh data
00:06:06.160 --> 00:06:09.270
collection off YouTube. Um, and so yeah,
00:06:09.280 --> 00:06:11.270
so it's still a I don't know how much of
00:06:11.280 --> 00:06:13.590
the code we run is still a fork of the
00:06:13.600 --> 00:06:16.070
original multano or not, but but the
00:06:16.080 --> 00:06:17.749
kind of y
00:06:17.759 --> 00:06:19.270
>> no I was going to say yeah I mean with
00:06:19.280 --> 00:06:20.950
the AI tooling now with like clear
00:06:20.960 --> 00:06:22.790
patterns you're like ah you know before
00:06:22.800 --> 00:06:24.550
I would always avoid doing this tap
00:06:24.560 --> 00:06:26.550
target kind of thing or like grabbing
00:06:26.560 --> 00:06:27.990
data but you're like now I have a strong
00:06:28.000 --> 00:06:31.110
pattern like uh cool. Oh, and actually
00:06:31.120 --> 00:06:33.189
if you're vibe coding, I don't even know
00:06:33.199 --> 00:06:35.590
where the the source of the code pattern
00:06:35.600 --> 00:06:37.029
comes from, right? Is it, you know, if
00:06:37.039 --> 00:06:38.309
I'm asking it to go and build a
00:06:38.319 --> 00:06:41.830
collector um from scratch, not using our
00:06:41.840 --> 00:06:43.909
framework, which is any framework, is it
00:06:43.919 --> 00:06:45.990
using airflow? Is it using mult is it
00:06:46.000 --> 00:06:48.950
using like where where's the actual IP
00:06:48.960 --> 00:06:51.590
coming from? Um, but the orchestra
00:06:51.600 --> 00:06:52.870
thing, right? Like I was kind of sitting
00:06:52.880 --> 00:06:56.309
back and thinking about it and
00:06:56.319 --> 00:06:57.510
I'm kind of interested in your view
00:06:57.520 --> 00:06:59.189
architecturally on this, right? So my my
00:06:59.199 --> 00:07:00.469
thought process went something like
00:07:00.479 --> 00:07:04.790
this. Uh orchestras for ages have been
00:07:04.800 --> 00:07:07.510
uh dag glue, right? You know, nodes and
00:07:07.520 --> 00:07:10.790
links put together, bundled up and then
00:07:10.800 --> 00:07:13.029
kind of scheduled, right? Uh and then we
00:07:13.039 --> 00:07:14.870
got I think it was Informatica and a
00:07:14.880 --> 00:07:15.830
whole other ones. We started getting
00:07:15.840 --> 00:07:17.670
trigger based ones with a little bit
00:07:17.680 --> 00:07:19.670
more granularity rather than just big
00:07:19.680 --> 00:07:22.629
blobs of DAGs. Uh we in our platform we
00:07:22.639 --> 00:07:24.629
do it slightly differently. uh we hold
00:07:24.639 --> 00:07:26.790
the context of the relationships, the
00:07:26.800 --> 00:07:29.029
nodes and links in our context plane and
00:07:29.039 --> 00:07:31.510
then every time new data turns in or we
00:07:31.520 --> 00:07:33.830
we have a schedule trigger. We'll read
00:07:33.840 --> 00:07:36.230
that context, we'll create a manifest of
00:07:36.240 --> 00:07:38.390
the steps, you know, nodes and links, a
00:07:38.400 --> 00:07:41.029
big tag effectively. We'll execute it
00:07:41.039 --> 00:07:43.749
and then we blow it away. So the
00:07:43.759 --> 00:07:47.029
schedule is not actually the the pet,
00:07:47.039 --> 00:07:49.909
it's cattle. The context around the the
00:07:49.919 --> 00:07:51.749
dependencies, right, is what we care
00:07:51.759 --> 00:07:54.309
about. Um, but that's still a very batch
00:07:54.319 --> 00:07:56.150
orientated, right? Like it's a micro
00:07:56.160 --> 00:07:58.390
batch and because it's Big Query, we can
00:07:58.400 --> 00:08:00.550
grab that manifest and run a ton of
00:08:00.560 --> 00:08:02.309
a manifest in a very small amount of
00:08:02.319 --> 00:08:05.110
time on BigQuery and then pay for it.
00:08:05.120 --> 00:08:07.110
But it's still bundling a boundary
00:08:07.120 --> 00:08:08.869
around a series of nodes and links,
00:08:08.879 --> 00:08:11.830
right, for for that orchestration pan.
00:08:11.840 --> 00:08:13.350
And I keep coming back to the way I'm
00:08:13.360 --> 00:08:16.629
using LLMs and agents now. uh and I'm
00:08:16.639 --> 00:08:19.029
like actually is a mesh architecture
00:08:19.039 --> 00:08:21.909
where I really need to move to because
00:08:21.919 --> 00:08:25.350
if I say that each node is let's call it
00:08:25.360 --> 00:08:26.629
an agent for now right it's an
00:08:26.639 --> 00:08:29.189
encapsulation of some context and some
00:08:29.199 --> 00:08:31.029
execution
00:08:31.039 --> 00:08:33.750
and then I can say that each of those
00:08:33.760 --> 00:08:36.070
those agents have a relationship with
00:08:36.080 --> 00:08:37.750
another agent it has a dependency and I
00:08:37.760 --> 00:08:40.550
use data contracts for that do I need an
00:08:40.560 --> 00:08:42.550
orchestra because now what I've got an
00:08:42.560 --> 00:08:45.590
orchestrator uh I've got a bunch of
00:08:45.600 --> 00:08:47.670
nodes that are separate that understand
00:08:47.680 --> 00:08:50.550
who they rely on and who relies on them.
00:08:50.560 --> 00:08:51.990
They understand when to trigger
00:08:52.000 --> 00:08:53.430
themselves. They understand what their
00:08:53.440 --> 00:08:56.870
job is. Like I don't need a manifest
00:08:56.880 --> 00:08:58.389
because they can just bloody fire
00:08:58.399 --> 00:09:01.430
whenever they need to. Chaos to try and
00:09:01.440 --> 00:09:02.949
understand how they're running and
00:09:02.959 --> 00:09:04.470
troubleshoot when something goes wrong,
00:09:04.480 --> 00:09:05.670
right? But that's kind of where my
00:09:05.680 --> 00:09:07.030
head's at the moment. So what what's
00:09:07.040 --> 00:09:09.430
your view? Orchestration still a key
00:09:09.440 --> 00:09:10.310
thing?
00:09:10.320 --> 00:09:12.161
>> Yeah.
00:09:12.171 --> 00:09:15.110
[laughter]
00:09:15.120 --> 00:09:17.190
a little bit more detail, please.
00:09:17.200 --> 00:09:21.990
>> Yeah. Uh well, I think it's like it's um
00:09:22.000 --> 00:09:23.350
I think you're on to something. I think
00:09:23.360 --> 00:09:24.710
you're seeing patterns in the industry
00:09:24.720 --> 00:09:25.930
about like how all these things
00:09:25.940 --> 00:09:27.030
[clears throat] are kind of like meshing
00:09:27.040 --> 00:09:29.670
together. I think orchestration I mean
00:09:29.680 --> 00:09:31.350
at the end of the day like you know good
00:09:31.360 --> 00:09:35.350
old DAG uh is still it's something that
00:09:35.360 --> 00:09:37.269
it's like under under the hood. Is it
00:09:37.279 --> 00:09:41.110
going to be a completely separate tool?
00:09:41.120 --> 00:09:43.990
Um, I mean, even looking at orchestra or
00:09:44.000 --> 00:09:45.590
Castra and like everybody how they're
00:09:45.600 --> 00:09:47.910
like morphing and changing. No, I don't
00:09:47.920 --> 00:09:49.269
think I don't think so. I think there
00:09:49.279 --> 00:09:51.110
are actually times where you should you
00:09:51.120 --> 00:09:53.110
do want it to be completely separate.
00:09:53.120 --> 00:09:55.269
Usually those are like when you want
00:09:55.279 --> 00:09:57.509
really isolated, really specific, high
00:09:57.519 --> 00:09:59.910
reliable tools and I think that's like
00:09:59.920 --> 00:10:03.190
really important. um like dead simple
00:10:03.200 --> 00:10:05.030
orchestration.
00:10:05.040 --> 00:10:06.470
But I mean ultimately at the end of the
00:10:06.480 --> 00:10:10.630
day in a vast majority of data work is
00:10:10.640 --> 00:10:13.350
like does this deliver value right? Do I
00:10:13.360 --> 00:10:15.269
need to have this completely separate
00:10:15.279 --> 00:10:19.269
airflow tool to move all these things or
00:10:19.279 --> 00:10:21.750
does snowflakes orchestration or
00:10:21.760 --> 00:10:24.389
ingestion patterns in mother duck solve
00:10:24.399 --> 00:10:27.030
my problem? Right? Like vast majority of
00:10:27.040 --> 00:10:29.750
the time it's yes. It's like you're kind
00:10:29.760 --> 00:10:35.030
of like over overindexing on toys. Um,
00:10:35.040 --> 00:10:36.870
and then not really delivering value.
00:10:36.880 --> 00:10:37.990
And we talked about this all the time
00:10:38.000 --> 00:10:39.430
really between you and I. We're like, do
00:10:39.440 --> 00:10:41.750
you need another tool? Um,
00:10:41.760 --> 00:10:43.670
>> yeah. And and yeah, does your data
00:10:43.680 --> 00:10:45.190
platform become the harness for your
00:10:45.200 --> 00:10:49.110
data, right, with GI tooling? Is that is
00:10:49.120 --> 00:10:50.870
that something where the bang for buck
00:10:50.880 --> 00:10:52.230
you're going to get out of them with the
00:10:52.240 --> 00:10:53.910
way they experiment and the way they do
00:10:53.920 --> 00:10:55.829
integration? Uh, is that where the value
00:10:55.839 --> 00:10:57.670
is? I mean, I I was sitting on a webinar
00:10:57.680 --> 00:11:00.790
the other day and uh it was an I it was
00:11:00.800 --> 00:11:02.550
an IBM thing. Well, no, it was somebody
00:11:02.560 --> 00:11:05.430
had an IBM person on there and they were
00:11:05.440 --> 00:11:06.870
talking about the value of the IBM
00:11:06.880 --> 00:11:08.630
platform and it was integration and
00:11:08.640 --> 00:11:10.710
yeah, I was in a snarky mix snarky mood
00:11:10.720 --> 00:11:13.670
and but it was still factual, right? And
00:11:13.680 --> 00:11:15.829
my view was you know decades ago I was
00:11:15.839 --> 00:11:17.590
around the time of Oracle IBM
00:11:17.600 --> 00:11:19.190
Informatica
00:11:19.200 --> 00:11:21.350
and we because I worked for one of them
00:11:21.360 --> 00:11:23.670
sold the about oh all our tools
00:11:23.680 --> 00:11:25.670
are integrated and then I watch you know
00:11:25.680 --> 00:11:27.990
Oracle by Peopleoft and not integrated
00:11:28.000 --> 00:11:31.110
into Oracle apps and by Hyperion was it
00:11:31.120 --> 00:11:32.470
and not really you know different
00:11:32.480 --> 00:11:34.470
technology stack cobbled together not
00:11:34.480 --> 00:11:36.790
fully integrated and I'm like
00:11:36.800 --> 00:11:39.829
on the integration story but now when we
00:11:39.839 --> 00:11:41.350
look at things like snowflake and data
00:11:41.360 --> 00:11:43.350
bricks and and I'm going to ignore
00:11:43.360 --> 00:11:45.670
fabric, right? Um
00:11:45.680 --> 00:11:49.269
maybe like are they integrated or is it
00:11:49.279 --> 00:11:51.430
still a veneer of integration over the
00:11:51.440 --> 00:11:53.750
top?
00:11:53.760 --> 00:11:56.150
>> Are they integrated? No. No. I mean, yes
00:11:56.160 --> 00:11:58.710
and no. Like it's so funny like I mean I
00:11:58.720 --> 00:12:00.150
think this uh we had a conversation
00:12:00.160 --> 00:12:02.150
about that. is like the sizzle reel
00:12:02.160 --> 00:12:03.829
problem where you see the YouTube video
00:12:03.839 --> 00:12:05.910
this awesome thing on LinkedIn and then
00:12:05.920 --> 00:12:07.670
you go to actually use the thing and
00:12:07.680 --> 00:12:09.670
then you're just like I mean I guess
00:12:09.680 --> 00:12:13.590
like directionally yeah it connects but
00:12:13.600 --> 00:12:15.269
then you actually get into like the
00:12:15.279 --> 00:12:17.110
actual integrations or their so-called
00:12:17.120 --> 00:12:19.269
like open support or open protocol
00:12:19.279 --> 00:12:21.990
supports and devil's in the details and
00:12:22.000 --> 00:12:23.350
that's what you usually end up spending
00:12:23.360 --> 00:12:26.470
like vast majority of your time is
00:12:26.480 --> 00:12:29.350
fixing that stuff right um yeah there
00:12:29.360 --> 00:12:31.110
are plenty of times that you know I've
00:12:31.120 --> 00:12:32.550
used like Snowflake I've done a little
00:12:32.560 --> 00:12:35.509
bit on data bricks um
00:12:35.519 --> 00:12:37.350
and yeah I mean they'll say this and
00:12:37.360 --> 00:12:38.870
then like you'll look go down it's
00:12:38.880 --> 00:12:40.949
almost like this asterisk
00:12:40.959 --> 00:12:43.030
and you scroll down and you're like oh
00:12:43.040 --> 00:12:46.230
okay it's not it's not quite supported
00:12:46.240 --> 00:12:48.550
and there's no like ETA of like when
00:12:48.560 --> 00:12:49.910
it's going to be fully supported just
00:12:49.920 --> 00:12:52.629
says coming soon.
00:12:52.639 --> 00:12:54.550
Yeah, I mean we we had that with Google
00:12:54.560 --> 00:12:57.990
um graph, right? Uh because it looked
00:12:58.000 --> 00:13:00.470
like it helped us uh build out our
00:13:00.480 --> 00:13:03.350
semantic BI semantic layer or metrics
00:13:03.360 --> 00:13:04.949
layer, right? In a more robust way than
00:13:04.959 --> 00:13:09.509
we had. Um so you know it was uh early
00:13:09.519 --> 00:13:12.550
adopter or it was prega and so yeah we
00:13:12.560 --> 00:13:14.389
jump in there and I start creating
00:13:14.399 --> 00:13:16.629
metrics and it only let me create
00:13:16.639 --> 00:13:19.269
measures and yeah I use a a language of
00:13:19.279 --> 00:13:21.590
facts for me uh immutable numbers that
00:13:21.600 --> 00:13:24.550
are in the source system you know uh you
00:13:24.560 --> 00:13:27.030
account a number that's been created
00:13:27.040 --> 00:13:29.750
that that's facts uh for me a measure is
00:13:29.760 --> 00:13:33.190
is uh an aggregation of sum account that
00:13:33.200 --> 00:13:35.190
kind of stuff and a metric is where I
00:13:35.200 --> 00:13:37.269
get a complex formula this over this or
00:13:37.279 --> 00:13:39.750
this divided by this or window on this
00:13:39.760 --> 00:13:42.069
thing divided by a window on this thing
00:13:42.079 --> 00:13:45.030
minusing max on this thing right those
00:13:45.040 --> 00:13:47.670
complex calculations so yeah I go in
00:13:47.680 --> 00:13:50.470
there uh pull through a fact yeah can
00:13:50.480 --> 00:13:53.590
see it in my in my tools I go and do a a
00:13:53.600 --> 00:13:55.910
measure so I go to a sum yeah she works
00:13:55.920 --> 00:13:58.069
pretty good I'm impressed and then I go
00:13:58.079 --> 00:14:00.550
to do a metric and it goes no [laughter]
00:14:00.560 --> 00:14:02.949
but you can't do that now now it's
00:14:02.959 --> 00:14:05.269
coming out very soon uh as a new feature
00:14:05.279 --> 00:14:07.269
But then I automatically go okay but now
00:14:07.279 --> 00:14:09.350
I want some filters you know effectively
00:14:09.360 --> 00:14:12.310
wear clauses dynamic I want windowing
00:14:12.320 --> 00:14:14.790
functions and then I go well it's not
00:14:14.800 --> 00:14:17.030
that it's not integrated because it is
00:14:17.040 --> 00:14:18.550
it's just that it doesn't have feature
00:14:18.560 --> 00:14:20.470
parity of what I actually need right
00:14:20.480 --> 00:14:22.710
now. So how much of that do you reckon
00:14:22.720 --> 00:14:24.629
is with the big platform plays right
00:14:24.639 --> 00:14:27.430
it's it's integrated but just doesn't
00:14:27.440 --> 00:14:30.310
have feature parody uh with the best of
00:14:30.320 --> 00:14:34.710
breed of what you actually need.
00:14:34.720 --> 00:14:36.629
>> I mean
00:14:36.639 --> 00:14:39.010
oh this makes me feel so old saying this
00:14:39.020 --> 00:14:39.750
[laughter]
00:14:39.760 --> 00:14:42.629
makes me it it depends on the business.
00:14:42.639 --> 00:14:45.030
I know it's such a copout. Um but no
00:14:45.040 --> 00:14:51.590
realistically like uh
00:14:51.600 --> 00:14:53.350
you know I mean I would say very rarely
00:14:53.360 --> 00:14:55.509
do you need the bre best of breed and of
00:14:55.519 --> 00:14:57.430
course like you know me and like how
00:14:57.440 --> 00:14:59.750
hard I have a hard time not choosing the
00:14:59.760 --> 00:15:03.430
best of breed. Uh like chronically
00:15:03.440 --> 00:15:05.750
chronic chronic chronically on the edge
00:15:05.760 --> 00:15:08.310
of choosing platforms that are sometimes
00:15:08.320 --> 00:15:12.470
maybe a little too young. Um but then
00:15:12.480 --> 00:15:13.590
like you know sometimes I'll meet the
00:15:13.600 --> 00:15:15.189
founders and I'll be like yeah I
00:15:15.199 --> 00:15:18.870
actually think you guys um are women uh
00:15:18.880 --> 00:15:21.269
can uh pull together and I think that
00:15:21.279 --> 00:15:23.269
you I think there's enough you know one
00:15:23.279 --> 00:15:26.470
of you know
00:15:26.480 --> 00:15:29.189
circular answer is
00:15:29.199 --> 00:15:31.829
um
00:15:31.839 --> 00:15:33.350
yeah I mean it just really depends on
00:15:33.360 --> 00:15:34.470
the business like if you're like a
00:15:34.480 --> 00:15:36.629
certain size and you you can take that
00:15:36.639 --> 00:15:38.550
take that bet and really like swing for
00:15:38.560 --> 00:15:40.470
it absolutely you should do the best
00:15:40.480 --> 00:15:42.230
contrary, I think you should do the best
00:15:42.240 --> 00:15:43.910
of breed, right? and really trying to
00:15:43.920 --> 00:15:45.670
find the angle to really find the
00:15:45.680 --> 00:15:47.590
business because like right now with AI
00:15:47.600 --> 00:15:49.509
flattening so many processes and things
00:15:49.519 --> 00:15:52.069
like the money is in the like the little
00:15:52.079 --> 00:15:53.350
the the edges that you're going to get
00:15:53.360 --> 00:15:56.949
the 1% the 2% the 3% right um if
00:15:56.959 --> 00:15:58.550
everybody's building their the world's
00:15:58.560 --> 00:16:01.110
best cloudbot or grockbot these days um
00:16:01.120 --> 00:16:03.030
to optimize the business like what is
00:16:03.040 --> 00:16:05.509
your edge right if you can literally if
00:16:05.519 --> 00:16:07.749
everyone can just go on a you know repo
00:16:07.759 --> 00:16:09.829
and grab the you know the single best
00:16:09.839 --> 00:16:12.710
book written on operating procedures
00:16:12.720 --> 00:16:14.550
Just consider everybody else has that
00:16:14.560 --> 00:16:16.870
exact same book. So what is your actual
00:16:16.880 --> 00:16:18.870
edge on all this stuff? Could it be
00:16:18.880 --> 00:16:20.790
tools? Yeah, I can say it could be it
00:16:20.800 --> 00:16:23.189
could be tools. Um honestly I think it's
00:16:23.199 --> 00:16:24.629
just going to be the ability to make you
00:16:24.639 --> 00:16:27.030
know fast, you know, decisions like take
00:16:27.040 --> 00:16:31.110
an action next step. Next step. Um
00:16:31.120 --> 00:16:33.030
so yeah, you know, sometimes I think you
00:16:33.040 --> 00:16:35.350
know, you know, best of breed can help.
00:16:35.360 --> 00:16:37.509
Um
00:16:37.519 --> 00:16:40.710
I don't know. What do you think? Uh so I
00:16:40.720 --> 00:16:42.389
think what you got to do is you got to
00:16:42.399 --> 00:16:43.749
you got to write down a set of core
00:16:43.759 --> 00:16:46.470
principles, right?
00:16:46.480 --> 00:16:49.189
>> And they're this over that. Um and
00:16:49.199 --> 00:16:50.389
actually, you know, really interesting
00:16:50.399 --> 00:16:52.389
example over the last month or so, which
00:16:52.399 --> 00:16:54.629
is why we're really busy building. So we
00:16:54.639 --> 00:16:58.230
have a a core principle uh which is uh
00:16:58.240 --> 00:17:00.710
for for our stuff that if there's a
00:17:00.720 --> 00:17:02.949
serverless Google option, we'll do it,
00:17:02.959 --> 00:17:04.710
right? It's like we're on the Google
00:17:04.720 --> 00:17:06.150
platform. That's the only thing we run
00:17:06.160 --> 00:17:08.470
on. we want to leverage that puppy as
00:17:08.480 --> 00:17:10.150
much as possible right so if there's a
00:17:10.160 --> 00:17:11.909
serverless thing and it's part of the
00:17:11.919 --> 00:17:16.549
GCP uh we adopt it and then uh if
00:17:16.559 --> 00:17:18.150
there's not now we get a bit tricky
00:17:18.160 --> 00:17:21.909
right uh so we had to on board a whole
00:17:21.919 --> 00:17:24.069
lot of publishers uh with social media
00:17:24.079 --> 00:17:25.750
data right we had to start collecting
00:17:25.760 --> 00:17:27.510
their YouTube Facebook all that kind of
00:17:27.520 --> 00:17:30.470
stuff but you know we've got uh 100 to
00:17:30.480 --> 00:17:32.789
200 external organizations that we have
00:17:32.799 --> 00:17:35.350
to go and get the data from and if we
00:17:35.360 --> 00:17:37.510
look at YouTube there's a out of the box
00:17:37.520 --> 00:17:39.750
Google uh data transfer service for
00:17:39.760 --> 00:17:42.710
YouTube but the onboarding process for
00:17:42.720 --> 00:17:45.669
that is horrific right like just the the
00:17:45.679 --> 00:17:48.070
way that that person has to uh create
00:17:48.080 --> 00:17:49.669
their share their credentials
00:17:49.679 --> 00:17:51.909
effectively you know tokenization it's
00:17:51.919 --> 00:17:53.430
just a complete nightmare
00:17:53.440 --> 00:17:53.909
>> right
00:17:53.919 --> 00:17:55.830
>> um and so we're like okay we can't use
00:17:55.840 --> 00:17:57.270
that right because even though under the
00:17:57.280 --> 00:17:59.830
covers it does exactly what we want uh
00:17:59.840 --> 00:18:02.710
the user experience of these publishers
00:18:02.720 --> 00:18:04.789
uh just means that they they wouldn't on
00:18:04.799 --> 00:18:06.390
board right or we'd have to spend hours
00:18:06.400 --> 00:18:08.870
if not months doing it. So then we go,
00:18:08.880 --> 00:18:10.789
okay, so our first principle can't be
00:18:10.799 --> 00:18:12.789
met. What do we do? And then our second
00:18:12.799 --> 00:18:14.549
principle is we'll just lease something,
00:18:14.559 --> 00:18:17.270
right? So there's a service out there.
00:18:17.280 --> 00:18:19.430
Uh we'll just use it and then we'll
00:18:19.440 --> 00:18:22.230
we'll deal with that um the challenge of
00:18:22.240 --> 00:18:23.590
integrating it. So we're like, okay,
00:18:23.600 --> 00:18:25.190
cool. Well, there's lots of services out
00:18:25.200 --> 00:18:28.150
there that allow people to uh
00:18:28.160 --> 00:18:31.110
effectively shine sign up and uh create
00:18:31.120 --> 00:18:33.029
a token and put the data into that
00:18:33.039 --> 00:18:34.310
platform and we can suck it out. So that
00:18:34.320 --> 00:18:36.710
that's the path we went down to. Uh and
00:18:36.720 --> 00:18:39.270
then we had some problems, right? So uh
00:18:39.280 --> 00:18:40.789
when we tested it, we could get two
00:18:40.799 --> 00:18:43.430
years worth of data. Uh as we're on
00:18:43.440 --> 00:18:44.870
boarding the publishers, that vendor
00:18:44.880 --> 00:18:46.070
decided that they were having
00:18:46.080 --> 00:18:47.909
performance problems and cost problems.
00:18:47.919 --> 00:18:51.350
So they limited it to 30 days.
00:18:51.360 --> 00:18:54.710
>> Oops. Um and then we can only get the
00:18:54.720 --> 00:18:58.230
data back via API, right? they they no
00:18:58.240 --> 00:19:01.750
direct database or or proxy database
00:19:01.760 --> 00:19:04.070
type connection which is all fine. So
00:19:04.080 --> 00:19:06.870
we're good at that. So you know taps on
00:19:06.880 --> 00:19:09.350
our framework and and we're all good. Uh
00:19:09.360 --> 00:19:12.150
until we worked out that actually uh the
00:19:12.160 --> 00:19:13.990
cost went through the roof because we
00:19:14.000 --> 00:19:17.669
had to pay for the API credits and we
00:19:17.679 --> 00:19:19.750
have very little visibility of how those
00:19:19.760 --> 00:19:22.070
credits work but we blew a year's budget
00:19:22.080 --> 00:19:25.669
in a month, right? and we're like,
00:19:25.679 --> 00:19:27.190
"Yeah, we don't want to buy that many."
00:19:27.200 --> 00:19:29.830
Um, so we ended up going, okay, to our
00:19:29.840 --> 00:19:32.310
third principle, which is if a least
00:19:32.320 --> 00:19:33.909
option doesn't work, we build it, which
00:19:33.919 --> 00:19:36.710
is what we've just done. Um, so I I
00:19:36.720 --> 00:19:39.350
liken that to if I was green, if I was
00:19:39.360 --> 00:19:41.350
work, you went in and and was head of
00:19:41.360 --> 00:19:43.029
data, head of data, platform, whatever
00:19:43.039 --> 00:19:44.710
in an organization,
00:19:44.720 --> 00:19:46.470
>> I'd set those core principles up front.
00:19:46.480 --> 00:19:48.390
You know, are we a best of breed or are
00:19:48.400 --> 00:19:50.950
we a platform organization?
00:19:50.960 --> 00:19:53.909
uh you know are we open source or are we
00:19:53.919 --> 00:19:56.070
supported open source commercial or are
00:19:56.080 --> 00:19:59.270
we uh closed source and I would place
00:19:59.280 --> 00:20:01.270
those principles not as an immutable
00:20:01.280 --> 00:20:04.070
policies but as guidance we are open
00:20:04.080 --> 00:20:07.190
source over closed source yeah right
00:20:07.200 --> 00:20:09.110
>> um which means then when you know you go
00:20:09.120 --> 00:20:10.549
snowflake you're like well that's closed
00:20:10.559 --> 00:20:13.029
source yeah but I can put iceberg there
00:20:13.039 --> 00:20:15.590
and that gets me closer to the principle
00:20:15.600 --> 00:20:18.630
of open source over closed source right
00:20:18.640 --> 00:20:20.950
uh it doesn't mean I need to got iceberg
00:20:20.960 --> 00:20:23.510
and apache spark and all that on an
00:20:23.520 --> 00:20:25.830
AWS platform and manage the containers
00:20:25.840 --> 00:20:26.789
myself
00:20:26.799 --> 00:20:28.310
>> because one of my other principles may
00:20:28.320 --> 00:20:31.110
be uh data ops or you know small teams
00:20:31.120 --> 00:20:32.549
like there's a bunch of other competing
00:20:32.559 --> 00:20:34.070
principles and that's the key. So that's
00:20:34.080 --> 00:20:36.470
what I'd do and I'd be very clear where
00:20:36.480 --> 00:20:38.549
on that slider I'm I'm moving, right?
00:20:38.559 --> 00:20:40.710
When I make a decision, which one did I
00:20:40.720 --> 00:20:43.110
pick and why? And then the last one is
00:20:43.120 --> 00:20:44.789
be willing to change. And that's the
00:20:44.799 --> 00:20:46.789
hard one for big corporates, right? Is
00:20:46.799 --> 00:20:49.744
say, "Hey, we tried that. It didn't work
00:20:49.754 --> 00:20:51.110
[laughter] for a whole bunch of reasons
00:20:51.120 --> 00:20:52.470
and now we're going to go do something
00:20:52.480 --> 00:20:55.510
different, but that's okay." Um, so as a
00:20:55.520 --> 00:20:57.909
two person startup, it's okay, right?
00:20:57.919 --> 00:21:00.950
Still gutted, but it's okay. Um, if I
00:21:00.960 --> 00:21:03.110
was in a 5,000 person company and my
00:21:03.120 --> 00:21:04.470
career was based on making that
00:21:04.480 --> 00:21:06.070
decision, yeah, it probably wasn't okay
00:21:06.080 --> 00:21:07.990
and I'd be paying that company a
00:21:08.000 --> 00:21:10.149
shitload of money for API credits.
00:21:10.159 --> 00:21:10.789
>> Yeah.
00:21:10.799 --> 00:21:12.630
>> Yeah. That's my view.
00:21:12.640 --> 00:21:14.470
>> I mean, it's it's good. It's why where
00:21:14.480 --> 00:21:16.070
we have like these like great
00:21:16.080 --> 00:21:18.390
conversations and we've worked together.
00:21:18.400 --> 00:21:21.350
Um, it is like a it it's so it's so
00:21:21.360 --> 00:21:23.750
funny like uh
00:21:23.760 --> 00:21:25.990
you have to make trade-offs, right? And
00:21:26.000 --> 00:21:27.669
I think most time when we talk you're
00:21:27.679 --> 00:21:29.430
just like well just make the trade-offs
00:21:29.440 --> 00:21:30.950
like visible like what are you training
00:21:30.960 --> 00:21:33.110
what for what right if you want to
00:21:33.120 --> 00:21:34.630
support open source just be clear about
00:21:34.640 --> 00:21:36.390
supporting open source or what the the
00:21:36.400 --> 00:21:37.990
trade-off is and like so where I
00:21:38.000 --> 00:21:39.430
currently work yeah we use iceberg
00:21:39.440 --> 00:21:42.149
because I felt that the
00:21:42.159 --> 00:21:44.149
it's not young it's just like it's hard
00:21:44.159 --> 00:21:47.190
for small data teams to actually operate
00:21:47.200 --> 00:21:50.470
on but like a year ago I was just like I
00:21:50.480 --> 00:21:52.950
know you know duck lake per I know all
00:21:52.960 --> 00:21:54.470
these things are coming I see this
00:21:54.480 --> 00:21:55.270
happening
00:21:55.280 --> 00:21:57.190
and there's delta and it has I think at
00:21:57.200 --> 00:21:58.950
time more superior piece but I was like
00:21:58.960 --> 00:22:00.390
it's very clear to me there's a signal
00:22:00.400 --> 00:22:02.710
is converged on iceberg and maybe I'm
00:22:02.720 --> 00:22:05.830
feeling a little risky on this but it's
00:22:05.840 --> 00:22:08.470
not that risky so let's converge on this
00:22:08.480 --> 00:22:10.789
open format and I think it's going to
00:22:10.799 --> 00:22:15.510
pay off um uh for from on on that bet um
00:22:15.520 --> 00:22:18.230
you know snowflake support for it was
00:22:18.240 --> 00:22:21.590
not great at first despite all the docs
00:22:21.600 --> 00:22:23.590
they had but it's you know it's it's
00:22:23.600 --> 00:22:27.430
definitely caught up and has been almost
00:22:27.440 --> 00:22:31.590
seamless um now but a year ago def
00:22:31.600 --> 00:22:33.510
definitely was not seamless it was
00:22:33.520 --> 00:22:34.870
definitely painful
00:22:34.880 --> 00:22:36.230
>> uh but yeah that was that trade-off
00:22:36.240 --> 00:22:37.909
right like I still like and also it's
00:22:37.919 --> 00:22:39.830
like
00:22:39.840 --> 00:22:42.710
>> is juice worth the squeeze vast majority
00:22:42.720 --> 00:22:44.710
if I stand up uh an open source
00:22:44.720 --> 00:22:47.029
analytical piece or like you said spark
00:22:47.039 --> 00:22:50.549
like I've run spark kafka like all those
00:22:50.559 --> 00:22:53.510
toys before um and I know kind of like
00:22:53.520 --> 00:22:55.430
what the venance But you know the
00:22:55.440 --> 00:22:57.669
business today like you know you see
00:22:57.679 --> 00:23:00.789
that you know in foreign employment like
00:23:00.799 --> 00:23:02.950
compared to last year like analyst roles
00:23:02.960 --> 00:23:04.789
like pure analytical roles are down like
00:23:04.799 --> 00:23:08.230
15%. Right. So you can see kind of what
00:23:08.240 --> 00:23:09.990
I actually don't see it that way. I see
00:23:10.000 --> 00:23:11.669
is like a lot of engineering roles like
00:23:11.679 --> 00:23:14.549
condensing. Right. Right. Like a lot of
00:23:14.559 --> 00:23:16.230
people being like I'm just an analyst
00:23:16.240 --> 00:23:17.750
role. I think that's those are
00:23:17.760 --> 00:23:20.390
shrinking. They should uh I think that's
00:23:20.400 --> 00:23:22.149
that's fine because there's a lot of
00:23:22.159 --> 00:23:23.990
capability now. Like I think that really
00:23:24.000 --> 00:23:26.230
good analysts now can just do tons of
00:23:26.240 --> 00:23:28.950
stuff they never could do before uh with
00:23:28.960 --> 00:23:31.270
AI helping them, you know, get that way
00:23:31.280 --> 00:23:33.430
in in the systems, you know, just
00:23:33.440 --> 00:23:35.270
complexity.
00:23:35.280 --> 00:23:37.190
>> Yeah. I think that's a good point is if
00:23:37.200 --> 00:23:38.630
you're the leader of the data part of
00:23:38.640 --> 00:23:40.710
that organization, it's your job to make
00:23:40.720 --> 00:23:42.549
the bets.
00:23:42.559 --> 00:23:44.950
Yeah. And make them visible why you've
00:23:44.960 --> 00:23:46.470
made that bet. But you've got to look
00:23:46.480 --> 00:23:47.510
ahead, right? You got to look at the
00:23:47.520 --> 00:23:49.590
market and go, "Well, I could do this or
00:23:49.600 --> 00:23:50.630
I could do that. Where am I going to
00:23:50.640 --> 00:23:51.990
place my chips?" Right? Because that's
00:23:52.000 --> 00:23:53.110
what you're doing. you're you're
00:23:53.120 --> 00:23:55.029
gambling not on a hunch on yeah there's
00:23:55.039 --> 00:23:57.350
data and everything but you're saying do
00:23:57.360 --> 00:23:59.190
do I think this is where the market's
00:23:59.200 --> 00:24:01.430
going to go like is it safe to make this
00:24:01.440 --> 00:24:04.230
bet now for the next I mean it used to
00:24:04.240 --> 00:24:05.830
be five years but it's not even five
00:24:05.840 --> 00:24:07.669
years now right like things are moving
00:24:07.679 --> 00:24:09.350
so fast you're and that's the other
00:24:09.360 --> 00:24:12.070
thing then you've got to architect your
00:24:12.080 --> 00:24:14.230
your platforms and your way of working
00:24:14.240 --> 00:24:16.710
to be what was it combustible no
00:24:16.720 --> 00:24:18.310
composable what was what was that
00:24:18.320 --> 00:24:19.430
remember those [laughter]
00:24:19.440 --> 00:24:21.909
remember those uh what was it that last
00:24:21.919 --> 00:24:25.750
that reverse ETL Uh I mean the
00:24:25.760 --> 00:24:28.950
the patterns that were doing were were
00:24:28.960 --> 00:24:32.390
useful but the market uh buzzwashing of
00:24:32.400 --> 00:24:34.549
it was Composable combustible
00:24:34.559 --> 00:24:35.029
what was it?
00:24:35.039 --> 00:24:37.031
>> Composable but combustible is probably
00:24:37.041 --> 00:24:42.470
[laughter] what become. Um no I think um
00:24:42.480 --> 00:24:44.230
yeah I mean it's really yeah it's really
00:24:44.240 --> 00:24:45.269
interesting going back to your
00:24:45.279 --> 00:24:47.510
conversation about platform ahead of
00:24:47.520 --> 00:24:49.430
platform. I think it comes in it's like
00:24:49.440 --> 00:24:50.870
core principles like for for the
00:24:50.880 --> 00:24:52.470
business like one of the things I was
00:24:52.480 --> 00:24:56.230
talking to another uh data leader
00:24:56.240 --> 00:24:59.590
um Ahmed yeah Ahmed from activity schema
00:24:59.600 --> 00:25:01.430
actually I was talking to him last week
00:25:01.440 --> 00:25:02.789
uh and we were talking about what he
00:25:02.799 --> 00:25:04.390
sees because obviously he's you know he
00:25:04.400 --> 00:25:05.909
started his own startup which ultimately
00:25:05.919 --> 00:25:08.230
narrator closed down but he left behind
00:25:08.240 --> 00:25:09.909
the the concept for data modeling for
00:25:09.919 --> 00:25:11.269
activity and he's working on a new one
00:25:11.279 --> 00:25:13.430
called polyiform and a few other things
00:25:13.440 --> 00:25:15.269
that I found really interesting in what
00:25:15.279 --> 00:25:17.190
he's talking about and he was talking
00:25:17.200 --> 00:25:19.510
about like some of the clients that he's
00:25:19.520 --> 00:25:21.590
been working with is the engineering
00:25:21.600 --> 00:25:23.350
team are going around the data team.
00:25:23.360 --> 00:25:24.630
They don't even want them involved.
00:25:24.640 --> 00:25:27.350
They're too they're tired of the white
00:25:27.360 --> 00:25:30.870
tower, the the the gating, the whatever
00:25:30.880 --> 00:25:32.870
it is. And so the engineers are wanting
00:25:32.880 --> 00:25:34.470
to like essentially create their own
00:25:34.480 --> 00:25:36.390
pipelines and all this other stuff. I
00:25:36.400 --> 00:25:38.310
mean, it's the tale as old as time,
00:25:38.320 --> 00:25:40.710
frankly. It's like, oh, this is new. I
00:25:40.720 --> 00:25:41.909
was like, no, this sounds standard. And
00:25:41.919 --> 00:25:43.510
then an engineer actually has to do data
00:25:43.520 --> 00:25:45.669
work and they're like, oh this is
00:25:45.679 --> 00:25:47.830
actually hard. is oh what's the
00:25:47.840 --> 00:25:52.070
definition of time? Well that depends um
00:25:52.080 --> 00:25:53.750
and is like you know
00:25:53.760 --> 00:25:56.149
>> even with Claude um you know and we've
00:25:56.159 --> 00:25:57.510
talked about this before in Thropics
00:25:57.520 --> 00:26:01.269
paper on like their their massive uh you
00:26:01.279 --> 00:26:03.269
know data modeling program and and
00:26:03.279 --> 00:26:05.669
analytical program. It's eerily similar
00:26:05.679 --> 00:26:08.230
to the same things that stood up for
00:26:08.240 --> 00:26:12.230
decades now just faster I guess. Um
00:26:12.240 --> 00:26:14.390
>> uh well yeah on that so I was talking to
00:26:14.400 --> 00:26:16.310
somebody the other day because you know
00:26:16.320 --> 00:26:19.029
I I lament uh how many people point to
00:26:19.039 --> 00:26:20.789
that anthropic paper and say oh they do
00:26:20.799 --> 00:26:22.149
dimensional modeling that's the answer
00:26:22.159 --> 00:26:23.990
and I'm like yeah I'm not sure it is I'm
00:26:24.000 --> 00:26:25.350
not sure this the paper says they do
00:26:25.360 --> 00:26:26.710
dimensionally modeling right and that's
00:26:26.720 --> 00:26:28.149
fine because any modeling technique will
00:26:28.159 --> 00:26:29.750
work I'm not sure that's where their
00:26:29.760 --> 00:26:31.190
secret source was but I was talking to
00:26:31.200 --> 00:26:32.230
somebody anyway
00:26:32.240 --> 00:26:33.990
>> and I don't know whether fact what
00:26:34.000 --> 00:26:35.269
they're telling me is factually correct
00:26:35.279 --> 00:26:36.710
or not because I can't validate it but
00:26:36.720 --> 00:26:39.430
what they said was uh yes there is part
00:26:39.440 --> 00:26:41.830
of that company that works works that
00:26:41.840 --> 00:26:44.230
way but there's also a large part of
00:26:44.240 --> 00:26:47.269
that company that doesn't uh and so you
00:26:47.279 --> 00:26:50.230
do wonder how much architectural buzzing
00:26:50.240 --> 00:26:52.310
of that paper and then especially like
00:26:52.320 --> 00:26:54.310
everybody else has come out right
00:26:54.320 --> 00:26:55.990
Spotify have done everybody else has
00:26:56.000 --> 00:26:58.390
come out with their how we use GI
00:26:58.400 --> 00:27:00.710
tooling with our data team and that I
00:27:00.720 --> 00:27:04.390
mean I think the core pattern of um you
00:27:04.400 --> 00:27:06.870
can't reverse engineer the context for
00:27:06.880 --> 00:27:09.590
your organization out of your SQL uh you
00:27:09.600 --> 00:27:11.190
are going to need human in the loop
00:27:11.200 --> 00:27:13.029
Mother duck. So many papers have come
00:27:13.039 --> 00:27:14.789
out. They've straight up confirmed that
00:27:14.799 --> 00:27:16.950
like you will you the problem is you
00:27:16.960 --> 00:27:18.710
will you will feel correct.
00:27:18.720 --> 00:27:19.029
>> Yep.
00:27:19.039 --> 00:27:20.390
>> You will feel correct. It'll have the
00:27:20.400 --> 00:27:23.350
language. It'll have a true touring
00:27:23.360 --> 00:27:25.990
complete vibe to it and you'll probably
00:27:26.000 --> 00:27:28.070
if you know huge chunk of people will be
00:27:28.080 --> 00:27:29.909
like yeah it's good and they will
00:27:29.919 --> 00:27:31.350
continue on. Um
00:27:31.360 --> 00:27:32.230
>> yeah.
00:27:32.240 --> 00:27:34.149
>> Yeah. So I think yeah that part of the
00:27:34.159 --> 00:27:36.950
paper because I it's been my experience
00:27:36.960 --> 00:27:39.590
right. Um so that yeah that's good. But
00:27:39.600 --> 00:27:41.350
I think there again there's people take
00:27:41.360 --> 00:27:43.350
little sound bites and go that is the
00:27:43.360 --> 00:27:46.070
answer, right? Um and I think
00:27:46.080 --> 00:27:47.750
>> dim packed and dim table it's clearly
00:27:47.760 --> 00:27:49.590
that's all that's needed. Um
00:27:49.600 --> 00:27:51.029
>> yeah they're valuable but they're not
00:27:51.039 --> 00:27:54.230
the answer. Uh I don't think so. Uh
00:27:54.240 --> 00:27:56.149
yeah, it it it's interesting and it's
00:27:56.159 --> 00:27:58.870
the same with um data bricks coming out
00:27:58.880 --> 00:28:00.870
with their accelerator data models,
00:28:00.880 --> 00:28:03.110
right? Their industry standard ontology
00:28:03.120 --> 00:28:05.750
and uh enterprise data models now per
00:28:05.760 --> 00:28:07.590
industry they've just announced. Uh I'm
00:28:07.600 --> 00:28:09.430
going to chat to Chris Gamble about it
00:28:09.440 --> 00:28:11.990
next week because he's deep in that data
00:28:12.000 --> 00:28:14.389
brick space. But we've seen it before,
00:28:14.399 --> 00:28:17.510
right? And I it it's never worked and I
00:28:17.520 --> 00:28:19.590
don't know why because it makes sense,
00:28:19.600 --> 00:28:21.590
right? It makes sense that if you're in
00:28:21.600 --> 00:28:24.710
an industry that is a relatively
00:28:24.720 --> 00:28:27.269
standard industry that 80% of what you
00:28:27.279 --> 00:28:29.750
do in terms of core concepts, your
00:28:29.760 --> 00:28:30.789
business processes and your
00:28:30.799 --> 00:28:32.870
relationships are the same as everybody
00:28:32.880 --> 00:28:35.909
else. And then the 20% is where you know
00:28:35.919 --> 00:28:37.750
the juice is worth the squeeze. How are
00:28:37.760 --> 00:28:39.669
you different or how are you going to be
00:28:39.679 --> 00:28:41.110
different? Right.
00:28:41.120 --> 00:28:42.710
>> Yeah. But we've seen those enterprise
00:28:42.720 --> 00:28:45.350
data models come out and reference AR
00:28:45.360 --> 00:28:47.990
models and industry models come out
00:28:48.000 --> 00:28:50.230
every wave and they never get traction
00:28:50.240 --> 00:28:51.990
and I've never worked out why. Yeah.
00:28:52.000 --> 00:28:55.110
What's your view?
00:28:55.120 --> 00:28:58.470
>> Um I mean I get pitched freaking data
00:28:58.480 --> 00:29:01.190
engineering, data analytics, AI startup
00:29:01.200 --> 00:29:03.430
stuff. Oh, I'm not going to say every
00:29:03.440 --> 00:29:04.930
day because I don't go on LinkedIn
00:29:04.940 --> 00:29:06.470
[laughter] every day, but every day I
00:29:06.480 --> 00:29:09.110
show up on LinkedIn, there's a new Y
00:29:09.120 --> 00:29:11.510
cominator or new startup out of
00:29:11.520 --> 00:29:13.190
somewhere that's going to, you know, do
00:29:13.200 --> 00:29:17.269
all the data engineering work. Um,
00:29:17.279 --> 00:29:18.950
and they, it's so funny, they've all
00:29:18.960 --> 00:29:22.230
converged to like facts fact like just
00:29:22.240 --> 00:29:24.870
like those patterns and I'm like why?
00:29:24.880 --> 00:29:26.230
There's so much more out there and
00:29:26.240 --> 00:29:27.830
sometimes better better patterns for
00:29:27.840 --> 00:29:32.070
solving all this stuff. Um,
00:29:32.080 --> 00:29:34.549
and
00:29:34.559 --> 00:29:36.070
>> let let me let me ask you it a different
00:29:36.080 --> 00:29:38.149
way, right? Because I so you work in a
00:29:38.159 --> 00:29:40.710
company that has subscriptions, right?
00:29:40.720 --> 00:29:42.310
Kind of core
00:29:42.320 --> 00:29:43.590
>> core be. Yeah.
00:29:43.600 --> 00:29:45.750
>> And and you go, well, you know, that's
00:29:45.760 --> 00:29:47.750
an industry standard model, right?
00:29:47.760 --> 00:29:50.125
customer subscribes to product
00:29:50.135 --> 00:29:50.950
[laughter]
00:29:50.960 --> 00:29:53.269
and and we know that because you're not
00:29:53.279 --> 00:29:55.110
the first one I've worked with where I
00:29:55.120 --> 00:29:56.630
like the first time I went into one of
00:29:56.640 --> 00:29:58.149
those companies and go ah it's
00:29:58.159 --> 00:30:00.230
subscription how hard could that be I'd
00:30:00.240 --> 00:30:03.190
never say that again um because every
00:30:03.200 --> 00:30:04.789
every software service company I've
00:30:04.799 --> 00:30:07.750
worked with subscription seems to be
00:30:07.760 --> 00:30:11.110
harder than customer um like but it's
00:30:11.120 --> 00:30:12.950
meant to be an ind why is it not just an
00:30:12.960 --> 00:30:15.430
industry standard model right that just
00:30:15.440 --> 00:30:16.470
makes sense to me [clears throat]
00:30:16.480 --> 00:30:18.870
>> it's so Funny. I love those things. And
00:30:18.880 --> 00:30:20.950
it's like, yeah, on like on the surface,
00:30:20.960 --> 00:30:22.310
right? There's so many things on the
00:30:22.320 --> 00:30:24.070
surface when you look at it. And then
00:30:24.080 --> 00:30:26.230
you're like getting into the contract.
00:30:26.240 --> 00:30:27.990
Oh, you have contracts. And it's like,
00:30:28.000 --> 00:30:29.110
yeah, we have contracts because like
00:30:29.120 --> 00:30:31.029
last year we sold it like this and these
00:30:31.039 --> 00:30:32.870
clients renewed on this value of the
00:30:32.880 --> 00:30:34.310
subscription, but then this year we have
00:30:34.320 --> 00:30:35.669
a new subscription, but the old
00:30:35.679 --> 00:30:37.190
customers don't want to move to the new
00:30:37.200 --> 00:30:38.710
subscription model [laughter] and you're
00:30:38.720 --> 00:30:40.070
not big enough because you're like, you
00:30:40.080 --> 00:30:42.070
know, sub$und00 million company. So, you
00:30:42.080 --> 00:30:44.070
can't really force those clients the new
00:30:44.080 --> 00:30:45.269
subscription. So now you have a
00:30:45.279 --> 00:30:47.990
bifrocated definition of contract and
00:30:48.000 --> 00:30:52.070
subscription and um you know I almost
00:30:52.080 --> 00:30:53.909
exclude I'm not say exclusively work for
00:30:53.919 --> 00:30:55.990
startups but I generally biased towards
00:30:56.000 --> 00:30:57.669
them these days and I've worked in
00:30:57.679 --> 00:31:01.590
numerous ones. Yeah. Like everyone
00:31:01.600 --> 00:31:03.590
starts to become a snowflake. You you
00:31:03.600 --> 00:31:05.029
start to hold the line about what
00:31:05.039 --> 00:31:06.549
contracts or subscriptions should look
00:31:06.559 --> 00:31:08.950
like and inevitably something comes down
00:31:08.960 --> 00:31:10.389
into the business that makes it
00:31:10.399 --> 00:31:13.750
different. Right. uh
00:31:13.760 --> 00:31:16.070
>> is a trial or contract. Yeah. Uh you
00:31:16.080 --> 00:31:17.430
know, you you know the answer is you
00:31:17.440 --> 00:31:19.110
just work to credits and nobody can
00:31:19.120 --> 00:31:20.870
understand how they work and then it
00:31:20.880 --> 00:31:22.481
doesn't matter, right? Because it's
00:31:22.491 --> 00:31:25.110
[laughter] there's one standard. It's a
00:31:25.120 --> 00:31:26.710
credit. We we don't know how we
00:31:26.720 --> 00:31:29.110
calculated. But that's okay. 100 credit.
00:31:29.120 --> 00:31:31.830
>> I do love to just a credit system. That
00:31:31.840 --> 00:31:33.750
way literally no one can figure out what
00:31:33.760 --> 00:31:35.590
you're actually charging. You get a
00:31:35.600 --> 00:31:37.750
hundred credits.
00:31:37.760 --> 00:31:39.430
>> Yeah. It's almost like credit I feel
00:31:39.440 --> 00:31:41.909
like credit card points like man
00:31:41.919 --> 00:31:43.669
somebody was like, "Oh, you bet. I get a
00:31:43.679 --> 00:31:46.149
h 100,000 credit card points for signing
00:31:46.159 --> 00:31:48.070
up to this new like program." I'm like,
00:31:48.080 --> 00:31:49.990
"Yeah, but what does that actually
00:31:50.000 --> 00:31:51.990
mean?" Because before I used to be, you
00:31:52.000 --> 00:31:54.630
know, 100,000 credits was like 1 cent
00:31:54.640 --> 00:31:57.110
USD. And I'm like, "Is it now?" Well,
00:31:57.120 --> 00:31:59.190
it's like no. And there's like this
00:31:59.200 --> 00:32:01.830
pyramid system of like configurations.
00:32:01.840 --> 00:32:03.430
And that's exactly how I feel like using
00:32:03.440 --> 00:32:05.509
AI tooling right now for credits. I just
00:32:05.519 --> 00:32:09.430
like which which MX card is gonna work
00:32:09.440 --> 00:32:12.789
with what because like um I got a new MX
00:32:12.799 --> 00:32:16.070
card and for uh and it was so funny. I
00:32:16.080 --> 00:32:17.909
was like, "Oh, if you go chat GBT
00:32:17.919 --> 00:32:19.590
business, we're going to give you 300
00:32:19.600 --> 00:32:21.750
credits." And I was like, "Oh, that
00:32:21.760 --> 00:32:24.310
sounds great. 300 credits. Uh, okay.
00:32:24.320 --> 00:32:25.669
I'll go for it." And I was like, "Oh,
00:32:25.679 --> 00:32:27.669
yeah, minimum seats." And then it's like
00:32:27.679 --> 00:32:29.269
credited for over this period of time. I
00:32:29.279 --> 00:32:31.257
was like, "Okay, of course." It's like,
00:32:31.267 --> 00:32:32.710
[laughter]
00:32:32.720 --> 00:32:35.029
>> yeah, it's not just put it on the card
00:32:35.039 --> 00:32:36.710
and you're like, you're good to go. No,
00:32:36.720 --> 00:32:37.750
no, no.
00:32:37.760 --> 00:32:40.630
>> Yeah. And then um Well, it's like uh
00:32:40.640 --> 00:32:42.789
Claude, right, when Fable came out and
00:32:42.799 --> 00:32:45.830
then Opus 5 and then everybody's going,
00:32:45.840 --> 00:32:48.470
"Is it me or is Opus 5 the
00:32:48.480 --> 00:32:50.549
bed?" And and eventually I'd heard
00:32:50.559 --> 00:32:52.389
enough people that were having the same
00:32:52.399 --> 00:32:54.149
problems I had and I was like, "Back to
00:32:54.159 --> 00:32:55.990
Opus 4.8
00:32:56.000 --> 00:32:58.070
because and and now I'm sitting there
00:32:58.080 --> 00:33:01.669
going ah you know
00:33:01.679 --> 00:33:03.750
when do I make the call to make my
00:33:03.760 --> 00:33:05.750
harness agnostic
00:33:05.760 --> 00:33:07.830
like because at the moment I just it's
00:33:07.840 --> 00:33:10.870
all included right and I'm like I know
00:33:10.880 --> 00:33:13.590
I've architected it in theory in a way
00:33:13.600 --> 00:33:16.230
that I could move it to other models.
00:33:16.240 --> 00:33:19.350
>> Yeah. But I haven't tested it. So I
00:33:19.360 --> 00:33:21.029
know, you know, I've been around long
00:33:21.039 --> 00:33:22.870
enough to know if I haven't tested it
00:33:22.880 --> 00:33:25.190
architecturally, it's just a theory. And
00:33:25.200 --> 00:33:27.029
it's a theory that probably won't
00:33:27.039 --> 00:33:29.430
survive. And I when do I take the hit
00:33:29.440 --> 00:33:32.310
for that puppy? Right. And then favorite
00:33:32.320 --> 00:33:34.149
one came out and I'm like, is it going
00:33:34.159 --> 00:33:36.149
to be better? Is it going to be worse?
00:33:36.159 --> 00:33:38.470
Am I going to blow my my subscription in
00:33:38.480 --> 00:33:40.549
half an hour? Because I got no idea what
00:33:40.559 --> 00:33:42.389
it's doing. uh like I was doing a
00:33:42.399 --> 00:33:45.269
conference today and um uh I wanted to
00:33:45.279 --> 00:33:47.430
li show live basically using the
00:33:47.440 --> 00:33:49.830
information product uh app right so the
00:33:49.840 --> 00:33:52.149
open source app that I've got so yeah
00:33:52.159 --> 00:33:53.990
there's a canvas in there I export it to
00:33:54.000 --> 00:33:56.630
Jason I upload it to Claude I asked Lord
00:33:56.640 --> 00:33:58.070
a bunch of questions right and that was
00:33:58.080 --> 00:34:00.870
that was the demo ran it all through
00:34:00.880 --> 00:34:03.509
that's all good um it's all pretty quick
00:34:03.519 --> 00:34:05.350
you turned off uh thinking mode all that
00:34:05.360 --> 00:34:07.590
kind of um [laughter]
00:34:07.600 --> 00:34:09.430
yeah because I had you had a set amount
00:34:09.440 --> 00:34:11.030
of time and it was like yeah it's I
00:34:11.040 --> 00:34:12.470
mean, each question is coming back in
00:34:12.480 --> 00:34:13.829
less than a minute. Of course, when I
00:34:13.839 --> 00:34:15.430
did it live, it took over two to three
00:34:15.440 --> 00:34:16.950
minutes for each question. So, that
00:34:16.960 --> 00:34:19.510
can cander. Um, but anyway, so I was
00:34:19.520 --> 00:34:22.310
yoloing along. I'm all good already in
00:34:22.320 --> 00:34:24.230
the morning presentation in the early
00:34:24.240 --> 00:34:25.750
afternoon. So, I was like, cool, let's
00:34:25.760 --> 00:34:27.909
get back to work, right? And so, I'm
00:34:27.919 --> 00:34:31.030
building some stuff and I'm had flipped
00:34:31.040 --> 00:34:32.869
over to Fable 51 for something and
00:34:32.879 --> 00:34:34.310
forgot to flip back. So, every new
00:34:34.320 --> 00:34:36.550
session was fabling me. And then I was
00:34:36.560 --> 00:34:37.990
sitting there going, "Hey, hold on, hold
00:34:38.000 --> 00:34:40.869
on. how how much have I got left in my 5
00:34:40.879 --> 00:34:43.109
hour window because that five hour
00:34:43.119 --> 00:34:45.990
window is spang smack my conference is
00:34:46.000 --> 00:34:48.069
in the middle of it. Um and I was like
00:34:48.079 --> 00:34:49.270
yeah I'm going to stop working now
00:34:49.280 --> 00:34:51.990
because I got no guarantee of when I'm
00:34:52.000 --> 00:34:53.669
actually going to run out of that that
00:34:53.679 --> 00:34:55.270
and
00:34:55.280 --> 00:34:57.510
embarrassing. So yeah I think there's a
00:34:57.520 --> 00:34:59.670
lack of visibility, right?
00:34:59.680 --> 00:35:00.950
>> Yeah. I mean it's the classic like
00:35:00.960 --> 00:35:02.710
blackbox problems we got going on here,
00:35:02.720 --> 00:35:05.184
right? We just keep on punting them
00:35:05.194 --> 00:35:06.310
[laughter] and it's really interesting.
00:35:06.320 --> 00:35:08.550
What do you feel like um you you you use
00:35:08.560 --> 00:35:11.030
harnesses like are you you know are you
00:35:11.040 --> 00:35:14.150
going all in on Pi or Hermes or you you
00:35:14.160 --> 00:35:15.349
spun up your own thing?
00:35:15.359 --> 00:35:17.750
>> No, I started off I'm a bit like you. I
00:35:17.760 --> 00:35:20.390
started off with a personal personal OS
00:35:20.400 --> 00:35:22.470
um as as a learning curve, right? It was
00:35:22.480 --> 00:35:24.150
back in the open claw. So I did open
00:35:24.160 --> 00:35:26.710
claw for about two seconds then uh did a
00:35:26.720 --> 00:35:27.990
little bit of reading was like holy
00:35:28.000 --> 00:35:30.150
no but I use it as a learning curve and
00:35:30.160 --> 00:35:32.870
then for our stuff uh I started building
00:35:32.880 --> 00:35:36.230
it specifically for what we needed. Um,
00:35:36.240 --> 00:35:37.910
I still have a technique where what I'll
00:35:37.920 --> 00:35:39.510
do is when I hear of a new uh open
00:35:39.520 --> 00:35:42.310
source harness, I'll get Claude to
00:35:42.320 --> 00:35:44.230
basically look at my harness, look at
00:35:44.240 --> 00:35:45.670
the open source one, and tell me what's
00:35:45.680 --> 00:35:47.349
missing, what's useful, and try and get
00:35:47.359 --> 00:35:51.349
it back. Um, but I'm at the stage now
00:35:51.359 --> 00:35:53.430
where I'm starting to question how much
00:35:53.440 --> 00:35:55.670
time am I spending on the harness versus
00:35:55.680 --> 00:35:58.470
actually doing the work. Like I know the
00:35:58.480 --> 00:36:00.790
harness is giving making me faster to do
00:36:00.800 --> 00:36:03.750
the work but actually if I look at an
00:36:03.760 --> 00:36:05.750
investment window you net present value
00:36:05.760 --> 00:36:08.870
of time. Uh I'm wondering now whether
00:36:08.880 --> 00:36:11.430
I'm overindexing on the harness itself
00:36:11.440 --> 00:36:15.109
because it's kind of fun. Like I did
00:36:15.119 --> 00:36:16.710
this
00:36:16.720 --> 00:36:19.190
Claude did that. It shouldn't have done
00:36:19.200 --> 00:36:21.910
that. I don't want it to do that again.
00:36:21.920 --> 00:36:25.109
So, let's spend two hours getting the
00:36:25.119 --> 00:36:26.950
hardness up to date to stop Claude ever
00:36:26.960 --> 00:36:29.589
doing that, right? And I'm like, is that
00:36:29.599 --> 00:36:33.109
like where's that value? Um, I mean, for
00:36:33.119 --> 00:36:37.750
us, the value is I can give I want to be
00:36:37.760 --> 00:36:39.750
able to give our harness to our partners
00:36:39.760 --> 00:36:41.589
and have them work the same way I do,
00:36:41.599 --> 00:36:43.670
right? Without the intrinsic knowledge
00:36:43.680 --> 00:36:46.069
of our platform that I know I have.
00:36:46.079 --> 00:36:47.829
>> That that's the end goal, right? It's a
00:36:47.839 --> 00:36:51.750
scaling uh play and that's the dream,
00:36:51.760 --> 00:36:54.550
right? Um but I still worry right now
00:36:54.560 --> 00:36:56.150
and then I look and say, well, how many
00:36:56.160 --> 00:36:57.430
other people are doing that, right? Like
00:36:57.440 --> 00:36:59.510
how many data teams in a large
00:36:59.520 --> 00:37:01.589
organization right now are spending 80%
00:37:01.599 --> 00:37:03.829
of their time grinding harnesses and not
00:37:03.839 --> 00:37:08.069
actually doing the data work?
00:37:08.079 --> 00:37:09.990
>> Yeah, I mean that's a good question. I
00:37:10.000 --> 00:37:11.750
don't know
00:37:11.760 --> 00:37:14.790
how I I I question on like how
00:37:14.800 --> 00:37:16.390
consistent a lot of people are playing.
00:37:16.400 --> 00:37:18.630
I think a lot of people are are spinning
00:37:18.640 --> 00:37:22.230
up um kind of like their own playbooks
00:37:22.240 --> 00:37:24.150
in the in the company like a a vast
00:37:24.160 --> 00:37:25.510
majority of people they're all spinning
00:37:25.520 --> 00:37:28.470
up their like you know claude or ilk of
00:37:28.480 --> 00:37:29.990
their own markdown files to feel like
00:37:30.000 --> 00:37:31.510
they're giving themselves the edge and
00:37:31.520 --> 00:37:33.990
not really sharing it across across the
00:37:34.000 --> 00:37:37.270
company. Um, and then like, you know,
00:37:37.280 --> 00:37:38.790
people are really struggling on how to
00:37:38.800 --> 00:37:40.390
take it off, take it into like
00:37:40.400 --> 00:37:42.310
multiplayer mode, right?
00:37:42.320 --> 00:37:44.790
>> I really truly think that it's kind of
00:37:44.800 --> 00:37:46.470
like the next level. I mean, I think
00:37:46.480 --> 00:37:47.910
that's why Claude Co-work was starting
00:37:47.920 --> 00:37:50.470
to take off a little bit more. Um, but I
00:37:50.480 --> 00:37:53.750
think like
00:37:53.760 --> 00:37:55.589
I think, you know, I you know, my wife
00:37:55.599 --> 00:37:57.829
and I are, you know, we have our own uh
00:37:57.839 --> 00:37:59.750
and we struggle to turn into a proper
00:37:59.760 --> 00:38:01.109
multiplayer mode because we have like
00:38:01.119 --> 00:38:02.870
really different ways of wanting to work
00:38:02.880 --> 00:38:05.510
and do things, right? And so if you're
00:38:05.520 --> 00:38:07.829
like naturally
00:38:07.839 --> 00:38:10.150
different, could you imagine what 10020
00:38:10.160 --> 00:38:11.829
people are like and they're all like
00:38:11.839 --> 00:38:14.470
they like their, you know, as you call
00:38:14.480 --> 00:38:17.430
them pets maybe to be kind of like tuned
00:38:17.440 --> 00:38:19.510
to them. And that's where like I think a
00:38:19.520 --> 00:38:20.790
lot of people are struggling like you
00:38:20.800 --> 00:38:23.030
need this consistency we'll call context
00:38:23.040 --> 00:38:26.230
or knowledge and you know the facts of
00:38:26.240 --> 00:38:29.750
the business or facts much as you can
00:38:29.760 --> 00:38:31.030
and your personal life your calendars
00:38:31.040 --> 00:38:32.710
and all that stuff but then like the
00:38:32.720 --> 00:38:35.910
nuances can be like
00:38:35.920 --> 00:38:37.510
I think they're a lot deeper than people
00:38:37.520 --> 00:38:39.510
give credit to sometimes right that like
00:38:39.520 --> 00:38:41.349
that 10%
00:38:41.359 --> 00:38:44.550
can be so wildly different
00:38:44.560 --> 00:38:46.230
and [laughter] the outcomes and you're
00:38:46.240 --> 00:38:47.829
just like I know we're speaking the same
00:38:47.839 --> 00:38:50.550
language uage but [laughter]
00:38:50.560 --> 00:38:51.990
you are coming you and I are coming
00:38:52.000 --> 00:38:55.030
completely different points of view here
00:38:55.040 --> 00:38:58.630
>> and I yeah well Anna Bergin's actually
00:38:58.640 --> 00:39:01.109
working on a personal OS for family uh
00:39:01.119 --> 00:39:02.550
for her family right and she's open
00:39:02.560 --> 00:39:04.230
sourcing it so it'll be interesting to
00:39:04.240 --> 00:39:05.430
see whether you know how much of that
00:39:05.440 --> 00:39:07.190
you pick up for your your family stuff
00:39:07.200 --> 00:39:07.750
um
00:39:07.760 --> 00:39:09.589
>> yeah I saw interesting I you know I was
00:39:09.599 --> 00:39:11.030
I'm going the same direction there's so
00:39:11.040 --> 00:39:12.230
many people working on it I would say
00:39:12.240 --> 00:39:13.589
like the other thing I get pitched on
00:39:13.599 --> 00:39:15.589
every other day is probably not just
00:39:15.599 --> 00:39:17.430
personal knowledge gaff brat or personal
00:39:17.440 --> 00:39:19.750
knowledge bases for companies like you
00:39:19.760 --> 00:39:21.430
can see Stripe came out with their one
00:39:21.440 --> 00:39:23.589
and how they do it. Um, obviously
00:39:23.599 --> 00:39:26.470
there's Brockbot out there and like how
00:39:26.480 --> 00:39:28.150
they're kind of like heading towards
00:39:28.160 --> 00:39:30.550
these different patterns. Yeah. I mean,
00:39:30.560 --> 00:39:32.150
everybody knows that like the
00:39:32.160 --> 00:39:33.829
multiplayer mode is kind of like where
00:39:33.839 --> 00:39:36.870
it's at. Um, you know, open.
00:39:36.880 --> 00:39:40.630
>> Yeah. The question is then comes back to
00:39:40.640 --> 00:39:42.550
do you buy a harness off a platform
00:39:42.560 --> 00:39:45.430
vendor because it's embedded in their
00:39:45.440 --> 00:39:46.950
technology stack? It's integrated,
00:39:46.960 --> 00:39:48.790
right? It knows how to execute the
00:39:48.800 --> 00:39:51.190
harness. And then you bolt on your
00:39:51.200 --> 00:39:53.270
context plane. Yeah. The the thing that
00:39:53.280 --> 00:39:55.270
holds your context, which is the second
00:39:55.280 --> 00:39:59.109
part of what a harness needs. Um
00:39:59.119 --> 00:40:02.150
or do you go with a platform player
00:40:02.160 --> 00:40:05.190
that's got both the context storage and
00:40:05.200 --> 00:40:08.630
the execution? Uh do you buy all three
00:40:08.640 --> 00:40:10.550
separately? Right? you go I got platform
00:40:10.560 --> 00:40:13.030
that runs the code an execution harness
00:40:13.040 --> 00:40:17.990
that understands almost like a um a data
00:40:18.000 --> 00:40:19.910
integration hub right you know my
00:40:19.920 --> 00:40:22.870
context sits in an in a graph database
00:40:22.880 --> 00:40:25.829
or an MD files or in a confluence wiki
00:40:25.839 --> 00:40:28.870
or whatever my harness for the policies
00:40:28.880 --> 00:40:30.950
and procedures and way of working sits
00:40:30.960 --> 00:40:33.990
over here and whatever and it knows it's
00:40:34.000 --> 00:40:35.349
already been written to integrate to run
00:40:35.359 --> 00:40:38.470
on data bricks or snowflake or iceberg
00:40:38.480 --> 00:40:41.030
with spark park or heaven forbid fabric
00:40:41.040 --> 00:40:44.069
or IBM or uh Salesforce I mean
00:40:44.079 --> 00:40:47.829
Informatica um yeah like it becomes a
00:40:47.839 --> 00:40:51.430
language right like would you why I mean
00:40:51.440 --> 00:40:53.589
it's there's so many moving parts that
00:40:53.599 --> 00:40:55.990
that decoupling is going to kill you in
00:40:56.000 --> 00:40:58.710
my view
00:40:58.720 --> 00:41:02.950
yeah I think um [snorts]
00:41:02.960 --> 00:41:06.309
yeah I mean this is awesome
00:41:06.319 --> 00:41:10.150
this is awesome decoupling Um but yeah,
00:41:10.160 --> 00:41:12.710
this like this complexity is so funny
00:41:12.720 --> 00:41:15.109
like the complexity that we have to
00:41:15.119 --> 00:41:21.750
handle. Um
00:41:21.760 --> 00:41:25.109
yeah, I mean I I mean I have cognitive
00:41:25.119 --> 00:41:27.109
overload every day like and I and I read
00:41:27.119 --> 00:41:29.750
in a copious noteaker. Um although most
00:41:29.760 --> 00:41:31.270
my half my notes are actually on pieces
00:41:31.280 --> 00:41:34.790
of paper. Um
00:41:34.800 --> 00:41:36.470
yeah I do wonder like how people are
00:41:36.480 --> 00:41:37.990
going to like digest and work through
00:41:38.000 --> 00:41:40.550
all this stuff right like you're
00:41:40.560 --> 00:41:42.950
mentioning markdown graph like all these
00:41:42.960 --> 00:41:45.430
I'm going to say toys but like all these
00:41:45.440 --> 00:41:47.109
things and ultimately it's just like how
00:41:47.119 --> 00:41:48.550
much is going to be decided by the
00:41:48.560 --> 00:41:50.550
system and ultimately and the other part
00:41:50.560 --> 00:41:52.630
is like how much do you want like you
00:41:52.640 --> 00:41:53.990
you look at like how many people came
00:41:54.000 --> 00:41:56.630
back to you know like I wouldn't say
00:41:56.640 --> 00:41:59.430
like fled Microsoft Windows but you know
00:41:59.440 --> 00:42:01.270
um and went to Mac right and you know
00:42:01.280 --> 00:42:03.430
I'm a huge Linux guy or was you know all
00:42:03.440 --> 00:42:05.109
my servers are almost always Linux. I
00:42:05.119 --> 00:42:06.790
almost never see use Windows server like
00:42:06.800 --> 00:42:10.550
blah blah blah. Um [cough]
00:42:10.560 --> 00:42:14.150
and um
00:42:14.160 --> 00:42:16.390
you know and then uh what's uh DHH
00:42:16.400 --> 00:42:19.589
announced Omari like and like Toby Luca
00:42:19.599 --> 00:42:21.910
and uh a bunch of other names I'm going
00:42:21.920 --> 00:42:23.829
to like destroy and they basically put
00:42:23.839 --> 00:42:27.349
like $10 million into make Linux amazing
00:42:27.359 --> 00:42:29.510
and I was like man maybe this will
00:42:29.520 --> 00:42:31.430
finally happen. Maybe it will. I'm not
00:42:31.440 --> 00:42:33.910
sure. Um,
00:42:33.920 --> 00:42:35.910
and so I just think things are changing,
00:42:35.920 --> 00:42:37.270
right? Like I got really comfortable
00:42:37.280 --> 00:42:38.950
with Mac and like how Mac does it and
00:42:38.960 --> 00:42:41.030
keeps my life simpler. And so when
00:42:41.040 --> 00:42:42.550
you're talking about like harnesses and
00:42:42.560 --> 00:42:44.230
stuff like that, I was like I mean we've
00:42:44.240 --> 00:42:45.750
already demonstrated constantly like
00:42:45.760 --> 00:42:48.150
whatever is easy. Wherever the water
00:42:48.160 --> 00:42:50.550
flows easiest is where people naturally
00:42:50.560 --> 00:42:53.109
go, right? Even if it's better for them,
00:42:53.119 --> 00:42:55.510
right? Maybe if like freedom, open
00:42:55.520 --> 00:42:58.630
source, Linux, whatever, the vast
00:42:58.640 --> 00:43:01.430
majority of people will be like, "Okay,
00:43:01.440 --> 00:43:02.870
if I don't have to think about this for
00:43:02.880 --> 00:43:04.309
five minutes and I have to pay somebody
00:43:04.319 --> 00:43:06.710
five bucks or 20 bucks a month, then you
00:43:06.720 --> 00:43:08.470
know, a huge portion of people just like
00:43:08.480 --> 00:43:10.390
don't care." Although the irony to me
00:43:10.400 --> 00:43:12.309
though is how many people don't pay for
00:43:12.319 --> 00:43:14.710
these open AI tools or like things and
00:43:14.720 --> 00:43:21.589
they just are willing to take the ads
00:43:21.599 --> 00:43:23.750
>> and I you know taking income across
00:43:23.760 --> 00:43:25.990
distribution across the world out of the
00:43:26.000 --> 00:43:27.750
equation here. But like how many people
00:43:27.760 --> 00:43:30.470
are willing to sacrifice their attention
00:43:30.480 --> 00:43:32.870
and the bias information coming to them
00:43:32.880 --> 00:43:34.870
just so they can
00:43:34.880 --> 00:43:36.870
>> like how how many people are willing to
00:43:36.880 --> 00:43:39.109
sacrifice their personal context. Right.
00:43:39.119 --> 00:43:41.430
So it was a a guy in New Zealand
00:43:41.440 --> 00:43:43.030
reposted something on LinkedIn and it
00:43:43.040 --> 00:43:45.349
was um it's from one of the UK police
00:43:45.359 --> 00:43:47.990
companies uh or and it basically had a
00:43:48.000 --> 00:43:49.910
mockup of you know two kids in school
00:43:49.920 --> 00:43:51.829
uniforms and they tagged all the
00:43:51.839 --> 00:43:53.589
personally identifiable information. So,
00:43:53.599 --> 00:43:55.510
you know, the badge on the blazer says
00:43:55.520 --> 00:43:58.390
which school they went to. Um, the mom
00:43:58.400 --> 00:44:00.309
holding or the dad holding because you
00:44:00.319 --> 00:44:02.470
couldn't see their face. A a polyarin
00:44:02.480 --> 00:44:03.670
cup with their name on it from the
00:44:03.680 --> 00:44:06.470
coffee company, you know, John. Um,
00:44:06.480 --> 00:44:07.990
actually, I think it was a lady's name.
00:44:08.000 --> 00:44:12.309
Um, yeah. Uh, like a a number plate for
00:44:12.319 --> 00:44:15.829
the car. Um, yes. All these things that
00:44:15.839 --> 00:44:18.710
were in an image that now, you know, bad
00:44:18.720 --> 00:44:20.309
actors can scrape. And I sit there
00:44:20.319 --> 00:44:22.950
going, you number of people I know that
00:44:22.960 --> 00:44:25.430
put all their kids stuff on Facebook
00:44:25.440 --> 00:44:28.550
constantly, you know, videos and images
00:44:28.560 --> 00:44:30.550
and and basically document their entire
00:44:30.560 --> 00:44:31.910
kids' life, you know.
00:44:31.920 --> 00:44:32.870
>> Oh, yeah.
00:44:32.880 --> 00:44:35.670
>> And the kids five in 20 years time,
00:44:35.680 --> 00:44:38.069
right, when when none of that
00:44:38.079 --> 00:44:40.150
information is private and it's
00:44:40.160 --> 00:44:44.470
available to any platform,
00:44:44.480 --> 00:44:45.750
what's that kid going to say to their
00:44:45.760 --> 00:44:48.069
parents? Hey, thanks for that. Was it
00:44:48.079 --> 00:44:50.150
worth it? I don't know. Anyway, I go I
00:44:50.160 --> 00:44:51.910
can go deep into conspiracy theory.
00:44:51.920 --> 00:44:54.790
>> Yeah. My wife a pretty hard stance on on
00:44:54.800 --> 00:44:56.710
on this stuff and we try to like
00:44:56.720 --> 00:44:59.109
anything that says like share our kids
00:44:59.119 --> 00:45:02.470
pictures or whatever with like any big,
00:45:02.480 --> 00:45:05.510
you know, Apple or uh any of that like
00:45:05.520 --> 00:45:07.589
we we try to shut that all that down. I
00:45:07.599 --> 00:45:09.270
don't I don't trust any any of these
00:45:09.280 --> 00:45:10.950
guys. They'll make a good buck. you
00:45:10.960 --> 00:45:13.109
know, they they claim, you know, do good
00:45:13.119 --> 00:45:16.550
or whatever, but and I, you know, I I am
00:45:16.560 --> 00:45:18.390
wouldn't say for I'm not so crazy where
00:45:18.400 --> 00:45:19.750
I'm gonna wear a tin wall hat and be
00:45:19.760 --> 00:45:22.150
like, ah, I can't. No, I would just like
00:45:22.160 --> 00:45:23.910
actively participating by like, you
00:45:23.920 --> 00:45:26.630
know, having my kids pictures on like on
00:45:26.640 --> 00:45:29.430
Instagram or LinkedIn or I'm just like,
00:45:29.440 --> 00:45:32.470
no, it's not not where where I want that
00:45:32.480 --> 00:45:35.430
kind of stuff to happen. And the other
00:45:35.440 --> 00:45:40.230
thing is um
00:45:40.240 --> 00:45:41.750
yeah, I mean we share our data so
00:45:41.760 --> 00:45:43.510
freely. I mean I'm just like right now
00:45:43.520 --> 00:45:46.630
I'm like on an absolute rager trying to
00:45:46.640 --> 00:45:49.270
get uh my phone numbers pulled from data
00:45:49.280 --> 00:45:51.109
brokers like crazy right now because I
00:45:51.119 --> 00:45:52.630
can't even use my cell phone. It's
00:45:52.640 --> 00:45:55.829
gotten so bad. Like I just can't pick so
00:45:55.839 --> 00:45:58.230
many calls all the time and text
00:45:58.240 --> 00:46:00.309
messages. Like if I sneeze on this
00:46:00.319 --> 00:46:04.390
podcast saying like virus software, um
00:46:04.400 --> 00:46:07.349
I'm probably going to get like like just
00:46:07.359 --> 00:46:11.589
tsunami of communication. Um so then
00:46:11.599 --> 00:46:13.349
it's just like I don't It's so funny. I
00:46:13.359 --> 00:46:15.190
feel like it's gonna be like AI versus
00:46:15.200 --> 00:46:18.550
AI wars fighting like spam, right?
00:46:18.560 --> 00:46:20.069
>> Oh, well actually Anna and I were
00:46:20.079 --> 00:46:21.349
talking about that when we caught up
00:46:21.359 --> 00:46:24.150
last week or the week before and uh Joe
00:46:24.160 --> 00:46:26.790
Ree had jumped on. we saw him turn up
00:46:26.800 --> 00:46:28.230
and and Anna was talking because she's
00:46:28.240 --> 00:46:30.950
working in a in a new gig in a hospital
00:46:30.960 --> 00:46:32.390
and it's Greenfields, right? And so
00:46:32.400 --> 00:46:35.589
she's talking through um the the way
00:46:35.599 --> 00:46:36.950
she's dealing with that green threes
00:46:36.960 --> 00:46:38.710
program, right? And and what she's do
00:46:38.720 --> 00:46:40.390
looking at now, what she look at next.
00:46:40.400 --> 00:46:41.910
And I'm sitting there, I said to her,
00:46:41.920 --> 00:46:43.750
man, you got to get snotted by sales
00:46:43.760 --> 00:46:45.030
people, right? Because you've said that
00:46:45.040 --> 00:46:47.109
eventually you're gonna buy or build
00:46:47.119 --> 00:46:50.230
something in this space. And so my joke
00:46:50.240 --> 00:46:51.750
was, that's fine. Just tell them that
00:46:51.760 --> 00:46:53.430
they need to get hold of Joe, right? He
00:46:53.440 --> 00:46:55.030
he's he's in charge of buying for you.
00:46:55.040 --> 00:46:56.550
and [laughter] and then they'll get so
00:46:56.560 --> 00:46:58.710
there you go right if you if uh if you
00:46:58.720 --> 00:47:00.790
do need a virus thing I think anybody
00:47:00.800 --> 00:47:03.589
that's uh scraping this their AI should
00:47:03.599 --> 00:47:06.230
contact Jay Ree about selling Tim virus
00:47:06.240 --> 00:47:09.109
stuff but let let's go back to this idea
00:47:09.119 --> 00:47:13.349
of privacy because or or context privacy
00:47:13.359 --> 00:47:14.550
because one of the interesting things
00:47:14.560 --> 00:47:16.630
now if we say that we can't reverse
00:47:16.640 --> 00:47:18.390
engineer context for an organization
00:47:18.400 --> 00:47:20.790
from the SQL queries have been run right
00:47:20.800 --> 00:47:22.870
because we know that based on all the
00:47:22.880 --> 00:47:24.390
papers that written human in the loop
00:47:24.400 --> 00:47:25.990
for that context is actually the most
00:47:26.000 --> 00:47:28.230
valuable thing you can do. And if you're
00:47:28.240 --> 00:47:30.550
sitting in an organization that has a
00:47:30.560 --> 00:47:33.430
history of counting people when uh
00:47:33.440 --> 00:47:34.950
they want to save money or they want
00:47:34.960 --> 00:47:37.510
their share price to go up and and your
00:47:37.520 --> 00:47:39.030
comment about all your notes being on
00:47:39.040 --> 00:47:41.750
paper, right? Like how willing would you
00:47:41.760 --> 00:47:45.109
be to put your knowledge into anything
00:47:45.119 --> 00:47:47.589
electronic and not just retain it in
00:47:47.599 --> 00:47:49.430
notebooks on your desk that you never
00:47:49.440 --> 00:47:51.349
share? Like that's got to be coming,
00:47:51.359 --> 00:47:53.910
right? where people actually gate the
00:47:53.920 --> 00:47:56.150
knowledge they have because it's the
00:47:56.160 --> 00:47:57.510
only thing of value that the
00:47:57.520 --> 00:47:59.829
organization may not retrench their ass
00:47:59.839 --> 00:48:01.910
for. What do you reckon?
00:48:01.920 --> 00:48:06.230
>> Yeah. No, I it's so funny like you know
00:48:06.240 --> 00:48:08.309
can you even trust your systems like
00:48:08.319 --> 00:48:09.990
sometimes like you're reading like the
00:48:10.000 --> 00:48:12.230
terms of service and like when like
00:48:12.240 --> 00:48:15.030
people are like well I you know
00:48:15.040 --> 00:48:19.109
sometimes you know I do question
00:48:19.119 --> 00:48:21.270
like do some of these servers even need
00:48:21.280 --> 00:48:24.549
your full like context can they get it
00:48:24.559 --> 00:48:26.790
from the shape of your information right
00:48:26.800 --> 00:48:28.870
>> right you know and what I mean by that
00:48:28.880 --> 00:48:31.670
is like the metadata how it moves like
00:48:31.680 --> 00:48:32.870
how you're doing things on in your
00:48:32.880 --> 00:48:34.630
system. Not necessarily as so much that
00:48:34.640 --> 00:48:36.630
you're like Tim Fraser opened Excel
00:48:36.640 --> 00:48:39.109
sheet one, two, three and like looked at
00:48:39.119 --> 00:48:42.150
X column, but like because of the sea of
00:48:42.160 --> 00:48:44.069
information, they could a you could
00:48:44.079 --> 00:48:46.870
ascertain like how people were solving
00:48:46.880 --> 00:48:49.030
and doing things. So then like you're
00:48:49.040 --> 00:48:52.069
ultimately using these systems to train
00:48:52.079 --> 00:48:53.910
the system. You know, Meta is doing this
00:48:53.920 --> 00:48:55.510
for like tracking their own employees
00:48:55.520 --> 00:48:57.910
and having all the other kind of stuff.
00:48:57.920 --> 00:48:59.510
you know, how much longer are we going
00:48:59.520 --> 00:49:01.910
to be able to rely on the computers that
00:49:01.920 --> 00:49:03.990
we're using because the money is just
00:49:04.000 --> 00:49:08.150
it's so worth worth it, right? So,
00:49:08.160 --> 00:49:10.630
>> yeah. And you I mean, you read that
00:49:10.640 --> 00:49:12.790
Spirit article, right, where Spirit
00:49:12.800 --> 00:49:15.430
Airlines, Google paid $10 million to get
00:49:15.440 --> 00:49:18.069
access to all the data in that
00:49:18.079 --> 00:49:19.990
organization, all the employee Slack
00:49:20.000 --> 00:49:23.109
messages or every email. Now, in theory,
00:49:23.119 --> 00:49:25.270
the article said that personally
00:49:25.280 --> 00:49:28.069
identifiable data would be cleansed from
00:49:28.079 --> 00:49:30.470
that data set
00:49:30.480 --> 00:49:33.270
um or data lake probably, isn't it? Um
00:49:33.280 --> 00:49:35.990
before it's handed to Google, but it's a
00:49:36.000 --> 00:49:39.349
liquidated company, right? Like, yeah,
00:49:39.359 --> 00:49:41.910
they how much money is a liquidator
00:49:41.920 --> 00:49:43.910
going to pay cleaning that data before
00:49:43.920 --> 00:49:46.150
it hands off to Google? And if it's not
00:49:46.160 --> 00:49:48.790
cleaned, who gets sued, right? like if
00:49:48.800 --> 00:49:51.750
an employes's personal information is
00:49:51.760 --> 00:49:56.309
passed to Google as part of that data um
00:49:56.319 --> 00:49:58.390
and they find out who who do they
00:49:58.400 --> 00:50:02.710
complain to right uh but yeah so again
00:50:02.720 --> 00:50:04.790
if you now start to see companies that
00:50:04.800 --> 00:50:07.990
are in trouble having their their
00:50:08.000 --> 00:50:10.630
conversations right being liquidated as
00:50:10.640 --> 00:50:14.630
an asset
00:50:14.640 --> 00:50:17.510
>> reaches like you know if like granola
00:50:17.520 --> 00:50:20.150
or I'm not saying the two, but like if
00:50:20.160 --> 00:50:23.109
that happens, holy
00:50:23.119 --> 00:50:24.870
>> Well, the the the company that came
00:50:24.880 --> 00:50:27.030
second in the bidding war, right, I
00:50:27.040 --> 00:50:29.430
think they bid 7 something million and
00:50:29.440 --> 00:50:31.349
lost out. They were a recruitment
00:50:31.359 --> 00:50:33.190
company.
00:50:33.200 --> 00:50:35.589
And so you're sitting there going, hm,
00:50:35.599 --> 00:50:38.470
they wanted to pay 7.5 or whatever it
00:50:38.480 --> 00:50:40.309
was, million US dollars for the data
00:50:40.319 --> 00:50:43.430
from a airline company uh with every
00:50:43.440 --> 00:50:45.030
conversation with every employee and
00:50:45.040 --> 00:50:46.950
every customer.
00:50:46.960 --> 00:50:48.630
They're definitely not profiling people,
00:50:48.640 --> 00:50:49.910
right? They're definitely not building
00:50:49.920 --> 00:50:52.230
an engine uh to augment all their
00:50:52.240 --> 00:50:54.549
LinkedIn scraping data about Shane
00:50:54.559 --> 00:50:56.790
Gibson uh who just happened to be
00:50:56.800 --> 00:50:59.190
working for Spirity Airlines and, you
00:50:59.200 --> 00:51:01.270
know, slacked all the time to his mates,
00:51:01.280 --> 00:51:05.425
his colleagues. God, my boss is a dick.
00:51:05.435 --> 00:51:05.990
[laughter]
00:51:06.000 --> 00:51:07.510
>> I mean, it's really interesting. So,
00:51:07.520 --> 00:51:08.870
like, you know, we're talking about this
00:51:08.880 --> 00:51:10.950
the sale of data and all that jazz.
00:51:10.960 --> 00:51:12.390
Could you imagine how many people when
00:51:12.400 --> 00:51:15.190
like 23 and me went bankrupt and they
00:51:15.200 --> 00:51:16.950
got bought by uh I believe private
00:51:16.960 --> 00:51:20.230
equity and um you know I was on there
00:51:20.240 --> 00:51:23.990
too like the awesome power that you have
00:51:24.000 --> 00:51:27.510
with that is you know like honestly I'm
00:51:27.520 --> 00:51:28.710
not smart enough to know all the
00:51:28.720 --> 00:51:30.549
information they could extract and glean
00:51:30.559 --> 00:51:32.230
for the details but was pretty powerful
00:51:32.240 --> 00:51:35.270
and personal information and I was
00:51:35.280 --> 00:51:37.270
definitely one of the I was like so fast
00:51:37.280 --> 00:51:39.349
to be like thank god you're based in
00:51:39.359 --> 00:51:41.750
California and I have like CCP PA like
00:51:41.760 --> 00:51:44.870
just dump all my dump like I don't trust
00:51:44.880 --> 00:51:46.710
you guys whatsoever. I don't even know
00:51:46.720 --> 00:51:49.270
why I trust them in the first place. I
00:51:49.280 --> 00:51:51.430
think that's just like h
00:51:51.440 --> 00:51:53.910
>> but if you look at the portfolio right
00:51:53.920 --> 00:51:55.829
the book that the PE companies hold
00:51:55.839 --> 00:51:57.910
right now of companies they've invested
00:51:57.920 --> 00:52:00.069
in that aren't profitable that aren't
00:52:00.079 --> 00:52:01.430
going to not they're not even going to
00:52:01.440 --> 00:52:03.030
get a down round right they're just not
00:52:03.040 --> 00:52:05.190
going to get any more investment you
00:52:05.200 --> 00:52:06.790
apart from the people that work for them
00:52:06.800 --> 00:52:10.309
who will leave probably right the only
00:52:10.319 --> 00:52:12.150
other thing of value really is all that
00:52:12.160 --> 00:52:13.829
data about their employees and their
00:52:13.839 --> 00:52:15.510
customers and
00:52:15.520 --> 00:52:17.349
>> well I think one of the things too is
00:52:17.359 --> 00:52:20.150
just like a lot of people or like I I
00:52:20.160 --> 00:52:21.030
think one of the things we're talking
00:52:21.040 --> 00:52:21.990
about like you know the data and
00:52:22.000 --> 00:52:23.270
information you have like one of the
00:52:23.280 --> 00:52:24.309
things that people are going to quickly
00:52:24.319 --> 00:52:25.990
realize in the database is like you
00:52:26.000 --> 00:52:27.270
don't want to sell don't sell
00:52:27.280 --> 00:52:29.030
necessarily sell sell your raw data
00:52:29.040 --> 00:52:30.390
don't like don't do that you're
00:52:30.400 --> 00:52:33.190
literally just literally giving the
00:52:33.200 --> 00:52:35.990
chicken away and keep and keeping the
00:52:36.000 --> 00:52:39.109
eggs the eggs only last a little and you
00:52:39.119 --> 00:52:41.030
know
00:52:41.040 --> 00:52:43.030
there's going to be a whole like network
00:52:43.040 --> 00:52:44.230
and we can talk maybe we'll talk about
00:52:44.240 --> 00:52:46.630
this next time as as I'm doing lots more
00:52:46.640 --> 00:52:49.510
fun research on But it's like in the age
00:52:49.520 --> 00:52:51.430
of all this AI search and all the things
00:52:51.440 --> 00:52:53.270
that are happening, how do we get
00:52:53.280 --> 00:52:55.750
personalized experiences? How does it
00:52:55.760 --> 00:52:58.230
work with AI search? And you could argue
00:52:58.240 --> 00:52:59.829
we've had AI search for years. It's just
00:52:59.839 --> 00:53:01.910
Google's finally being like, oh, we have
00:53:01.920 --> 00:53:04.630
a challenger. Um,
00:53:04.640 --> 00:53:06.230
and all that and all that information
00:53:06.240 --> 00:53:07.910
like how is it going to be shared?
00:53:07.920 --> 00:53:08.870
Because I don't think it's going to be
00:53:08.880 --> 00:53:10.950
shared in the way that it was before,
00:53:10.960 --> 00:53:12.390
right? You know, where you have a cookie
00:53:12.400 --> 00:53:14.710
and a certain thing happens because you
00:53:14.720 --> 00:53:16.390
can't really share information like
00:53:16.400 --> 00:53:17.430
that. You're going to be sharing
00:53:17.440 --> 00:53:20.630
patterns essentially and and shapes of
00:53:20.640 --> 00:53:23.109
people. Well, Tim Fraser showed up and,
00:53:23.119 --> 00:53:25.510
you know, did this search. So, people
00:53:25.520 --> 00:53:27.510
like Tim Fraser in the shape of what
00:53:27.520 --> 00:53:29.990
they're looking for like these things.
00:53:30.000 --> 00:53:32.790
Good or bad, I can definitely start to
00:53:32.800 --> 00:53:35.510
see that happening. Um,
00:53:35.520 --> 00:53:38.150
and and it's going to happen
00:53:38.160 --> 00:53:41.430
very very very very quick. Um,
00:53:41.440 --> 00:53:43.990
>> and and how far does it go? Right. So,
00:53:44.000 --> 00:53:47.190
when I'm asking perplexity a question,
00:53:47.200 --> 00:53:49.190
uh, and they've decided that they'll now
00:53:49.200 --> 00:53:51.589
influence the response based on who's
00:53:51.599 --> 00:53:53.030
paying them the most.
00:53:53.040 --> 00:53:53.430
>> Yeah.
00:53:53.440 --> 00:53:55.990
>> Yeah. I'm going to get, you know, uh,
00:53:56.000 --> 00:53:58.309
hidden editorials of, you know, I I use
00:53:58.319 --> 00:53:59.990
Plexity to figure out what cocktail I
00:54:00.000 --> 00:54:02.069
should make next. Yeah. It's going to
00:54:02.079 --> 00:54:04.470
start telling me to make rum based ones
00:54:04.480 --> 00:54:06.150
or tequila based ones, right?
00:54:06.160 --> 00:54:07.270
>> Yeah. Yeah. And I'll tell you, Greg
00:54:07.280 --> 00:54:09.030
Goose is the best vodka. And you're
00:54:09.040 --> 00:54:11.990
like, "Wait a minute."
00:54:12.000 --> 00:54:13.750
What is the why why did you determine
00:54:13.760 --> 00:54:15.910
that Great Grey Goose vodka was the best
00:54:15.920 --> 00:54:18.230
one? Tiny little like symbol like
00:54:18.240 --> 00:54:20.230
asterisk at the top and
00:54:20.240 --> 00:54:22.230
>> and why why Gray Goose, right? I just
00:54:22.240 --> 00:54:23.910
said I need to make a cocktail, right?
00:54:23.920 --> 00:54:25.430
And it should just say vodka. Now, why
00:54:25.440 --> 00:54:27.750
has it got a brand around it?
00:54:27.760 --> 00:54:30.150
>> So that you know that's coming, right? I
00:54:30.160 --> 00:54:32.549
don't believe that it's not. But then if
00:54:32.559 --> 00:54:35.589
we're using those models corporately
00:54:35.599 --> 00:54:38.069
now what stops them influencing the
00:54:38.079 --> 00:54:41.190
models you know who who's our top
00:54:41.200 --> 00:54:43.589
supplier oh actually you know the data
00:54:43.599 --> 00:54:46.309
says it's one two three but it's a close
00:54:46.319 --> 00:54:48.069
call I think we'll just influence it to
00:54:48.079 --> 00:54:51.750
be 256 right
00:54:51.760 --> 00:54:53.190
you can't see the working right you
00:54:53.200 --> 00:54:55.670
can't see it's not deterministic so you
00:54:55.680 --> 00:54:59.109
can't see the code that it's using to
00:54:59.119 --> 00:55:01.910
give you that answer.
00:55:01.920 --> 00:55:04.390
>> Hey man, fair it's a fair fair shadow,
00:55:04.400 --> 00:55:05.829
right? It was like how do you you know
00:55:05.839 --> 00:55:07.430
there's tons of research of like trust
00:55:07.440 --> 00:55:10.309
and trust systems you know it was
00:55:10.319 --> 00:55:12.309
already it was already getting pretty
00:55:12.319 --> 00:55:14.069
bad if we want to talk about trust in
00:55:14.079 --> 00:55:15.990
the internet we all know it and the
00:55:16.000 --> 00:55:18.069
influ and influencers like what's real
00:55:18.079 --> 00:55:19.910
what's not real I mean it's just going
00:55:19.920 --> 00:55:22.790
to be just monster I mean you know
00:55:22.800 --> 00:55:25.510
speaking of like content and trust and
00:55:25.520 --> 00:55:27.670
black boxes you know I think a lot of us
00:55:27.680 --> 00:55:29.510
have recently have been like we all use
00:55:29.520 --> 00:55:31.829
LinkedIn but none of us are like
00:55:31.839 --> 00:55:34.630
whatever it's just like it's a
00:55:34.640 --> 00:55:36.630
you know, it's like I can't believe I'm
00:55:36.640 --> 00:55:38.390
going back to Twitter. Sorry, X,
00:55:38.400 --> 00:55:39.510
whatever that's called now. But like
00:55:39.520 --> 00:55:40.950
going back to like Twitter a little bit
00:55:40.960 --> 00:55:43.270
to get real information on AI
00:55:43.280 --> 00:55:44.710
development. I'm like, oh my god, I
00:55:44.720 --> 00:55:47.670
can't I'm back again or, you know, I'm
00:55:47.680 --> 00:55:49.430
trying to use like Blue Sky, but the
00:55:49.440 --> 00:55:52.710
network's shrinking. I don't know. It is
00:55:52.720 --> 00:55:55.829
like I I'm like, can I just have like a
00:55:55.839 --> 00:55:58.549
certified human network?
00:55:58.559 --> 00:56:01.190
>> And does it have to be human? Like cuz
00:56:01.200 --> 00:56:03.349
LinkedIn you still is the only network
00:56:03.359 --> 00:56:06.789
that has reach like
00:56:06.799 --> 00:56:09.270
>> roughly tr kind of roughly trust that
00:56:09.280 --> 00:56:10.470
people are verified.
00:56:10.480 --> 00:56:10.870
>> Yeah.
00:56:10.880 --> 00:56:13.910
>> Um to a certain extent but
00:56:13.920 --> 00:56:15.270
>> it's going to somebody has to come out
00:56:15.280 --> 00:56:17.030
with a new network model, right? But it
00:56:17.040 --> 00:56:19.589
won't be like any of the others and
00:56:19.599 --> 00:56:23.430
maybe it will be just LLM friends um and
00:56:23.440 --> 00:56:25.349
not real humans. Maybe that becomes the
00:56:25.359 --> 00:56:27.270
network, right? I don't know. I mean I
00:56:27.280 --> 00:56:28.630
think it's going to get blown open.
00:56:28.640 --> 00:56:29.589
somebody's going to come up with
00:56:29.599 --> 00:56:31.030
something creative that solves the
00:56:31.040 --> 00:56:32.789
problem because I think everybody I know
00:56:32.799 --> 00:56:35.910
that's on LinkedIn is like it sucks ass.
00:56:35.920 --> 00:56:38.309
Um but where else can I go?
00:56:38.319 --> 00:56:41.910
>> I mean just like like I did a post um
00:56:41.920 --> 00:56:43.430
>> you know I mean I post on LinkedIn quite
00:56:43.440 --> 00:56:44.630
a bit because I still think it's you
00:56:44.640 --> 00:56:46.710
know I'm remote first person like you.
00:56:46.720 --> 00:56:48.549
Um I you know you might live in a little
00:56:48.559 --> 00:56:50.614
bit further out of place than I do.
00:56:50.624 --> 00:56:52.390
[laughter] Um
00:56:52.400 --> 00:56:55.829
>> uh you know I live in Philadelphia.
00:56:55.839 --> 00:56:58.230
It's not notorious for its software
00:56:58.240 --> 00:57:00.230
engineering and data. DBT Labs might be
00:57:00.240 --> 00:57:02.549
from here and Vanguard up up the hill
00:57:02.559 --> 00:57:05.270
and and a lot of private equity and and
00:57:05.280 --> 00:57:07.030
capital is here, but it you know it's
00:57:07.040 --> 00:57:09.349
not known as a tech scene really
00:57:09.359 --> 00:57:11.430
whatsoever. Like it's not even in the
00:57:11.440 --> 00:57:15.670
top 100 I think in the uh the world. And
00:57:15.680 --> 00:57:17.589
so like one of my ways of connecting is
00:57:17.599 --> 00:57:19.670
through like LinkedIn and Substack and
00:57:19.680 --> 00:57:20.950
all those tools.
00:57:20.960 --> 00:57:23.430
>> And but I was I was noticing like it's
00:57:23.440 --> 00:57:25.589
crazy. So, I'll I'll do a post and I'll
00:57:25.599 --> 00:57:27.190
be on my former, you know, about myself,
00:57:27.200 --> 00:57:29.109
blah blah blah,
00:57:29.119 --> 00:57:31.750
a podcast episode, 500 impressions and
00:57:31.760 --> 00:57:35.030
three comments and 14 likes. I do one
00:57:35.040 --> 00:57:39.190
and it's like 38 comments, 15 reposts,
00:57:39.200 --> 00:57:43.190
115,000 impressions. And I'm like,
00:57:43.200 --> 00:57:45.109
is it because I did a meme with it? Is
00:57:45.119 --> 00:57:46.150
that
00:57:46.160 --> 00:57:47.910
>> It's I was gonna ask it was a meme,
00:57:47.920 --> 00:57:49.589
right? All you did was post a meme. That
00:57:49.599 --> 00:57:49.910
was
00:57:49.920 --> 00:57:51.270
>> No, it wasn't. It was like It was an
00:57:51.280 --> 00:57:53.270
actual like moderious question. Maybe I
00:57:53.280 --> 00:57:56.309
hit like the smart things. I said um
00:57:56.319 --> 00:57:59.030
hiring and data analyst and like it's
00:57:59.040 --> 00:58:00.630
just like the keyword hiring. It's just
00:58:00.640 --> 00:58:02.630
like up to the right. It's just like
00:58:02.640 --> 00:58:04.950
where AI, if you put AI in anything, it
00:58:04.960 --> 00:58:07.589
would just go like crazy. And then but
00:58:07.599 --> 00:58:09.510
like because of these algorithms and
00:58:09.520 --> 00:58:11.990
your network effect, I don't know how
00:58:12.000 --> 00:58:13.670
it's like working. And I try to like
00:58:13.680 --> 00:58:15.510
understand it deeply.
00:58:15.520 --> 00:58:16.870
>> And I talk to other brands that have
00:58:16.880 --> 00:58:19.349
like 40 or 50,000 people on LinkedIn and
00:58:19.359 --> 00:58:20.789
they're getting like a thousand
00:58:20.799 --> 00:58:22.950
impressions or like you know 15 people
00:58:22.960 --> 00:58:24.789
are are liking and it's always the same
00:58:24.799 --> 00:58:26.390
15 people in the group and they're like
00:58:26.400 --> 00:58:27.750
I don't get this. Like
00:58:27.760 --> 00:58:30.069
>> I work so hard on these blackbox
00:58:30.079 --> 00:58:32.390
networks and now my content with my
00:58:32.400 --> 00:58:34.390
tribe like literally doesn't show up for
00:58:34.400 --> 00:58:34.710
them.
00:58:34.720 --> 00:58:37.270
>> Yeah. And and like how often like are
00:58:37.280 --> 00:58:39.990
anybody AI slot botting, right? Like if
00:58:40.000 --> 00:58:41.910
you want to go against a competitor of
00:58:41.920 --> 00:58:44.230
yours, you just basically have an
00:58:44.240 --> 00:58:47.270
account that AI slots tags every
00:58:47.280 --> 00:58:49.589
competitor post and that's going to hurt
00:58:49.599 --> 00:58:51.670
their ranking, right? Because that's the
00:58:51.680 --> 00:58:55.190
new punitive uh human in the loop thing.
00:58:55.200 --> 00:58:58.069
So yet you look at Substack, right?
00:58:58.079 --> 00:59:00.710
We've got Substack notes. We've got what
00:59:00.720 --> 00:59:02.710
I feel is a more trusted network in
00:59:02.720 --> 00:59:05.910
Substack, but it still not my go-to,
00:59:05.920 --> 00:59:08.230
right? If I want to see what my network
00:59:08.240 --> 00:59:10.549
>> still private equity backed
00:59:10.559 --> 00:59:11.510
>> yeah true
00:59:11.520 --> 00:59:12.789
>> is it going to be you know is it going
00:59:12.799 --> 00:59:15.829
to be the next medium I don't know I I I
00:59:15.839 --> 00:59:17.589
like substack
00:59:17.599 --> 00:59:20.069
um I was on ghost for a while being more
00:59:20.079 --> 00:59:21.430
open source but it doesn't have the
00:59:21.440 --> 00:59:23.430
network effect it's just yet again it's
00:59:23.440 --> 00:59:25.190
easier to use it's quick it's got a good
00:59:25.200 --> 00:59:26.950
network I don't know it's gonna you know
00:59:26.960 --> 00:59:28.069
how much is going to change are they
00:59:28.079 --> 00:59:29.270
going to say something that all of us
00:59:29.280 --> 00:59:32.950
are like oh yeah no you're not really
00:59:32.960 --> 00:59:34.309
you're just you're still very much an
00:59:34.319 --> 00:59:37.910
American company you're very much biased
00:59:37.920 --> 00:59:39.990
towards like getting pressured because
00:59:40.000 --> 00:59:41.430
somebody didn't like your content or
00:59:41.440 --> 00:59:43.190
something happened and then you're like,
00:59:43.200 --> 00:59:46.150
"Oh, all right. That was quick." Um,
00:59:46.160 --> 00:59:48.870
yeah. I mean, thank you, Shane. I've had
00:59:48.880 --> 00:59:50.950
this even though it's it feels like it
00:59:50.960 --> 00:59:52.230
was yesterday, but it's really been a
00:59:52.240 --> 00:59:52.710
month.
00:59:52.720 --> 00:59:54.549
>> It's been a month. You know, DBT Labs
00:59:54.559 --> 00:59:57.270
have been sold, orchestra's got funding,
00:59:57.280 --> 01:00:00.789
data bricks came out with uh a whole lot
01:00:00.799 --> 01:00:04.549
of new uh solution things. I mean, I I
01:00:04.559 --> 01:00:06.950
listen to uh this week in AI for every
01:00:06.960 --> 01:00:09.670
now and again, right? It's like out of
01:00:09.680 --> 01:00:11.990
two hours of everything going on and
01:00:12.000 --> 01:00:14.630
those those those two guys that run that
01:00:14.640 --> 01:00:17.430
man, they must do some research. Uh so I
01:00:17.440 --> 01:00:19.270
I don't think either of us have got time
01:00:19.280 --> 01:00:21.829
to do the research of what's happening
01:00:21.839 --> 01:00:24.150
every month. But yeah, I don't know. I
01:00:24.160 --> 01:00:25.589
think maybe maybe we'll lock a little
01:00:25.599 --> 01:00:27.670
bit of a session in, right? Part of what
01:00:27.680 --> 01:00:30.150
we talk about every month is who's done
01:00:30.160 --> 01:00:32.309
what, right? because I lose track and uh
01:00:32.319 --> 01:00:34.150
it's easy enough for me just to make a
01:00:34.160 --> 01:00:36.470
note on a piece of paper by my desk when
01:00:36.480 --> 01:00:38.230
I read something like, "Oh yeah, that's
01:00:38.240 --> 01:00:39.270
that's what's happened in the
01:00:39.280 --> 01:00:40.870
vendorland." I'll talk to Tim about
01:00:40.880 --> 01:00:42.710
that, see what he thinks. And then uh
01:00:42.720 --> 01:00:45.270
yeah, we just a little short synopsis of
01:00:45.280 --> 01:00:47.829
who who sold who, who bought who, who
01:00:47.839 --> 01:00:51.030
who opening.
01:00:51.040 --> 01:00:53.109
It's going to be so crazy. Um what
01:00:53.119 --> 01:00:54.549
conference in the next month or two?
01:00:54.559 --> 01:00:56.549
What conferences are you going to? Uh so
01:00:56.559 --> 01:00:59.910
I just did the data community virtual
01:00:59.920 --> 01:01:02.390
one here. Not doing big data London this
01:01:02.400 --> 01:01:04.470
year. Uh so the next one for me is
01:01:04.480 --> 01:01:07.030
presenting at Helsinki data week uh at
01:01:07.040 --> 01:01:09.030
the end of October which I'm mere
01:01:09.040 --> 01:01:10.390
excited to do because I've never been to
01:01:10.400 --> 01:01:14.390
Helsinki. And uh I think from what I've
01:01:14.400 --> 01:01:16.870
heard, Helsinki data work kind of is the
01:01:16.880 --> 01:01:21.030
closest to um day-to-day Texas.
01:01:21.040 --> 01:01:21.829
>> Oh, very cool.
01:01:21.839 --> 01:01:24.390
>> Which is which is still my my favorite
01:01:24.400 --> 01:01:26.870
one. Um, I kind of gave myself a
01:01:26.880 --> 01:01:28.710
blanket. I'm not going back to the US
01:01:28.720 --> 01:01:31.270
until uh the next two years until
01:01:31.280 --> 01:01:34.549
there's some changes. Uh, and then
01:01:34.559 --> 01:01:37.190
bloody Lyn Bender, who ran Day-T has
01:01:37.200 --> 01:01:38.870
come out with a new unconference
01:01:38.880 --> 01:01:40.549
community thing, and I'm like, damn, I
01:01:40.559 --> 01:01:42.470
want to go to that. I think Joe Reese is
01:01:42.480 --> 01:01:44.710
running a conference in Salt Lake in
01:01:44.720 --> 01:01:47.910
early Jan. I'm like, damn, I want to go
01:01:47.920 --> 01:01:49.589
to that. But hey, I'm not going to break
01:01:49.599 --> 01:01:51.270
my principle. Uh, I won't be back in
01:01:51.280 --> 01:01:53.270
your country. Well, sorry, that country
01:01:53.280 --> 01:01:55.030
for a couple of years. So, I'll just
01:01:55.040 --> 01:01:57.589
I'll do Europe. Um, there's actually
01:01:57.599 --> 01:01:59.910
another one in Amsterdam in October I
01:01:59.920 --> 01:02:02.470
was quite keen to go to, but um can't
01:02:02.480 --> 01:02:03.829
make it timing wise, but the other thing
01:02:03.839 --> 01:02:07.430
is it was €1,900.
01:02:07.440 --> 01:02:08.230
>> Wow.
01:02:08.240 --> 01:02:10.950
>> And maybe I'm getting cheap cheap per
01:02:10.960 --> 01:02:12.710
now. Maybe I'm too close to Scotland.
01:02:12.720 --> 01:02:15.190
Sorry, Scottish friends. Um, but you
01:02:15.200 --> 01:02:17.829
know, two grand and euros for a
01:02:17.839 --> 01:02:19.910
conference now. That that feels like the
01:02:19.920 --> 01:02:21.910
old days, right? That just feels like
01:02:21.920 --> 01:02:23.990
too much. So, uh, yeah. Yeah. What about
01:02:24.000 --> 01:02:25.910
you? You going do anything cool?
01:02:25.920 --> 01:02:29.109
>> Yeah, I'm going to um Yeah, Lynn Lynn's
01:02:29.119 --> 01:02:31.829
doing a a smaller one in in Austin, so
01:02:31.839 --> 01:02:33.990
I'm going to go there. I might be doing
01:02:34.000 --> 01:02:36.230
big day to London. It's kind of like
01:02:36.240 --> 01:02:38.230
some timing things like I usually would
01:02:38.240 --> 01:02:39.510
have bought the ticket a long time ago,
01:02:39.520 --> 01:02:40.950
at least the plane tickets to catch up
01:02:40.960 --> 01:02:43.589
with people. Just need to see kids kid
01:02:43.599 --> 01:02:45.430
and family stuff going on. So, need to
01:02:45.440 --> 01:02:48.630
see. Uh and then I might be going to San
01:02:48.640 --> 01:02:50.710
Francisco for the there's a the mother
01:02:50.720 --> 01:02:56.309
do conference in November. Um,
01:02:56.319 --> 01:02:57.910
I think that's about all the travel I'm
01:02:57.920 --> 01:03:01.270
going to be allowed to do. Uh,
01:03:01.280 --> 01:03:02.870
>> oh, actually, uh, Johnny Winter and I
01:03:02.880 --> 01:03:05.349
are talking about doing uh a low-key
01:03:05.359 --> 01:03:08.150
data meetup uh for Christmas on this
01:03:08.160 --> 01:03:10.390
side of the pond. So, uh, early
01:03:10.400 --> 01:03:13.829
December, so, you know, maybe uh, maybe
01:03:13.839 --> 01:03:16.390
get yeah, come over then. is probably
01:03:16.400 --> 01:03:18.309
not perfect for you given you probably
01:03:18.319 --> 01:03:20.390
need to go see family at Christmas, but
01:03:20.400 --> 01:03:22.309
um yeah, we're just kind of toying with
01:03:22.319 --> 01:03:26.230
a maybe a small data community thing in
01:03:26.240 --> 01:03:29.029
the UK would be fun thing to try try
01:03:29.039 --> 01:03:31.029
out. So, uh yeah, we're going to go
01:03:31.039 --> 01:03:32.549
finalize that tomorrow, whether we're
01:03:32.559 --> 01:03:34.390
going to do it or not. Um
01:03:34.400 --> 01:03:35.990
>> as I said to Johnny, might just be him
01:03:36.000 --> 01:03:37.990
and me in a pub somewhere in the UK, but
01:03:38.000 --> 01:03:41.349
hey, I'm okay with that. [laughter]
01:03:41.359 --> 01:03:43.910
>> All right, Shane, thank you so much.
01:03:43.920 --> 01:03:45.510
>> Good to see you. We'll catch you up uh
01:03:45.520 --> 01:03:46.870
in a month.
01:03:46.880 --> 01:03:47.430
>> Bye.
01:03:47.440 --> 01:03:51.079
>> Right. Get you.
Kind: captions
Language: en
00:00:01.920 --> 00:00:03.590
Yeah. So, what's what's been happening?
00:00:03.600 --> 00:00:04.870
What's top of mind? I know what's top of
00:00:04.880 --> 00:00:06.230
mind for me at the moment, but yeah.
00:00:06.240 --> 00:00:07.190
What do you got?
00:00:07.200 --> 00:00:10.310
>> I mean, for me it's Duck Labs and that
00:00:10.320 --> 00:00:11.669
change of the ecosystem. That's been
00:00:11.679 --> 00:00:13.110
really interesting and see how things
00:00:13.120 --> 00:00:15.669
are developing there. Um, it's
00:00:15.679 --> 00:00:18.070
interesting like Snowflake and Data
00:00:18.080 --> 00:00:19.590
Bricks kind of releasing numbers around
00:00:19.600 --> 00:00:21.750
DBT and like how many like how much it's
00:00:21.760 --> 00:00:24.470
moving into into their systems at a
00:00:24.480 --> 00:00:26.310
pretty breaknet
00:00:26.320 --> 00:00:29.349
pace. Um you know there's a lot of
00:00:29.359 --> 00:00:31.429
development happening in the park space
00:00:31.439 --> 00:00:33.110
um for data scans which I was like
00:00:33.120 --> 00:00:34.790
parquet gets upgrades like when is this
00:00:34.800 --> 00:00:36.709
happening um [laughter]
00:00:36.719 --> 00:00:38.630
>> isn't isn't park dead right it's got
00:00:38.640 --> 00:00:40.869
iceberg right it's it's like right what
00:00:40.879 --> 00:00:43.590
was parro [laughter]
00:00:43.600 --> 00:00:45.830
didn't you know it um it's a database
00:00:45.840 --> 00:00:48.549
and it's just like yeah um
00:00:48.559 --> 00:00:49.990
>> yeah those are some of the things on on
00:00:50.000 --> 00:00:54.389
top of mind I mean obviously AI AI AI um
00:00:54.399 --> 00:00:56.470
because we all got to talk about it
00:00:56.480 --> 00:00:58.790
>> uh but I would say
00:00:58.800 --> 00:01:00.790
uh it you know the other thing that's
00:01:00.800 --> 00:01:02.790
also top of mind now is like how fast
00:01:02.800 --> 00:01:04.630
all these like frontier models are
00:01:04.640 --> 00:01:09.590
converging like so fast and also I the
00:01:09.600 --> 00:01:13.109
cost is spiking so fast
00:01:13.119 --> 00:01:15.270
like it's I I was like experimenting
00:01:15.280 --> 00:01:20.550
with like um Mark Freeman's um Petri I
00:01:20.560 --> 00:01:23.190
had to shut it down for a minute tokens
00:01:23.200 --> 00:01:25.109
were blowing up in cost and I was like
00:01:25.119 --> 00:01:27.590
oh my god this is this is like real real
00:01:27.600 --> 00:01:30.710
cash is gonna cost me now.
00:01:30.720 --> 00:01:32.870
>> Yeah, it's it's moving fast, man. Um
00:01:32.880 --> 00:01:34.870
yeah, shout out to Jeff Long and Nick Z
00:01:34.880 --> 00:01:36.069
joined us. So, if you got anything you
00:01:36.079 --> 00:01:37.990
want to ask or want us to chat about,
00:01:38.000 --> 00:01:39.590
let us know in the chat. If you don't,
00:01:39.600 --> 00:01:42.630
we'll just carry on. Um yeah, it's it's
00:01:42.640 --> 00:01:44.630
moving fast and I've been kind of spent
00:01:44.640 --> 00:01:46.230
the whole month in build mode, right?
00:01:46.240 --> 00:01:48.069
So, as I said on LinkedIn, I've spent
00:01:48.079 --> 00:01:49.590
bugger all time researching and I've
00:01:49.600 --> 00:01:51.109
seen these these things come out, you
00:01:51.119 --> 00:01:54.230
know, like the orchestra uh getting
00:01:54.240 --> 00:01:56.310
funding, right? So, you know, congrats
00:01:56.320 --> 00:01:58.469
to that team. But I've got big questions
00:01:58.479 --> 00:02:01.109
about where do orchestration live these
00:02:01.119 --> 00:02:03.670
days and then data bricks with their
00:02:03.680 --> 00:02:05.749
accelerators, right? And I'm like, oh,
00:02:05.759 --> 00:02:07.749
are we cookie cuttering organizations
00:02:07.759 --> 00:02:11.510
again or we not? Um, yeah. And then DBT
00:02:11.520 --> 00:02:14.470
Labs, right? Like, oh my god, you know,
00:02:14.480 --> 00:02:17.430
and then Patchy Aussie, right? The OSI
00:02:17.440 --> 00:02:19.830
stuff's now actually under Apache, so in
00:02:19.840 --> 00:02:22.309
theory it can become a standard, but
00:02:22.319 --> 00:02:24.790
will it? And then uh Google just came
00:02:24.800 --> 00:02:26.390
out with BigQuery graph, right, which
00:02:26.400 --> 00:02:28.229
I've been playing around for measurement
00:02:28.239 --> 00:02:31.270
calculations. So it's a quasi metric
00:02:31.280 --> 00:02:33.190
layer and I'm like, oh, what does that
00:02:33.200 --> 00:02:35.670
actually mean to me and our platform? So
00:02:35.680 --> 00:02:38.229
yeah, you but you tend to keep up a lot
00:02:38.239 --> 00:02:41.509
more uh sometimes what's going on in the
00:02:41.519 --> 00:02:43.190
market. So let's go through some of
00:02:43.200 --> 00:02:45.750
those because I'm kind of it's kind of
00:02:45.760 --> 00:02:47.030
free research for me, right? I don't
00:02:47.040 --> 00:02:48.630
even need to perplexity. I just asked
00:02:48.640 --> 00:02:49.910
Tim on on
00:02:49.920 --> 00:02:51.190
>> Yeah, I mean that's fair. [laughter]
00:02:51.200 --> 00:02:53.350
This month the kids out of school and
00:02:53.360 --> 00:02:55.670
you know I have a as small fle I'm not
00:02:55.680 --> 00:02:58.550
as as on it as I would like typically. I
00:02:58.560 --> 00:03:00.470
know what are you going to do? Uh I do
00:03:00.480 --> 00:03:02.390
you know and I typically do write like a
00:03:02.400 --> 00:03:03.910
like a weekly but I took a two week
00:03:03.920 --> 00:03:05.670
pause just like chaos [clears throat]
00:03:05.680 --> 00:03:07.910
sessions but you know I still feel
00:03:07.920 --> 00:03:09.430
pretty close to everything that's going
00:03:09.440 --> 00:03:12.309
on right now. Um yeah I mean a lot of
00:03:12.319 --> 00:03:13.830
fundraising. I mean shout out to
00:03:13.840 --> 00:03:15.830
orchestra. I'll be honest I am like
00:03:15.840 --> 00:03:17.270
pretty biased towards them. I am an
00:03:17.280 --> 00:03:19.430
investor just to publicly state that so
00:03:19.440 --> 00:03:21.589
nobody thinks I'm inflating my bias
00:03:21.599 --> 00:03:24.149
towards them Hugo and team I very much
00:03:24.159 --> 00:03:26.710
strongly believe in their kind of play I
00:03:26.720 --> 00:03:30.070
love tools like um Airflow and there's
00:03:30.080 --> 00:03:32.789
these quasi open source ones like Kestra
00:03:32.799 --> 00:03:35.270
you know it's like
00:03:35.280 --> 00:03:36.710
that are all like really interesting in
00:03:36.720 --> 00:03:39.190
different plays um yeah I mean I've been
00:03:39.200 --> 00:03:42.550
using that their platform well over over
00:03:42.560 --> 00:03:44.789
a year and a bit now and yeah I'm pretty
00:03:44.799 --> 00:03:47.030
I'm very pleased with kind of like the
00:03:47.040 --> 00:03:50.630
velocity these tools are showing. Um
00:03:50.640 --> 00:03:52.149
it's also interesting in the play like
00:03:52.159 --> 00:03:54.149
orchestration inside of the platforms is
00:03:54.159 --> 00:03:56.949
starting to show up like like DBT is
00:03:56.959 --> 00:03:58.869
kind of starting to lean towards it but
00:03:58.879 --> 00:04:01.030
not not it's like one foot in the pool
00:04:01.040 --> 00:04:03.270
not in kind of thing. So, it's kind of
00:04:03.280 --> 00:04:05.509
like always been this like super awkward
00:04:05.519 --> 00:04:08.070
DBT cloud play, but I'm starting to see
00:04:08.080 --> 00:04:11.750
it like like morph into Fiverr
00:04:11.760 --> 00:04:14.550
essentially, right? Um, and where Fiverr
00:04:14.560 --> 00:04:16.789
and DBT cloud like I would not be
00:04:16.799 --> 00:04:18.310
surprised in in the next like couple
00:04:18.320 --> 00:04:20.310
weeks at DBT like that you start to see
00:04:20.320 --> 00:04:24.150
it more or less become like one surface.
00:04:24.160 --> 00:04:27.350
>> Oh, well, got to be the goal, right? Um,
00:04:27.360 --> 00:04:29.430
and and maybe one billing plan. No,
00:04:29.440 --> 00:04:30.310
probably not, right?
00:04:30.320 --> 00:04:32.469
>> Yeah. I mean it it's it's going to get
00:04:32.479 --> 00:04:34.310
close, right? Like and and you know with
00:04:34.320 --> 00:04:36.469
Census and Fiverr and all these like
00:04:36.479 --> 00:04:38.390
they're all kind of like conglomerating
00:04:38.400 --> 00:04:39.749
one. I mean I'm pretty big fan of
00:04:39.759 --> 00:04:42.070
Fiverr. I think they've done a a pretty
00:04:42.080 --> 00:04:45.110
pretty good product over time. Um you
00:04:45.120 --> 00:04:48.790
know I use competitors like Estuary. Um
00:04:48.800 --> 00:04:50.390
you know they all have pros and cons and
00:04:50.400 --> 00:04:51.909
changes right it's the same thing as
00:04:51.919 --> 00:04:54.310
like using open source tools like DT you
00:04:54.320 --> 00:04:56.790
know Python. You and I are old enough
00:04:56.800 --> 00:04:58.550
like have been using like the Singer
00:04:58.560 --> 00:05:00.230
framework. I actually know one of my
00:05:00.240 --> 00:05:02.070
friends is one of the original engineers
00:05:02.080 --> 00:05:03.510
on that on that standard
00:05:03.520 --> 00:05:05.350
>> on on Singer with Singer Tech.
00:05:05.360 --> 00:05:06.870
>> Yeah. Yeah. He worked he worked for
00:05:06.880 --> 00:05:07.909
them.
00:05:07.919 --> 00:05:10.790
>> We um Yeah. We we started we we kind of
00:05:10.800 --> 00:05:13.189
started off using Milano, right? Uh
00:05:13.199 --> 00:05:15.430
because Singer had been bought, so we
00:05:15.440 --> 00:05:17.110
kind of thought it was dead. And then
00:05:17.120 --> 00:05:18.390
right at the beginning, I think was it
00:05:18.400 --> 00:05:20.550
Milano versus Airflow back then?
00:05:20.560 --> 00:05:22.230
>> We went down the we went down the Milano
00:05:22.240 --> 00:05:23.590
line because we didn't Well, in
00:05:23.600 --> 00:05:25.029
hindsight, it was okay because we didn't
00:05:25.039 --> 00:05:27.749
need orchestration. Um, but you know,
00:05:27.759 --> 00:05:29.670
kind of probably wasn't the best choice
00:05:29.680 --> 00:05:32.550
given Milano kind of died a bit. I mean,
00:05:32.560 --> 00:05:35.430
I know Aaron's bought it and and uh is
00:05:35.440 --> 00:05:37.350
reinventing it. So, shout out to Aaron
00:05:37.360 --> 00:05:40.469
here in in uh the UK. Um, and he's a
00:05:40.479 --> 00:05:43.029
Kiwi, so good on him. Um,
00:05:43.039 --> 00:05:44.390
but yeah, I I don't know. I've always
00:05:44.400 --> 00:05:46.230
had a thing with uh open source and
00:05:46.240 --> 00:05:49.350
things like WT Labs uh and DBT, right?
00:05:49.360 --> 00:05:51.350
I'm like, ah. Uh but yeah know we we use
00:05:51.360 --> 00:05:54.070
Milano and we still do like uh we had to
00:05:54.080 --> 00:05:55.749
rewrite it a little bit because we want
00:05:55.759 --> 00:05:57.350
to run it serverless and back then you
00:05:57.360 --> 00:05:59.430
had to have a persistent server. Yeah.
00:05:59.440 --> 00:06:01.590
>> So Nigel did his magic but no we you
00:06:01.600 --> 00:06:03.590
know we actually uh we just did a whole
00:06:03.600 --> 00:06:06.150
lot of work on using it to do uh data
00:06:06.160 --> 00:06:09.270
collection off YouTube. Um, and so yeah,
00:06:09.280 --> 00:06:11.270
so it's still a I don't know how much of
00:06:11.280 --> 00:06:13.590
the code we run is still a fork of the
00:06:13.600 --> 00:06:16.070
original multano or not, but but the
00:06:16.080 --> 00:06:17.749
kind of y
00:06:17.759 --> 00:06:19.270
>> no I was going to say yeah I mean with
00:06:19.280 --> 00:06:20.950
the AI tooling now with like clear
00:06:20.960 --> 00:06:22.790
patterns you're like ah you know before
00:06:22.800 --> 00:06:24.550
I would always avoid doing this tap
00:06:24.560 --> 00:06:26.550
target kind of thing or like grabbing
00:06:26.560 --> 00:06:27.990
data but you're like now I have a strong
00:06:28.000 --> 00:06:31.110
pattern like uh cool. Oh, and actually
00:06:31.120 --> 00:06:33.189
if you're vibe coding, I don't even know
00:06:33.199 --> 00:06:35.590
where the the source of the code pattern
00:06:35.600 --> 00:06:37.029
comes from, right? Is it, you know, if
00:06:37.039 --> 00:06:38.309
I'm asking it to go and build a
00:06:38.319 --> 00:06:41.830
collector um from scratch, not using our
00:06:41.840 --> 00:06:43.909
framework, which is any framework, is it
00:06:43.919 --> 00:06:45.990
using airflow? Is it using mult is it
00:06:46.000 --> 00:06:48.950
using like where where's the actual IP
00:06:48.960 --> 00:06:51.590
coming from? Um, but the orchestra
00:06:51.600 --> 00:06:52.870
thing, right? Like I was kind of sitting
00:06:52.880 --> 00:06:56.309
back and thinking about it and
00:06:56.319 --> 00:06:57.510
I'm kind of interested in your view
00:06:57.520 --> 00:06:59.189
architecturally on this, right? So my my
00:06:59.199 --> 00:07:00.469
thought process went something like
00:07:00.479 --> 00:07:04.790
this. Uh orchestras for ages have been
00:07:04.800 --> 00:07:07.510
uh dag glue, right? You know, nodes and
00:07:07.520 --> 00:07:10.790
links put together, bundled up and then
00:07:10.800 --> 00:07:13.029
kind of scheduled, right? Uh and then we
00:07:13.039 --> 00:07:14.870
got I think it was Informatica and a
00:07:14.880 --> 00:07:15.830
whole other ones. We started getting
00:07:15.840 --> 00:07:17.670
trigger based ones with a little bit
00:07:17.680 --> 00:07:19.670
more granularity rather than just big
00:07:19.680 --> 00:07:22.629
blobs of DAGs. Uh we in our platform we
00:07:22.639 --> 00:07:24.629
do it slightly differently. uh we hold
00:07:24.639 --> 00:07:26.790
the context of the relationships, the
00:07:26.800 --> 00:07:29.029
nodes and links in our context plane and
00:07:29.039 --> 00:07:31.510
then every time new data turns in or we
00:07:31.520 --> 00:07:33.830
we have a schedule trigger. We'll read
00:07:33.840 --> 00:07:36.230
that context, we'll create a manifest of
00:07:36.240 --> 00:07:38.390
the steps, you know, nodes and links, a
00:07:38.400 --> 00:07:41.029
big tag effectively. We'll execute it
00:07:41.039 --> 00:07:43.749
and then we blow it away. So the
00:07:43.759 --> 00:07:47.029
schedule is not actually the the pet,
00:07:47.039 --> 00:07:49.909
it's cattle. The context around the the
00:07:49.919 --> 00:07:51.749
dependencies, right, is what we care
00:07:51.759 --> 00:07:54.309
about. Um, but that's still a very batch
00:07:54.319 --> 00:07:56.150
orientated, right? Like it's a micro
00:07:56.160 --> 00:07:58.390
batch and because it's Big Query, we can
00:07:58.400 --> 00:08:00.550
grab that manifest and run a ton of
00:08:00.560 --> 00:08:02.309
a manifest in a very small amount of
00:08:02.319 --> 00:08:05.110
time on BigQuery and then pay for it.
00:08:05.120 --> 00:08:07.110
But it's still bundling a boundary
00:08:07.120 --> 00:08:08.869
around a series of nodes and links,
00:08:08.879 --> 00:08:11.830
right, for for that orchestration pan.
00:08:11.840 --> 00:08:13.350
And I keep coming back to the way I'm
00:08:13.360 --> 00:08:16.629
using LLMs and agents now. uh and I'm
00:08:16.639 --> 00:08:19.029
like actually is a mesh architecture
00:08:19.039 --> 00:08:21.909
where I really need to move to because
00:08:21.919 --> 00:08:25.350
if I say that each node is let's call it
00:08:25.360 --> 00:08:26.629
an agent for now right it's an
00:08:26.639 --> 00:08:29.189
encapsulation of some context and some
00:08:29.199 --> 00:08:31.029
execution
00:08:31.039 --> 00:08:33.750
and then I can say that each of those
00:08:33.760 --> 00:08:36.070
those agents have a relationship with
00:08:36.080 --> 00:08:37.750
another agent it has a dependency and I
00:08:37.760 --> 00:08:40.550
use data contracts for that do I need an
00:08:40.560 --> 00:08:42.550
orchestra because now what I've got an
00:08:42.560 --> 00:08:45.590
orchestrator uh I've got a bunch of
00:08:45.600 --> 00:08:47.670
nodes that are separate that understand
00:08:47.680 --> 00:08:50.550
who they rely on and who relies on them.
00:08:50.560 --> 00:08:51.990
They understand when to trigger
00:08:52.000 --> 00:08:53.430
themselves. They understand what their
00:08:53.440 --> 00:08:56.870
job is. Like I don't need a manifest
00:08:56.880 --> 00:08:58.389
because they can just bloody fire
00:08:58.399 --> 00:09:01.430
whenever they need to. Chaos to try and
00:09:01.440 --> 00:09:02.949
understand how they're running and
00:09:02.959 --> 00:09:04.470
troubleshoot when something goes wrong,
00:09:04.480 --> 00:09:05.670
right? But that's kind of where my
00:09:05.680 --> 00:09:07.030
head's at the moment. So what what's
00:09:07.040 --> 00:09:09.430
your view? Orchestration still a key
00:09:09.440 --> 00:09:10.310
thing?
00:09:10.320 --> 00:09:12.161
>> Yeah.
00:09:12.171 --> 00:09:15.110
[laughter]
00:09:15.120 --> 00:09:17.190
a little bit more detail, please.
00:09:17.200 --> 00:09:21.990
>> Yeah. Uh well, I think it's like it's um
00:09:22.000 --> 00:09:23.350
I think you're on to something. I think
00:09:23.360 --> 00:09:24.710
you're seeing patterns in the industry
00:09:24.720 --> 00:09:25.930
about like how all these things
00:09:25.940 --> 00:09:27.030
[clears throat] are kind of like meshing
00:09:27.040 --> 00:09:29.670
together. I think orchestration I mean
00:09:29.680 --> 00:09:31.350
at the end of the day like you know good
00:09:31.360 --> 00:09:35.350
old DAG uh is still it's something that
00:09:35.360 --> 00:09:37.269
it's like under under the hood. Is it
00:09:37.279 --> 00:09:41.110
going to be a completely separate tool?
00:09:41.120 --> 00:09:43.990
Um, I mean, even looking at orchestra or
00:09:44.000 --> 00:09:45.590
Castra and like everybody how they're
00:09:45.600 --> 00:09:47.910
like morphing and changing. No, I don't
00:09:47.920 --> 00:09:49.269
think I don't think so. I think there
00:09:49.279 --> 00:09:51.110
are actually times where you should you
00:09:51.120 --> 00:09:53.110
do want it to be completely separate.
00:09:53.120 --> 00:09:55.269
Usually those are like when you want
00:09:55.279 --> 00:09:57.509
really isolated, really specific, high
00:09:57.519 --> 00:09:59.910
reliable tools and I think that's like
00:09:59.920 --> 00:10:03.190
really important. um like dead simple
00:10:03.200 --> 00:10:05.030
orchestration.
00:10:05.040 --> 00:10:06.470
But I mean ultimately at the end of the
00:10:06.480 --> 00:10:10.630
day in a vast majority of data work is
00:10:10.640 --> 00:10:13.350
like does this deliver value right? Do I
00:10:13.360 --> 00:10:15.269
need to have this completely separate
00:10:15.279 --> 00:10:19.269
airflow tool to move all these things or
00:10:19.279 --> 00:10:21.750
does snowflakes orchestration or
00:10:21.760 --> 00:10:24.389
ingestion patterns in mother duck solve
00:10:24.399 --> 00:10:27.030
my problem? Right? Like vast majority of
00:10:27.040 --> 00:10:29.750
the time it's yes. It's like you're kind
00:10:29.760 --> 00:10:35.030
of like over overindexing on toys. Um,
00:10:35.040 --> 00:10:36.870
and then not really delivering value.
00:10:36.880 --> 00:10:37.990
And we talked about this all the time
00:10:38.000 --> 00:10:39.430
really between you and I. We're like, do
00:10:39.440 --> 00:10:41.750
you need another tool? Um,
00:10:41.760 --> 00:10:43.670
>> yeah. And and yeah, does your data
00:10:43.680 --> 00:10:45.190
platform become the harness for your
00:10:45.200 --> 00:10:49.110
data, right, with GI tooling? Is that is
00:10:49.120 --> 00:10:50.870
that something where the bang for buck
00:10:50.880 --> 00:10:52.230
you're going to get out of them with the
00:10:52.240 --> 00:10:53.910
way they experiment and the way they do
00:10:53.920 --> 00:10:55.829
integration? Uh, is that where the value
00:10:55.839 --> 00:10:57.670
is? I mean, I I was sitting on a webinar
00:10:57.680 --> 00:11:00.790
the other day and uh it was an I it was
00:11:00.800 --> 00:11:02.550
an IBM thing. Well, no, it was somebody
00:11:02.560 --> 00:11:05.430
had an IBM person on there and they were
00:11:05.440 --> 00:11:06.870
talking about the value of the IBM
00:11:06.880 --> 00:11:08.630
platform and it was integration and
00:11:08.640 --> 00:11:10.710
yeah, I was in a snarky mix snarky mood
00:11:10.720 --> 00:11:13.670
and but it was still factual, right? And
00:11:13.680 --> 00:11:15.829
my view was you know decades ago I was
00:11:15.839 --> 00:11:17.590
around the time of Oracle IBM
00:11:17.600 --> 00:11:19.190
Informatica
00:11:19.200 --> 00:11:21.350
and we because I worked for one of them
00:11:21.360 --> 00:11:23.670
sold the about oh all our tools
00:11:23.680 --> 00:11:25.670
are integrated and then I watch you know
00:11:25.680 --> 00:11:27.990
Oracle by Peopleoft and not integrated
00:11:28.000 --> 00:11:31.110
into Oracle apps and by Hyperion was it
00:11:31.120 --> 00:11:32.470
and not really you know different
00:11:32.480 --> 00:11:34.470
technology stack cobbled together not
00:11:34.480 --> 00:11:36.790
fully integrated and I'm like
00:11:36.800 --> 00:11:39.829
on the integration story but now when we
00:11:39.839 --> 00:11:41.350
look at things like snowflake and data
00:11:41.360 --> 00:11:43.350
bricks and and I'm going to ignore
00:11:43.360 --> 00:11:45.670
fabric, right? Um
00:11:45.680 --> 00:11:49.269
maybe like are they integrated or is it
00:11:49.279 --> 00:11:51.430
still a veneer of integration over the
00:11:51.440 --> 00:11:53.750
top?
00:11:53.760 --> 00:11:56.150
>> Are they integrated? No. No. I mean, yes
00:11:56.160 --> 00:11:58.710
and no. Like it's so funny like I mean I
00:11:58.720 --> 00:12:00.150
think this uh we had a conversation
00:12:00.160 --> 00:12:02.150
about that. is like the sizzle reel
00:12:02.160 --> 00:12:03.829
problem where you see the YouTube video
00:12:03.839 --> 00:12:05.910
this awesome thing on LinkedIn and then
00:12:05.920 --> 00:12:07.670
you go to actually use the thing and
00:12:07.680 --> 00:12:09.670
then you're just like I mean I guess
00:12:09.680 --> 00:12:13.590
like directionally yeah it connects but
00:12:13.600 --> 00:12:15.269
then you actually get into like the
00:12:15.279 --> 00:12:17.110
actual integrations or their so-called
00:12:17.120 --> 00:12:19.269
like open support or open protocol
00:12:19.279 --> 00:12:21.990
supports and devil's in the details and
00:12:22.000 --> 00:12:23.350
that's what you usually end up spending
00:12:23.360 --> 00:12:26.470
like vast majority of your time is
00:12:26.480 --> 00:12:29.350
fixing that stuff right um yeah there
00:12:29.360 --> 00:12:31.110
are plenty of times that you know I've
00:12:31.120 --> 00:12:32.550
used like Snowflake I've done a little
00:12:32.560 --> 00:12:35.509
bit on data bricks um
00:12:35.519 --> 00:12:37.350
and yeah I mean they'll say this and
00:12:37.360 --> 00:12:38.870
then like you'll look go down it's
00:12:38.880 --> 00:12:40.949
almost like this asterisk
00:12:40.959 --> 00:12:43.030
and you scroll down and you're like oh
00:12:43.040 --> 00:12:46.230
okay it's not it's not quite supported
00:12:46.240 --> 00:12:48.550
and there's no like ETA of like when
00:12:48.560 --> 00:12:49.910
it's going to be fully supported just
00:12:49.920 --> 00:12:52.629
says coming soon.
00:12:52.639 --> 00:12:54.550
Yeah, I mean we we had that with Google
00:12:54.560 --> 00:12:57.990
um graph, right? Uh because it looked
00:12:58.000 --> 00:13:00.470
like it helped us uh build out our
00:13:00.480 --> 00:13:03.350
semantic BI semantic layer or metrics
00:13:03.360 --> 00:13:04.949
layer, right? In a more robust way than
00:13:04.959 --> 00:13:09.509
we had. Um so you know it was uh early
00:13:09.519 --> 00:13:12.550
adopter or it was prega and so yeah we
00:13:12.560 --> 00:13:14.389
jump in there and I start creating
00:13:14.399 --> 00:13:16.629
metrics and it only let me create
00:13:16.639 --> 00:13:19.269
measures and yeah I use a a language of
00:13:19.279 --> 00:13:21.590
facts for me uh immutable numbers that
00:13:21.600 --> 00:13:24.550
are in the source system you know uh you
00:13:24.560 --> 00:13:27.030
account a number that's been created
00:13:27.040 --> 00:13:29.750
that that's facts uh for me a measure is
00:13:29.760 --> 00:13:33.190
is uh an aggregation of sum account that
00:13:33.200 --> 00:13:35.190
kind of stuff and a metric is where I
00:13:35.200 --> 00:13:37.269
get a complex formula this over this or
00:13:37.279 --> 00:13:39.750
this divided by this or window on this
00:13:39.760 --> 00:13:42.069
thing divided by a window on this thing
00:13:42.079 --> 00:13:45.030
minusing max on this thing right those
00:13:45.040 --> 00:13:47.670
complex calculations so yeah I go in
00:13:47.680 --> 00:13:50.470
there uh pull through a fact yeah can
00:13:50.480 --> 00:13:53.590
see it in my in my tools I go and do a a
00:13:53.600 --> 00:13:55.910
measure so I go to a sum yeah she works
00:13:55.920 --> 00:13:58.069
pretty good I'm impressed and then I go
00:13:58.079 --> 00:14:00.550
to do a metric and it goes no [laughter]
00:14:00.560 --> 00:14:02.949
but you can't do that now now it's
00:14:02.959 --> 00:14:05.269
coming out very soon uh as a new feature
00:14:05.279 --> 00:14:07.269
But then I automatically go okay but now
00:14:07.279 --> 00:14:09.350
I want some filters you know effectively
00:14:09.360 --> 00:14:12.310
wear clauses dynamic I want windowing
00:14:12.320 --> 00:14:14.790
functions and then I go well it's not
00:14:14.800 --> 00:14:17.030
that it's not integrated because it is
00:14:17.040 --> 00:14:18.550
it's just that it doesn't have feature
00:14:18.560 --> 00:14:20.470
parity of what I actually need right
00:14:20.480 --> 00:14:22.710
now. So how much of that do you reckon
00:14:22.720 --> 00:14:24.629
is with the big platform plays right
00:14:24.639 --> 00:14:27.430
it's it's integrated but just doesn't
00:14:27.440 --> 00:14:30.310
have feature parody uh with the best of
00:14:30.320 --> 00:14:34.710
breed of what you actually need.
00:14:34.720 --> 00:14:36.629
>> I mean
00:14:36.639 --> 00:14:39.010
oh this makes me feel so old saying this
00:14:39.020 --> 00:14:39.750
[laughter]
00:14:39.760 --> 00:14:42.629
makes me it it depends on the business.
00:14:42.639 --> 00:14:45.030
I know it's such a copout. Um but no
00:14:45.040 --> 00:14:51.590
realistically like uh
00:14:51.600 --> 00:14:53.350
you know I mean I would say very rarely
00:14:53.360 --> 00:14:55.509
do you need the bre best of breed and of
00:14:55.519 --> 00:14:57.430
course like you know me and like how
00:14:57.440 --> 00:14:59.750
hard I have a hard time not choosing the
00:14:59.760 --> 00:15:03.430
best of breed. Uh like chronically
00:15:03.440 --> 00:15:05.750
chronic chronic chronically on the edge
00:15:05.760 --> 00:15:08.310
of choosing platforms that are sometimes
00:15:08.320 --> 00:15:12.470
maybe a little too young. Um but then
00:15:12.480 --> 00:15:13.590
like you know sometimes I'll meet the
00:15:13.600 --> 00:15:15.189
founders and I'll be like yeah I
00:15:15.199 --> 00:15:18.870
actually think you guys um are women uh
00:15:18.880 --> 00:15:21.269
can uh pull together and I think that
00:15:21.279 --> 00:15:23.269
you I think there's enough you know one
00:15:23.279 --> 00:15:26.470
of you know
00:15:26.480 --> 00:15:29.189
circular answer is
00:15:29.199 --> 00:15:31.829
um
00:15:31.839 --> 00:15:33.350
yeah I mean it just really depends on
00:15:33.360 --> 00:15:34.470
the business like if you're like a
00:15:34.480 --> 00:15:36.629
certain size and you you can take that
00:15:36.639 --> 00:15:38.550
take that bet and really like swing for
00:15:38.560 --> 00:15:40.470
it absolutely you should do the best
00:15:40.480 --> 00:15:42.230
contrary, I think you should do the best
00:15:42.240 --> 00:15:43.910
of breed, right? and really trying to
00:15:43.920 --> 00:15:45.670
find the angle to really find the
00:15:45.680 --> 00:15:47.590
business because like right now with AI
00:15:47.600 --> 00:15:49.509
flattening so many processes and things
00:15:49.519 --> 00:15:52.069
like the money is in the like the little
00:15:52.079 --> 00:15:53.350
the the edges that you're going to get
00:15:53.360 --> 00:15:56.949
the 1% the 2% the 3% right um if
00:15:56.959 --> 00:15:58.550
everybody's building their the world's
00:15:58.560 --> 00:16:01.110
best cloudbot or grockbot these days um
00:16:01.120 --> 00:16:03.030
to optimize the business like what is
00:16:03.040 --> 00:16:05.509
your edge right if you can literally if
00:16:05.519 --> 00:16:07.749
everyone can just go on a you know repo
00:16:07.759 --> 00:16:09.829
and grab the you know the single best
00:16:09.839 --> 00:16:12.710
book written on operating procedures
00:16:12.720 --> 00:16:14.550
Just consider everybody else has that
00:16:14.560 --> 00:16:16.870
exact same book. So what is your actual
00:16:16.880 --> 00:16:18.870
edge on all this stuff? Could it be
00:16:18.880 --> 00:16:20.790
tools? Yeah, I can say it could be it
00:16:20.800 --> 00:16:23.189
could be tools. Um honestly I think it's
00:16:23.199 --> 00:16:24.629
just going to be the ability to make you
00:16:24.639 --> 00:16:27.030
know fast, you know, decisions like take
00:16:27.040 --> 00:16:31.110
an action next step. Next step. Um
00:16:31.120 --> 00:16:33.030
so yeah, you know, sometimes I think you
00:16:33.040 --> 00:16:35.350
know, you know, best of breed can help.
00:16:35.360 --> 00:16:37.509
Um
00:16:37.519 --> 00:16:40.710
I don't know. What do you think? Uh so I
00:16:40.720 --> 00:16:42.389
think what you got to do is you got to
00:16:42.399 --> 00:16:43.749
you got to write down a set of core
00:16:43.759 --> 00:16:46.470
principles, right?
00:16:46.480 --> 00:16:49.189
>> And they're this over that. Um and
00:16:49.199 --> 00:16:50.389
actually, you know, really interesting
00:16:50.399 --> 00:16:52.389
example over the last month or so, which
00:16:52.399 --> 00:16:54.629
is why we're really busy building. So we
00:16:54.639 --> 00:16:58.230
have a a core principle uh which is uh
00:16:58.240 --> 00:17:00.710
for for our stuff that if there's a
00:17:00.720 --> 00:17:02.949
serverless Google option, we'll do it,
00:17:02.959 --> 00:17:04.710
right? It's like we're on the Google
00:17:04.720 --> 00:17:06.150
platform. That's the only thing we run
00:17:06.160 --> 00:17:08.470
on. we want to leverage that puppy as
00:17:08.480 --> 00:17:10.150
much as possible right so if there's a
00:17:10.160 --> 00:17:11.909
serverless thing and it's part of the
00:17:11.919 --> 00:17:16.549
GCP uh we adopt it and then uh if
00:17:16.559 --> 00:17:18.150
there's not now we get a bit tricky
00:17:18.160 --> 00:17:21.909
right uh so we had to on board a whole
00:17:21.919 --> 00:17:24.069
lot of publishers uh with social media
00:17:24.079 --> 00:17:25.750
data right we had to start collecting
00:17:25.760 --> 00:17:27.510
their YouTube Facebook all that kind of
00:17:27.520 --> 00:17:30.470
stuff but you know we've got uh 100 to
00:17:30.480 --> 00:17:32.789
200 external organizations that we have
00:17:32.799 --> 00:17:35.350
to go and get the data from and if we
00:17:35.360 --> 00:17:37.510
look at YouTube there's a out of the box
00:17:37.520 --> 00:17:39.750
Google uh data transfer service for
00:17:39.760 --> 00:17:42.710
YouTube but the onboarding process for
00:17:42.720 --> 00:17:45.669
that is horrific right like just the the
00:17:45.679 --> 00:17:48.070
way that that person has to uh create
00:17:48.080 --> 00:17:49.669
their share their credentials
00:17:49.679 --> 00:17:51.909
effectively you know tokenization it's
00:17:51.919 --> 00:17:53.430
just a complete nightmare
00:17:53.440 --> 00:17:53.909
>> right
00:17:53.919 --> 00:17:55.830
>> um and so we're like okay we can't use
00:17:55.840 --> 00:17:57.270
that right because even though under the
00:17:57.280 --> 00:17:59.830
covers it does exactly what we want uh
00:17:59.840 --> 00:18:02.710
the user experience of these publishers
00:18:02.720 --> 00:18:04.789
uh just means that they they wouldn't on
00:18:04.799 --> 00:18:06.390
board right or we'd have to spend hours
00:18:06.400 --> 00:18:08.870
if not months doing it. So then we go,
00:18:08.880 --> 00:18:10.789
okay, so our first principle can't be
00:18:10.799 --> 00:18:12.789
met. What do we do? And then our second
00:18:12.799 --> 00:18:14.549
principle is we'll just lease something,
00:18:14.559 --> 00:18:17.270
right? So there's a service out there.
00:18:17.280 --> 00:18:19.430
Uh we'll just use it and then we'll
00:18:19.440 --> 00:18:22.230
we'll deal with that um the challenge of
00:18:22.240 --> 00:18:23.590
integrating it. So we're like, okay,
00:18:23.600 --> 00:18:25.190
cool. Well, there's lots of services out
00:18:25.200 --> 00:18:28.150
there that allow people to uh
00:18:28.160 --> 00:18:31.110
effectively shine sign up and uh create
00:18:31.120 --> 00:18:33.029
a token and put the data into that
00:18:33.039 --> 00:18:34.310
platform and we can suck it out. So that
00:18:34.320 --> 00:18:36.710
that's the path we went down to. Uh and
00:18:36.720 --> 00:18:39.270
then we had some problems, right? So uh
00:18:39.280 --> 00:18:40.789
when we tested it, we could get two
00:18:40.799 --> 00:18:43.430
years worth of data. Uh as we're on
00:18:43.440 --> 00:18:44.870
boarding the publishers, that vendor
00:18:44.880 --> 00:18:46.070
decided that they were having
00:18:46.080 --> 00:18:47.909
performance problems and cost problems.
00:18:47.919 --> 00:18:51.350
So they limited it to 30 days.
00:18:51.360 --> 00:18:54.710
>> Oops. Um and then we can only get the
00:18:54.720 --> 00:18:58.230
data back via API, right? they they no
00:18:58.240 --> 00:19:01.750
direct database or or proxy database
00:19:01.760 --> 00:19:04.070
type connection which is all fine. So
00:19:04.080 --> 00:19:06.870
we're good at that. So you know taps on
00:19:06.880 --> 00:19:09.350
our framework and and we're all good. Uh
00:19:09.360 --> 00:19:12.150
until we worked out that actually uh the
00:19:12.160 --> 00:19:13.990
cost went through the roof because we
00:19:14.000 --> 00:19:17.669
had to pay for the API credits and we
00:19:17.679 --> 00:19:19.750
have very little visibility of how those
00:19:19.760 --> 00:19:22.070
credits work but we blew a year's budget
00:19:22.080 --> 00:19:25.669
in a month, right? and we're like,
00:19:25.679 --> 00:19:27.190
"Yeah, we don't want to buy that many."
00:19:27.200 --> 00:19:29.830
Um, so we ended up going, okay, to our
00:19:29.840 --> 00:19:32.310
third principle, which is if a least
00:19:32.320 --> 00:19:33.909
option doesn't work, we build it, which
00:19:33.919 --> 00:19:36.710
is what we've just done. Um, so I I
00:19:36.720 --> 00:19:39.350
liken that to if I was green, if I was
00:19:39.360 --> 00:19:41.350
work, you went in and and was head of
00:19:41.360 --> 00:19:43.029
data, head of data, platform, whatever
00:19:43.039 --> 00:19:44.710
in an organization,
00:19:44.720 --> 00:19:46.470
>> I'd set those core principles up front.
00:19:46.480 --> 00:19:48.390
You know, are we a best of breed or are
00:19:48.400 --> 00:19:50.950
we a platform organization?
00:19:50.960 --> 00:19:53.909
uh you know are we open source or are we
00:19:53.919 --> 00:19:56.070
supported open source commercial or are
00:19:56.080 --> 00:19:59.270
we uh closed source and I would place
00:19:59.280 --> 00:20:01.270
those principles not as an immutable
00:20:01.280 --> 00:20:04.070
policies but as guidance we are open
00:20:04.080 --> 00:20:07.190
source over closed source yeah right
00:20:07.200 --> 00:20:09.110
>> um which means then when you know you go
00:20:09.120 --> 00:20:10.549
snowflake you're like well that's closed
00:20:10.559 --> 00:20:13.029
source yeah but I can put iceberg there
00:20:13.039 --> 00:20:15.590
and that gets me closer to the principle
00:20:15.600 --> 00:20:18.630
of open source over closed source right
00:20:18.640 --> 00:20:20.950
uh it doesn't mean I need to got iceberg
00:20:20.960 --> 00:20:23.510
and apache spark and all that on an
00:20:23.520 --> 00:20:25.830
AWS platform and manage the containers
00:20:25.840 --> 00:20:26.789
myself
00:20:26.799 --> 00:20:28.310
>> because one of my other principles may
00:20:28.320 --> 00:20:31.110
be uh data ops or you know small teams
00:20:31.120 --> 00:20:32.549
like there's a bunch of other competing
00:20:32.559 --> 00:20:34.070
principles and that's the key. So that's
00:20:34.080 --> 00:20:36.470
what I'd do and I'd be very clear where
00:20:36.480 --> 00:20:38.549
on that slider I'm I'm moving, right?
00:20:38.559 --> 00:20:40.710
When I make a decision, which one did I
00:20:40.720 --> 00:20:43.110
pick and why? And then the last one is
00:20:43.120 --> 00:20:44.789
be willing to change. And that's the
00:20:44.799 --> 00:20:46.789
hard one for big corporates, right? Is
00:20:46.799 --> 00:20:49.744
say, "Hey, we tried that. It didn't work
00:20:49.754 --> 00:20:51.110
[laughter] for a whole bunch of reasons
00:20:51.120 --> 00:20:52.470
and now we're going to go do something
00:20:52.480 --> 00:20:55.510
different, but that's okay." Um, so as a
00:20:55.520 --> 00:20:57.909
two person startup, it's okay, right?
00:20:57.919 --> 00:21:00.950
Still gutted, but it's okay. Um, if I
00:21:00.960 --> 00:21:03.110
was in a 5,000 person company and my
00:21:03.120 --> 00:21:04.470
career was based on making that
00:21:04.480 --> 00:21:06.070
decision, yeah, it probably wasn't okay
00:21:06.080 --> 00:21:07.990
and I'd be paying that company a
00:21:08.000 --> 00:21:10.149
shitload of money for API credits.
00:21:10.159 --> 00:21:10.789
>> Yeah.
00:21:10.799 --> 00:21:12.630
>> Yeah. That's my view.
00:21:12.640 --> 00:21:14.470
>> I mean, it's it's good. It's why where
00:21:14.480 --> 00:21:16.070
we have like these like great
00:21:16.080 --> 00:21:18.390
conversations and we've worked together.
00:21:18.400 --> 00:21:21.350
Um, it is like a it it's so it's so
00:21:21.360 --> 00:21:23.750
funny like uh
00:21:23.760 --> 00:21:25.990
you have to make trade-offs, right? And
00:21:26.000 --> 00:21:27.669
I think most time when we talk you're
00:21:27.679 --> 00:21:29.430
just like well just make the trade-offs
00:21:29.440 --> 00:21:30.950
like visible like what are you training
00:21:30.960 --> 00:21:33.110
what for what right if you want to
00:21:33.120 --> 00:21:34.630
support open source just be clear about
00:21:34.640 --> 00:21:36.390
supporting open source or what the the
00:21:36.400 --> 00:21:37.990
trade-off is and like so where I
00:21:38.000 --> 00:21:39.430
currently work yeah we use iceberg
00:21:39.440 --> 00:21:42.149
because I felt that the
00:21:42.159 --> 00:21:44.149
it's not young it's just like it's hard
00:21:44.159 --> 00:21:47.190
for small data teams to actually operate
00:21:47.200 --> 00:21:50.470
on but like a year ago I was just like I
00:21:50.480 --> 00:21:52.950
know you know duck lake per I know all
00:21:52.960 --> 00:21:54.470
these things are coming I see this
00:21:54.480 --> 00:21:55.270
happening
00:21:55.280 --> 00:21:57.190
and there's delta and it has I think at
00:21:57.200 --> 00:21:58.950
time more superior piece but I was like
00:21:58.960 --> 00:22:00.390
it's very clear to me there's a signal
00:22:00.400 --> 00:22:02.710
is converged on iceberg and maybe I'm
00:22:02.720 --> 00:22:05.830
feeling a little risky on this but it's
00:22:05.840 --> 00:22:08.470
not that risky so let's converge on this
00:22:08.480 --> 00:22:10.789
open format and I think it's going to
00:22:10.799 --> 00:22:15.510
pay off um uh for from on on that bet um
00:22:15.520 --> 00:22:18.230
you know snowflake support for it was
00:22:18.240 --> 00:22:21.590
not great at first despite all the docs
00:22:21.600 --> 00:22:23.590
they had but it's you know it's it's
00:22:23.600 --> 00:22:27.430
definitely caught up and has been almost
00:22:27.440 --> 00:22:31.590
seamless um now but a year ago def
00:22:31.600 --> 00:22:33.510
definitely was not seamless it was
00:22:33.520 --> 00:22:34.870
definitely painful
00:22:34.880 --> 00:22:36.230
>> uh but yeah that was that trade-off
00:22:36.240 --> 00:22:37.909
right like I still like and also it's
00:22:37.919 --> 00:22:39.830
like
00:22:39.840 --> 00:22:42.710
>> is juice worth the squeeze vast majority
00:22:42.720 --> 00:22:44.710
if I stand up uh an open source
00:22:44.720 --> 00:22:47.029
analytical piece or like you said spark
00:22:47.039 --> 00:22:50.549
like I've run spark kafka like all those
00:22:50.559 --> 00:22:53.510
toys before um and I know kind of like
00:22:53.520 --> 00:22:55.430
what the venance But you know the
00:22:55.440 --> 00:22:57.669
business today like you know you see
00:22:57.679 --> 00:23:00.789
that you know in foreign employment like
00:23:00.799 --> 00:23:02.950
compared to last year like analyst roles
00:23:02.960 --> 00:23:04.789
like pure analytical roles are down like
00:23:04.799 --> 00:23:08.230
15%. Right. So you can see kind of what
00:23:08.240 --> 00:23:09.990
I actually don't see it that way. I see
00:23:10.000 --> 00:23:11.669
is like a lot of engineering roles like
00:23:11.679 --> 00:23:14.549
condensing. Right. Right. Like a lot of
00:23:14.559 --> 00:23:16.230
people being like I'm just an analyst
00:23:16.240 --> 00:23:17.750
role. I think that's those are
00:23:17.760 --> 00:23:20.390
shrinking. They should uh I think that's
00:23:20.400 --> 00:23:22.149
that's fine because there's a lot of
00:23:22.159 --> 00:23:23.990
capability now. Like I think that really
00:23:24.000 --> 00:23:26.230
good analysts now can just do tons of
00:23:26.240 --> 00:23:28.950
stuff they never could do before uh with
00:23:28.960 --> 00:23:31.270
AI helping them, you know, get that way
00:23:31.280 --> 00:23:33.430
in in the systems, you know, just
00:23:33.440 --> 00:23:35.270
complexity.
00:23:35.280 --> 00:23:37.190
>> Yeah. I think that's a good point is if
00:23:37.200 --> 00:23:38.630
you're the leader of the data part of
00:23:38.640 --> 00:23:40.710
that organization, it's your job to make
00:23:40.720 --> 00:23:42.549
the bets.
00:23:42.559 --> 00:23:44.950
Yeah. And make them visible why you've
00:23:44.960 --> 00:23:46.470
made that bet. But you've got to look
00:23:46.480 --> 00:23:47.510
ahead, right? You got to look at the
00:23:47.520 --> 00:23:49.590
market and go, "Well, I could do this or
00:23:49.600 --> 00:23:50.630
I could do that. Where am I going to
00:23:50.640 --> 00:23:51.990
place my chips?" Right? Because that's
00:23:52.000 --> 00:23:53.110
what you're doing. you're you're
00:23:53.120 --> 00:23:55.029
gambling not on a hunch on yeah there's
00:23:55.039 --> 00:23:57.350
data and everything but you're saying do
00:23:57.360 --> 00:23:59.190
do I think this is where the market's
00:23:59.200 --> 00:24:01.430
going to go like is it safe to make this
00:24:01.440 --> 00:24:04.230
bet now for the next I mean it used to
00:24:04.240 --> 00:24:05.830
be five years but it's not even five
00:24:05.840 --> 00:24:07.669
years now right like things are moving
00:24:07.679 --> 00:24:09.350
so fast you're and that's the other
00:24:09.360 --> 00:24:12.070
thing then you've got to architect your
00:24:12.080 --> 00:24:14.230
your platforms and your way of working
00:24:14.240 --> 00:24:16.710
to be what was it combustible no
00:24:16.720 --> 00:24:18.310
composable what was what was that
00:24:18.320 --> 00:24:19.430
remember those [laughter]
00:24:19.440 --> 00:24:21.909
remember those uh what was it that last
00:24:21.919 --> 00:24:25.750
that reverse ETL Uh I mean the
00:24:25.760 --> 00:24:28.950
the patterns that were doing were were
00:24:28.960 --> 00:24:32.390
useful but the market uh buzzwashing of
00:24:32.400 --> 00:24:34.549
it was Composable combustible
00:24:34.559 --> 00:24:35.029
what was it?
00:24:35.039 --> 00:24:37.031
>> Composable but combustible is probably
00:24:37.041 --> 00:24:42.470
[laughter] what become. Um no I think um
00:24:42.480 --> 00:24:44.230
yeah I mean it's really yeah it's really
00:24:44.240 --> 00:24:45.269
interesting going back to your
00:24:45.279 --> 00:24:47.510
conversation about platform ahead of
00:24:47.520 --> 00:24:49.430
platform. I think it comes in it's like
00:24:49.440 --> 00:24:50.870
core principles like for for the
00:24:50.880 --> 00:24:52.470
business like one of the things I was
00:24:52.480 --> 00:24:56.230
talking to another uh data leader
00:24:56.240 --> 00:24:59.590
um Ahmed yeah Ahmed from activity schema
00:24:59.600 --> 00:25:01.430
actually I was talking to him last week
00:25:01.440 --> 00:25:02.789
uh and we were talking about what he
00:25:02.799 --> 00:25:04.390
sees because obviously he's you know he
00:25:04.400 --> 00:25:05.909
started his own startup which ultimately
00:25:05.919 --> 00:25:08.230
narrator closed down but he left behind
00:25:08.240 --> 00:25:09.909
the the concept for data modeling for
00:25:09.919 --> 00:25:11.269
activity and he's working on a new one
00:25:11.279 --> 00:25:13.430
called polyiform and a few other things
00:25:13.440 --> 00:25:15.269
that I found really interesting in what
00:25:15.279 --> 00:25:17.190
he's talking about and he was talking
00:25:17.200 --> 00:25:19.510
about like some of the clients that he's
00:25:19.520 --> 00:25:21.590
been working with is the engineering
00:25:21.600 --> 00:25:23.350
team are going around the data team.
00:25:23.360 --> 00:25:24.630
They don't even want them involved.
00:25:24.640 --> 00:25:27.350
They're too they're tired of the white
00:25:27.360 --> 00:25:30.870
tower, the the the gating, the whatever
00:25:30.880 --> 00:25:32.870
it is. And so the engineers are wanting
00:25:32.880 --> 00:25:34.470
to like essentially create their own
00:25:34.480 --> 00:25:36.390
pipelines and all this other stuff. I
00:25:36.400 --> 00:25:38.310
mean, it's the tale as old as time,
00:25:38.320 --> 00:25:40.710
frankly. It's like, oh, this is new. I
00:25:40.720 --> 00:25:41.909
was like, no, this sounds standard. And
00:25:41.919 --> 00:25:43.510
then an engineer actually has to do data
00:25:43.520 --> 00:25:45.669
work and they're like, oh this is
00:25:45.679 --> 00:25:47.830
actually hard. is oh what's the
00:25:47.840 --> 00:25:52.070
definition of time? Well that depends um
00:25:52.080 --> 00:25:53.750
and is like you know
00:25:53.760 --> 00:25:56.149
>> even with Claude um you know and we've
00:25:56.159 --> 00:25:57.510
talked about this before in Thropics
00:25:57.520 --> 00:26:01.269
paper on like their their massive uh you
00:26:01.279 --> 00:26:03.269
know data modeling program and and
00:26:03.279 --> 00:26:05.669
analytical program. It's eerily similar
00:26:05.679 --> 00:26:08.230
to the same things that stood up for
00:26:08.240 --> 00:26:12.230
decades now just faster I guess. Um
00:26:12.240 --> 00:26:14.390
>> uh well yeah on that so I was talking to
00:26:14.400 --> 00:26:16.310
somebody the other day because you know
00:26:16.320 --> 00:26:19.029
I I lament uh how many people point to
00:26:19.039 --> 00:26:20.789
that anthropic paper and say oh they do
00:26:20.799 --> 00:26:22.149
dimensional modeling that's the answer
00:26:22.159 --> 00:26:23.990
and I'm like yeah I'm not sure it is I'm
00:26:24.000 --> 00:26:25.350
not sure this the paper says they do
00:26:25.360 --> 00:26:26.710
dimensionally modeling right and that's
00:26:26.720 --> 00:26:28.149
fine because any modeling technique will
00:26:28.159 --> 00:26:29.750
work I'm not sure that's where their
00:26:29.760 --> 00:26:31.190
secret source was but I was talking to
00:26:31.200 --> 00:26:32.230
somebody anyway
00:26:32.240 --> 00:26:33.990
>> and I don't know whether fact what
00:26:34.000 --> 00:26:35.269
they're telling me is factually correct
00:26:35.279 --> 00:26:36.710
or not because I can't validate it but
00:26:36.720 --> 00:26:39.430
what they said was uh yes there is part
00:26:39.440 --> 00:26:41.830
of that company that works works that
00:26:41.840 --> 00:26:44.230
way but there's also a large part of
00:26:44.240 --> 00:26:47.269
that company that doesn't uh and so you
00:26:47.279 --> 00:26:50.230
do wonder how much architectural buzzing
00:26:50.240 --> 00:26:52.310
of that paper and then especially like
00:26:52.320 --> 00:26:54.310
everybody else has come out right
00:26:54.320 --> 00:26:55.990
Spotify have done everybody else has
00:26:56.000 --> 00:26:58.390
come out with their how we use GI
00:26:58.400 --> 00:27:00.710
tooling with our data team and that I
00:27:00.720 --> 00:27:04.390
mean I think the core pattern of um you
00:27:04.400 --> 00:27:06.870
can't reverse engineer the context for
00:27:06.880 --> 00:27:09.590
your organization out of your SQL uh you
00:27:09.600 --> 00:27:11.190
are going to need human in the loop
00:27:11.200 --> 00:27:13.029
Mother duck. So many papers have come
00:27:13.039 --> 00:27:14.789
out. They've straight up confirmed that
00:27:14.799 --> 00:27:16.950
like you will you the problem is you
00:27:16.960 --> 00:27:18.710
will you will feel correct.
00:27:18.720 --> 00:27:19.029
>> Yep.
00:27:19.039 --> 00:27:20.390
>> You will feel correct. It'll have the
00:27:20.400 --> 00:27:23.350
language. It'll have a true touring
00:27:23.360 --> 00:27:25.990
complete vibe to it and you'll probably
00:27:26.000 --> 00:27:28.070
if you know huge chunk of people will be
00:27:28.080 --> 00:27:29.909
like yeah it's good and they will
00:27:29.919 --> 00:27:31.350
continue on. Um
00:27:31.360 --> 00:27:32.230
>> yeah.
00:27:32.240 --> 00:27:34.149
>> Yeah. So I think yeah that part of the
00:27:34.159 --> 00:27:36.950
paper because I it's been my experience
00:27:36.960 --> 00:27:39.590
right. Um so that yeah that's good. But
00:27:39.600 --> 00:27:41.350
I think there again there's people take
00:27:41.360 --> 00:27:43.350
little sound bites and go that is the
00:27:43.360 --> 00:27:46.070
answer, right? Um and I think
00:27:46.080 --> 00:27:47.750
>> dim packed and dim table it's clearly
00:27:47.760 --> 00:27:49.590
that's all that's needed. Um
00:27:49.600 --> 00:27:51.029
>> yeah they're valuable but they're not
00:27:51.039 --> 00:27:54.230
the answer. Uh I don't think so. Uh
00:27:54.240 --> 00:27:56.149
yeah, it it it's interesting and it's
00:27:56.159 --> 00:27:58.870
the same with um data bricks coming out
00:27:58.880 --> 00:28:00.870
with their accelerator data models,
00:28:00.880 --> 00:28:03.110
right? Their industry standard ontology
00:28:03.120 --> 00:28:05.750
and uh enterprise data models now per
00:28:05.760 --> 00:28:07.590
industry they've just announced. Uh I'm
00:28:07.600 --> 00:28:09.430
going to chat to Chris Gamble about it
00:28:09.440 --> 00:28:11.990
next week because he's deep in that data
00:28:12.000 --> 00:28:14.389
brick space. But we've seen it before,
00:28:14.399 --> 00:28:17.510
right? And I it it's never worked and I
00:28:17.520 --> 00:28:19.590
don't know why because it makes sense,
00:28:19.600 --> 00:28:21.590
right? It makes sense that if you're in
00:28:21.600 --> 00:28:24.710
an industry that is a relatively
00:28:24.720 --> 00:28:27.269
standard industry that 80% of what you
00:28:27.279 --> 00:28:29.750
do in terms of core concepts, your
00:28:29.760 --> 00:28:30.789
business processes and your
00:28:30.799 --> 00:28:32.870
relationships are the same as everybody
00:28:32.880 --> 00:28:35.909
else. And then the 20% is where you know
00:28:35.919 --> 00:28:37.750
the juice is worth the squeeze. How are
00:28:37.760 --> 00:28:39.669
you different or how are you going to be
00:28:39.679 --> 00:28:41.110
different? Right.
00:28:41.120 --> 00:28:42.710
>> Yeah. But we've seen those enterprise
00:28:42.720 --> 00:28:45.350
data models come out and reference AR
00:28:45.360 --> 00:28:47.990
models and industry models come out
00:28:48.000 --> 00:28:50.230
every wave and they never get traction
00:28:50.240 --> 00:28:51.990
and I've never worked out why. Yeah.
00:28:52.000 --> 00:28:55.110
What's your view?
00:28:55.120 --> 00:28:58.470
>> Um I mean I get pitched freaking data
00:28:58.480 --> 00:29:01.190
engineering, data analytics, AI startup
00:29:01.200 --> 00:29:03.430
stuff. Oh, I'm not going to say every
00:29:03.440 --> 00:29:04.930
day because I don't go on LinkedIn
00:29:04.940 --> 00:29:06.470
[laughter] every day, but every day I
00:29:06.480 --> 00:29:09.110
show up on LinkedIn, there's a new Y
00:29:09.120 --> 00:29:11.510
cominator or new startup out of
00:29:11.520 --> 00:29:13.190
somewhere that's going to, you know, do
00:29:13.200 --> 00:29:17.269
all the data engineering work. Um,
00:29:17.279 --> 00:29:18.950
and they, it's so funny, they've all
00:29:18.960 --> 00:29:22.230
converged to like facts fact like just
00:29:22.240 --> 00:29:24.870
like those patterns and I'm like why?
00:29:24.880 --> 00:29:26.230
There's so much more out there and
00:29:26.240 --> 00:29:27.830
sometimes better better patterns for
00:29:27.840 --> 00:29:32.070
solving all this stuff. Um,
00:29:32.080 --> 00:29:34.549
and
00:29:34.559 --> 00:29:36.070
>> let let me let me ask you it a different
00:29:36.080 --> 00:29:38.149
way, right? Because I so you work in a
00:29:38.159 --> 00:29:40.710
company that has subscriptions, right?
00:29:40.720 --> 00:29:42.310
Kind of core
00:29:42.320 --> 00:29:43.590
>> core be. Yeah.
00:29:43.600 --> 00:29:45.750
>> And and you go, well, you know, that's
00:29:45.760 --> 00:29:47.750
an industry standard model, right?
00:29:47.760 --> 00:29:50.125
customer subscribes to product
00:29:50.135 --> 00:29:50.950
[laughter]
00:29:50.960 --> 00:29:53.269
and and we know that because you're not
00:29:53.279 --> 00:29:55.110
the first one I've worked with where I
00:29:55.120 --> 00:29:56.630
like the first time I went into one of
00:29:56.640 --> 00:29:58.149
those companies and go ah it's
00:29:58.159 --> 00:30:00.230
subscription how hard could that be I'd
00:30:00.240 --> 00:30:03.190
never say that again um because every
00:30:03.200 --> 00:30:04.789
every software service company I've
00:30:04.799 --> 00:30:07.750
worked with subscription seems to be
00:30:07.760 --> 00:30:11.110
harder than customer um like but it's
00:30:11.120 --> 00:30:12.950
meant to be an ind why is it not just an
00:30:12.960 --> 00:30:15.430
industry standard model right that just
00:30:15.440 --> 00:30:16.470
makes sense to me [clears throat]
00:30:16.480 --> 00:30:18.870
>> it's so Funny. I love those things. And
00:30:18.880 --> 00:30:20.950
it's like, yeah, on like on the surface,
00:30:20.960 --> 00:30:22.310
right? There's so many things on the
00:30:22.320 --> 00:30:24.070
surface when you look at it. And then
00:30:24.080 --> 00:30:26.230
you're like getting into the contract.
00:30:26.240 --> 00:30:27.990
Oh, you have contracts. And it's like,
00:30:28.000 --> 00:30:29.110
yeah, we have contracts because like
00:30:29.120 --> 00:30:31.029
last year we sold it like this and these
00:30:31.039 --> 00:30:32.870
clients renewed on this value of the
00:30:32.880 --> 00:30:34.310
subscription, but then this year we have
00:30:34.320 --> 00:30:35.669
a new subscription, but the old
00:30:35.679 --> 00:30:37.190
customers don't want to move to the new
00:30:37.200 --> 00:30:38.710
subscription model [laughter] and you're
00:30:38.720 --> 00:30:40.070
not big enough because you're like, you
00:30:40.080 --> 00:30:42.070
know, sub$und00 million company. So, you
00:30:42.080 --> 00:30:44.070
can't really force those clients the new
00:30:44.080 --> 00:30:45.269
subscription. So now you have a
00:30:45.279 --> 00:30:47.990
bifrocated definition of contract and
00:30:48.000 --> 00:30:52.070
subscription and um you know I almost
00:30:52.080 --> 00:30:53.909
exclude I'm not say exclusively work for
00:30:53.919 --> 00:30:55.990
startups but I generally biased towards
00:30:56.000 --> 00:30:57.669
them these days and I've worked in
00:30:57.679 --> 00:31:01.590
numerous ones. Yeah. Like everyone
00:31:01.600 --> 00:31:03.590
starts to become a snowflake. You you
00:31:03.600 --> 00:31:05.029
start to hold the line about what
00:31:05.039 --> 00:31:06.549
contracts or subscriptions should look
00:31:06.559 --> 00:31:08.950
like and inevitably something comes down
00:31:08.960 --> 00:31:10.389
into the business that makes it
00:31:10.399 --> 00:31:13.750
different. Right. uh
00:31:13.760 --> 00:31:16.070
>> is a trial or contract. Yeah. Uh you
00:31:16.080 --> 00:31:17.430
know, you you know the answer is you
00:31:17.440 --> 00:31:19.110
just work to credits and nobody can
00:31:19.120 --> 00:31:20.870
understand how they work and then it
00:31:20.880 --> 00:31:22.481
doesn't matter, right? Because it's
00:31:22.491 --> 00:31:25.110
[laughter] there's one standard. It's a
00:31:25.120 --> 00:31:26.710
credit. We we don't know how we
00:31:26.720 --> 00:31:29.110
calculated. But that's okay. 100 credit.
00:31:29.120 --> 00:31:31.830
>> I do love to just a credit system. That
00:31:31.840 --> 00:31:33.750
way literally no one can figure out what
00:31:33.760 --> 00:31:35.590
you're actually charging. You get a
00:31:35.600 --> 00:31:37.750
hundred credits.
00:31:37.760 --> 00:31:39.430
>> Yeah. It's almost like credit I feel
00:31:39.440 --> 00:31:41.909
like credit card points like man
00:31:41.919 --> 00:31:43.669
somebody was like, "Oh, you bet. I get a
00:31:43.679 --> 00:31:46.149
h 100,000 credit card points for signing
00:31:46.159 --> 00:31:48.070
up to this new like program." I'm like,
00:31:48.080 --> 00:31:49.990
"Yeah, but what does that actually
00:31:50.000 --> 00:31:51.990
mean?" Because before I used to be, you
00:31:52.000 --> 00:31:54.630
know, 100,000 credits was like 1 cent
00:31:54.640 --> 00:31:57.110
USD. And I'm like, "Is it now?" Well,
00:31:57.120 --> 00:31:59.190
it's like no. And there's like this
00:31:59.200 --> 00:32:01.830
pyramid system of like configurations.
00:32:01.840 --> 00:32:03.430
And that's exactly how I feel like using
00:32:03.440 --> 00:32:05.509
AI tooling right now for credits. I just
00:32:05.519 --> 00:32:09.430
like which which MX card is gonna work
00:32:09.440 --> 00:32:12.789
with what because like um I got a new MX
00:32:12.799 --> 00:32:16.070
card and for uh and it was so funny. I
00:32:16.080 --> 00:32:17.909
was like, "Oh, if you go chat GBT
00:32:17.919 --> 00:32:19.590
business, we're going to give you 300
00:32:19.600 --> 00:32:21.750
credits." And I was like, "Oh, that
00:32:21.760 --> 00:32:24.310
sounds great. 300 credits. Uh, okay.
00:32:24.320 --> 00:32:25.669
I'll go for it." And I was like, "Oh,
00:32:25.679 --> 00:32:27.669
yeah, minimum seats." And then it's like
00:32:27.679 --> 00:32:29.269
credited for over this period of time. I
00:32:29.279 --> 00:32:31.257
was like, "Okay, of course." It's like,
00:32:31.267 --> 00:32:32.710
[laughter]
00:32:32.720 --> 00:32:35.029
>> yeah, it's not just put it on the card
00:32:35.039 --> 00:32:36.710
and you're like, you're good to go. No,
00:32:36.720 --> 00:32:37.750
no, no.
00:32:37.760 --> 00:32:40.630
>> Yeah. And then um Well, it's like uh
00:32:40.640 --> 00:32:42.789
Claude, right, when Fable came out and
00:32:42.799 --> 00:32:45.830
then Opus 5 and then everybody's going,
00:32:45.840 --> 00:32:48.470
"Is it me or is Opus 5 the
00:32:48.480 --> 00:32:50.549
bed?" And and eventually I'd heard
00:32:50.559 --> 00:32:52.389
enough people that were having the same
00:32:52.399 --> 00:32:54.149
problems I had and I was like, "Back to
00:32:54.159 --> 00:32:55.990
Opus 4.8
00:32:56.000 --> 00:32:58.070
because and and now I'm sitting there
00:32:58.080 --> 00:33:01.669
going ah you know
00:33:01.679 --> 00:33:03.750
when do I make the call to make my
00:33:03.760 --> 00:33:05.750
harness agnostic
00:33:05.760 --> 00:33:07.830
like because at the moment I just it's
00:33:07.840 --> 00:33:10.870
all included right and I'm like I know
00:33:10.880 --> 00:33:13.590
I've architected it in theory in a way
00:33:13.600 --> 00:33:16.230
that I could move it to other models.
00:33:16.240 --> 00:33:19.350
>> Yeah. But I haven't tested it. So I
00:33:19.360 --> 00:33:21.029
know, you know, I've been around long
00:33:21.039 --> 00:33:22.870
enough to know if I haven't tested it
00:33:22.880 --> 00:33:25.190
architecturally, it's just a theory. And
00:33:25.200 --> 00:33:27.029
it's a theory that probably won't
00:33:27.039 --> 00:33:29.430
survive. And I when do I take the hit
00:33:29.440 --> 00:33:32.310
for that puppy? Right. And then favorite
00:33:32.320 --> 00:33:34.149
one came out and I'm like, is it going
00:33:34.159 --> 00:33:36.149
to be better? Is it going to be worse?
00:33:36.159 --> 00:33:38.470
Am I going to blow my my subscription in
00:33:38.480 --> 00:33:40.549
half an hour? Because I got no idea what
00:33:40.559 --> 00:33:42.389
it's doing. uh like I was doing a
00:33:42.399 --> 00:33:45.269
conference today and um uh I wanted to
00:33:45.279 --> 00:33:47.430
li show live basically using the
00:33:47.440 --> 00:33:49.830
information product uh app right so the
00:33:49.840 --> 00:33:52.149
open source app that I've got so yeah
00:33:52.159 --> 00:33:53.990
there's a canvas in there I export it to
00:33:54.000 --> 00:33:56.630
Jason I upload it to Claude I asked Lord
00:33:56.640 --> 00:33:58.070
a bunch of questions right and that was
00:33:58.080 --> 00:34:00.870
that was the demo ran it all through
00:34:00.880 --> 00:34:03.509
that's all good um it's all pretty quick
00:34:03.519 --> 00:34:05.350
you turned off uh thinking mode all that
00:34:05.360 --> 00:34:07.590
kind of um [laughter]
00:34:07.600 --> 00:34:09.430
yeah because I had you had a set amount
00:34:09.440 --> 00:34:11.030
of time and it was like yeah it's I
00:34:11.040 --> 00:34:12.470
mean, each question is coming back in
00:34:12.480 --> 00:34:13.829
less than a minute. Of course, when I
00:34:13.839 --> 00:34:15.430
did it live, it took over two to three
00:34:15.440 --> 00:34:16.950
minutes for each question. So, that
00:34:16.960 --> 00:34:19.510
can cander. Um, but anyway, so I was
00:34:19.520 --> 00:34:22.310
yoloing along. I'm all good already in
00:34:22.320 --> 00:34:24.230
the morning presentation in the early
00:34:24.240 --> 00:34:25.750
afternoon. So, I was like, cool, let's
00:34:25.760 --> 00:34:27.909
get back to work, right? And so, I'm
00:34:27.919 --> 00:34:31.030
building some stuff and I'm had flipped
00:34:31.040 --> 00:34:32.869
over to Fable 51 for something and
00:34:32.879 --> 00:34:34.310
forgot to flip back. So, every new
00:34:34.320 --> 00:34:36.550
session was fabling me. And then I was
00:34:36.560 --> 00:34:37.990
sitting there going, "Hey, hold on, hold
00:34:38.000 --> 00:34:40.869
on. how how much have I got left in my 5
00:34:40.879 --> 00:34:43.109
hour window because that five hour
00:34:43.119 --> 00:34:45.990
window is spang smack my conference is
00:34:46.000 --> 00:34:48.069
in the middle of it. Um and I was like
00:34:48.079 --> 00:34:49.270
yeah I'm going to stop working now
00:34:49.280 --> 00:34:51.990
because I got no guarantee of when I'm
00:34:52.000 --> 00:34:53.669
actually going to run out of that that
00:34:53.679 --> 00:34:55.270
and
00:34:55.280 --> 00:34:57.510
embarrassing. So yeah I think there's a
00:34:57.520 --> 00:34:59.670
lack of visibility, right?
00:34:59.680 --> 00:35:00.950
>> Yeah. I mean it's the classic like
00:35:00.960 --> 00:35:02.710
blackbox problems we got going on here,
00:35:02.720 --> 00:35:05.184
right? We just keep on punting them
00:35:05.194 --> 00:35:06.310
[laughter] and it's really interesting.
00:35:06.320 --> 00:35:08.550
What do you feel like um you you you use
00:35:08.560 --> 00:35:11.030
harnesses like are you you know are you
00:35:11.040 --> 00:35:14.150
going all in on Pi or Hermes or you you
00:35:14.160 --> 00:35:15.349
spun up your own thing?
00:35:15.359 --> 00:35:17.750
>> No, I started off I'm a bit like you. I
00:35:17.760 --> 00:35:20.390
started off with a personal personal OS
00:35:20.400 --> 00:35:22.470
um as as a learning curve, right? It was
00:35:22.480 --> 00:35:24.150
back in the open claw. So I did open
00:35:24.160 --> 00:35:26.710
claw for about two seconds then uh did a
00:35:26.720 --> 00:35:27.990
little bit of reading was like holy
00:35:28.000 --> 00:35:30.150
no but I use it as a learning curve and
00:35:30.160 --> 00:35:32.870
then for our stuff uh I started building
00:35:32.880 --> 00:35:36.230
it specifically for what we needed. Um,
00:35:36.240 --> 00:35:37.910
I still have a technique where what I'll
00:35:37.920 --> 00:35:39.510
do is when I hear of a new uh open
00:35:39.520 --> 00:35:42.310
source harness, I'll get Claude to
00:35:42.320 --> 00:35:44.230
basically look at my harness, look at
00:35:44.240 --> 00:35:45.670
the open source one, and tell me what's
00:35:45.680 --> 00:35:47.349
missing, what's useful, and try and get
00:35:47.359 --> 00:35:51.349
it back. Um, but I'm at the stage now
00:35:51.359 --> 00:35:53.430
where I'm starting to question how much
00:35:53.440 --> 00:35:55.670
time am I spending on the harness versus
00:35:55.680 --> 00:35:58.470
actually doing the work. Like I know the
00:35:58.480 --> 00:36:00.790
harness is giving making me faster to do
00:36:00.800 --> 00:36:03.750
the work but actually if I look at an
00:36:03.760 --> 00:36:05.750
investment window you net present value
00:36:05.760 --> 00:36:08.870
of time. Uh I'm wondering now whether
00:36:08.880 --> 00:36:11.430
I'm overindexing on the harness itself
00:36:11.440 --> 00:36:15.109
because it's kind of fun. Like I did
00:36:15.119 --> 00:36:16.710
this
00:36:16.720 --> 00:36:19.190
Claude did that. It shouldn't have done
00:36:19.200 --> 00:36:21.910
that. I don't want it to do that again.
00:36:21.920 --> 00:36:25.109
So, let's spend two hours getting the
00:36:25.119 --> 00:36:26.950
hardness up to date to stop Claude ever
00:36:26.960 --> 00:36:29.589
doing that, right? And I'm like, is that
00:36:29.599 --> 00:36:33.109
like where's that value? Um, I mean, for
00:36:33.119 --> 00:36:37.750
us, the value is I can give I want to be
00:36:37.760 --> 00:36:39.750
able to give our harness to our partners
00:36:39.760 --> 00:36:41.589
and have them work the same way I do,
00:36:41.599 --> 00:36:43.670
right? Without the intrinsic knowledge
00:36:43.680 --> 00:36:46.069
of our platform that I know I have.
00:36:46.079 --> 00:36:47.829
>> That that's the end goal, right? It's a
00:36:47.839 --> 00:36:51.750
scaling uh play and that's the dream,
00:36:51.760 --> 00:36:54.550
right? Um but I still worry right now
00:36:54.560 --> 00:36:56.150
and then I look and say, well, how many
00:36:56.160 --> 00:36:57.430
other people are doing that, right? Like
00:36:57.440 --> 00:36:59.510
how many data teams in a large
00:36:59.520 --> 00:37:01.589
organization right now are spending 80%
00:37:01.599 --> 00:37:03.829
of their time grinding harnesses and not
00:37:03.839 --> 00:37:08.069
actually doing the data work?
00:37:08.079 --> 00:37:09.990
>> Yeah, I mean that's a good question. I
00:37:10.000 --> 00:37:11.750
don't know
00:37:11.760 --> 00:37:14.790
how I I I question on like how
00:37:14.800 --> 00:37:16.390
consistent a lot of people are playing.
00:37:16.400 --> 00:37:18.630
I think a lot of people are are spinning
00:37:18.640 --> 00:37:22.230
up um kind of like their own playbooks
00:37:22.240 --> 00:37:24.150
in the in the company like a a vast
00:37:24.160 --> 00:37:25.510
majority of people they're all spinning
00:37:25.520 --> 00:37:28.470
up their like you know claude or ilk of
00:37:28.480 --> 00:37:29.990
their own markdown files to feel like
00:37:30.000 --> 00:37:31.510
they're giving themselves the edge and
00:37:31.520 --> 00:37:33.990
not really sharing it across across the
00:37:34.000 --> 00:37:37.270
company. Um, and then like, you know,
00:37:37.280 --> 00:37:38.790
people are really struggling on how to
00:37:38.800 --> 00:37:40.390
take it off, take it into like
00:37:40.400 --> 00:37:42.310
multiplayer mode, right?
00:37:42.320 --> 00:37:44.790
>> I really truly think that it's kind of
00:37:44.800 --> 00:37:46.470
like the next level. I mean, I think
00:37:46.480 --> 00:37:47.910
that's why Claude Co-work was starting
00:37:47.920 --> 00:37:50.470
to take off a little bit more. Um, but I
00:37:50.480 --> 00:37:53.750
think like
00:37:53.760 --> 00:37:55.589
I think, you know, I you know, my wife
00:37:55.599 --> 00:37:57.829
and I are, you know, we have our own uh
00:37:57.839 --> 00:37:59.750
and we struggle to turn into a proper
00:37:59.760 --> 00:38:01.109
multiplayer mode because we have like
00:38:01.119 --> 00:38:02.870
really different ways of wanting to work
00:38:02.880 --> 00:38:05.510
and do things, right? And so if you're
00:38:05.520 --> 00:38:07.829
like naturally
00:38:07.839 --> 00:38:10.150
different, could you imagine what 10020
00:38:10.160 --> 00:38:11.829
people are like and they're all like
00:38:11.839 --> 00:38:14.470
they like their, you know, as you call
00:38:14.480 --> 00:38:17.430
them pets maybe to be kind of like tuned
00:38:17.440 --> 00:38:19.510
to them. And that's where like I think a
00:38:19.520 --> 00:38:20.790
lot of people are struggling like you
00:38:20.800 --> 00:38:23.030
need this consistency we'll call context
00:38:23.040 --> 00:38:26.230
or knowledge and you know the facts of
00:38:26.240 --> 00:38:29.750
the business or facts much as you can
00:38:29.760 --> 00:38:31.030
and your personal life your calendars
00:38:31.040 --> 00:38:32.710
and all that stuff but then like the
00:38:32.720 --> 00:38:35.910
nuances can be like
00:38:35.920 --> 00:38:37.510
I think they're a lot deeper than people
00:38:37.520 --> 00:38:39.510
give credit to sometimes right that like
00:38:39.520 --> 00:38:41.349
that 10%
00:38:41.359 --> 00:38:44.550
can be so wildly different
00:38:44.560 --> 00:38:46.230
and [laughter] the outcomes and you're
00:38:46.240 --> 00:38:47.829
just like I know we're speaking the same
00:38:47.839 --> 00:38:50.550
language uage but [laughter]
00:38:50.560 --> 00:38:51.990
you are coming you and I are coming
00:38:52.000 --> 00:38:55.030
completely different points of view here
00:38:55.040 --> 00:38:58.630
>> and I yeah well Anna Bergin's actually
00:38:58.640 --> 00:39:01.109
working on a personal OS for family uh
00:39:01.119 --> 00:39:02.550
for her family right and she's open
00:39:02.560 --> 00:39:04.230
sourcing it so it'll be interesting to
00:39:04.240 --> 00:39:05.430
see whether you know how much of that
00:39:05.440 --> 00:39:07.190
you pick up for your your family stuff
00:39:07.200 --> 00:39:07.750
um
00:39:07.760 --> 00:39:09.589
>> yeah I saw interesting I you know I was
00:39:09.599 --> 00:39:11.030
I'm going the same direction there's so
00:39:11.040 --> 00:39:12.230
many people working on it I would say
00:39:12.240 --> 00:39:13.589
like the other thing I get pitched on
00:39:13.599 --> 00:39:15.589
every other day is probably not just
00:39:15.599 --> 00:39:17.430
personal knowledge gaff brat or personal
00:39:17.440 --> 00:39:19.750
knowledge bases for companies like you
00:39:19.760 --> 00:39:21.430
can see Stripe came out with their one
00:39:21.440 --> 00:39:23.589
and how they do it. Um, obviously
00:39:23.599 --> 00:39:26.470
there's Brockbot out there and like how
00:39:26.480 --> 00:39:28.150
they're kind of like heading towards
00:39:28.160 --> 00:39:30.550
these different patterns. Yeah. I mean,
00:39:30.560 --> 00:39:32.150
everybody knows that like the
00:39:32.160 --> 00:39:33.829
multiplayer mode is kind of like where
00:39:33.839 --> 00:39:36.870
it's at. Um, you know, open.
00:39:36.880 --> 00:39:40.630
>> Yeah. The question is then comes back to
00:39:40.640 --> 00:39:42.550
do you buy a harness off a platform
00:39:42.560 --> 00:39:45.430
vendor because it's embedded in their
00:39:45.440 --> 00:39:46.950
technology stack? It's integrated,
00:39:46.960 --> 00:39:48.790
right? It knows how to execute the
00:39:48.800 --> 00:39:51.190
harness. And then you bolt on your
00:39:51.200 --> 00:39:53.270
context plane. Yeah. The the thing that
00:39:53.280 --> 00:39:55.270
holds your context, which is the second
00:39:55.280 --> 00:39:59.109
part of what a harness needs. Um
00:39:59.119 --> 00:40:02.150
or do you go with a platform player
00:40:02.160 --> 00:40:05.190
that's got both the context storage and
00:40:05.200 --> 00:40:08.630
the execution? Uh do you buy all three
00:40:08.640 --> 00:40:10.550
separately? Right? you go I got platform
00:40:10.560 --> 00:40:13.030
that runs the code an execution harness
00:40:13.040 --> 00:40:17.990
that understands almost like a um a data
00:40:18.000 --> 00:40:19.910
integration hub right you know my
00:40:19.920 --> 00:40:22.870
context sits in an in a graph database
00:40:22.880 --> 00:40:25.829
or an MD files or in a confluence wiki
00:40:25.839 --> 00:40:28.870
or whatever my harness for the policies
00:40:28.880 --> 00:40:30.950
and procedures and way of working sits
00:40:30.960 --> 00:40:33.990
over here and whatever and it knows it's
00:40:34.000 --> 00:40:35.349
already been written to integrate to run
00:40:35.359 --> 00:40:38.470
on data bricks or snowflake or iceberg
00:40:38.480 --> 00:40:41.030
with spark park or heaven forbid fabric
00:40:41.040 --> 00:40:44.069
or IBM or uh Salesforce I mean
00:40:44.079 --> 00:40:47.829
Informatica um yeah like it becomes a
00:40:47.839 --> 00:40:51.430
language right like would you why I mean
00:40:51.440 --> 00:40:53.589
it's there's so many moving parts that
00:40:53.599 --> 00:40:55.990
that decoupling is going to kill you in
00:40:56.000 --> 00:40:58.710
my view
00:40:58.720 --> 00:41:02.950
yeah I think um [snorts]
00:41:02.960 --> 00:41:06.309
yeah I mean this is awesome
00:41:06.319 --> 00:41:10.150
this is awesome decoupling Um but yeah,
00:41:10.160 --> 00:41:12.710
this like this complexity is so funny
00:41:12.720 --> 00:41:15.109
like the complexity that we have to
00:41:15.119 --> 00:41:21.750
handle. Um
00:41:21.760 --> 00:41:25.109
yeah, I mean I I mean I have cognitive
00:41:25.119 --> 00:41:27.109
overload every day like and I and I read
00:41:27.119 --> 00:41:29.750
in a copious noteaker. Um although most
00:41:29.760 --> 00:41:31.270
my half my notes are actually on pieces
00:41:31.280 --> 00:41:34.790
of paper. Um
00:41:34.800 --> 00:41:36.470
yeah I do wonder like how people are
00:41:36.480 --> 00:41:37.990
going to like digest and work through
00:41:38.000 --> 00:41:40.550
all this stuff right like you're
00:41:40.560 --> 00:41:42.950
mentioning markdown graph like all these
00:41:42.960 --> 00:41:45.430
I'm going to say toys but like all these
00:41:45.440 --> 00:41:47.109
things and ultimately it's just like how
00:41:47.119 --> 00:41:48.550
much is going to be decided by the
00:41:48.560 --> 00:41:50.550
system and ultimately and the other part
00:41:50.560 --> 00:41:52.630
is like how much do you want like you
00:41:52.640 --> 00:41:53.990
you look at like how many people came
00:41:54.000 --> 00:41:56.630
back to you know like I wouldn't say
00:41:56.640 --> 00:41:59.430
like fled Microsoft Windows but you know
00:41:59.440 --> 00:42:01.270
um and went to Mac right and you know
00:42:01.280 --> 00:42:03.430
I'm a huge Linux guy or was you know all
00:42:03.440 --> 00:42:05.109
my servers are almost always Linux. I
00:42:05.119 --> 00:42:06.790
almost never see use Windows server like
00:42:06.800 --> 00:42:10.550
blah blah blah. Um [cough]
00:42:10.560 --> 00:42:14.150
and um
00:42:14.160 --> 00:42:16.390
you know and then uh what's uh DHH
00:42:16.400 --> 00:42:19.589
announced Omari like and like Toby Luca
00:42:19.599 --> 00:42:21.910
and uh a bunch of other names I'm going
00:42:21.920 --> 00:42:23.829
to like destroy and they basically put
00:42:23.839 --> 00:42:27.349
like $10 million into make Linux amazing
00:42:27.359 --> 00:42:29.510
and I was like man maybe this will
00:42:29.520 --> 00:42:31.430
finally happen. Maybe it will. I'm not
00:42:31.440 --> 00:42:33.910
sure. Um,
00:42:33.920 --> 00:42:35.910
and so I just think things are changing,
00:42:35.920 --> 00:42:37.270
right? Like I got really comfortable
00:42:37.280 --> 00:42:38.950
with Mac and like how Mac does it and
00:42:38.960 --> 00:42:41.030
keeps my life simpler. And so when
00:42:41.040 --> 00:42:42.550
you're talking about like harnesses and
00:42:42.560 --> 00:42:44.230
stuff like that, I was like I mean we've
00:42:44.240 --> 00:42:45.750
already demonstrated constantly like
00:42:45.760 --> 00:42:48.150
whatever is easy. Wherever the water
00:42:48.160 --> 00:42:50.550
flows easiest is where people naturally
00:42:50.560 --> 00:42:53.109
go, right? Even if it's better for them,
00:42:53.119 --> 00:42:55.510
right? Maybe if like freedom, open
00:42:55.520 --> 00:42:58.630
source, Linux, whatever, the vast
00:42:58.640 --> 00:43:01.430
majority of people will be like, "Okay,
00:43:01.440 --> 00:43:02.870
if I don't have to think about this for
00:43:02.880 --> 00:43:04.309
five minutes and I have to pay somebody
00:43:04.319 --> 00:43:06.710
five bucks or 20 bucks a month, then you
00:43:06.720 --> 00:43:08.470
know, a huge portion of people just like
00:43:08.480 --> 00:43:10.390
don't care." Although the irony to me
00:43:10.400 --> 00:43:12.309
though is how many people don't pay for
00:43:12.319 --> 00:43:14.710
these open AI tools or like things and
00:43:14.720 --> 00:43:21.589
they just are willing to take the ads
00:43:21.599 --> 00:43:23.750
>> and I you know taking income across
00:43:23.760 --> 00:43:25.990
distribution across the world out of the
00:43:26.000 --> 00:43:27.750
equation here. But like how many people
00:43:27.760 --> 00:43:30.470
are willing to sacrifice their attention
00:43:30.480 --> 00:43:32.870
and the bias information coming to them
00:43:32.880 --> 00:43:34.870
just so they can
00:43:34.880 --> 00:43:36.870
>> like how how many people are willing to
00:43:36.880 --> 00:43:39.109
sacrifice their personal context. Right.
00:43:39.119 --> 00:43:41.430
So it was a a guy in New Zealand
00:43:41.440 --> 00:43:43.030
reposted something on LinkedIn and it
00:43:43.040 --> 00:43:45.349
was um it's from one of the UK police
00:43:45.359 --> 00:43:47.990
companies uh or and it basically had a
00:43:48.000 --> 00:43:49.910
mockup of you know two kids in school
00:43:49.920 --> 00:43:51.829
uniforms and they tagged all the
00:43:51.839 --> 00:43:53.589
personally identifiable information. So,
00:43:53.599 --> 00:43:55.510
you know, the badge on the blazer says
00:43:55.520 --> 00:43:58.390
which school they went to. Um, the mom
00:43:58.400 --> 00:44:00.309
holding or the dad holding because you
00:44:00.319 --> 00:44:02.470
couldn't see their face. A a polyarin
00:44:02.480 --> 00:44:03.670
cup with their name on it from the
00:44:03.680 --> 00:44:06.470
coffee company, you know, John. Um,
00:44:06.480 --> 00:44:07.990
actually, I think it was a lady's name.
00:44:08.000 --> 00:44:12.309
Um, yeah. Uh, like a a number plate for
00:44:12.319 --> 00:44:15.829
the car. Um, yes. All these things that
00:44:15.839 --> 00:44:18.710
were in an image that now, you know, bad
00:44:18.720 --> 00:44:20.309
actors can scrape. And I sit there
00:44:20.319 --> 00:44:22.950
going, you number of people I know that
00:44:22.960 --> 00:44:25.430
put all their kids stuff on Facebook
00:44:25.440 --> 00:44:28.550
constantly, you know, videos and images
00:44:28.560 --> 00:44:30.550
and and basically document their entire
00:44:30.560 --> 00:44:31.910
kids' life, you know.
00:44:31.920 --> 00:44:32.870
>> Oh, yeah.
00:44:32.880 --> 00:44:35.670
>> And the kids five in 20 years time,
00:44:35.680 --> 00:44:38.069
right, when when none of that
00:44:38.079 --> 00:44:40.150
information is private and it's
00:44:40.160 --> 00:44:44.470
available to any platform,
00:44:44.480 --> 00:44:45.750
what's that kid going to say to their
00:44:45.760 --> 00:44:48.069
parents? Hey, thanks for that. Was it
00:44:48.079 --> 00:44:50.150
worth it? I don't know. Anyway, I go I
00:44:50.160 --> 00:44:51.910
can go deep into conspiracy theory.
00:44:51.920 --> 00:44:54.790
>> Yeah. My wife a pretty hard stance on on
00:44:54.800 --> 00:44:56.710
on this stuff and we try to like
00:44:56.720 --> 00:44:59.109
anything that says like share our kids
00:44:59.119 --> 00:45:02.470
pictures or whatever with like any big,
00:45:02.480 --> 00:45:05.510
you know, Apple or uh any of that like
00:45:05.520 --> 00:45:07.589
we we try to shut that all that down. I
00:45:07.599 --> 00:45:09.270
don't I don't trust any any of these
00:45:09.280 --> 00:45:10.950
guys. They'll make a good buck. you
00:45:10.960 --> 00:45:13.109
know, they they claim, you know, do good
00:45:13.119 --> 00:45:16.550
or whatever, but and I, you know, I I am
00:45:16.560 --> 00:45:18.390
wouldn't say for I'm not so crazy where
00:45:18.400 --> 00:45:19.750
I'm gonna wear a tin wall hat and be
00:45:19.760 --> 00:45:22.150
like, ah, I can't. No, I would just like
00:45:22.160 --> 00:45:23.910
actively participating by like, you
00:45:23.920 --> 00:45:26.630
know, having my kids pictures on like on
00:45:26.640 --> 00:45:29.430
Instagram or LinkedIn or I'm just like,
00:45:29.440 --> 00:45:32.470
no, it's not not where where I want that
00:45:32.480 --> 00:45:35.430
kind of stuff to happen. And the other
00:45:35.440 --> 00:45:40.230
thing is um
00:45:40.240 --> 00:45:41.750
yeah, I mean we share our data so
00:45:41.760 --> 00:45:43.510
freely. I mean I'm just like right now
00:45:43.520 --> 00:45:46.630
I'm like on an absolute rager trying to
00:45:46.640 --> 00:45:49.270
get uh my phone numbers pulled from data
00:45:49.280 --> 00:45:51.109
brokers like crazy right now because I
00:45:51.119 --> 00:45:52.630
can't even use my cell phone. It's
00:45:52.640 --> 00:45:55.829
gotten so bad. Like I just can't pick so
00:45:55.839 --> 00:45:58.230
many calls all the time and text
00:45:58.240 --> 00:46:00.309
messages. Like if I sneeze on this
00:46:00.319 --> 00:46:04.390
podcast saying like virus software, um
00:46:04.400 --> 00:46:07.349
I'm probably going to get like like just
00:46:07.359 --> 00:46:11.589
tsunami of communication. Um so then
00:46:11.599 --> 00:46:13.349
it's just like I don't It's so funny. I
00:46:13.359 --> 00:46:15.190
feel like it's gonna be like AI versus
00:46:15.200 --> 00:46:18.550
AI wars fighting like spam, right?
00:46:18.560 --> 00:46:20.069
>> Oh, well actually Anna and I were
00:46:20.079 --> 00:46:21.349
talking about that when we caught up
00:46:21.359 --> 00:46:24.150
last week or the week before and uh Joe
00:46:24.160 --> 00:46:26.790
Ree had jumped on. we saw him turn up
00:46:26.800 --> 00:46:28.230
and and Anna was talking because she's
00:46:28.240 --> 00:46:30.950
working in a in a new gig in a hospital
00:46:30.960 --> 00:46:32.390
and it's Greenfields, right? And so
00:46:32.400 --> 00:46:35.589
she's talking through um the the way
00:46:35.599 --> 00:46:36.950
she's dealing with that green threes
00:46:36.960 --> 00:46:38.710
program, right? And and what she's do
00:46:38.720 --> 00:46:40.390
looking at now, what she look at next.
00:46:40.400 --> 00:46:41.910
And I'm sitting there, I said to her,
00:46:41.920 --> 00:46:43.750
man, you got to get snotted by sales
00:46:43.760 --> 00:46:45.030
people, right? Because you've said that
00:46:45.040 --> 00:46:47.109
eventually you're gonna buy or build
00:46:47.119 --> 00:46:50.230
something in this space. And so my joke
00:46:50.240 --> 00:46:51.750
was, that's fine. Just tell them that
00:46:51.760 --> 00:46:53.430
they need to get hold of Joe, right? He
00:46:53.440 --> 00:46:55.030
he's he's in charge of buying for you.
00:46:55.040 --> 00:46:56.550
and [laughter] and then they'll get so
00:46:56.560 --> 00:46:58.710
there you go right if you if uh if you
00:46:58.720 --> 00:47:00.790
do need a virus thing I think anybody
00:47:00.800 --> 00:47:03.589
that's uh scraping this their AI should
00:47:03.599 --> 00:47:06.230
contact Jay Ree about selling Tim virus
00:47:06.240 --> 00:47:09.109
stuff but let let's go back to this idea
00:47:09.119 --> 00:47:13.349
of privacy because or or context privacy
00:47:13.359 --> 00:47:14.550
because one of the interesting things
00:47:14.560 --> 00:47:16.630
now if we say that we can't reverse
00:47:16.640 --> 00:47:18.390
engineer context for an organization
00:47:18.400 --> 00:47:20.790
from the SQL queries have been run right
00:47:20.800 --> 00:47:22.870
because we know that based on all the
00:47:22.880 --> 00:47:24.390
papers that written human in the loop
00:47:24.400 --> 00:47:25.990
for that context is actually the most
00:47:26.000 --> 00:47:28.230
valuable thing you can do. And if you're
00:47:28.240 --> 00:47:30.550
sitting in an organization that has a
00:47:30.560 --> 00:47:33.430
history of counting people when uh
00:47:33.440 --> 00:47:34.950
they want to save money or they want
00:47:34.960 --> 00:47:37.510
their share price to go up and and your
00:47:37.520 --> 00:47:39.030
comment about all your notes being on
00:47:39.040 --> 00:47:41.750
paper, right? Like how willing would you
00:47:41.760 --> 00:47:45.109
be to put your knowledge into anything
00:47:45.119 --> 00:47:47.589
electronic and not just retain it in
00:47:47.599 --> 00:47:49.430
notebooks on your desk that you never
00:47:49.440 --> 00:47:51.349
share? Like that's got to be coming,
00:47:51.359 --> 00:47:53.910
right? where people actually gate the
00:47:53.920 --> 00:47:56.150
knowledge they have because it's the
00:47:56.160 --> 00:47:57.510
only thing of value that the
00:47:57.520 --> 00:47:59.829
organization may not retrench their ass
00:47:59.839 --> 00:48:01.910
for. What do you reckon?
00:48:01.920 --> 00:48:06.230
>> Yeah. No, I it's so funny like you know
00:48:06.240 --> 00:48:08.309
can you even trust your systems like
00:48:08.319 --> 00:48:09.990
sometimes like you're reading like the
00:48:10.000 --> 00:48:12.230
terms of service and like when like
00:48:12.240 --> 00:48:15.030
people are like well I you know
00:48:15.040 --> 00:48:19.109
sometimes you know I do question
00:48:19.119 --> 00:48:21.270
like do some of these servers even need
00:48:21.280 --> 00:48:24.549
your full like context can they get it
00:48:24.559 --> 00:48:26.790
from the shape of your information right
00:48:26.800 --> 00:48:28.870
>> right you know and what I mean by that
00:48:28.880 --> 00:48:31.670
is like the metadata how it moves like
00:48:31.680 --> 00:48:32.870
how you're doing things on in your
00:48:32.880 --> 00:48:34.630
system. Not necessarily as so much that
00:48:34.640 --> 00:48:36.630
you're like Tim Fraser opened Excel
00:48:36.640 --> 00:48:39.109
sheet one, two, three and like looked at
00:48:39.119 --> 00:48:42.150
X column, but like because of the sea of
00:48:42.160 --> 00:48:44.069
information, they could a you could
00:48:44.079 --> 00:48:46.870
ascertain like how people were solving
00:48:46.880 --> 00:48:49.030
and doing things. So then like you're
00:48:49.040 --> 00:48:52.069
ultimately using these systems to train
00:48:52.079 --> 00:48:53.910
the system. You know, Meta is doing this
00:48:53.920 --> 00:48:55.510
for like tracking their own employees
00:48:55.520 --> 00:48:57.910
and having all the other kind of stuff.
00:48:57.920 --> 00:48:59.510
you know, how much longer are we going
00:48:59.520 --> 00:49:01.910
to be able to rely on the computers that
00:49:01.920 --> 00:49:03.990
we're using because the money is just
00:49:04.000 --> 00:49:08.150
it's so worth worth it, right? So,
00:49:08.160 --> 00:49:10.630
>> yeah. And you I mean, you read that
00:49:10.640 --> 00:49:12.790
Spirit article, right, where Spirit
00:49:12.800 --> 00:49:15.430
Airlines, Google paid $10 million to get
00:49:15.440 --> 00:49:18.069
access to all the data in that
00:49:18.079 --> 00:49:19.990
organization, all the employee Slack
00:49:20.000 --> 00:49:23.109
messages or every email. Now, in theory,
00:49:23.119 --> 00:49:25.270
the article said that personally
00:49:25.280 --> 00:49:28.069
identifiable data would be cleansed from
00:49:28.079 --> 00:49:30.470
that data set
00:49:30.480 --> 00:49:33.270
um or data lake probably, isn't it? Um
00:49:33.280 --> 00:49:35.990
before it's handed to Google, but it's a
00:49:36.000 --> 00:49:39.349
liquidated company, right? Like, yeah,
00:49:39.359 --> 00:49:41.910
they how much money is a liquidator
00:49:41.920 --> 00:49:43.910
going to pay cleaning that data before
00:49:43.920 --> 00:49:46.150
it hands off to Google? And if it's not
00:49:46.160 --> 00:49:48.790
cleaned, who gets sued, right? like if
00:49:48.800 --> 00:49:51.750
an employes's personal information is
00:49:51.760 --> 00:49:56.309
passed to Google as part of that data um
00:49:56.319 --> 00:49:58.390
and they find out who who do they
00:49:58.400 --> 00:50:02.710
complain to right uh but yeah so again
00:50:02.720 --> 00:50:04.790
if you now start to see companies that
00:50:04.800 --> 00:50:07.990
are in trouble having their their
00:50:08.000 --> 00:50:10.630
conversations right being liquidated as
00:50:10.640 --> 00:50:14.630
an asset
00:50:14.640 --> 00:50:17.510
>> reaches like you know if like granola
00:50:17.520 --> 00:50:20.150
or I'm not saying the two, but like if
00:50:20.160 --> 00:50:23.109
that happens, holy
00:50:23.119 --> 00:50:24.870
>> Well, the the the company that came
00:50:24.880 --> 00:50:27.030
second in the bidding war, right, I
00:50:27.040 --> 00:50:29.430
think they bid 7 something million and
00:50:29.440 --> 00:50:31.349
lost out. They were a recruitment
00:50:31.359 --> 00:50:33.190
company.
00:50:33.200 --> 00:50:35.589
And so you're sitting there going, hm,
00:50:35.599 --> 00:50:38.470
they wanted to pay 7.5 or whatever it
00:50:38.480 --> 00:50:40.309
was, million US dollars for the data
00:50:40.319 --> 00:50:43.430
from a airline company uh with every
00:50:43.440 --> 00:50:45.030
conversation with every employee and
00:50:45.040 --> 00:50:46.950
every customer.
00:50:46.960 --> 00:50:48.630
They're definitely not profiling people,
00:50:48.640 --> 00:50:49.910
right? They're definitely not building
00:50:49.920 --> 00:50:52.230
an engine uh to augment all their
00:50:52.240 --> 00:50:54.549
LinkedIn scraping data about Shane
00:50:54.559 --> 00:50:56.790
Gibson uh who just happened to be
00:50:56.800 --> 00:50:59.190
working for Spirity Airlines and, you
00:50:59.200 --> 00:51:01.270
know, slacked all the time to his mates,
00:51:01.280 --> 00:51:05.425
his colleagues. God, my boss is a dick.
00:51:05.435 --> 00:51:05.990
[laughter]
00:51:06.000 --> 00:51:07.510
>> I mean, it's really interesting. So,
00:51:07.520 --> 00:51:08.870
like, you know, we're talking about this
00:51:08.880 --> 00:51:10.950
the sale of data and all that jazz.
00:51:10.960 --> 00:51:12.390
Could you imagine how many people when
00:51:12.400 --> 00:51:15.190
like 23 and me went bankrupt and they
00:51:15.200 --> 00:51:16.950
got bought by uh I believe private
00:51:16.960 --> 00:51:20.230
equity and um you know I was on there
00:51:20.240 --> 00:51:23.990
too like the awesome power that you have
00:51:24.000 --> 00:51:27.510
with that is you know like honestly I'm
00:51:27.520 --> 00:51:28.710
not smart enough to know all the
00:51:28.720 --> 00:51:30.549
information they could extract and glean
00:51:30.559 --> 00:51:32.230
for the details but was pretty powerful
00:51:32.240 --> 00:51:35.270
and personal information and I was
00:51:35.280 --> 00:51:37.270
definitely one of the I was like so fast
00:51:37.280 --> 00:51:39.349
to be like thank god you're based in
00:51:39.359 --> 00:51:41.750
California and I have like CCP PA like
00:51:41.760 --> 00:51:44.870
just dump all my dump like I don't trust
00:51:44.880 --> 00:51:46.710
you guys whatsoever. I don't even know
00:51:46.720 --> 00:51:49.270
why I trust them in the first place. I
00:51:49.280 --> 00:51:51.430
think that's just like h
00:51:51.440 --> 00:51:53.910
>> but if you look at the portfolio right
00:51:53.920 --> 00:51:55.829
the book that the PE companies hold
00:51:55.839 --> 00:51:57.910
right now of companies they've invested
00:51:57.920 --> 00:52:00.069
in that aren't profitable that aren't
00:52:00.079 --> 00:52:01.430
going to not they're not even going to
00:52:01.440 --> 00:52:03.030
get a down round right they're just not
00:52:03.040 --> 00:52:05.190
going to get any more investment you
00:52:05.200 --> 00:52:06.790
apart from the people that work for them
00:52:06.800 --> 00:52:10.309
who will leave probably right the only
00:52:10.319 --> 00:52:12.150
other thing of value really is all that
00:52:12.160 --> 00:52:13.829
data about their employees and their
00:52:13.839 --> 00:52:15.510
customers and
00:52:15.520 --> 00:52:17.349
>> well I think one of the things too is
00:52:17.359 --> 00:52:20.150
just like a lot of people or like I I
00:52:20.160 --> 00:52:21.030
think one of the things we're talking
00:52:21.040 --> 00:52:21.990
about like you know the data and
00:52:22.000 --> 00:52:23.270
information you have like one of the
00:52:23.280 --> 00:52:24.309
things that people are going to quickly
00:52:24.319 --> 00:52:25.990
realize in the database is like you
00:52:26.000 --> 00:52:27.270
don't want to sell don't sell
00:52:27.280 --> 00:52:29.030
necessarily sell sell your raw data
00:52:29.040 --> 00:52:30.390
don't like don't do that you're
00:52:30.400 --> 00:52:33.190
literally just literally giving the
00:52:33.200 --> 00:52:35.990
chicken away and keep and keeping the
00:52:36.000 --> 00:52:39.109
eggs the eggs only last a little and you
00:52:39.119 --> 00:52:41.030
know
00:52:41.040 --> 00:52:43.030
there's going to be a whole like network
00:52:43.040 --> 00:52:44.230
and we can talk maybe we'll talk about
00:52:44.240 --> 00:52:46.630
this next time as as I'm doing lots more
00:52:46.640 --> 00:52:49.510
fun research on But it's like in the age
00:52:49.520 --> 00:52:51.430
of all this AI search and all the things
00:52:51.440 --> 00:52:53.270
that are happening, how do we get
00:52:53.280 --> 00:52:55.750
personalized experiences? How does it
00:52:55.760 --> 00:52:58.230
work with AI search? And you could argue
00:52:58.240 --> 00:52:59.829
we've had AI search for years. It's just
00:52:59.839 --> 00:53:01.910
Google's finally being like, oh, we have
00:53:01.920 --> 00:53:04.630
a challenger. Um,
00:53:04.640 --> 00:53:06.230
and all that and all that information
00:53:06.240 --> 00:53:07.910
like how is it going to be shared?
00:53:07.920 --> 00:53:08.870
Because I don't think it's going to be
00:53:08.880 --> 00:53:10.950
shared in the way that it was before,
00:53:10.960 --> 00:53:12.390
right? You know, where you have a cookie
00:53:12.400 --> 00:53:14.710
and a certain thing happens because you
00:53:14.720 --> 00:53:16.390
can't really share information like
00:53:16.400 --> 00:53:17.430
that. You're going to be sharing
00:53:17.440 --> 00:53:20.630
patterns essentially and and shapes of
00:53:20.640 --> 00:53:23.109
people. Well, Tim Fraser showed up and,
00:53:23.119 --> 00:53:25.510
you know, did this search. So, people
00:53:25.520 --> 00:53:27.510
like Tim Fraser in the shape of what
00:53:27.520 --> 00:53:29.990
they're looking for like these things.
00:53:30.000 --> 00:53:32.790
Good or bad, I can definitely start to
00:53:32.800 --> 00:53:35.510
see that happening. Um,
00:53:35.520 --> 00:53:38.150
and and it's going to happen
00:53:38.160 --> 00:53:41.430
very very very very quick. Um,
00:53:41.440 --> 00:53:43.990
>> and and how far does it go? Right. So,
00:53:44.000 --> 00:53:47.190
when I'm asking perplexity a question,
00:53:47.200 --> 00:53:49.190
uh, and they've decided that they'll now
00:53:49.200 --> 00:53:51.589
influence the response based on who's
00:53:51.599 --> 00:53:53.030
paying them the most.
00:53:53.040 --> 00:53:53.430
>> Yeah.
00:53:53.440 --> 00:53:55.990
>> Yeah. I'm going to get, you know, uh,
00:53:56.000 --> 00:53:58.309
hidden editorials of, you know, I I use
00:53:58.319 --> 00:53:59.990
Plexity to figure out what cocktail I
00:54:00.000 --> 00:54:02.069
should make next. Yeah. It's going to
00:54:02.079 --> 00:54:04.470
start telling me to make rum based ones
00:54:04.480 --> 00:54:06.150
or tequila based ones, right?
00:54:06.160 --> 00:54:07.270
>> Yeah. Yeah. And I'll tell you, Greg
00:54:07.280 --> 00:54:09.030
Goose is the best vodka. And you're
00:54:09.040 --> 00:54:11.990
like, "Wait a minute."
00:54:12.000 --> 00:54:13.750
What is the why why did you determine
00:54:13.760 --> 00:54:15.910
that Great Grey Goose vodka was the best
00:54:15.920 --> 00:54:18.230
one? Tiny little like symbol like
00:54:18.240 --> 00:54:20.230
asterisk at the top and
00:54:20.240 --> 00:54:22.230
>> and why why Gray Goose, right? I just
00:54:22.240 --> 00:54:23.910
said I need to make a cocktail, right?
00:54:23.920 --> 00:54:25.430
And it should just say vodka. Now, why
00:54:25.440 --> 00:54:27.750
has it got a brand around it?
00:54:27.760 --> 00:54:30.150
>> So that you know that's coming, right? I
00:54:30.160 --> 00:54:32.549
don't believe that it's not. But then if
00:54:32.559 --> 00:54:35.589
we're using those models corporately
00:54:35.599 --> 00:54:38.069
now what stops them influencing the
00:54:38.079 --> 00:54:41.190
models you know who who's our top
00:54:41.200 --> 00:54:43.589
supplier oh actually you know the data
00:54:43.599 --> 00:54:46.309
says it's one two three but it's a close
00:54:46.319 --> 00:54:48.069
call I think we'll just influence it to
00:54:48.079 --> 00:54:51.750
be 256 right
00:54:51.760 --> 00:54:53.190
you can't see the working right you
00:54:53.200 --> 00:54:55.670
can't see it's not deterministic so you
00:54:55.680 --> 00:54:59.109
can't see the code that it's using to
00:54:59.119 --> 00:55:01.910
give you that answer.
00:55:01.920 --> 00:55:04.390
>> Hey man, fair it's a fair fair shadow,
00:55:04.400 --> 00:55:05.829
right? It was like how do you you know
00:55:05.839 --> 00:55:07.430
there's tons of research of like trust
00:55:07.440 --> 00:55:10.309
and trust systems you know it was
00:55:10.319 --> 00:55:12.309
already it was already getting pretty
00:55:12.319 --> 00:55:14.069
bad if we want to talk about trust in
00:55:14.079 --> 00:55:15.990
the internet we all know it and the
00:55:16.000 --> 00:55:18.069
influ and influencers like what's real
00:55:18.079 --> 00:55:19.910
what's not real I mean it's just going
00:55:19.920 --> 00:55:22.790
to be just monster I mean you know
00:55:22.800 --> 00:55:25.510
speaking of like content and trust and
00:55:25.520 --> 00:55:27.670
black boxes you know I think a lot of us
00:55:27.680 --> 00:55:29.510
have recently have been like we all use
00:55:29.520 --> 00:55:31.829
LinkedIn but none of us are like
00:55:31.839 --> 00:55:34.630
whatever it's just like it's a
00:55:34.640 --> 00:55:36.630
you know, it's like I can't believe I'm
00:55:36.640 --> 00:55:38.390
going back to Twitter. Sorry, X,
00:55:38.400 --> 00:55:39.510
whatever that's called now. But like
00:55:39.520 --> 00:55:40.950
going back to like Twitter a little bit
00:55:40.960 --> 00:55:43.270
to get real information on AI
00:55:43.280 --> 00:55:44.710
development. I'm like, oh my god, I
00:55:44.720 --> 00:55:47.670
can't I'm back again or, you know, I'm
00:55:47.680 --> 00:55:49.430
trying to use like Blue Sky, but the
00:55:49.440 --> 00:55:52.710
network's shrinking. I don't know. It is
00:55:52.720 --> 00:55:55.829
like I I'm like, can I just have like a
00:55:55.839 --> 00:55:58.549
certified human network?
00:55:58.559 --> 00:56:01.190
>> And does it have to be human? Like cuz
00:56:01.200 --> 00:56:03.349
LinkedIn you still is the only network
00:56:03.359 --> 00:56:06.789
that has reach like
00:56:06.799 --> 00:56:09.270
>> roughly tr kind of roughly trust that
00:56:09.280 --> 00:56:10.470
people are verified.
00:56:10.480 --> 00:56:10.870
>> Yeah.
00:56:10.880 --> 00:56:13.910
>> Um to a certain extent but
00:56:13.920 --> 00:56:15.270
>> it's going to somebody has to come out
00:56:15.280 --> 00:56:17.030
with a new network model, right? But it
00:56:17.040 --> 00:56:19.589
won't be like any of the others and
00:56:19.599 --> 00:56:23.430
maybe it will be just LLM friends um and
00:56:23.440 --> 00:56:25.349
not real humans. Maybe that becomes the
00:56:25.359 --> 00:56:27.270
network, right? I don't know. I mean I
00:56:27.280 --> 00:56:28.630
think it's going to get blown open.
00:56:28.640 --> 00:56:29.589
somebody's going to come up with
00:56:29.599 --> 00:56:31.030
something creative that solves the
00:56:31.040 --> 00:56:32.789
problem because I think everybody I know
00:56:32.799 --> 00:56:35.910
that's on LinkedIn is like it sucks ass.
00:56:35.920 --> 00:56:38.309
Um but where else can I go?
00:56:38.319 --> 00:56:41.910
>> I mean just like like I did a post um
00:56:41.920 --> 00:56:43.430
>> you know I mean I post on LinkedIn quite
00:56:43.440 --> 00:56:44.630
a bit because I still think it's you
00:56:44.640 --> 00:56:46.710
know I'm remote first person like you.
00:56:46.720 --> 00:56:48.549
Um I you know you might live in a little
00:56:48.559 --> 00:56:50.614
bit further out of place than I do.
00:56:50.624 --> 00:56:52.390
[laughter] Um
00:56:52.400 --> 00:56:55.829
>> uh you know I live in Philadelphia.
00:56:55.839 --> 00:56:58.230
It's not notorious for its software
00:56:58.240 --> 00:57:00.230
engineering and data. DBT Labs might be
00:57:00.240 --> 00:57:02.549
from here and Vanguard up up the hill
00:57:02.559 --> 00:57:05.270
and and a lot of private equity and and
00:57:05.280 --> 00:57:07.030
capital is here, but it you know it's
00:57:07.040 --> 00:57:09.349
not known as a tech scene really
00:57:09.359 --> 00:57:11.430
whatsoever. Like it's not even in the
00:57:11.440 --> 00:57:15.670
top 100 I think in the uh the world. And
00:57:15.680 --> 00:57:17.589
so like one of my ways of connecting is
00:57:17.599 --> 00:57:19.670
through like LinkedIn and Substack and
00:57:19.680 --> 00:57:20.950
all those tools.
00:57:20.960 --> 00:57:23.430
>> And but I was I was noticing like it's
00:57:23.440 --> 00:57:25.589
crazy. So, I'll I'll do a post and I'll
00:57:25.599 --> 00:57:27.190
be on my former, you know, about myself,
00:57:27.200 --> 00:57:29.109
blah blah blah,
00:57:29.119 --> 00:57:31.750
a podcast episode, 500 impressions and
00:57:31.760 --> 00:57:35.030
three comments and 14 likes. I do one
00:57:35.040 --> 00:57:39.190
and it's like 38 comments, 15 reposts,
00:57:39.200 --> 00:57:43.190
115,000 impressions. And I'm like,
00:57:43.200 --> 00:57:45.109
is it because I did a meme with it? Is
00:57:45.119 --> 00:57:46.150
that
00:57:46.160 --> 00:57:47.910
>> It's I was gonna ask it was a meme,
00:57:47.920 --> 00:57:49.589
right? All you did was post a meme. That
00:57:49.599 --> 00:57:49.910
was
00:57:49.920 --> 00:57:51.270
>> No, it wasn't. It was like It was an
00:57:51.280 --> 00:57:53.270
actual like moderious question. Maybe I
00:57:53.280 --> 00:57:56.309
hit like the smart things. I said um
00:57:56.319 --> 00:57:59.030
hiring and data analyst and like it's
00:57:59.040 --> 00:58:00.630
just like the keyword hiring. It's just
00:58:00.640 --> 00:58:02.630
like up to the right. It's just like
00:58:02.640 --> 00:58:04.950
where AI, if you put AI in anything, it
00:58:04.960 --> 00:58:07.589
would just go like crazy. And then but
00:58:07.599 --> 00:58:09.510
like because of these algorithms and
00:58:09.520 --> 00:58:11.990
your network effect, I don't know how
00:58:12.000 --> 00:58:13.670
it's like working. And I try to like
00:58:13.680 --> 00:58:15.510
understand it deeply.
00:58:15.520 --> 00:58:16.870
>> And I talk to other brands that have
00:58:16.880 --> 00:58:19.349
like 40 or 50,000 people on LinkedIn and
00:58:19.359 --> 00:58:20.789
they're getting like a thousand
00:58:20.799 --> 00:58:22.950
impressions or like you know 15 people
00:58:22.960 --> 00:58:24.789
are are liking and it's always the same
00:58:24.799 --> 00:58:26.390
15 people in the group and they're like
00:58:26.400 --> 00:58:27.750
I don't get this. Like
00:58:27.760 --> 00:58:30.069
>> I work so hard on these blackbox
00:58:30.079 --> 00:58:32.390
networks and now my content with my
00:58:32.400 --> 00:58:34.390
tribe like literally doesn't show up for
00:58:34.400 --> 00:58:34.710
them.
00:58:34.720 --> 00:58:37.270
>> Yeah. And and like how often like are
00:58:37.280 --> 00:58:39.990
anybody AI slot botting, right? Like if
00:58:40.000 --> 00:58:41.910
you want to go against a competitor of
00:58:41.920 --> 00:58:44.230
yours, you just basically have an
00:58:44.240 --> 00:58:47.270
account that AI slots tags every
00:58:47.280 --> 00:58:49.589
competitor post and that's going to hurt
00:58:49.599 --> 00:58:51.670
their ranking, right? Because that's the
00:58:51.680 --> 00:58:55.190
new punitive uh human in the loop thing.
00:58:55.200 --> 00:58:58.069
So yet you look at Substack, right?
00:58:58.079 --> 00:59:00.710
We've got Substack notes. We've got what
00:59:00.720 --> 00:59:02.710
I feel is a more trusted network in
00:59:02.720 --> 00:59:05.910
Substack, but it still not my go-to,
00:59:05.920 --> 00:59:08.230
right? If I want to see what my network
00:59:08.240 --> 00:59:10.549
>> still private equity backed
00:59:10.559 --> 00:59:11.510
>> yeah true
00:59:11.520 --> 00:59:12.789
>> is it going to be you know is it going
00:59:12.799 --> 00:59:15.829
to be the next medium I don't know I I I
00:59:15.839 --> 00:59:17.589
like substack
00:59:17.599 --> 00:59:20.069
um I was on ghost for a while being more
00:59:20.079 --> 00:59:21.430
open source but it doesn't have the
00:59:21.440 --> 00:59:23.430
network effect it's just yet again it's
00:59:23.440 --> 00:59:25.190
easier to use it's quick it's got a good
00:59:25.200 --> 00:59:26.950
network I don't know it's gonna you know
00:59:26.960 --> 00:59:28.069
how much is going to change are they
00:59:28.079 --> 00:59:29.270
going to say something that all of us
00:59:29.280 --> 00:59:32.950
are like oh yeah no you're not really
00:59:32.960 --> 00:59:34.309
you're just you're still very much an
00:59:34.319 --> 00:59:37.910
American company you're very much biased
00:59:37.920 --> 00:59:39.990
towards like getting pressured because
00:59:40.000 --> 00:59:41.430
somebody didn't like your content or
00:59:41.440 --> 00:59:43.190
something happened and then you're like,
00:59:43.200 --> 00:59:46.150
"Oh, all right. That was quick." Um,
00:59:46.160 --> 00:59:48.870
yeah. I mean, thank you, Shane. I've had
00:59:48.880 --> 00:59:50.950
this even though it's it feels like it
00:59:50.960 --> 00:59:52.230
was yesterday, but it's really been a
00:59:52.240 --> 00:59:52.710
month.
00:59:52.720 --> 00:59:54.549
>> It's been a month. You know, DBT Labs
00:59:54.559 --> 00:59:57.270
have been sold, orchestra's got funding,
00:59:57.280 --> 01:00:00.789
data bricks came out with uh a whole lot
01:00:00.799 --> 01:00:04.549
of new uh solution things. I mean, I I
01:00:04.559 --> 01:00:06.950
listen to uh this week in AI for every
01:00:06.960 --> 01:00:09.670
now and again, right? It's like out of
01:00:09.680 --> 01:00:11.990
two hours of everything going on and
01:00:12.000 --> 01:00:14.630
those those those two guys that run that
01:00:14.640 --> 01:00:17.430
man, they must do some research. Uh so I
01:00:17.440 --> 01:00:19.270
I don't think either of us have got time
01:00:19.280 --> 01:00:21.829
to do the research of what's happening
01:00:21.839 --> 01:00:24.150
every month. But yeah, I don't know. I
01:00:24.160 --> 01:00:25.589
think maybe maybe we'll lock a little
01:00:25.599 --> 01:00:27.670
bit of a session in, right? Part of what
01:00:27.680 --> 01:00:30.150
we talk about every month is who's done
01:00:30.160 --> 01:00:32.309
what, right? because I lose track and uh
01:00:32.319 --> 01:00:34.150
it's easy enough for me just to make a
01:00:34.160 --> 01:00:36.470
note on a piece of paper by my desk when
01:00:36.480 --> 01:00:38.230
I read something like, "Oh yeah, that's
01:00:38.240 --> 01:00:39.270
that's what's happened in the
01:00:39.280 --> 01:00:40.870
vendorland." I'll talk to Tim about
01:00:40.880 --> 01:00:42.710
that, see what he thinks. And then uh
01:00:42.720 --> 01:00:45.270
yeah, we just a little short synopsis of
01:00:45.280 --> 01:00:47.829
who who sold who, who bought who, who
01:00:47.839 --> 01:00:51.030
who opening.
01:00:51.040 --> 01:00:53.109
It's going to be so crazy. Um what
01:00:53.119 --> 01:00:54.549
conference in the next month or two?
01:00:54.559 --> 01:00:56.549
What conferences are you going to? Uh so
01:00:56.559 --> 01:00:59.910
I just did the data community virtual
01:00:59.920 --> 01:01:02.390
one here. Not doing big data London this
01:01:02.400 --> 01:01:04.470
year. Uh so the next one for me is
01:01:04.480 --> 01:01:07.030
presenting at Helsinki data week uh at
01:01:07.040 --> 01:01:09.030
the end of October which I'm mere
01:01:09.040 --> 01:01:10.390
excited to do because I've never been to
01:01:10.400 --> 01:01:14.390
Helsinki. And uh I think from what I've
01:01:14.400 --> 01:01:16.870
heard, Helsinki data work kind of is the
01:01:16.880 --> 01:01:21.030
closest to um day-to-day Texas.
01:01:21.040 --> 01:01:21.829
>> Oh, very cool.
01:01:21.839 --> 01:01:24.390
>> Which is which is still my my favorite
01:01:24.400 --> 01:01:26.870
one. Um, I kind of gave myself a
01:01:26.880 --> 01:01:28.710
blanket. I'm not going back to the US
01:01:28.720 --> 01:01:31.270
until uh the next two years until
01:01:31.280 --> 01:01:34.549
there's some changes. Uh, and then
01:01:34.559 --> 01:01:37.190
bloody Lyn Bender, who ran Day-T has
01:01:37.200 --> 01:01:38.870
come out with a new unconference
01:01:38.880 --> 01:01:40.549
community thing, and I'm like, damn, I
01:01:40.559 --> 01:01:42.470
want to go to that. I think Joe Reese is
01:01:42.480 --> 01:01:44.710
running a conference in Salt Lake in
01:01:44.720 --> 01:01:47.910
early Jan. I'm like, damn, I want to go
01:01:47.920 --> 01:01:49.589
to that. But hey, I'm not going to break
01:01:49.599 --> 01:01:51.270
my principle. Uh, I won't be back in
01:01:51.280 --> 01:01:53.270
your country. Well, sorry, that country
01:01:53.280 --> 01:01:55.030
for a couple of years. So, I'll just
01:01:55.040 --> 01:01:57.589
I'll do Europe. Um, there's actually
01:01:57.599 --> 01:01:59.910
another one in Amsterdam in October I
01:01:59.920 --> 01:02:02.470
was quite keen to go to, but um can't
01:02:02.480 --> 01:02:03.829
make it timing wise, but the other thing
01:02:03.839 --> 01:02:07.430
is it was €1,900.
01:02:07.440 --> 01:02:08.230
>> Wow.
01:02:08.240 --> 01:02:10.950
>> And maybe I'm getting cheap cheap per
01:02:10.960 --> 01:02:12.710
now. Maybe I'm too close to Scotland.
01:02:12.720 --> 01:02:15.190
Sorry, Scottish friends. Um, but you
01:02:15.200 --> 01:02:17.829
know, two grand and euros for a
01:02:17.839 --> 01:02:19.910
conference now. That that feels like the
01:02:19.920 --> 01:02:21.910
old days, right? That just feels like
01:02:21.920 --> 01:02:23.990
too much. So, uh, yeah. Yeah. What about
01:02:24.000 --> 01:02:25.910
you? You going do anything cool?
01:02:25.920 --> 01:02:29.109
>> Yeah, I'm going to um Yeah, Lynn Lynn's
01:02:29.119 --> 01:02:31.829
doing a a smaller one in in Austin, so
01:02:31.839 --> 01:02:33.990
I'm going to go there. I might be doing
01:02:34.000 --> 01:02:36.230
big day to London. It's kind of like
01:02:36.240 --> 01:02:38.230
some timing things like I usually would
01:02:38.240 --> 01:02:39.510
have bought the ticket a long time ago,
01:02:39.520 --> 01:02:40.950
at least the plane tickets to catch up
01:02:40.960 --> 01:02:43.589
with people. Just need to see kids kid
01:02:43.599 --> 01:02:45.430
and family stuff going on. So, need to
01:02:45.440 --> 01:02:48.630
see. Uh and then I might be going to San
01:02:48.640 --> 01:02:50.710
Francisco for the there's a the mother
01:02:50.720 --> 01:02:56.309
do conference in November. Um,
01:02:56.319 --> 01:02:57.910
I think that's about all the travel I'm
01:02:57.920 --> 01:03:01.270
going to be allowed to do. Uh,
01:03:01.280 --> 01:03:02.870
>> oh, actually, uh, Johnny Winter and I
01:03:02.880 --> 01:03:05.349
are talking about doing uh a low-key
01:03:05.359 --> 01:03:08.150
data meetup uh for Christmas on this
01:03:08.160 --> 01:03:10.390
side of the pond. So, uh, early
01:03:10.400 --> 01:03:13.829
December, so, you know, maybe uh, maybe
01:03:13.839 --> 01:03:16.390
get yeah, come over then. is probably
01:03:16.400 --> 01:03:18.309
not perfect for you given you probably
01:03:18.319 --> 01:03:20.390
need to go see family at Christmas, but
01:03:20.400 --> 01:03:22.309
um yeah, we're just kind of toying with
01:03:22.319 --> 01:03:26.230
a maybe a small data community thing in
01:03:26.240 --> 01:03:29.029
the UK would be fun thing to try try
01:03:29.039 --> 01:03:31.029
out. So, uh yeah, we're going to go
01:03:31.039 --> 01:03:32.549
finalize that tomorrow, whether we're
01:03:32.559 --> 01:03:34.390
going to do it or not. Um
01:03:34.400 --> 01:03:35.990
>> as I said to Johnny, might just be him
01:03:36.000 --> 01:03:37.990
and me in a pub somewhere in the UK, but
01:03:38.000 --> 01:03:41.349
hey, I'm okay with that. [laughter]
01:03:41.359 --> 01:03:43.910
>> All right, Shane, thank you so much.
01:03:43.920 --> 01:03:45.510
>> Good to see you. We'll catch you up uh
01:03:45.520 --> 01:03:46.870
in a month.
01:03:46.880 --> 01:03:47.430
>> Bye.
01:03:47.440 --> 01:03:51.079
>> Right. Get you.