Xata

1
00:00:05,049 --> 00:00:08,449
Michael: Hello and welcome to PostgresFM,
a weekly show about all things PostgreSQL.

2
00:00:08,529 --> 00:00:11,320
I am Michael, founder of pgMustard,
and I'm joined as always by

3
00:00:11,320 --> 00:00:13,140
Nik, founder of Postgres.ai.

4
00:00:13,140 --> 00:00:13,160
Hey, Nik

5
00:00:13,718 --> 00:00:14,328
Nik: Well, Michael

6
00:00:14,908 --> 00:00:17,823
Michael: Hello, and we have
a special guest today, Tudor

7
00:00:17,903 --> 00:00:20,561
Golobenco from Xata, CTO there.

8
00:00:20,881 --> 00:00:21,921
Really nice to have you, Tudor.

9
00:00:21,921 --> 00:00:22,691
Thanks for joining us

10
00:00:23,133 --> 00:00:24,553
Tudor: hi Michael, hi Nick, hi everyone.

11
00:00:24,693 --> 00:00:26,283
Thank you so much for having me

12
00:00:27,020 --> 00:00:28,260
Michael: It is our pleasure.

13
00:00:28,350 --> 00:00:31,370
So to kick us off, I wonder if
you could give us a bit of the

14
00:00:31,400 --> 00:00:35,320
background story of Xata and take
us through to where we are today

15
00:00:36,095 --> 00:00:40,377
Tudor: Yeah, so I will actually start
with what it is today because we did a

16
00:00:40,407 --> 00:00:45,747
change once, like a pivot or like a--
I call it more like a technical pivot.

17
00:00:46,227 --> 00:00:53,696
So it's a Postgres platform which has
as differentiating features a very fast

18
00:00:53,726 --> 00:00:59,826
copy-on-write branching, so it can create
a branching in a matter of seconds, even

19
00:00:59,836 --> 00:01:07,256
for databases that, that are terabytes in
sizes, and scale to zero which means that

20
00:01:07,806 --> 00:01:12,926
if a database is inactive, the compute
part shuts down, and then on the next

21
00:01:12,936 --> 00:01:15,286
connection, it automatically wakes up.

22
00:01:15,316 --> 00:01:17,166
And this wake up is very fast.

23
00:01:17,176 --> 00:01:18,726
We're targeting one second.

24
00:01:19,316 --> 00:01:25,556
So that makes it very convenient,
especially for developer workflows.

25
00:01:25,556 --> 00:01:30,876
So these non-production use cases, things
like if you want a Postgres branch for

26
00:01:30,876 --> 00:01:37,006
each pull request or if you want to create
an, sort of an ephemeral database for each

27
00:01:37,056 --> 00:01:40,176
agent run or for your agent and so on.

28
00:01:40,786 --> 00:01:43,556
Now, these are our more common use cases.

29
00:01:44,276 --> 00:01:48,606
But on the Xata platform we can and
do run production workloads as well.

30
00:01:48,606 --> 00:01:52,933
So the differentiating features are
on the developer experience if you

31
00:01:52,933 --> 00:01:57,243
want, but you can use it as a normal
Postgres hosting provider as well.

32
00:01:57,973 --> 00:02:00,183
So that's what Xata is today.

33
00:02:00,303 --> 00:02:02,363
It didn't really start like that.

34
00:02:02,413 --> 00:02:09,063
The initial idea five years ago my
partner Monica Sarbu, she wanted to do

35
00:02:09,063 --> 00:02:11,473
a database that's really easy to use.

36
00:02:11,473 --> 00:02:15,933
It was a lot of inspiration from Airbase
or spreadsheets and because they were

37
00:02:15,933 --> 00:02:22,053
like like developer using Airbase as a
database or Google Sheets as a database.

38
00:02:22,453 --> 00:02:27,693
So the initial idea was to make
one that's as easy as to use as a

39
00:02:27,703 --> 00:02:31,503
spreadsheet but that scales up so you
don't have to switch at some point.

40
00:02:31,503 --> 00:02:33,553
So having features from both.

41
00:02:34,113 --> 00:02:39,133
And we did that for a while
with some moderate success.

42
00:02:39,243 --> 00:02:44,383
But at some point, we decided
that it's pretty hard to build a

43
00:02:44,443 --> 00:02:47,443
long-term business on that audience.

44
00:02:47,493 --> 00:02:51,503
And it n-never really took
off as a primary database

45
00:02:51,513 --> 00:02:53,993
for, like for more production

46
00:02:54,407 --> 00:02:54,757
Nik: This…

47
00:02:55,133 --> 00:02:57,163
Tudor: use as teams and things like this.

48
00:02:57,217 --> 00:02:59,367
Nik: Apologies for interrupting,
because you're packing a lot of

49
00:02:59,377 --> 00:03:01,573
stuff in one a very short time.

50
00:03:01,943 --> 00:03:05,953
I just wanted to pause a little bit here
and it- this reminds me a bit IO, right?

51
00:03:05,963 --> 00:03:08,833
Th- they also like drag and drop CSV file.

52
00:03:09,173 --> 00:03:13,473
You don't think about schema or
anything, but it's Postgres behind it.

53
00:03:13,683 --> 00:03:16,123
So you had similar thing, right?

54
00:03:16,655 --> 00:03:19,605
Tudor: Yeah, but p.io
were actual Postgres.

55
00:03:20,055 --> 00:03:23,635
Ours, we were actually hiding
Postgres behind our own API.

56
00:03:23,635 --> 00:03:29,065
So it was a REST API that you
interacted with data a bit like

57
00:03:29,095 --> 00:03:31,315
PostgREST or something like this.

58
00:03:31,865 --> 00:03:36,445
Nicer, I would say, but but
that, that type of interfaces.

59
00:03:36,445 --> 00:03:36,835
Yeah

60
00:03:37,239 --> 00:03:40,529
Nik: So it's I like this idea
a lot but Bit.io was closed.

61
00:03:40,529 --> 00:03:43,929
I remember like a very short notice,
like two weeks notice or something.

62
00:03:43,929 --> 00:03:46,219
People were like outraged.

63
00:03:46,229 --> 00:03:51,216
That was yet another not fully Postgres
platform, but I have something,

64
00:03:51,226 --> 00:03:54,146
like I store my data somewhere and
but then it's closed and I have

65
00:03:54,156 --> 00:03:55,686
very short notice to grab my data.

66
00:03:55,836 --> 00:03:59,096
And you are telling us right now that
also this model didn't work well, right?

67
00:04:00,176 --> 00:04:00,486
In terms of-

68
00:04:00,524 --> 00:04:01,124
Tudor: yeah.

69
00:04:01,794 --> 00:04:05,454
Yeah, like there's definitely
companies that do-- that address

70
00:04:05,494 --> 00:04:07,464
this sort of audience successfully.

71
00:04:07,834 --> 00:04:10,734
Like I'm thinking of
Supabase, for example, right?

72
00:04:11,044 --> 00:04:16,234
But they have like millions of developers
and such, and out of those millions,

73
00:04:16,234 --> 00:04:22,004
you're going to have a small minority
that will become large databases, so

74
00:04:22,004 --> 00:04:24,044
they will pay a lot and so on, right?

75
00:04:24,364 --> 00:04:26,804
And we had a few tens of thousand.

76
00:04:26,854 --> 00:04:29,914
I think at the peak we had 50,000 users.

77
00:04:30,804 --> 00:04:35,884
Turns out of those, there's
not enough paying customers

78
00:04:36,034 --> 00:04:37,644
to sustain such a business.

79
00:04:37,694 --> 00:04:38,804
It didn't work for

80
00:04:38,890 --> 00:04:39,560
Nik: Thank you for sharing.

81
00:04:39,560 --> 00:04:40,760
This is super interesting.

82
00:04:40,930 --> 00:04:43,050
50,000 it's actually a lot.

83
00:04:43,570 --> 00:04:43,830
But

84
00:04:44,010 --> 00:04:44,250
Tudor: yeah.

85
00:04:44,270 --> 00:04:45,170
It was quite a lot.

86
00:04:45,330 --> 00:04:48,550
We had the growth in the number
of users, but that didn't

87
00:04:48,550 --> 00:04:50,670
reflect in the growth in revenue.

88
00:04:50,968 --> 00:04:52,878
Nik: Many of them are not paying.

89
00:04:53,128 --> 00:04:53,428
Most

90
00:04:53,530 --> 00:04:53,980
Tudor: Yeah.

91
00:04:54,488 --> 00:04:54,758
Nik: Yeah

92
00:04:55,636 --> 00:04:56,536
I feel you well.

93
00:04:56,536 --> 00:04:59,866
I understand this problem also, like
from different kinds of businesses.

94
00:04:59,866 --> 00:05:00,146
Yeah.

95
00:05:00,405 --> 00:05:02,435
It's hard to find business model on this.

96
00:05:03,425 --> 00:05:04,685
Unless you're Supabase.

97
00:05:04,685 --> 00:05:05,725
This is a good comparison.

98
00:05:05,725 --> 00:05:06,415
Thank you so much.

99
00:05:06,425 --> 00:05:06,765
It's super.

100
00:05:06,855 --> 00:05:11,365
Like before th-this recording I
said we were, we're going to focus

101
00:05:11,365 --> 00:05:14,785
on technical stuff, but the start
is very different than what…

102
00:05:15,655 --> 00:05:16,315
and I like it.

103
00:05:16,315 --> 00:05:16,335
Yeah.

104
00:05:16,645 --> 00:05:17,425
Thank you so much.

105
00:05:18,190 --> 00:05:21,490
Michael: I think it's really useful
context to then dive into the technical

106
00:05:21,490 --> 00:05:26,610
side because I remember one of your
posts that I thought was really good

107
00:05:26,720 --> 00:05:29,912
about the architecture which I guess
might have been the architecture all

108
00:05:29,932 --> 00:05:34,222
along, but it explained how you could
offer a free plan and how the free

109
00:05:34,242 --> 00:05:38,002
plan didn't actually cost you very much
because of your architectural choices.

110
00:05:38,382 --> 00:05:42,282
And that, that was a really nice post
from the business perspective but also

111
00:05:42,282 --> 00:05:45,312
from a technical perspective, like
understanding how those trade-offs

112
00:05:45,312 --> 00:05:46,732
actually help you as a business.

113
00:05:47,012 --> 00:05:49,962
Tudor: Yeah, we-- So on the new
platform, we don't have a free tier.

114
00:05:49,966 --> 00:05:53,756
this is again, the business side, and
our experience with the old platform.

115
00:05:53,766 --> 00:05:56,146
So we call the old platform Zeta Lite.

116
00:05:56,156 --> 00:05:57,216
It's now closed.

117
00:05:57,266 --> 00:06:01,316
We migrated the users that we
could and so on, but it's now

118
00:06:01,316 --> 00:06:02,826
closed, so now it's only Zeta

119
00:06:03,040 --> 00:06:08,750
but the old platform had a had a free
tier and when doing the new platform, we

120
00:06:08,750 --> 00:06:11,180
were not very eager to add another one.

121
00:06:11,290 --> 00:06:15,660
Although with our current architecture,
we can do it again because we have

122
00:06:15,660 --> 00:06:19,080
this scale to zero, which is also very
efficient from that point of view.

123
00:06:19,800 --> 00:06:24,670
But we're not very eager because it adds a
lot of noise and like I was saying earlier

124
00:06:24,690 --> 00:06:27,760
you have some growth, but as long as…

125
00:06:27,830 --> 00:06:31,230
If that doesn't really result
in in the type of business that

126
00:06:31,230 --> 00:06:35,000
you want, you get a bunch of
feedback that's not really useful.

127
00:06:35,280 --> 00:06:38,910
And it confuses things a little
bit more on the business side.

128
00:06:39,010 --> 00:06:41,800
We might come back to that
decision and maybe eventually

129
00:06:41,800 --> 00:06:44,890
we'll add again a free tier plan,
but currently we don't have one.

130
00:06:45,926 --> 00:06:49,306
Nik: Yeah, let's shift to more
technical stuff because I know you

131
00:06:49,306 --> 00:06:53,096
use CloudNativePG, you have branching,
you have very interesting open

132
00:06:53,096 --> 00:06:56,506
source tools like pgStream developed
and others like pgRole, right?

133
00:06:56,992 --> 00:07:00,292
I'm very curious to talk
with you about any of them.

134
00:07:01,266 --> 00:07:04,426
Tudor: Yeah, maybe we
start at the Xata Store.

135
00:07:04,636 --> 00:07:07,926
It's our storage engine that
makes a lot of this possible.

136
00:07:08,256 --> 00:07:11,976
And then we can go about how
we use CloudNativePG to get

137
00:07:12,026 --> 00:07:13,736
the very fast wake-up times.

138
00:07:13,746 --> 00:07:15,196
That's also, I think, interesting.

139
00:07:15,576 --> 00:07:17,296
Nik: So copy on write and branching,

140
00:07:18,036 --> 00:07:18,106
right?

141
00:07:18,686 --> 00:07:19,146
Tudor: yeah.

142
00:07:19,367 --> 00:07:24,557
so we do copy-on-write branching, but
we do it at the underlying volume level.

143
00:07:24,757 --> 00:07:28,997
So because our feature set is
pretty similar with Neon, right?

144
00:07:28,997 --> 00:07:32,507
Also like copy-on-write branching
and then scale to zero with

145
00:07:32,507 --> 00:07:34,027
fast wake-up times, these

146
00:07:34,105 --> 00:07:36,045
Nik: decouple storage and compute, right?

147
00:07:36,647 --> 00:07:37,397
Tudor: Yeah, the

148
00:07:37,485 --> 00:07:40,225
Nik: And you re- and you recently
made everything open source,

149
00:07:40,705 --> 00:07:42,805
Apache 2.0 or no, or not everything

150
00:07:42,861 --> 00:07:47,821
Tudor: it not quite everything, but
enough to run the whole platform yourself.

151
00:07:48,341 --> 00:07:53,751
But it won't scale to the same levels that
we scale our own platform, and you won't

152
00:07:53,751 --> 00:07:56,891
get as fast times as with our platform.

153
00:07:56,891 --> 00:08:01,151
I can go into more details about
exactly why, but we have op-open

154
00:08:01,151 --> 00:08:04,831
source enough to be, to be running
it yourself and to be useful.

155
00:08:05,391 --> 00:08:07,641
But it's not quite everything
open source in there

156
00:08:08,961 --> 00:08:09,261
Nik: Cool.

157
00:08:09,291 --> 00:08:13,471
So if we talk about storage,
like what's interesting there

158
00:08:13,708 --> 00:08:17,048
Tudor: Yeah, so Zeta Store, we call
it like the Zeta Store and it's-- this

159
00:08:17,048 --> 00:08:19,218
is like one of the proprietary pieces.

160
00:08:19,938 --> 00:08:22,898
And we have quite a
long history into that.

161
00:08:22,998 --> 00:08:28,108
We tried a bunch of products and
Kubernetes tools for managing

162
00:08:28,138 --> 00:08:29,928
storage, the storage layer.

163
00:08:30,598 --> 00:08:35,368
As in the end, we decided to build
our own, but also use a bunch of ZFS

164
00:08:35,368 --> 00:08:37,308
functionality, so it's based on that.

165
00:08:37,798 --> 00:08:42,648
The way it works is, and I'm not sure
how familiar you are with ZFS, it's

166
00:08:43,022 --> 00:08:44,412
Nik: let me interrupt you here.

167
00:08:44,982 --> 00:08:49,052
my company, Postgres.ai, has Database
Lab Engine, which implemented

168
00:08:49,392 --> 00:08:54,192
database branching in 2019 before
Neon and before Xata, right?

169
00:08:54,252 --> 00:08:57,612
And I have a lot of questions related
to your experience with database

170
00:08:57,612 --> 00:09:01,502
branching and how you convince
users that it's really needed.

171
00:09:01,742 --> 00:09:05,582
Because we do it seven years and
some of them are solved very easily,

172
00:09:05,582 --> 00:09:08,812
but some of them still struggle
and are like, "Oh, production data,

173
00:09:08,842 --> 00:09:11,232
we need to change it," and so on.

174
00:09:11,242 --> 00:09:15,382
Let's talk about this separately,
but I just to give you context

175
00:09:15,442 --> 00:09:16,542
that you probably don't have.

176
00:09:17,482 --> 00:09:17,922
Tudor: yeah.

177
00:09:17,922 --> 00:09:17,932
Yeah

178
00:09:18,232 --> 00:09:19,262
Nik: And it's on ZFS.

179
00:09:19,370 --> 00:09:19,380
Yeah.

180
00:09:19,926 --> 00:09:22,006
Tudor: Oh, are you using ZFS?

181
00:09:22,006 --> 00:09:24,056
I thought you are using BTRFS.

182
00:09:24,540 --> 00:09:25,096
Uh, no

183
00:09:25,096 --> 00:09:27,606
Nik: it was not stable when
we started, and I don't know

184
00:09:27,660 --> 00:09:29,160
Tudor: Gotcha.

185
00:09:29,160 --> 00:09:29,580
Okay.

186
00:09:29,620 --> 00:09:30,890
I was actually remembering

187
00:09:30,900 --> 00:09:31,360
wrong then

188
00:09:31,716 --> 00:09:32,766
Nik: It was actually interesting.

189
00:09:32,766 --> 00:09:36,826
When I created it first, it was out
came out of ex-experience with GitLab.

190
00:09:37,326 --> 00:09:41,176
We just needed to give backend
engineers capabilities to experiment

191
00:09:41,216 --> 00:09:47,256
and see ex-explain plans not touching
a few database experts, right?

192
00:09:47,256 --> 00:09:48,216
And it's, it was great.

193
00:09:48,216 --> 00:09:48,916
It worked great.

194
00:09:48,926 --> 00:09:54,136
And then we scaled it and the problem
always was that first of all, production

195
00:09:54,136 --> 00:09:56,716
data how to protect this, right?

196
00:09:56,866 --> 00:10:02,526
And also when Neon started working
with branching, it's very different.

197
00:10:02,526 --> 00:10:07,806
So we do it very locally, and we achieve
static cost for all clones and branches.

198
00:10:07,806 --> 00:10:11,486
So you have, you pay once and you
have everything, like mu-multi-tenancy

199
00:10:11,486 --> 00:10:13,186
in one virtual machine, basically.

200
00:10:13,756 --> 00:10:17,906
You have very different model, so I'm very
curious how it works and what's your…

201
00:10:18,416 --> 00:10:20,116
what do you observe how people use it?

202
00:10:20,556 --> 00:10:25,456
And because my honest take, I
spent a lot of time in this area,

203
00:10:25,836 --> 00:10:29,686
market is still not understanding
that branching is super powerful.

204
00:10:29,996 --> 00:10:30,966
And I hope AI will

205
00:10:30,974 --> 00:10:31,354
Tudor: I think--

206
00:10:31,566 --> 00:10:32,136
Nik: but yeah.

207
00:10:32,786 --> 00:10:36,806
Tudor: I think it's it's
it's growing pretty nicely.

208
00:10:37,286 --> 00:10:42,216
But yeah the, like the use cases that
we see most often are branch per PR.

209
00:10:42,886 --> 00:10:47,859
So they do PR and have some
sort of preview environments.

210
00:10:48,039 --> 00:10:52,459
This could be, like Vercel or
their own stuff or different

211
00:10:52,459 --> 00:10:58,899
other platforms for ephemeral
environments or preview environments.

212
00:10:59,459 --> 00:11:04,369
But then they want also the database to
be part of that with the data, right?

213
00:11:04,439 --> 00:11:07,889
So then a platform like
ours makes that possible.

214
00:11:07,929 --> 00:11:10,279
You have the database with the data.

215
00:11:10,769 --> 00:11:13,489
It doesn't have to be the
exact production data.

216
00:11:13,899 --> 00:11:19,089
And the way we typically recommend
people to do is we tell them,

217
00:11:19,089 --> 00:11:20,909
"Look, keep production where it is."

218
00:11:20,949 --> 00:11:22,199
Let's say it's RDS.

219
00:11:22,769 --> 00:11:25,399
And then we have, this is
why we develop pgStream.

220
00:11:26,143 --> 00:11:28,303
We create a replica of it.

221
00:11:28,473 --> 00:11:31,603
And pgStream can copy the data once.

222
00:11:31,793 --> 00:11:35,993
So for small databases, we recommend
people to set up a nightly job

223
00:11:36,123 --> 00:11:38,903
or a weekly job or something
like this because it's simpler.

224
00:11:39,583 --> 00:11:44,563
For large databases, pgStream can
also do the initial snapshot and then

225
00:11:44,563 --> 00:11:47,403
keep it up via a logical replication.

226
00:11:48,185 --> 00:11:48,205
Nik: And DDL?

227
00:11:48,225 --> 00:11:48,765
And

228
00:11:50,021 --> 00:11:51,001
Tudor: and DDL,

229
00:11:51,413 --> 00:11:52,223
Nik: you solve DDL.

230
00:11:52,223 --> 00:11:56,813
I looked at it briefly, and I'm
very curious, like, how… if

231
00:11:56,823 --> 00:11:59,293
you do it and what's your scale?

232
00:11:59,303 --> 00:12:00,603
Is it working really well?

233
00:12:00,623 --> 00:12:03,633
If it's working really well,
why it doesn't go to core

234
00:12:03,633 --> 00:12:05,423
of Postgres in that form?

235
00:12:06,289 --> 00:12:06,549
Tudor: Yeah.

236
00:12:06,569 --> 00:12:10,447
I think there's some work in
progress to make it in in, In co-

237
00:12:10,799 --> 00:12:15,239
Nik: it-- there are several, there,
there are several ways, and if you

238
00:12:15,239 --> 00:12:18,715
check recent activities it's not
super progress-- it's not progressing

239
00:12:19,267 --> 00:12:20,437
Tudor: And not progressing, yeah.

240
00:12:20,541 --> 00:12:25,191
actually a little bit of
backstory on, on pgStream.

241
00:12:25,831 --> 00:12:30,981
We were doing in the old product,
so Zetalight, we were doing, we were

242
00:12:30,981 --> 00:12:34,941
offering this REST API, and then
most of it was powered by Postgres.

243
00:12:35,531 --> 00:12:41,801
But we also had a free text search
with, BM25 and relevance and so on.

244
00:12:42,181 --> 00:12:44,801
And that was actually
searched from Elasticsearch.

245
00:12:44,811 --> 00:12:48,711
So we wanted to have this replication
between Postgres and Elasticsearch.

246
00:12:49,561 --> 00:12:54,638
And we could do, because we had like our
own API for modifying the schema and such,

247
00:12:54,638 --> 00:12:56,778
you could do it in both at the same time.

248
00:12:57,558 --> 00:13:04,568
But that's also tricky because if
you replicate data but the schema

249
00:13:04,568 --> 00:13:08,618
changes come out of order, then
again you have a problem because

250
00:13:08,618 --> 00:13:12,908
it's not going to fit the schema, the
classical CDC type of problems, right?

251
00:13:13,328 --> 00:13:16,458
So it's actually better if the
schema changes are in the same

252
00:13:16,458 --> 00:13:18,968
replication stream as the actual data.

253
00:13:19,258 --> 00:13:20,328
It's better that way.

254
00:13:20,768 --> 00:13:24,708
So that's why we did this event
hook approach, listening for schema

255
00:13:24,708 --> 00:13:27,258
changes and then apply them that.

256
00:13:27,738 --> 00:13:31,898
There, with Zetalight we had this
advantage that we, you couldn't

257
00:13:31,898 --> 00:13:34,358
do arbitrary schemas in Postgres.

258
00:13:34,358 --> 00:13:38,798
It it was a particular set
of data types and so on.

259
00:13:38,808 --> 00:13:42,778
It was a little bit more into
our control, so we knew we could

260
00:13:42,788 --> 00:13:46,358
test all possible schema changes
and make sure they replicate

261
00:13:46,358 --> 00:13:48,258
correctly and these sort of things.

262
00:13:48,878 --> 00:13:51,808
But then we generalized this
approach a little bit more.

263
00:13:52,186 --> 00:13:52,436
Nik: Yeah.

264
00:13:52,436 --> 00:13:53,286
So that's interesting.

265
00:13:53,286 --> 00:13:56,026
So you have limited set of
what's happening, right?

266
00:13:56,776 --> 00:14:00,176
And then using event triggers, you, Yeah.

267
00:14:00,286 --> 00:14:01,576
Tudor: Yeah, that's our approach.

268
00:14:01,576 --> 00:14:04,376
And of course, it needs-- we need
to make sure it's in the right

269
00:14:04,556 --> 00:14:06,526
place in the replication stream.

270
00:14:07,506 --> 00:14:12,426
And the way we do it is we write
in a kind of, in a shadow table.

271
00:14:12,436 --> 00:14:17,446
We write what has changed and which
w- with what altered statement that

272
00:14:17,456 --> 00:14:22,606
happened, and that gets put into
the logical replication stream.

273
00:14:23,066 --> 00:14:27,056
And then pgStream reads it
from there and applies it on

274
00:14:27,056 --> 00:14:28,846
the target in the right order.

275
00:14:28,896 --> 00:14:30,276
That's the general approach.

276
00:14:30,398 --> 00:14:33,468
Nik: And I'm very curious at what
scale it is working and are you

277
00:14:33,478 --> 00:14:37,458
fully satisfied or there are some
like plans to improve or something?

278
00:14:37,654 --> 00:14:38,084
Tudor: Yeah.

279
00:14:38,124 --> 00:14:43,274
We have people using pgStream at
like four or five terabytes of data.

280
00:14:43,714 --> 00:14:48,024
And the initial snapshot takes like
24 hours or something like this.

281
00:14:48,024 --> 00:14:51,864
And then And it b- that depends on a
number of factors or something like

282
00:14:51,864 --> 00:14:53,894
this, and then it's kept up to date.

283
00:14:53,924 --> 00:14:58,054
But then it, like it, it's from
company to company, and sometimes

284
00:14:58,054 --> 00:15:01,864
we have to work on pgStream a little
bit , to make it work for them.

285
00:15:01,944 --> 00:15:06,584
But in the meantime, we had we it
had enough battles, let's say that

286
00:15:06,624 --> 00:15:10,584
now we're feeling like, it covers
already the majority of, uh, of,

287
00:15:10,634 --> 00:15:11,764
cases pretty well.

288
00:15:12,008 --> 00:15:12,148
Nik: Yeah.

289
00:15:12,288 --> 00:15:13,368
Battle proven already, right?

290
00:15:13,964 --> 00:15:14,964
Tudor: Yeah, to some degree.

291
00:15:15,014 --> 00:15:17,224
Like there, there are still problems.

292
00:15:17,234 --> 00:15:19,394
The CDC logical replication

293
00:15:19,444 --> 00:15:19,824
Nik: oh yes

294
00:15:19,886 --> 00:15:24,406
Tudor: is still quite a pla- I w- I would
love if DDL would be supported in core

295
00:15:24,942 --> 00:15:28,632
Nik: everyone waits for it s- but
it's super not trivial as I saw

296
00:15:28,662 --> 00:15:30,622
last time when I checked this topic.

297
00:15:31,252 --> 00:15:31,812
That's cool.

298
00:15:31,822 --> 00:15:35,752
I like s-small non-non-technical
side note side question.

299
00:15:36,272 --> 00:15:39,002
These pieces are… Like you
mentioned some proprietary stuff

300
00:15:39,002 --> 00:15:41,082
about p- pgroll and pgstream

301
00:15:41,152 --> 00:15:42,622
Tudor: Yeah, these are fully open source,

302
00:15:42,692 --> 00:15:44,952
Nik: yeah, that's and Apache 2.0, right?

303
00:15:45,134 --> 00:15:45,534
Tudor: Yes.

304
00:15:45,644 --> 00:15:45,984
yeah.

305
00:15:46,684 --> 00:15:50,164
Yeah, and especially pgStream,
it's important for us to be open

306
00:15:50,164 --> 00:15:53,654
source because we recommend people
to run it in their environment

307
00:15:53,964 --> 00:15:55,884
because it also does anonymization.

308
00:15:55,884 --> 00:15:58,354
This is the, you know, like
kind of what we started from.

309
00:15:58,664 --> 00:16:03,314
It's important to do anonymization so
that when you do this copy-on-write

310
00:16:03,314 --> 00:16:08,604
branching, the data is already anonymized
and you have branches, very fast branches

311
00:16:09,664 --> 00:16:12,114
that don't contain the sensitive data.

312
00:16:12,234 --> 00:16:15,934
Like it's up to you what what
transformation rules you set and so on.

313
00:16:15,944 --> 00:16:21,784
But in many cases, it's like 80% of
the data is fine as it is, and you have

314
00:16:21,784 --> 00:16:26,534
some data that it's sensitive and you
need to transform it and make sure it's

315
00:16:26,534 --> 00:16:28,704
not in any developer copy and so on.

316
00:16:28,714 --> 00:16:33,324
And this I think is the, is the
perfect balance of things if you want.

317
00:16:33,614 --> 00:16:36,764
But it does depend on
this replication story.

318
00:16:37,054 --> 00:16:41,144
And we want to run it in their environment
so they have full control over it.

319
00:16:41,234 --> 00:16:43,124
That's why it needs to
be open source really.

320
00:16:43,916 --> 00:16:44,226
Nik: Yeah.

321
00:16:44,546 --> 00:16:45,566
this answers my question

322
00:16:46,534 --> 00:16:49,834
Michael: on the anonymization
front, I saw you acquired a company.

323
00:16:49,834 --> 00:16:51,264
I saw some blog posts I think you wrote.

324
00:16:51,264 --> 00:16:52,534
Was it beginning of this year?

325
00:16:52,594 --> 00:16:54,974
Is that what you've then
ro- rolled into this

326
00:16:55,756 --> 00:16:57,486
Tudor: Yeah, partially partially.

327
00:16:57,566 --> 00:17:02,386
The it's called Privacy Dynamics, this
company, and they were, like, really

328
00:17:02,386 --> 00:17:04,986
deep into the anonymization front.

329
00:17:05,406 --> 00:17:07,176
it's like there's levels to this stuff.

330
00:17:07,196 --> 00:17:12,256
In pgStream we already had
the anonymization, which was

331
00:17:12,306 --> 00:17:15,246
simple column rule-based.

332
00:17:15,256 --> 00:17:19,586
Like you can say, okay, this
column, you can hash it like this

333
00:17:19,866 --> 00:17:22,516
deterministically and, it's hashed.

334
00:17:23,136 --> 00:17:28,686
But then like for example, for HIPAA
compliance and so on, you need to do

335
00:17:28,686 --> 00:17:35,116
the type of anonymization where if you,
le-let's say there's patient's data and

336
00:17:35,116 --> 00:17:42,866
you have a woman that's 46 years old
and so on, and has this email address

337
00:17:42,866 --> 00:17:46,966
or something like this, and if you just
anonymize the email address, but it's

338
00:17:46,966 --> 00:17:52,126
the only woman in the data set with
those characteristics then you haven't

339
00:17:52,146 --> 00:17:56,126
really anonymized anything because
you can, like even if the name is not

340
00:17:56,126 --> 00:17:59,966
there, there's enough information,
unique information there that you

341
00:17:59,966 --> 00:18:03,616
can still work backwards what it is.

342
00:18:03,966 --> 00:18:10,206
So they doing these sort of things
where they see exactly combining the

343
00:18:10,206 --> 00:18:15,016
data set into groups and figuring
out, okay, we need to anonymize this

344
00:18:15,016 --> 00:18:17,796
data so that no group is really small.

345
00:18:17,916 --> 00:18:22,566
Every group has more than 10
data points and such, so you

346
00:18:22,566 --> 00:18:24,316
cannot find the individual.

347
00:18:24,426 --> 00:18:28,526
And this is like more like, like HIPAA
compliant sort of things and such.

348
00:18:28,946 --> 00:18:32,496
So that's not, that's also something
that's not in pgStream at the moment.

349
00:18:32,506 --> 00:18:36,826
Maybe we will roll out more of those
features in there that we set up

350
00:18:36,836 --> 00:18:41,836
more like manually for customers that
actually need this complex stuff.

351
00:18:43,232 --> 00:18:44,042
Nik: Yeah, that's great.

352
00:18:44,642 --> 00:18:47,832
So these two pieces are,
they're interesting themselves.

353
00:18:47,832 --> 00:18:50,422
They could they could be
separate episodes, honestly.

354
00:18:50,902 --> 00:18:54,612
This is what I… Yeah, because if we
dive deeper, there are so many levels.

355
00:18:54,612 --> 00:18:56,382
As I said I'm in that space as well.

356
00:18:56,842 --> 00:19:00,342
We usually use some other
tools, like we didn't build.

357
00:19:00,342 --> 00:19:02,792
And we-- I remember also
there was a company, Tonika.

358
00:19:02,792 --> 00:19:04,292
Yeah I'm curious what happened to it.

359
00:19:04,362 --> 00:19:05,232
Als-also

360
00:19:05,238 --> 00:19:06,868
Tudor: They still exist, I think, right?

361
00:19:07,122 --> 00:19:07,372
Nik: Yeah.

362
00:19:07,422 --> 00:19:10,312
So a lot of… if you look to
what they build, you understand

363
00:19:10,312 --> 00:19:11,942
that so many layers go there.

364
00:19:12,042 --> 00:19:12,402
It's like

365
00:19:12,650 --> 00:19:12,880
Tudor: yeah.

366
00:19:12,880 --> 00:19:13,520
Absolutely.

367
00:19:13,570 --> 00:19:13,920
Absolutely.

368
00:19:14,110 --> 00:19:14,440
Yeah.

369
00:19:14,530 --> 00:19:18,700
And Tonic AI I think they still exist,
but it's just proprietary and they

370
00:19:18,700 --> 00:19:23,040
open source like a simple version
of it, but it doesn't scale at all.

371
00:19:23,080 --> 00:19:23,630
So it's,

372
00:19:23,660 --> 00:19:26,570
Nik: and important to also
remember, like use cases are

373
00:19:26,570 --> 00:19:28,080
very different for branching.

374
00:19:28,080 --> 00:19:33,610
If you just want to… Like you said you
don't recommend to take production and if

375
00:19:33,610 --> 00:19:37,070
it's like simple development it's better
to have some like preview environments.

376
00:19:37,080 --> 00:19:39,890
It can be some synthetic
data, for example, right?

377
00:19:39,900 --> 00:19:41,420
You don't need to synchronize it all.

378
00:19:41,760 --> 00:19:45,879
But If you bring data for a various
like closer to production testing,

379
00:19:45,889 --> 00:19:50,429
and especially if it's re- related to
performance testing, I'm-- I, I was

380
00:19:50,429 --> 00:19:55,359
always concerned about what plans we
will be getting because if you change

381
00:19:55,359 --> 00:19:57,769
data, I stop trusting those plans.

382
00:19:58,333 --> 00:19:58,993
Tudor: Yeah, of course.

383
00:19:59,412 --> 00:20:05,302
Nik: So I usually try to convince people
to keep like red data a- as is production

384
00:20:05,302 --> 00:20:09,322
data and just limit access and consider
it as part of production and still have

385
00:20:09,342 --> 00:20:14,262
branching and test it closer to production
because in… this is the only way you can

386
00:20:14,552 --> 00:20:17,732
have really realistic performance testing.

387
00:20:17,782 --> 00:20:18,862
What you think about this?

388
00:20:19,918 --> 00:20:20,248
Tudor: Yeah.

389
00:20:20,518 --> 00:20:25,958
So there, there is this deterministic
hashing which basically means if you

390
00:20:26,808 --> 00:20:31,168
hash a value, it will always hash
to the same thing, which is also

391
00:20:31,168 --> 00:20:33,258
useful for referential integrity.

392
00:20:33,258 --> 00:20:36,488
Let's say, like if you have a foreign
key on a string column or something

393
00:20:36,488 --> 00:20:42,358
like this, then you want it to
anonymize the same in both places.

394
00:20:42,828 --> 00:20:44,888
So that helps a little bit with this.

395
00:20:44,998 --> 00:20:48,978
But yeah, with the Postgres planner,
there can be a lot of nuances around

396
00:20:49,096 --> 00:20:51,306
Nik: we, as I said, this
could be whole episode.

397
00:20:51,506 --> 00:20:55,216
If we dive deeper, like when you
say hashing, okay pgStream takes

398
00:20:55,216 --> 00:20:56,906
data from RDS, understand this.

399
00:20:57,516 --> 00:21:00,226
You have ability, like
you copy anyway, right?

400
00:21:00,516 --> 00:21:05,256
But if you talk about your own platform,
you have physical data directory,

401
00:21:05,926 --> 00:21:06,346
Tudor: You're right.

402
00:21:06,346 --> 00:21:06,746
Yes.

403
00:21:07,136 --> 00:21:07,486
Yeah

404
00:21:08,266 --> 00:21:12,936
Nik: W- when you touch it with any
update, with anything, it's better,

405
00:21:12,986 --> 00:21:16,556
like you have, like we have think
cloning branching on physical level.

406
00:21:16,996 --> 00:21:20,096
And if you start touching it, you
already will Mi- Michael will tell

407
00:21:20,096 --> 00:21:23,596
better what, what will happen with plans
because he builds pgMustard, right?

408
00:21:23,656 --> 00:21:25,716
Which is focusing only on, on the plans.

409
00:21:26,306 --> 00:21:30,236
So if you physically - change
something, you like, different

410
00:21:30,246 --> 00:21:32,986
number of blocks and so on, it's
already different plans, right?

411
00:21:33,536 --> 00:21:33,856
Tudor: Yeah

412
00:21:34,176 --> 00:21:37,086
Nik: Relpages can… reltuples
numbers will become different.

413
00:21:37,086 --> 00:21:38,496
Statistics might start different.

414
00:21:38,586 --> 00:21:44,206
Even if you don't, if you say update
some row set column equals the same

415
00:21:44,206 --> 00:21:48,296
column ID something, logically nothing
changed, but physically it changed.

416
00:21:48,296 --> 00:21:52,216
And eventually if you massive
scale, it can drift plans

417
00:21:53,090 --> 00:21:55,530
Tudor: Yeah, and logical
replication already like kind

418
00:21:55,530 --> 00:21:56,970
of throws that away because

419
00:21:57,270 --> 00:21:57,410
Nik: Right.

420
00:21:57,620 --> 00:22:02,650
But if it's, if we go back to ZFS
and thin cloning, the copy-on-write,

421
00:22:03,240 --> 00:22:06,800
I, I-- my point was always like,
let's not touch the data and we will

422
00:22:06,810 --> 00:22:08,840
have ability to have this, the exact

423
00:22:08,938 --> 00:22:11,268
Tudor: and you can you can still do that.

424
00:22:11,278 --> 00:22:14,528
You can have… like if you run
production on Xata what we typically

425
00:22:14,528 --> 00:22:20,208
recommend is have a prod project and
then something like a staging project

426
00:22:20,208 --> 00:22:23,858
or something like this, and pgStream
between them to do the anonymization.

427
00:22:24,608 --> 00:22:28,198
But then the prod project still
supports copy-on-write branching,

428
00:22:28,248 --> 00:22:30,478
and that would be like ZFS

429
00:22:31,172 --> 00:22:32,222
Clone type of thing.

430
00:22:32,222 --> 00:22:36,562
And you can use that but you would
only use this, use it when you

431
00:22:36,562 --> 00:22:39,202
actually need it for like performance,

432
00:22:39,334 --> 00:22:42,114
Nik: And like I, I see Michael
wants to ask something.

433
00:22:42,114 --> 00:22:45,794
But I will ask very last question because
I sit on this topic seven years already,

434
00:22:45,794 --> 00:22:51,084
so I like, I'm super curious to find
guys who have like also spent time there.

435
00:22:51,504 --> 00:22:52,504
Last question from me.

436
00:22:52,504 --> 00:22:53,464
Unit economics.

437
00:22:53,494 --> 00:23:00,114
If we want to build like pipelines which
will test performance plans, right?

438
00:23:00,594 --> 00:23:06,694
If we keep data in Zetta production
and build these pipelines will

439
00:23:06,754 --> 00:23:10,504
each every CI pipeline lead to
provisioning on the whole node?

440
00:23:10,894 --> 00:23:13,434
Will I be paying separately
for whole… For if I run

441
00:23:13,434 --> 00:23:17,484
10 pipe- or 100 pipelines, you…
I need to pay for every one, right?

442
00:23:18,474 --> 00:23:18,884
Or it's

443
00:23:18,954 --> 00:23:22,044
Tudor: Because we do the separation
of storage and compute, right?

444
00:23:22,044 --> 00:23:26,084
And on the storage side,
there's this thin copy, right?

445
00:23:26,084 --> 00:23:27,974
So you only pay for the difference.

446
00:23:28,014 --> 00:23:28,694
That's on the

447
00:23:28,752 --> 00:23:30,272
Nik: so it basically doesn't change.

448
00:23:30,282 --> 00:23:32,532
If I don't change data,
it's the same price.

449
00:23:32,532 --> 00:23:32,572
I

450
00:23:32,696 --> 00:23:33,236
Tudor: Right.

451
00:23:33,856 --> 00:23:36,376
With, of course, some gotchas
there as well, but at the

452
00:23:36,376 --> 00:23:37,586
high level, let's say that.

453
00:23:38,056 --> 00:23:43,446
And on the compute side they would we use
Kubernetes so that would be-- they will

454
00:23:43,446 --> 00:23:49,316
be starting on different compute nodes
potentially, but not each new branch

455
00:23:49,316 --> 00:23:51,776
will be a new node necessarily, right?

456
00:23:51,786 --> 00:23:53,756
They will fill existing nodes and such.

457
00:23:54,356 --> 00:23:58,726
And there you only pay if you use
our Xata Cloud platform because

458
00:23:58,736 --> 00:24:01,336
you also have bring your own cloud,
which is a little bit different.

459
00:24:01,616 --> 00:24:07,056
But if you use our cloud platform you only
pay for the time the branch is active.

460
00:24:07,466 --> 00:24:12,156
And with the scale to zero that that,
it's like for development it makes a

461
00:24:12,156 --> 00:24:16,796
lot of sense because it wakes up in a
second when you connect to it, right?

462
00:24:16,796 --> 00:24:19,946
And you can set like a five
minutes expiration time.

463
00:24:19,946 --> 00:24:22,736
This is like a, a lot of our customers do.

464
00:24:23,526 --> 00:24:25,166
It's like you pay almost nothing.

465
00:24:25,266 --> 00:24:26,736
I actually have a blog post.

466
00:24:26,816 --> 00:24:29,716
You can run thousand… It's
just how the math works.

467
00:24:29,716 --> 00:24:35,866
You can run a thousand branches for
one dollar basically a tenth of a cent

468
00:24:35,896 --> 00:24:38,496
each if they run for five minutes.

469
00:24:38,723 --> 00:24:39,573
Nik: yeah, that's great.

470
00:24:39,793 --> 00:24:43,213
So this is much better than what
I observe with thing cloning

471
00:24:43,213 --> 00:24:45,603
at, in Aurora and Neon, right?

472
00:24:45,643 --> 00:24:48,573
Because there is absolutely
it, it's linear scaling.

473
00:24:48,573 --> 00:24:53,253
If you wa- if you want many
pipelines in CI to test, you

474
00:24:53,283 --> 00:24:54,983
amplify your spending budgets.

475
00:24:55,063 --> 00:24:59,313
And this limits, This is super important
because this makes engineers to

476
00:24:59,313 --> 00:25:01,203
stop testing enough, and they start

477
00:25:01,483 --> 00:25:01,833
Tudor: Yeah.

478
00:25:01,833 --> 00:25:02,213
Yeah.

479
00:25:02,293 --> 00:25:05,663
And it discouraged to have a
branch for each PR, which is like

480
00:25:05,853 --> 00:25:07,333
Nik: so that's the kudos for that.

481
00:25:07,333 --> 00:25:07,973
This is cool.

482
00:25:08,003 --> 00:25:11,123
I, as I understand, it's not like a
constant price, but definitely you've

483
00:25:11,123 --> 00:25:14,563
thought about this and because this
is super important to make testing

484
00:25:14,573 --> 00:25:17,433
more like intensive and complete.

485
00:25:17,655 --> 00:25:17,765
Tudor: Yeah.

486
00:25:18,675 --> 00:25:18,975
Yeah.

487
00:25:19,295 --> 00:25:19,635
Yeah.

488
00:25:20,045 --> 00:25:20,295
Yeah.

489
00:25:20,295 --> 00:25:24,755
So definitely Neon ch-- Neon,
like you get 20 branches for free.

490
00:25:24,805 --> 00:25:26,135
You still pay the compute.

491
00:25:26,175 --> 00:25:27,275
They're not completely free.

492
00:25:27,275 --> 00:25:34,205
You would pay for the compute but above
20 branches you pay a fixed amount for

493
00:25:34,205 --> 00:25:36,355
each of them, even if they are inactive.

494
00:25:36,755 --> 00:25:37,765
So we don't do that.

495
00:25:37,825 --> 00:25:38,825
We don't charge that.

496
00:25:38,845 --> 00:25:43,285
That's why you can have a thousand
branches with only $1, which I don't

497
00:25:43,285 --> 00:25:47,015
know, maybe in a year we'll have
another call and I will say that

498
00:25:47,015 --> 00:25:48,565
does- that doesn't work economically.

499
00:25:51,545 --> 00:25:52,055
Nik: I you start to add nuances.

500
00:25:52,055 --> 00:25:53,745
I know which exactly which nuance.

501
00:25:53,745 --> 00:25:58,895
If you have some branch or clone
or like it holds snapshot, and

502
00:25:58,895 --> 00:26:02,525
if production runs away into the
future, you keep… You start paying

503
00:26:02,525 --> 00:26:04,245
for all data to be stored, right?

504
00:26:04,371 --> 00:26:04,961
Tudor: Yeah, Exactly.

505
00:26:05,195 --> 00:26:05,815
Nik: That it is.

506
00:26:05,815 --> 00:26:07,405
I know this problem very well

507
00:26:08,001 --> 00:26:12,461
Tudor: Yeah like sometimes hard to
explain because you create a branch,

508
00:26:13,081 --> 00:26:17,031
its cost is zero on the storage, and
then you delete all the data into it,

509
00:26:17,821 --> 00:26:22,751
and then the-- you would expect-- you
wouldn't expect the cost to go up, but

510
00:26:22,751 --> 00:26:26,561
that's actually what happens because now
it's different from the parent, right?

511
00:26:27,081 --> 00:26:31,011
So it's like, "Hey, why am I paying
more by deleting data," right?

512
00:26:31,041 --> 00:26:32,391
That, that can be confusing

513
00:26:32,853 --> 00:26:36,513
Nik: this is exactly what we
also ha-had and still having.

514
00:26:36,533 --> 00:26:37,563
Yeah, that's cool.

515
00:26:38,453 --> 00:26:38,713
Cool.

516
00:26:38,773 --> 00:26:38,993
Yeah.

517
00:26:39,543 --> 00:26:42,503
Yeah, Michael, I, I
apologize you had questions.

518
00:26:42,563 --> 00:26:44,833
I, I'm just super excited to talk about

519
00:26:44,879 --> 00:26:47,779
Michael: I was gonna ask more,
but actually now I'm thinking,

520
00:26:47,779 --> 00:26:52,939
I think this is great in terms
of using Xata for your developer,

521
00:26:53,418 --> 00:26:57,318
environments, let's say like
pre-production those early maybe per

522
00:26:57,348 --> 00:26:59,118
agent, per branch, that kind of thing.

523
00:26:59,618 --> 00:27:04,258
But I think you've also got some
interesting stuff on the developer

524
00:27:04,268 --> 00:27:08,928
experience of running Postgres in
production, like pgRole for example.

525
00:27:09,198 --> 00:27:12,418
But I suspect you'll have opinions
on a lot of the other things that

526
00:27:12,748 --> 00:27:16,708
are currently quite hard to do as a
developer running Postgres in production.

527
00:27:16,708 --> 00:27:19,568
So I'd, I wondered about getting
your opinion on some of those.

528
00:27:19,578 --> 00:27:23,398
So the big one for me, I think
still that's difficult for people

529
00:27:23,398 --> 00:27:25,428
to manage is major version upgrades.

530
00:27:25,858 --> 00:27:29,618
Like that, like things like that, I
wonder if you have any takes or any,

531
00:27:29,648 --> 00:27:34,098
any interesting features that most
managed services don't offer or things

532
00:27:34,108 --> 00:27:36,458
that you want to do in the future
to make things like that easier?

533
00:27:38,038 --> 00:27:42,448
Tudor: Yeah, we did put a lot of effort
into the schema changes part of it.

534
00:27:42,478 --> 00:27:48,358
It felt to us like one of the
biggest pain points in, in Postgres.

535
00:27:48,358 --> 00:27:54,948
There's a lot of gotchas there as well,
like you do a schema change and it works

536
00:27:54,948 --> 00:27:59,038
on your small database that you test with,
but then you run it in production and it

537
00:27:59,038 --> 00:28:01,138
locks the whole table for a long time.

538
00:28:01,138 --> 00:28:05,638
So then you have to do this
process and orchestrate multiple

539
00:28:05,678 --> 00:28:09,798
PRs, change the application, do
that, do the backfill, and so on.

540
00:28:10,548 --> 00:28:13,608
So that w- that's how,
like how we started pgroll.

541
00:28:13,748 --> 00:28:18,574
It's also related to the old platform,
which again, it was all about making

542
00:28:18,594 --> 00:28:21,744
the database as easy as possible to use.

543
00:28:22,009 --> 00:28:25,044
so then we had an API
to do schema changes.

544
00:28:25,214 --> 00:28:29,354
And that was not as, you couldn't
do as much as as Postgres can do.

545
00:28:29,534 --> 00:28:34,154
But everything that you did
there was guaranteed to be safe.

546
00:28:34,204 --> 00:28:36,584
You could never lock
your database and such.

547
00:28:37,624 --> 00:28:41,334
But then again, you hit the problem
that people want to do those things.

548
00:28:41,334 --> 00:28:44,344
So it's like that Postgres
functionality was not added for nothing.

549
00:28:44,344 --> 00:28:47,134
And then you end up having to
support everything, which was,

550
00:28:47,294 --> 00:28:48,754
whi- which was our problem.

551
00:28:49,264 --> 00:28:55,274
So yeah, pgroll is is like it's an attempt
to make that as as simple as possible.

552
00:28:55,674 --> 00:28:56,514
It's working.

553
00:28:56,534 --> 00:29:01,234
It also has some, some gotchas that
currently we're not super focused on

554
00:29:01,234 --> 00:29:05,384
it because we're focused primarily
on the Xata platform itself with copy

555
00:29:05,384 --> 00:29:09,254
and write branching and supporting
many agents and these sort of things.

556
00:29:09,254 --> 00:29:10,434
That's where our focus is.

557
00:29:10,444 --> 00:29:13,244
So we didn't have that much
time for pgroll lately.

558
00:29:13,964 --> 00:29:15,464
Yeah, that's wh- that's where we are.

559
00:29:15,970 --> 00:29:21,760
Nik: So now focus is to support
to give database to agents, right?

560
00:29:21,900 --> 00:29:22,530
To-- This is

561
00:29:22,560 --> 00:29:22,980
Tudor: Right.

562
00:29:23,200 --> 00:29:24,690
And that can be in two ways.

563
00:29:24,690 --> 00:29:28,340
One is during development,
like you give like I was saying

564
00:29:28,490 --> 00:29:29,530
earlier, branches per PR.

565
00:29:30,700 --> 00:29:34,660
Now with agents, you create a lot more
PRs, and we're seeing, like we're seeing

566
00:29:34,660 --> 00:29:39,490
companies that have two engineers, and
they have 250 open PRs, and they're

567
00:29:39,490 --> 00:29:41,560
actually all making progress and such.

568
00:29:42,110 --> 00:29:44,610
We see people that that don't do reviews.

569
00:29:44,620 --> 00:29:48,020
It's only AI reviews, and
they get merged automatically.

570
00:29:48,280 --> 00:29:49,690
The tests are running.

571
00:29:49,690 --> 00:29:53,850
A lot of these things are happening now
and will probably happen more, right?

572
00:29:54,130 --> 00:29:55,440
So that's one.

573
00:29:55,850 --> 00:30:00,960
And the second way is like AI
platforms that need a Postgres database

574
00:30:00,960 --> 00:30:03,220
as part of their product right?

575
00:30:03,260 --> 00:30:09,210
And then they want to spin up like an FML
database for that application, which could

576
00:30:09,220 --> 00:30:11,630
be an experiment that they throw away.

577
00:30:11,640 --> 00:30:15,780
So then you want these cheap
databases that spin up and down

578
00:30:16,510 --> 00:30:18,900
very dynamically and scale a lot,

579
00:30:19,340 --> 00:30:20,210
scaling to

580
00:30:20,492 --> 00:30:21,972
Nik: Yeah.

581
00:30:21,972 --> 00:30:24,492
Yeah, these guys even don't
need HA at all, right?

582
00:30:25,282 --> 00:30:27,672
It's just single node no failover

583
00:30:28,094 --> 00:30:28,894
Tudor: Yeah, exactly.

584
00:30:28,944 --> 00:30:29,554
Exactly.

585
00:30:29,614 --> 00:30:33,324
U-usually you don't need
replicas for those use cases, yes

586
00:30:33,956 --> 00:30:35,856
Nik: Yeah, so no high availability needed.

587
00:30:35,936 --> 00:30:37,736
Maybe just backups and that's it, right?

588
00:30:37,796 --> 00:30:40,016
Like you also feel this like new demand

589
00:30:40,076 --> 00:30:40,656
Tudor: yeah.

590
00:30:40,726 --> 00:30:44,116
Yeah, and we do take advantage
of that because the way we solve

591
00:30:44,146 --> 00:30:49,196
these very fast wake-up times is
by doing what we call warm pools.

592
00:30:49,246 --> 00:30:52,086
We keep CNPG clusters ready to go.

593
00:30:52,986 --> 00:30:59,606
And then when there's a connection for
a hibernated branch or database we take

594
00:30:59,606 --> 00:31:06,106
one already provision clustered from
from the pool, and we connect it to the

595
00:31:06,106 --> 00:31:11,453
right ZFS volume on the ZetaStore storage
node, and then serve it to the user.

596
00:31:11,473 --> 00:31:13,873
And that's how, that's
how it happens so fast.

597
00:31:14,697 --> 00:31:20,367
But this means the clusters in the
pool have to all look the same because

598
00:31:20,367 --> 00:31:23,637
if they don't look the same and
and they have, they have different

599
00:31:23,647 --> 00:31:27,247
RAM and so on, it's obvious, but
also different Postgres settings.

600
00:31:27,537 --> 00:31:30,667
If that causes a restart, then
you did nothing because you

601
00:31:30,677 --> 00:31:32,347
need to restart it then too.

602
00:31:32,407 --> 00:31:33,927
So we take advantage of that.

603
00:31:34,047 --> 00:31:37,487
The cluster pools have
no, , no read replicas.

604
00:31:37,557 --> 00:31:41,417
It's-- They're very fast,
but it is what it is,

605
00:31:41,487 --> 00:31:42,407
Nik: how fast it is?

606
00:31:43,137 --> 00:31:45,087
Tudor: So like we're targeting a second.

607
00:31:46,025 --> 00:31:49,945
pro-- Yeah, provisioning a, a new
cluster is the simplest operation

608
00:31:49,975 --> 00:31:52,425
because we just take one of
the, from the pool and give it.

609
00:31:52,455 --> 00:31:54,275
It's it's, yeah, also second.

610
00:31:54,505 --> 00:31:58,165
But it's that most of the cost
there is really the control plane.

611
00:31:58,175 --> 00:32:01,965
We have a single global control plane
region, so we have to go to it and so on.

612
00:32:02,485 --> 00:32:03,335
But that's fast.

613
00:32:03,615 --> 00:32:08,645
The more-- The wake-up is actually more
interesting because there you cannot

614
00:32:08,645 --> 00:32:10,955
just take one from the pool and serve it.

615
00:32:11,445 --> 00:32:14,625
You have to take one of the, from
the pool, connect it to the right

616
00:32:14,625 --> 00:32:16,535
volume, and then serve it, right?

617
00:32:16,555 --> 00:32:22,045
And this is what the, this is what
the Xata store enables be- and this

618
00:32:22,065 --> 00:32:26,451
separation of storage and compute to
some degree And have a bit of a trick

619
00:32:26,481 --> 00:32:31,581
that Cloud Native PG doesn't support any
of this, so we are maintaining kind of

620
00:32:31,661 --> 00:32:34,381
our fork of it to enable some of that.

621
00:32:34,401 --> 00:32:38,791
But there's the instance manager in in
Cloud Native PG, and we modified it to

622
00:32:38,791 --> 00:32:45,261
essentially wait for the PGDATA folder
to show up, and until then it's waiting.

623
00:32:45,261 --> 00:32:47,831
So the pod is completely provisioned.

624
00:32:47,831 --> 00:32:52,171
It's just waiting for that folder to
show up before letting Postgres start.

625
00:32:52,911 --> 00:32:56,761
And behind the scenes, we mount it
in the right place, and as soon as

626
00:32:56,761 --> 00:33:01,811
that shows up there, the instance
manager will let Postgres start.

627
00:33:02,291 --> 00:33:06,641
And Postgres starts in
300 milliseconds or so.

628
00:33:06,641 --> 00:33:10,361
If the, it depends as well if
the, it, if the checkpoint was

629
00:33:10,361 --> 00:33:11,791
done and then things like this.

630
00:33:12,341 --> 00:33:15,911
But if it doesn't have to do the
recovery, it's actually really fast.

631
00:33:15,971 --> 00:33:19,811
But what's expensive is provisioning
the whole stuff in Kubernetes

632
00:33:20,091 --> 00:33:24,791
around it, the pod and the init
containers and these sort of things.

633
00:33:24,855 --> 00:33:26,285
that's where the time was going.

634
00:33:26,695 --> 00:33:29,795
So that's why we do all that
thing, and then the connection

635
00:33:30,095 --> 00:33:33,315
the, the work that we do at the
connection time is actually minimal.

636
00:33:33,465 --> 00:33:34,465
That's why it's so fast.

637
00:33:34,541 --> 00:33:35,261
Nik: That's impressive.

638
00:33:35,331 --> 00:33:35,891
That's cool.

639
00:33:36,501 --> 00:33:37,021
I have a

640
00:33:37,061 --> 00:33:37,651
tricky question.

641
00:33:38,131 --> 00:33:38,231
Oh.

642
00:33:38,241 --> 00:33:39,261
okay, Michael first

643
00:33:39,539 --> 00:33:41,549
Michael: I maybe have
the same tricky question.

644
00:33:41,959 --> 00:33:47,659
I wondered about your-- i-in a
development environment, you probably

645
00:33:47,659 --> 00:33:52,089
want the same extensions as production,
or you might you might need them

646
00:33:52,089 --> 00:33:53,589
in order to test various things.

647
00:33:54,009 --> 00:33:57,579
And also, you might have changed
a lot of configuration parameters.

648
00:33:57,579 --> 00:33:58,769
I'm wondering how…

649
00:33:59,549 --> 00:34:05,579
maybe it's not quite relevant to the same
pooling system, but does that, that system

650
00:34:05,579 --> 00:34:12,149
you just described limit your use of
extensions and changing of configuration

651
00:34:12,955 --> 00:34:14,425
Nik: I just want superuser.

652
00:34:14,425 --> 00:34:14,735
That's

653
00:34:14,825 --> 00:34:15,295
Tudor: Oh, okay.

654
00:34:16,775 --> 00:34:17,655
two different questions.

655
00:34:17,655 --> 00:34:19,305
Let me start with the one from Michael.

656
00:34:19,726 --> 00:34:20,706
Yeah it does.

657
00:34:20,726 --> 00:34:24,826
Because the, the problem is
you cannot keep a cluster pool.

658
00:34:24,826 --> 00:34:25,796
This is how we call them.

659
00:34:25,826 --> 00:34:30,426
You cannot keep a cluster pool for each
combination of, like you have instance

660
00:34:30,436 --> 00:34:36,376
size, and then you s- have a Postgres
setting or list of - extensions and so on.

661
00:34:37,546 --> 00:34:39,766
Preload libraries is more relevant.

662
00:34:39,776 --> 00:34:42,716
Because the extensions, they
can be on the image, you just

663
00:34:42,716 --> 00:34:44,796
don't enable them or enable them.

664
00:34:44,796 --> 00:34:45,336
That's fine.

665
00:34:45,336 --> 00:34:46,316
That's not a problem.

666
00:34:46,316 --> 00:34:51,436
But if you need different set of
preload libraries, that is a problem.

667
00:34:51,736 --> 00:34:56,316
So we just chose to enable
more more common stuff in the

668
00:34:56,326 --> 00:34:58,286
preload libraries by default.

669
00:34:58,996 --> 00:35:02,086
And then you have to, like you have
to keep in mind that if you are not

670
00:35:02,116 --> 00:35:07,766
on the happy path and we don't have a
cluster pool for you, it still works.

671
00:35:07,766 --> 00:35:08,666
It's just slower.

672
00:35:08,906 --> 00:35:13,406
It will take a few seconds then s-
six to 10 seconds instead of one

673
00:35:13,436 --> 00:35:14,836
if you're not on the happy path.

674
00:35:15,346 --> 00:35:18,096
And then the other thing is
that we're-- we, we haven't done

675
00:35:18,096 --> 00:35:23,656
yet, but we want to do, which is
auto-scaling for the cluster pools.

676
00:35:24,416 --> 00:35:29,366
Because some some configurations,
if you want, are going to be a lot

677
00:35:29,366 --> 00:35:31,406
more common than others, right?

678
00:35:31,406 --> 00:35:36,076
The default settings that we
have, you do a, a cluster pool

679
00:35:36,076 --> 00:35:39,246
of 100 waiting to go, right?

680
00:35:39,506 --> 00:35:44,316
So that you can do 100 at once and
you don't-- y- it's not overloaded.

681
00:35:44,886 --> 00:35:48,926
If you have a more complex one, it
could be just one is enough, right?

682
00:35:48,966 --> 00:35:53,686
Because you start that one, and
then you immediately provision

683
00:35:53,686 --> 00:35:55,106
the replacement for it.

684
00:35:55,146 --> 00:36:00,136
That will take 10 seconds, . As long as
someone else doesn't do the same operation

685
00:36:00,136 --> 00:36:01,776
in those 10 seconds, you're okay.

686
00:36:01,776 --> 00:36:03,676
So it's enough to keep one around.

687
00:36:04,316 --> 00:36:08,666
But making this -- via static
configuration is definitely annoying.

688
00:36:08,666 --> 00:36:12,486
So that's why we want to do some
sort of auto-scaling based on that.

689
00:36:12,536 --> 00:36:18,266
So depending on the demand that we've
seen historically, we would make these

690
00:36:18,386 --> 00:36:23,756
pools smaller or bigger, and that would
automatically optimize costs for us.

691
00:36:23,946 --> 00:36:27,976
That, that's something that we haven't
done yet but want to look into it.

692
00:36:28,966 --> 00:36:34,746
Yeah, superuser is like I happen
to know your opinion on that.

693
00:36:34,776 --> 00:36:37,276
We don't give superuser to to our user.

694
00:36:37,276 --> 00:36:38,416
We give the, give up s-

695
00:36:39,028 --> 00:36:39,158
Nik: But will you think about it at least?

696
00:36:40,336 --> 00:36:42,516
Tudor: Yeah, the, the issue is
because we run on Kubernetes

697
00:36:44,632 --> 00:36:47,132
it's pretty hard to secure it at

698
00:36:47,132 --> 00:36:47,462
that level.

699
00:36:47,472 --> 00:36:49,692
If you can run copy program on

700
00:36:49,774 --> 00:36:51,464
Nik: Let's wrap up this question.

701
00:36:51,464 --> 00:36:53,984
I have much more interesting
and we don't have a lot of time.

702
00:36:54,024 --> 00:36:54,414
So

703
00:36:54,616 --> 00:36:54,626
Tudor: Mm-hmm.

704
00:36:54,834 --> 00:36:56,004
Nik: question to you, tricky one.

705
00:36:56,614 --> 00:36:59,984
If you started this platform to build
this once again right now from scratch,

706
00:37:00,034 --> 00:37:01,184
would you choose Kubernetes still?

707
00:37:03,106 --> 00:37:06,626
Tudor: We do use a lot of stuff
from Kubernetes and it's good.

708
00:37:06,706 --> 00:37:08,356
Yeah, I'm happy with-- Overall,

709
00:37:08,356 --> 00:37:08,526
I'm

710
00:37:08,728 --> 00:37:09,208
Nik: most important.

711
00:37:09,268 --> 00:37:09,868
Okay, cool.

712
00:37:10,178 --> 00:37:10,388
Are you

713
00:37:10,466 --> 00:37:10,716
Tudor: I'm happy

714
00:37:10,848 --> 00:37:11,818
Nik: cloud native PG?

715
00:37:13,492 --> 00:37:18,052
Tudor: I'm-- Yeah, you know, it's like
that's more on the fence, I would say

716
00:37:18,612 --> 00:37:19,202
for me.

717
00:37:19,324 --> 00:37:20,084
Nik: opinion about it

718
00:37:21,052 --> 00:37:22,492
Tudor: I know your opinion about it.

719
00:37:22,872 --> 00:37:24,522
Nik: Yeah and more is coming actually.

720
00:37:24,862 --> 00:37:28,282
Literally this weekend I didn't
work only on this PG SimCity

721
00:37:28,814 --> 00:37:29,934
Tudor: Oh, that's awesome by the way.

722
00:37:30,332 --> 00:37:30,972
Nik: Yeah, thank you.

723
00:37:31,552 --> 00:37:33,612
It's, it was a simple one prompt.

724
00:37:33,612 --> 00:37:34,539
It was unexpected.

725
00:37:34,605 --> 00:37:41,729
but this thing like what's happening,
we have a le- a, a winning leader among

726
00:37:42,079 --> 00:37:48,099
Kubernetes operators, which is completely
like ignoring the facts like about

727
00:37:48,099 --> 00:37:50,359
split brains and data loss and so on.

728
00:37:51,319 --> 00:37:54,259
Do you use synchronous replication
if you have multi-node setup?

729
00:37:55,171 --> 00:37:55,191
Tudor: It's configurable.

730
00:37:55,331 --> 00:37:55,481
It's

731
00:37:56,929 --> 00:37:59,559
Nik: It's configurable, but
it's not on def- by default,

732
00:38:00,301 --> 00:38:00,571
Right?

733
00:38:00,571 --> 00:38:02,391
It's not d- it's not on by default.

734
00:38:03,051 --> 00:38:06,781
And it w- which means I have demonstration
of split brain is still possible

735
00:38:06,781 --> 00:38:08,871
in multiple f- in multiple ways.

736
00:38:09,621 --> 00:38:12,151
Michael: Tudor, do you mean you
let your users configure it?

737
00:38:12,953 --> 00:38:13,393
Tudor: yeah.

738
00:38:13,689 --> 00:38:14,769
We can enable it for them

739
00:38:15,041 --> 00:38:18,221
Nik: And in Cloud Native it's not
enabled by default, it's configurable.

740
00:38:18,221 --> 00:38:18,741
It's the same.

741
00:38:18,741 --> 00:38:19,741
Yeah, that's the problem.

742
00:38:19,741 --> 00:38:25,531
And so haven't you already… Like maybe
I'm over like maybe we don't need proper

743
00:38:25,531 --> 00:38:28,031
HA, split brain protection and so on.

744
00:38:28,071 --> 00:38:32,221
Like maybe this is like it's okay
to run this thing and if your users,

745
00:38:32,281 --> 00:38:35,571
if you don't know about your users
suffering may- at your scale, that

746
00:38:36,101 --> 00:38:37,491
this is good enough already, right?

747
00:38:37,491 --> 00:38:38,231
Like it's okay.

748
00:38:38,541 --> 00:38:39,511
What's your take on this?

749
00:38:40,299 --> 00:38:44,029
Tudor: No, I think it's definitely
a problem that like I'm, yeah I'm

750
00:38:44,029 --> 00:38:47,379
also a little bit disappointed
by the response to that issue.

751
00:38:47,429 --> 00:38:52,339
I was expecting, like to be closed sooner.

752
00:38:52,419 --> 00:38:57,989
I understand it's tricky because in
CNPG there's no there's no proxy layer.

753
00:38:58,229 --> 00:39:03,249
So they rely on Kubernetes services to
expose things and to move them around.

754
00:39:04,039 --> 00:39:07,729
That makes it, I think, a
little bit more complex for them

755
00:39:07,739 --> 00:39:10,269
compared to Patroni, for example.

756
00:39:11,739 --> 00:39:14,029
The funny thing is we do have a gateway

757
00:39:14,067 --> 00:39:16,287
Nik: it's not complex, it's impossible.

758
00:39:17,237 --> 00:39:18,037
It's impossible.

759
00:39:18,037 --> 00:39:20,327
You can… There is no
consensus algorithm used.

760
00:39:20,627 --> 00:39:24,447
There is only quorum, and there
is mixing of concepts like quorum

761
00:39:24,447 --> 00:39:26,147
is needed to handle transactions.

762
00:39:26,257 --> 00:39:29,397
It's not n- it's not for
deciding who is the primary.

763
00:39:30,137 --> 00:39:31,837
So I have demonstrations.

764
00:39:31,857 --> 00:39:33,947
You can have multiple--
You have two primaries.

765
00:39:34,207 --> 00:39:38,347
You can have split brain, you can have
data loss, and more to be published soon.

766
00:39:38,654 --> 00:39:42,804
I'm pissed off because m- because of
their popularity clients keep asking us.

767
00:39:43,184 --> 00:39:46,484
I literally have right now two asks
from two different companies to

768
00:39:46,484 --> 00:39:49,434
explain, should we use CloudNativePG
because it's super popular.

769
00:39:50,624 --> 00:39:53,524
And now, like I meet you
and you use it at scale.

770
00:39:54,114 --> 00:39:57,894
So you don't have problems with
no-nobody lost data, no-nobody

771
00:39:57,904 --> 00:40:00,004
dealt with split brains.

772
00:40:01,024 --> 00:40:07,194
Do we have… M-maybe we have already good
cloud resources which don't experience

773
00:40:07,194 --> 00:40:09,294
like outages, like maybe th-this is it.

774
00:40:09,734 --> 00:40:12,894
Maybe we don't need the HA
replicas may-maybe even, right?

775
00:40:12,904 --> 00:40:15,664
Like it's already good enough
to leave a single node.

776
00:40:16,504 --> 00:40:20,364
So there is no such a problem as HA,
high availability, no more such problems.

777
00:40:20,794 --> 00:40:24,894
Or you had such issues like split
brains or data losses because

778
00:40:25,284 --> 00:40:26,244
there's no consensus algorithm

779
00:40:27,924 --> 00:40:34,834
Tudor: Yeah, it's like I, like I would
have to go back and look how often.

780
00:40:34,884 --> 00:40:40,364
Like I do remember maybe one case
where to be honest, we couldn't

781
00:40:40,374 --> 00:40:42,504
figure out exactly what happened.

782
00:40:42,808 --> 00:40:45,728
in these things there's always
a combination of factors.

783
00:40:46,208 --> 00:40:49,898
But I do remember once we thought,
like this might actually be the

784
00:40:49,898 --> 00:40:52,528
split brain scenario hitting us.

785
00:40:52,538 --> 00:40:55,696
So like I, I think it's a bug really.

786
00:40:56,046 --> 00:40:59,806
Nik: Unfortunately, in their DNA, they
like, it's like this is a basic decision.

787
00:40:59,806 --> 00:41:01,166
If you read recent post.

788
00:41:01,226 --> 00:41:02,776
I have an answer unpublished.

789
00:41:02,866 --> 00:41:04,686
I'm going to publish it, so more stuff is

790
00:41:04,998 --> 00:41:05,708
Tudor: It's coming

791
00:41:05,796 --> 00:41:07,416
Nik: And I sent you a LinkedIn request.

792
00:41:07,416 --> 00:41:08,146
Please accept it.

793
00:41:08,146 --> 00:41:10,806
Let's talk, let's keep in touch
on this topic because I want the

794
00:41:10,806 --> 00:41:14,786
industry understand that consensus
algorithm is needed, and either

795
00:41:14,856 --> 00:41:19,356
CloudNativePG needs to be, to change
or people should stop using it.

796
00:41:20,286 --> 00:41:22,956
Or maybe we like, again,
like one node is enough.

797
00:41:23,126 --> 00:41:25,566
We don't need the HA, everything is good.

798
00:41:25,766 --> 00:41:31,316
AWS and Google Cloud they stopped being
unreliable as it was 10 to 15 years ago.

799
00:41:31,588 --> 00:41:35,958
Tudor: Yeah, on, on our side, we're
already I told you we are on a fork of it.

800
00:41:36,088 --> 00:41:39,488
It's not that I would, I would not
recommend people to use our fork

801
00:41:39,508 --> 00:41:43,598
because we would, we, we-- It's open
source as well, but we didn't make

802
00:41:43,598 --> 00:41:48,818
it with the goal of becoming, like a
CNPG successor or anything like that.

803
00:41:49,508 --> 00:41:55,098
But we did we did go a bit in a different
direction when it comes to backups.

804
00:41:55,268 --> 00:42:01,028
For example, we're using pgBackRest,
and we made it in the operator itself

805
00:42:01,028 --> 00:42:04,988
rather than the plug-in solution,
which has a number of disadvantages.

806
00:42:05,648 --> 00:42:10,888
So we have the, like we have the precedent
of us going a little bit in different

807
00:42:10,888 --> 00:42:13,198
directions regarding these things.

808
00:42:14,208 --> 00:42:16,088
Also, we do have a gateway anyway.

809
00:42:16,088 --> 00:42:18,528
We need a gateway, like a SQL gateway.

810
00:42:18,528 --> 00:42:24,008
So then, like the main benefit
that you are proxy-less actually

811
00:42:24,008 --> 00:42:25,578
doesn't really apply to us.

812
00:42:26,298 --> 00:42:29,448
So it might, that might be also
something that we would choose to

813
00:42:29,448 --> 00:42:30,938
do a little bit different there.

814
00:42:31,246 --> 00:42:31,446
Nik: Yeah.

815
00:42:31,446 --> 00:42:35,146
And by default CloudNative PG has
a Barman or, for backups or what?

816
00:42:35,198 --> 00:42:38,558
Tudor: I- it's a plugin that's-- It
used to be, Barman in the core, but

817
00:42:38,578 --> 00:42:40,948
then now it's Barman as a plugin.

818
00:42:41,728 --> 00:42:46,078
I think as of right now it's the
only, though, plugin that exists.

819
00:42:46,078 --> 00:42:47,858
I think there's one for pgBackRest,

820
00:42:47,908 --> 00:42:48,458
but in…

821
00:42:49,083 --> 00:42:52,273
Nik: So if you replace and if you
pl- if you have pgBackRest, this

822
00:42:52,273 --> 00:42:53,953
next step is just to add Patroni

823
00:42:55,279 --> 00:42:56,939
Tudor: Yeah, you're right

824
00:42:57,445 --> 00:42:59,495
Nik: And then my concerns
are fully resolved.

825
00:42:59,495 --> 00:43:01,155
I'm okay.

826
00:43:01,155 --> 00:43:02,775
Sensor algorithm works well and so on.

827
00:43:03,045 --> 00:43:06,145
But I understand it can be super
challenging at your scale because if

828
00:43:06,145 --> 00:43:11,235
you have a lot of smaller clusters
and so on ETCD or something, like you

829
00:43:11,235 --> 00:43:13,355
sh- you need to properly engineer it.

830
00:43:13,355 --> 00:43:17,945
I think it's done by some guys,
but it's not straightforward to

831
00:43:18,005 --> 00:43:19,255
not-- like, because we don't…

832
00:43:19,475 --> 00:43:24,395
If we have a lot of small clusters, we
cannot afford ETCD for everyone, right?

833
00:43:24,435 --> 00:43:26,815
We need to have shared
ETCD, and there are some

834
00:43:26,815 --> 00:43:27,265
limitations

835
00:43:28,085 --> 00:43:28,275
remember.

836
00:43:28,901 --> 00:43:29,471
Tudor: exactly.

837
00:43:29,545 --> 00:43:31,385
Nik: But it's like it
would be natural step.

838
00:43:31,695 --> 00:43:36,995
Like next time, like you bump into a
couple of other weird cases and maybe you

839
00:43:36,995 --> 00:43:40,255
should add Patroni, something like this.

840
00:43:40,317 --> 00:43:40,767
Tudor: Yeah.

841
00:43:40,887 --> 00:43:41,217
Yeah.

842
00:43:41,627 --> 00:43:46,267
but that's, like there's the advantage
that we don't really do this yet, but it

843
00:43:46,267 --> 00:43:51,247
does make sense to have a bit of different
configuration for your development

844
00:43:51,257 --> 00:43:53,827
branches versus your production branches.

845
00:43:54,487 --> 00:44:02,067
Because the the thousands of branches
and so on, the really large numbers don't

846
00:44:02,077 --> 00:44:06,087
even have replicas like you're saying,
because that like the point is that

847
00:44:06,117 --> 00:44:08,297
you shut them down, they come back up.

848
00:44:09,057 --> 00:44:12,437
The storage is separated and
it's secure no matter what.

849
00:44:12,437 --> 00:44:14,337
It doesn't depend on Postgres at all.

850
00:44:14,857 --> 00:44:19,067
So then, you just don't use read
replicas at all for those use cases.

851
00:44:19,257 --> 00:44:23,907
And the production ones are
just much fewer and then you can

852
00:44:23,957 --> 00:44:25,467
Nik: And computer's smaller, right?

853
00:44:25,467 --> 00:44:28,957
So like less bu- shared buffers is
small, everything like this right?

854
00:44:29,757 --> 00:44:34,707
But then you can fool the planner
and make it work as to choose

855
00:44:34,707 --> 00:44:36,037
the same plans as in production.

856
00:44:36,037 --> 00:44:40,517
This is magic I realized some time
ago, and it's super cool that you

857
00:44:40,517 --> 00:44:44,797
can s- have a tiny me- like memory
small, shared buffers is tiny, but

858
00:44:44,827 --> 00:44:46,647
you say effective cache size is huge.

859
00:44:46,957 --> 00:44:49,677
Other p- planning parameters
are huge, some other things, and

860
00:44:50,057 --> 00:44:52,347
you get exactly same plans as in

861
00:44:52,347 --> 00:44:52,867
production.

862
00:44:53,747 --> 00:44:57,687
Yeah, and then you need to just
adjust you, you need s- small mind

863
00:44:57,687 --> 00:45:03,477
shift and engineer minds to make them
focus on buffers numbers mo- more

864
00:45:03,477 --> 00:45:05,997
because timing might be different.

865
00:45:06,037 --> 00:45:09,467
This is what we talk about buffers
three years already or four years?

866
00:45:10,617 --> 00:45:10,827
Yeah

867
00:45:11,513 --> 00:45:12,783
Michael: I'm super conscious of time,

868
00:45:12,833 --> 00:45:15,603
but Tudor, is there any, any
last things or shout-outs you

869
00:45:15,603 --> 00:45:16,893
wanted to give on the Xata side?

870
00:45:16,893 --> 00:45:19,593
Like I'm very aware we haven't
even talked about DeltaX,

871
00:45:19,593 --> 00:45:20,893
which looks really promising.

872
00:45:21,457 --> 00:45:23,027
Tudor: Maybe we do another one in a few,

873
00:45:23,229 --> 00:45:24,339
Nik: Yeah, let's have part two

874
00:45:25,059 --> 00:45:25,599
quite soon.

875
00:45:25,649 --> 00:45:25,969
Yeah.

876
00:45:26,719 --> 00:45:27,729
It's, it was cool.

877
00:45:27,739 --> 00:45:28,789
And thank you for coming.

878
00:45:28,789 --> 00:45:30,149
I enjoyed to- a lot.

879
00:45:30,169 --> 00:45:34,169
I think we do similar things, like
in terms of where we ha- headed.

880
00:45:34,419 --> 00:45:36,329
I think everyone should
use database branching.

881
00:45:36,339 --> 00:45:39,838
It's super cool, we will have all
the links in show notes, right?

882
00:45:39,878 --> 00:45:42,808
And so I encourage everyone
to try out if you haven't.

883
00:45:42,808 --> 00:45:46,488
And I guess you just need to tell your
agent to try to go and use it, right?

884
00:45:46,508 --> 00:45:46,898
That's it.

885
00:45:47,976 --> 00:45:49,246
Tudor: Yeah, pretty much, yes.

886
00:45:49,658 --> 00:45:50,168
Nik: That's cool.

887
00:45:50,468 --> 00:45:51,238
Thank you for coming.

888
00:45:51,248 --> 00:45:52,538
Let's have part, part two soon.

889
00:45:54,120 --> 00:45:54,330
Michael: Yeah.

890
00:45:54,330 --> 00:45:55,330
Thanks so much, Judah

891
00:45:56,294 --> 00:45:56,684
Tudor: Thank you.

892
00:45:56,714 --> 00:45:57,664
It was really nice.

893
00:45:58,514 --> 00:45:59,024
Thanks both