How to Optimize Your Rails App Using SQLite - RUBY 674

WEBVTT
1
00:00:04.559 --> 00:00:08.919
Hey, folks, welcome back to another episode of Javis JavaScript Chamber.
2
00:00:09.000 --> 00:00:15.679
I am let me try that again. I am so sorry,
3
00:00:17.399 --> 00:00:22.280
I am so I don't know anyway, Hey folks, welcome
4
00:00:22.320 --> 00:00:26.359
back to another episode of the Ruby Rogues podcast. This week,
5
00:00:26.399 --> 00:00:28.679
on our panel, we have a Yoshinatya.
6
00:00:29.280 --> 00:00:29.559
Hello.
7
00:00:29.600 --> 00:00:34.560
Hello, I'm Charles Maxwood from top Endevs Ruby Geniuses dot Com.
8
00:00:34.719 --> 00:00:38.439
We have a special guest this week that is Steven Margheim. Stephen,
9
00:00:39.000 --> 00:00:39.799
welcome back.
10
00:00:41.039 --> 00:00:42.560
Yes, thank you for having me again.
11
00:00:43.799 --> 00:00:46.600
Yeah, we had you on I need to go look
12
00:00:46.679 --> 00:00:50.000
exactly when, but we had John. We talked about sequel
13
00:00:50.039 --> 00:00:51.880
Light last time, and I think it was mostly a
14
00:00:52.240 --> 00:00:54.200
can you really do all this stuff in sequel Light?
15
00:00:54.240 --> 00:01:00.079
And the answer was pretty much yeah. So yeah. Is
16
00:01:00.079 --> 00:01:02.039
there anything else that you want to bring up as
17
00:01:02.039 --> 00:01:06.359
far as an introduction before we dive in and start
18
00:01:06.400 --> 00:01:09.120
talking about more even more sequel Light awesomeness.
19
00:01:11.280 --> 00:01:15.680
No, not much. I'm happy to be back here, just
20
00:01:15.879 --> 00:01:20.640
braving the cold dark Berlin Winters. So happy to have
21
00:01:21.280 --> 00:01:25.400
the opportunity to chat with some other Ruby and Rails
22
00:01:26.040 --> 00:01:30.640
enthusiasts and hopefully maybe by the end of this talk
23
00:01:30.920 --> 00:01:36.079
convert you into Sequelight enthusiasts as well. Let's see awesome.
24
00:01:36.680 --> 00:01:43.120
So yeah, let me get the link into the comments
25
00:01:43.200 --> 00:01:47.079
for the videos. But yeah, we talked quite a bit
26
00:01:47.079 --> 00:01:51.400
about sequel Light last time. Are there any new developments
27
00:01:51.400 --> 00:01:53.840
as things go along with sequel Light or.
28
00:01:55.560 --> 00:02:00.280
Yeah, so there's there's been a one major development. So
29
00:02:01.040 --> 00:02:05.280
if I were to summarize the conversation last time, it
30
00:02:05.319 --> 00:02:09.680
would be can we really do this with SQLite in
31
00:02:09.840 --> 00:02:12.759
a web application built with Rails, And the answer is yes,
32
00:02:13.280 --> 00:02:18.120
but you need to finesse it. And the big change
33
00:02:18.360 --> 00:02:24.199
which is really exciting is that the release of Rails
34
00:02:24.280 --> 00:02:31.400
eight marks the first time that Rails and as far
35
00:02:31.439 --> 00:02:35.919
as I'm aware, any major web application framework has provided
36
00:02:36.680 --> 00:02:43.280
a fully production ready experience for a server driven web
37
00:02:43.319 --> 00:02:48.520
application backed entirely by seqlight. You can drive your cash
38
00:02:48.520 --> 00:02:53.960
through sqlight, your background jobs through sqlit, your web socket
39
00:02:54.280 --> 00:02:59.319
messages obviously your primary data, and you don't have to
40
00:02:59.639 --> 00:03:03.879
fiddle with any configuration. You don't have to fut's about
41
00:03:04.319 --> 00:03:11.280
with any setup or installation. You get a no compromises,
42
00:03:11.680 --> 00:03:17.560
high performance but incredibly lean, mean and stable application. When
43
00:03:17.639 --> 00:03:20.120
you run Rails new like you literally can run Rails
44
00:03:20.159 --> 00:03:23.400
new and come all deploy and you won't have any
45
00:03:23.400 --> 00:03:25.719
business logic, so it might not be very useful. But
46
00:03:26.120 --> 00:03:29.599
the bones of that application are as sturdy and as
47
00:03:29.639 --> 00:03:33.560
simple as they have ever been in the two decade
48
00:03:33.639 --> 00:03:34.400
history of RAILS.
49
00:03:35.960 --> 00:03:38.439
That's awesome. I have to admit that a lot of
50
00:03:38.439 --> 00:03:42.080
the stuff that's in Rails eight that just kind of
51
00:03:42.120 --> 00:03:48.479
comes baked in. I am so excited about. On Friday,
52
00:03:49.960 --> 00:03:53.759
I was how do I put it? I did some
53
00:03:53.879 --> 00:03:58.520
therapeutic coding and anger, and I thought, Okay, I'm just
54
00:03:58.560 --> 00:04:01.439
going to pull together this apple. I mean, admittedly I
55
00:04:01.479 --> 00:04:07.120
use postcris QOL, but I literally, yeah, I did the
56
00:04:07.159 --> 00:04:11.319
exact steps you were talking about. Other than that, and yeah,
57
00:04:11.319 --> 00:04:17.639
I had to deploy an accessory on Camal for the database,
58
00:04:18.839 --> 00:04:21.160
but then I only had one other accessory to deploy,
59
00:04:21.199 --> 00:04:22.959
and that was my Rails app. I mean, that was it,
60
00:04:23.639 --> 00:04:27.519
and it's it's so refreshing to be able to do that.
61
00:04:28.040 --> 00:04:29.680
And then I was looking at search and I was thinking,
62
00:04:29.759 --> 00:04:31.319
so I have to do elastic search, and it turns
63
00:04:31.360 --> 00:04:33.240
out that I am figuring out how to do the
64
00:04:33.319 --> 00:04:37.199
vector search in postgress. So even that's awesome. And I
65
00:04:37.199 --> 00:04:40.439
think there's a vector extension for sqlight isn't there. So
66
00:04:40.680 --> 00:04:43.879
there is I mean, it's it's I mean, this is
67
00:04:43.920 --> 00:04:46.959
the beautiful thing, right and so if you want to
68
00:04:47.000 --> 00:04:50.120
deploy it and make it run and all of that stuff,
69
00:04:50.160 --> 00:04:53.319
you definitely can. I have a question just to throw
70
00:04:53.360 --> 00:04:54.920
that out there. You said you can just do the
71
00:04:55.000 --> 00:04:59.879
camal deploy. Do you set up an accessory for sql
72
00:04:59.879 --> 00:05:02.079
I or do you just store it on a volume
73
00:05:02.720 --> 00:05:05.680
so that when you kill and restart your dock or
74
00:05:05.680 --> 00:05:07.560
it just connects to the same sequel light file.
75
00:05:08.639 --> 00:05:09.319
The latter.
76
00:05:11.639 --> 00:05:19.199
That's awesome, so then you only need one container, Yes, exactly.
77
00:05:18.319 --> 00:05:22.680
That's the cleost thought about sequel lighter thing is it's
78
00:05:22.720 --> 00:05:25.800
literally just a file. You're literally reading a file. I
79
00:05:25.839 --> 00:05:26.160
love that.
80
00:05:27.279 --> 00:05:27.519
Yep.
81
00:05:29.480 --> 00:05:33.279
Just a quick question about like the changes from RAEL
82
00:05:33.480 --> 00:05:37.000
seven to eight which kind of made a sequel light
83
00:05:37.000 --> 00:05:39.199
production ready. One thing that I know is something called
84
00:05:39.199 --> 00:05:42.000
a busy handler that was I think baked into the
85
00:05:42.040 --> 00:05:44.519
connection now and there's a couple other bits above that
86
00:05:44.519 --> 00:05:46.560
I remember reading a long blog push from music. Could
87
00:05:46.560 --> 00:05:50.040
you just summarize what are the changes from sys RAEL
88
00:05:50.120 --> 00:05:53.360
seven two to eight that now make sequel light production
89
00:05:53.439 --> 00:05:54.839
ready That wasn't the case earlier.
90
00:05:55.519 --> 00:06:02.680
Yeah, so there are two foundational changes that take rails
91
00:06:02.800 --> 00:06:06.839
from being You can sort of use it, but you
92
00:06:06.839 --> 00:06:09.279
need to work around some pain points to it works
93
00:06:09.279 --> 00:06:12.800
out of the box. So the first one concerns how
94
00:06:14.120 --> 00:06:20.920
sequalites transaction management system works in the context of a
95
00:06:21.079 --> 00:06:25.480
multi threaded web application. So I'm going to try to
96
00:06:25.519 --> 00:06:31.040
do this as succinctly as possible. But the default way
97
00:06:31.480 --> 00:06:37.959
that seqalite works with transactions, it calls working with deferred transactions.
98
00:06:38.000 --> 00:06:42.360
So to give one piece of context, seqalite only allows
99
00:06:42.560 --> 00:06:47.000
one writer to be doing a right operation at a time,
100
00:06:47.040 --> 00:06:50.279
so it has linear rights. Now, you can open up
101
00:06:50.360 --> 00:06:54.600
multiple connections to the database, and you can have connections
102
00:06:54.680 --> 00:06:58.319
reading in parallel, like that's totally fine. You can have
103
00:06:58.360 --> 00:07:00.959
five connections and they're all doing red and they can
104
00:07:01.079 --> 00:07:03.759
just do that at the same time. But if you
105
00:07:03.800 --> 00:07:06.279
have five connections open and they all try to write,
106
00:07:06.879 --> 00:07:09.240
have to line up and go one at a time.
107
00:07:10.800 --> 00:07:19.680
And when you use a transaction, the default behavior so
108
00:07:19.720 --> 00:07:25.040
if you just write begin transaction, some seql commands and
109
00:07:25.079 --> 00:07:31.560
then commit seql light doesn't attempt to acquire the right lock.
110
00:07:31.680 --> 00:07:35.079
This is the way in which it figures out which
111
00:07:35.120 --> 00:07:37.160
connection is able to write, like there's a lock, and
112
00:07:37.560 --> 00:07:40.600
each connection has to pass it around, so it doesn't
113
00:07:40.600 --> 00:07:43.480
attempt to acquire the right lock until it sees a
114
00:07:43.920 --> 00:07:47.959
right operation inside of your transaction. So if you do
115
00:07:48.240 --> 00:07:53.959
begin transactions select star, select star, update set from select,
116
00:07:54.920 --> 00:07:58.199
at that third operation, it will say, okay, let me
117
00:07:58.279 --> 00:08:03.480
try to acquire the right lock. And the problem that
118
00:08:03.519 --> 00:08:06.199
you hit and this is where anybody who tried to
119
00:08:06.319 --> 00:08:10.879
use seqalite in a Rail's application before Rails eight saw
120
00:08:10.959 --> 00:08:14.879
this error in their logs database is locked. You know,
121
00:08:14.959 --> 00:08:18.519
you got that error all the time, and that error
122
00:08:19.120 --> 00:08:25.560
was happening so often because when you begin the transaction,
123
00:08:26.399 --> 00:08:30.000
that is when seqalite takes a snapshot of your database
124
00:08:30.279 --> 00:08:34.120
to use for that transaction right to keep the isolation guaranteed.
125
00:08:35.480 --> 00:08:39.159
And so when you have a right operation inside of
126
00:08:39.200 --> 00:08:44.320
that transaction in order to ensure that isolation guarantee right
127
00:08:44.399 --> 00:08:49.240
for our acidic guarantees of this relational database, Sequalite does
128
00:08:49.279 --> 00:08:56.960
not allow that attempt to acquire the right lock to retry.
129
00:08:57.600 --> 00:09:00.000
So you get one shot. If the right block is
130
00:09:00.120 --> 00:09:04.639
being held by another connection, you immediately error. And so
131
00:09:04.919 --> 00:09:08.320
when you have a multi threaded web and you have
132
00:09:08.440 --> 00:09:12.919
many rights, there's no retry mechanism. It's like you either
133
00:09:13.320 --> 00:09:16.200
are able it's the lock is free and you can
134
00:09:16.240 --> 00:09:20.080
grab it, or it isn't and you error. And so
135
00:09:20.360 --> 00:09:24.679
the first change was to ensure that when RAILS generates
136
00:09:24.799 --> 00:09:31.480
transactions forceqlight, it explicitly makes them what seqlight calls immediate transactions.
137
00:09:31.480 --> 00:09:34.519
And all that that means is that you attempt to
138
00:09:34.639 --> 00:09:38.080
acquire the right lock at the begin transaction statement, not
139
00:09:38.200 --> 00:09:41.279
at the update statement. And the begin transaction statement is
140
00:09:41.320 --> 00:09:45.000
when you capture the snap shot, and so if the
141
00:09:45.080 --> 00:09:49.840
lock is being held, then we retry the begin transaction.
142
00:09:50.320 --> 00:09:52.679
So we retry taking a snap shot, and it's not
143
00:09:52.840 --> 00:09:55.639
until we can acquire the right lock for that particular
144
00:09:55.679 --> 00:09:58.840
connection that we begin the transaction. And then we have
145
00:09:58.879 --> 00:10:01.480
the right lock for the entire of the transaction, everything
146
00:10:01.519 --> 00:10:04.799
will be fine and we pass it along. So that
147
00:10:05.360 --> 00:10:09.279
immediately solved the major problem of just like getting a
148
00:10:09.320 --> 00:10:14.879
ton of databases locked aerrors by allowing our connection pool
149
00:10:15.200 --> 00:10:21.799
to deal with transactions. The second problem crept up thereafter,
150
00:10:22.440 --> 00:10:27.240
which is okay, now when you are retrying, so retries
151
00:10:27.360 --> 00:10:31.039
are happening at the sequlite level, and they're happening often
152
00:10:31.159 --> 00:10:34.080
in our multi threaded rails application with decent right tripe
153
00:10:34.240 --> 00:10:39.399
right through. But how does seqalite retry stuff? And it
154
00:10:39.519 --> 00:10:44.879
uses a very simple mechanism. Connection A has the right
155
00:10:44.919 --> 00:10:49.159
lock connection B attempts to acquire it can't seql. It says, okay,
156
00:10:49.480 --> 00:10:53.919
just sleep for one millisecond and then try again. So
157
00:10:53.960 --> 00:10:56.879
it sleeps one millisecond, tries again. It's still being held
158
00:10:57.039 --> 00:10:59.960
by A. So B is told sleep for two millisec
159
00:11:00.279 --> 00:11:02.919
and try again. And it has a kind of polynomial
160
00:11:03.120 --> 00:11:03.639
back off.
161
00:11:04.480 --> 00:11:08.960
It sounds like my kid's dad, Yeah, Dad, Dad.
162
00:11:09.840 --> 00:11:11.720
Are we there yet? Are we there yet? Are we
163
00:11:11.799 --> 00:11:12.279
there yet?
164
00:11:12.360 --> 00:11:14.840
And then they sit there and listen for a second. No,
165
00:11:14.919 --> 00:11:18.440
he's in there, dad, Right, that's the backup exactly.
166
00:11:20.159 --> 00:11:23.320
And this is simple and it works incredibly well, right,
167
00:11:23.399 --> 00:11:29.759
Like you can serialize concurrent operations like without the application
168
00:11:29.879 --> 00:11:32.559
needing to do any custom logic. That's great, right. The
169
00:11:32.639 --> 00:11:37.320
problem that's when I eventually go what yeah, exactly. The
170
00:11:37.399 --> 00:11:43.639
problem came from how Ruby and sequelite work together. Because
171
00:11:44.000 --> 00:11:47.120
the magic and the beauty of sequlite is that it
172
00:11:47.240 --> 00:11:51.080
is an embedded database. It's not a client server database,
173
00:11:51.120 --> 00:11:56.200
and this means that it runs inside of your application process.
174
00:11:56.240 --> 00:11:59.919
You don't spin up a Sequalite process that's running next
175
00:12:00.080 --> 00:12:05.080
to your Rails server process. It is code that's executed
176
00:12:05.120 --> 00:12:08.840
inside of that process. And what that meant for Rails
177
00:12:09.240 --> 00:12:14.440
or Ruby more generally is that, by default the way
178
00:12:14.559 --> 00:12:21.480
that Ruby is integrating with Sequalite, the Ruby interpreter did
179
00:12:21.519 --> 00:12:27.840
not see sequel lighte connection sleeping as io that it
180
00:12:27.879 --> 00:12:33.080
can release the GVL on. So the GVL gets retained
181
00:12:33.159 --> 00:12:36.960
as Seqlite retries. And so what that meant was if
182
00:12:37.000 --> 00:12:41.320
you spin up ten Puma workers and one Puma worker
183
00:12:41.399 --> 00:12:44.759
acquires the right lock and it's doing a little bit
184
00:12:44.840 --> 00:12:50.120
of longer running work, another Puma worker could and should
185
00:12:50.200 --> 00:12:54.519
theoretically be able to do Ruby work while Sequalite is
186
00:12:55.240 --> 00:12:59.399
doing Seqlite work, but they weren't able to coordinate correctly.
187
00:12:59.559 --> 00:13:04.559
So this is the busy handler thing. So what we did, I'm.
188
00:13:04.320 --> 00:13:06.240
Just going to back up for a second because you
189
00:13:06.279 --> 00:13:09.240
mentioned the GVL, yeah, which is something that we've talked
190
00:13:09.240 --> 00:13:11.120
about at length on this show before, but not for
191
00:13:11.200 --> 00:13:17.360
quite a while. So the gvl's the global vmlock and
192
00:13:17.399 --> 00:13:21.200
it says essentially, hey, I've got this thread that's trying
193
00:13:21.240 --> 00:13:25.399
to do a thing, and what Ruby does is when
194
00:13:25.440 --> 00:13:29.039
it's doing IO. Right, So if it's sending off to
195
00:13:29.080 --> 00:13:32.039
the network and waiting for a response or talking to
196
00:13:32.080 --> 00:13:34.840
the database and waiting for a response, what it does
197
00:13:34.919 --> 00:13:37.879
is it releases the GVL and lets something else, another
198
00:13:37.960 --> 00:13:42.320
thread pick things up and work. And so what Steven's
199
00:13:42.320 --> 00:13:49.080
talking about is essentially Ruby didn't detect this as IO
200
00:13:49.840 --> 00:13:52.320
is what he said, which meant that it wouldn't release
201
00:13:52.360 --> 00:13:54.960
the GVL. In other words, it's going to sit there
202
00:13:55.000 --> 00:14:00.440
and it's just go, I'm still waiting instead of you know,
203
00:14:00.919 --> 00:14:04.080
letting another thread pick up work and go do work.
204
00:14:04.120 --> 00:14:06.600
Because if it's not doing iowork that it has to
205
00:14:06.639 --> 00:14:08.159
wait for, then it can do other things, right. It
206
00:14:08.159 --> 00:14:12.159
can calculate stuff or you know, sort your raise or whatever,
207
00:14:12.240 --> 00:14:15.480
right because that all happens in the VM. And so yeah,
208
00:14:15.480 --> 00:14:19.200
so what Stephen's talking about is you would have things
209
00:14:19.279 --> 00:14:21.519
essentially just hang there until they got an answer.
210
00:14:22.080 --> 00:14:27.200
Yeah, and this is particularly important for typical rails applications
211
00:14:27.240 --> 00:14:30.279
because your typical rails application is using Puma. Puma's going
212
00:14:30.320 --> 00:14:34.399
to spin up a number of worker threads. Let's just
213
00:14:34.399 --> 00:14:38.080
say it's ten. And the way that it is expected
214
00:14:38.120 --> 00:14:42.799
to work is that you are increasing your throughput by
215
00:14:42.840 --> 00:14:48.600
having high saturated concurrent work. So one Ruby thread can
216
00:14:49.519 --> 00:14:52.759
take an incoming HTTP request and do Ruby work to
217
00:14:53.000 --> 00:14:54.840
like parse the stream like get a sense, let me
218
00:14:54.879 --> 00:14:58.360
get the rails request object, and then it makes a
219
00:14:58.399 --> 00:15:01.320
call to the database. It's like okay, now that worker
220
00:15:01.440 --> 00:15:03.120
is going to wait for the database. So the next
221
00:15:03.159 --> 00:15:07.200
worker over can start and pick up the next request,
222
00:15:07.360 --> 00:15:09.840
and so your CPU can be doing work constantly. We
223
00:15:09.879 --> 00:15:12.639
can really saturate our resources, take advantage of all of
224
00:15:12.679 --> 00:15:18.639
our cores, and do high through put work by really
225
00:15:18.720 --> 00:15:23.600
leaning into concurrency. That's like essential to how Puma is performant.
226
00:15:24.039 --> 00:15:28.279
And you could imagine that if talking to your database
227
00:15:29.559 --> 00:15:32.840
did not give the opportunity to your other Puma workers
228
00:15:32.879 --> 00:15:36.360
to do the other parts of the web request processing,
229
00:15:36.840 --> 00:15:41.159
you have ten workers, but effectively you have one, Like
230
00:15:41.320 --> 00:15:46.399
you have removed the advantages of the concurrency. And so
231
00:15:47.039 --> 00:15:51.120
what we needed to do was instead of using sqlightes
232
00:15:51.159 --> 00:15:57.320
built in mechanism for retrying, we wrote they expose a
233
00:15:57.399 --> 00:16:02.399
lower level API to write your own and then that
234
00:16:02.480 --> 00:16:07.320
allows us to write our own in Ruby. And so
235
00:16:07.559 --> 00:16:11.759
instead of calling the OS level sleep, we're calling Ruby sleep,
236
00:16:11.799 --> 00:16:16.600
and Ruby sleep does properly communicate to the interpreter, Hey,
237
00:16:16.879 --> 00:16:19.639
I am just sleeping. That is IO, you can release
238
00:16:19.679 --> 00:16:24.279
the GVL. And the third related thing, just to finish
239
00:16:24.360 --> 00:16:27.000
off the triad is when we were doing that rewrite
240
00:16:27.000 --> 00:16:30.639
and writing it a Ruby, I was doing benchmarking and
241
00:16:30.919 --> 00:16:37.679
I saw that sequelights implementation had a sort of subtle
242
00:16:37.840 --> 00:16:42.639
nasty side effect that I call punishing older queries or
243
00:16:43.200 --> 00:16:46.600
older connections, where because it was a polynomial back off
244
00:16:47.720 --> 00:16:50.120
what that and it's just like telling you don't ask
245
00:16:50.200 --> 00:16:53.080
me again for another five seconds. If you had high
246
00:16:53.600 --> 00:17:01.879
right throughput, once a connection needed to retry two or
247
00:17:01.919 --> 00:17:04.519
three times, so now it's waiting five milliseconds and then
248
00:17:04.559 --> 00:17:10.200
ten milliseconds. New connections have a much higher likelihood of
249
00:17:10.240 --> 00:17:14.240
acquiring the lock because they will retry more immediately and
250
00:17:14.279 --> 00:17:16.559
then one millisecond than too, so they get to try
251
00:17:16.599 --> 00:17:21.400
four times before my older connection even checks in again.
252
00:17:21.480 --> 00:17:24.079
And so if I have a steady stream of new connections,
253
00:17:24.400 --> 00:17:27.400
that old connection will go from the fourth generation to
254
00:17:27.440 --> 00:17:29.640
the fifth to the sixth, and it'll you'll always have
255
00:17:29.720 --> 00:17:33.319
one which goes until it errs out. And so you're
256
00:17:33.440 --> 00:17:37.200
like P ninety nine will have like these big spikes
257
00:17:37.240 --> 00:17:40.680
because you'll have like one connection that just never acquires.
258
00:17:40.680 --> 00:17:44.799
So we also rewrote the logic to just have every
259
00:17:44.880 --> 00:17:49.359
connection every time check in every millisecond, so there's no
260
00:17:49.519 --> 00:17:54.039
penalty for being quote unquote older having more retries as
261
00:17:54.079 --> 00:17:57.720
compared to younger having fewer retries. And this flattened out
262
00:17:57.880 --> 00:18:01.839
the latency that we saw well at like P ninety
263
00:18:01.880 --> 00:18:05.480
nine and P ninety five. So those three fixes are
264
00:18:05.559 --> 00:18:11.480
the major changes in the Rail's eight release that make
265
00:18:12.519 --> 00:18:14.880
rails production ready out of the box. You're not going
266
00:18:14.920 --> 00:18:17.160
to get these database lock airs, You're not going to
267
00:18:17.240 --> 00:18:20.480
see these latency spikes in your P ninety five or
268
00:18:20.480 --> 00:18:23.960
your P ninety nine statistics. You're going to get a
269
00:18:24.039 --> 00:18:29.599
high performance, stable experience by running rails new.
270
00:18:31.319 --> 00:18:35.799
Awesome. So I have another question. I seem to remember
271
00:18:36.160 --> 00:18:39.000
seeing somewhere that you were fairly involved in a lot
272
00:18:39.000 --> 00:18:45.960
of this. So when sequel light exposed the lower level thing,
273
00:18:46.079 --> 00:18:51.359
what was that for rails, Like, did you or David
274
00:18:51.480 --> 00:18:54.640
or somebody else go and actually talk to him and say, hey,
275
00:18:54.680 --> 00:18:56.839
we have this problem, can you let us do this
276
00:18:56.960 --> 00:18:59.559
other thing? Or was that something was already there?
277
00:19:00.119 --> 00:19:03.759
That was something that was already there. This is one
278
00:19:03.759 --> 00:19:09.519
of the things I have come to really appreciate about Syqalite.
279
00:19:10.200 --> 00:19:13.400
You know, it's thirty years old at this point, and
280
00:19:13.440 --> 00:19:20.240
it's used everywhere, and so it really is today like
281
00:19:20.759 --> 00:19:26.319
very feature rich and very powerful. The big difference that
282
00:19:26.400 --> 00:19:29.599
trips people up, because it's quite uncommon in our modern
283
00:19:29.599 --> 00:19:36.480
software culture, is that Sequelite cares far more about backwards
284
00:19:36.519 --> 00:19:44.119
compatibility than it does about enabling newer, better defaults and features.
285
00:19:44.839 --> 00:19:47.720
And so what that means is that if you don't
286
00:19:47.720 --> 00:19:51.039
do anything, if you don't do any configuration or any tuning,
287
00:19:52.319 --> 00:19:57.960
when you are using Seqlite today, you are effectively using
288
00:19:58.000 --> 00:20:01.680
Sequlite as it existed in two thousand four, which is
289
00:20:01.720 --> 00:20:07.759
when version three was released. And so these myths and
290
00:20:07.839 --> 00:20:10.559
sort of fears that have grown up in the web
291
00:20:10.599 --> 00:20:13.640
development community like seqlite is not a production of any database,
292
00:20:13.799 --> 00:20:19.039
have a root in reality, because in two thousand and six,
293
00:20:19.240 --> 00:20:22.960
seqlit was not a good database for web applications. It
294
00:20:23.079 --> 00:20:27.920
was squarely built for embedded devices and that context. Over
295
00:20:27.920 --> 00:20:30.880
the last two decades, a lot of features and a
296
00:20:31.000 --> 00:20:34.440
lot of work has been done to make it a
297
00:20:34.599 --> 00:20:40.160
far more flexible and powerful database engine, but literally none
298
00:20:40.200 --> 00:20:42.720
of those features are enabled by default, you have to
299
00:20:42.880 --> 00:20:49.920
manually configure seqlite to be properly run in a web application.
300
00:20:50.039 --> 00:20:52.440
This is the other big thing that rails does. We
301
00:20:52.559 --> 00:20:56.680
started doing this in rail step two. But it has
302
00:20:56.920 --> 00:21:02.000
a set of six or eight configurations that it applies
303
00:21:02.079 --> 00:21:05.119
to every connection you make so that it ensures that
304
00:21:05.119 --> 00:21:11.839
seqalite is running in a web application friendly mode. So
305
00:21:12.079 --> 00:21:14.759
this is just one of many features that seqalite has
306
00:21:14.799 --> 00:21:19.279
had sort of added over the last two decades that
307
00:21:20.319 --> 00:21:24.400
you can take advantage of to make seqallite run really
308
00:21:24.400 --> 00:21:27.480
well in the context of a multi threaded web application.
309
00:21:28.440 --> 00:21:32.000
But you have to do the work to figure out
310
00:21:32.079 --> 00:21:35.200
how you need seqlite to behave for your context.
311
00:21:37.880 --> 00:21:38.680
It's interesting that.
312
00:21:38.640 --> 00:21:42.519
You summarize the connection parameters. Is that going down a
313
00:21:42.559 --> 00:21:45.440
depravat hold the configuration stuff.
314
00:21:45.920 --> 00:21:51.519
I can summarize the key points. So the first most
315
00:21:51.519 --> 00:21:58.240
important one is that the original implementation of sequalite's journal mode,
316
00:21:58.279 --> 00:22:03.079
so the way in which it deals with stashing data
317
00:22:03.119 --> 00:22:10.240
to keep durability guarantees, was limited in that it required
318
00:22:10.519 --> 00:22:15.240
any operation to be linear, so reads and writes both
319
00:22:15.319 --> 00:22:17.359
contended with each other, so you could only do one
320
00:22:17.440 --> 00:22:20.599
thing at a time, so you could open multiple connections,
321
00:22:20.640 --> 00:22:23.480
but it's kind of useless. You were single threaded quote
322
00:22:23.519 --> 00:22:27.079
unquote at the point of view ofc QUAD. Over the years,
323
00:22:27.079 --> 00:22:32.559
they added a new journaling mode, right aheadlogging, which is
324
00:22:32.599 --> 00:22:36.599
the same sort of algorithm that postgress uses, which allows
325
00:22:36.640 --> 00:22:40.319
for concurrent reads. So that's like major change number one.
326
00:22:40.359 --> 00:22:43.640
You need to be in right a headlogging mode. When
327
00:22:44.480 --> 00:22:48.759
you use rite a headlogging mode, you have a much
328
00:22:48.880 --> 00:22:53.839
naturally safer environment, and so if you don't care about
329
00:22:53.880 --> 00:22:59.160
like the absolute strictest durability constraints, you can flush to
330
00:22:59.279 --> 00:23:06.799
disk often and get much faster performance. So the other default,
331
00:23:06.839 --> 00:23:10.240
like by default sequelite after every right flushes to disk,
332
00:23:10.359 --> 00:23:14.200
so you're doing the file system calls on every right,
333
00:23:14.240 --> 00:23:18.160
which adds a lot of overhead. So RAILS also defaults
334
00:23:18.200 --> 00:23:23.359
to flushing to disk less often, which is encouraged when
335
00:23:23.359 --> 00:23:25.519
you're using right ahead logging, so you get better performance.
336
00:23:26.359 --> 00:23:31.720
It also expands the default size of the in memory
337
00:23:33.000 --> 00:23:36.359
portion sort of space that sqlight uses. This is another
338
00:23:36.880 --> 00:23:42.079
misconception people have right like, every database is working with
339
00:23:42.240 --> 00:23:45.319
data on disc, like that's how we get durability for
340
00:23:45.359 --> 00:23:50.920
the ACID guarantee. But no database exclusively uses data on disc.
341
00:23:51.079 --> 00:23:54.759
Every database generates a pool of memory. It's like, here's
342
00:23:54.759 --> 00:23:57.599
a memory space I'm going to use, and they keep
343
00:23:57.960 --> 00:24:01.079
all of the hot data in that pool of memory
344
00:24:01.119 --> 00:24:03.599
such that they can just read and write directly in memory.
345
00:24:04.480 --> 00:24:07.559
Sequalite does the same, and you can control how big
346
00:24:07.640 --> 00:24:11.359
that space is. And it's default from you know, ten
347
00:24:11.480 --> 00:24:15.759
years ago is paltry compared to the memory that modern
348
00:24:15.839 --> 00:24:20.480
machines have, so we bumped that up. So for a
349
00:24:20.519 --> 00:24:23.680
lot of databases, you can actually like run your entire
350
00:24:23.759 --> 00:24:26.720
database effectively in memory like you have. There's literally no
351
00:24:26.799 --> 00:24:30.359
way any database can beat the speed of Sequelite in
352
00:24:30.400 --> 00:24:36.079
that context. That's actually what the guys Waprus found. They
353
00:24:36.160 --> 00:24:39.400
poorted their client from using reddis to Sequelite. He messaged
354
00:24:39.400 --> 00:24:40.680
me at one point He's like, Hey, I think I'm
355
00:24:40.680 --> 00:24:44.240
doing something wrong because I ran these benchmarks and it's
356
00:24:44.279 --> 00:24:47.160
actually faster with Sequelite and we're switched to Sequlite because
357
00:24:47.160 --> 00:24:52.039
we need like richer query language and we just knew
358
00:24:52.039 --> 00:24:55.119
we were going to pay that with slower speeds, but
359
00:24:55.160 --> 00:24:57.559
somehow it's faster. What am I doing wrong in my benchmarks?
360
00:24:57.960 --> 00:24:59.640
And we dug into it, it's like, no, your whole
361
00:24:59.720 --> 00:25:06.440
data fits in memory, so Seqallite is a more expressive
362
00:25:06.480 --> 00:25:09.839
and be faster than reddish in this case. And then
363
00:25:11.559 --> 00:25:15.839
we do a few other like small little bits and bobs.
364
00:25:15.839 --> 00:25:18.079
I have a whole blog post on my on my
365
00:25:18.119 --> 00:25:21.920
blog that breaks down all of the different settings with
366
00:25:22.000 --> 00:25:27.200
links to the documentation and explanations. Just the last one
367
00:25:27.200 --> 00:25:30.319
out I'll flag here Rails has been doing this forever,
368
00:25:30.359 --> 00:25:35.079
though you have to manually. By default, Sequalite does not
369
00:25:35.279 --> 00:25:39.319
enforce foreign keys, so like do you. Originally when it
370
00:25:39.359 --> 00:25:43.079
was built, poor and key constraints were just like you know,
371
00:25:43.279 --> 00:25:46.480
just sugar. The engine didn't do anything with them. They
372
00:25:46.519 --> 00:25:49.720
added that feature later for backwards compatibility. It's not added
373
00:25:49.720 --> 00:25:52.440
by defaults, so you have to explicitly say sequel Ie,
374
00:25:52.440 --> 00:25:53.920
please enforce my foreign keys.
375
00:25:56.799 --> 00:25:58.880
Interesting with the title of the blog post that you're
376
00:25:59.000 --> 00:25:59.599
referred to.
377
00:26:01.440 --> 00:26:05.039
Let me look up the exact title. So my blog
378
00:26:05.119 --> 00:26:07.200
is at fractalmind dot get habio and I have a
379
00:26:07.240 --> 00:26:10.000
little search up at the top. So I'm going to
380
00:26:10.039 --> 00:26:17.640
search for pragmas which is the name. I think it's
381
00:26:17.680 --> 00:26:20.480
called fine Fine tuning your database.
382
00:26:22.000 --> 00:26:24.960
Cool, I can see it now. I shall dive into
383
00:26:24.960 --> 00:26:25.400
that later.
384
00:26:29.680 --> 00:26:30.599
Yeah, I need to.
385
00:26:31.839 --> 00:26:32.240
I want to.
386
00:26:32.359 --> 00:26:33.839
I just want to capture it too so that we
387
00:26:33.839 --> 00:26:35.119
can put it into the show notes.
388
00:26:35.160 --> 00:26:42.880
And so let's move on to something a bit more real,
389
00:26:42.920 --> 00:26:47.079
specifics of a solid trifecta, which I am quite excited
390
00:26:47.079 --> 00:26:51.240
about because it's like more stuff big into reels rather
391
00:26:51.319 --> 00:26:57.079
than reaching for third party dependencies. It was obviously kind
392
00:26:57.079 --> 00:27:02.559
of conceptualized with sequel in mind, which is awesome, but
393
00:27:02.680 --> 00:27:05.759
there's still quite a lot of postgress out there in
394
00:27:05.799 --> 00:27:09.920
the world. So my question is, how, like, when you
395
00:27:10.119 --> 00:27:13.519
think about web applications, how differently do you have to
396
00:27:13.559 --> 00:27:17.119
think when you're using sequel light and if you're using
397
00:27:17.440 --> 00:27:22.200
the solid trifector with postgress, are there any things you
398
00:27:22.279 --> 00:27:24.799
really need to be aware of? Because instantly things like
399
00:27:24.880 --> 00:27:28.240
solid cable where you're kind of pulling for messages, it's
400
00:27:28.240 --> 00:27:31.160
making my spidery sense tingle that this is less efficient
401
00:27:31.200 --> 00:27:34.880
than reddest But is that a misconception when it comes
402
00:27:34.920 --> 00:27:36.559
to postcast because in sequel light, I know you can
403
00:27:36.599 --> 00:27:37.720
pull that much. I'll be fine.
404
00:27:40.839 --> 00:27:46.039
Yeah, So there are differences. The short answer is yes,
405
00:27:46.519 --> 00:27:49.039
Like I think that your your spider sense there is
406
00:27:50.680 --> 00:27:53.759
is onto something right. So let's let's start with with
407
00:27:53.960 --> 00:27:58.880
action cable. So Rails has had a postcress specific action
408
00:27:59.000 --> 00:28:02.720
cable adapter for a while now, and that adapter uses
409
00:28:02.960 --> 00:28:10.759
Postgress is in built listen notify functionality, which is faster,
410
00:28:11.240 --> 00:28:16.559
it's more performance, it will scale more naturally as your
411
00:28:17.400 --> 00:28:21.960
throughput increases, but it does come with a trade off
412
00:28:22.359 --> 00:28:24.839
of like a maximum message size of I think like
413
00:28:25.079 --> 00:28:38.559
eight kilobytes. There's a gem that expands that that max size.
414
00:28:38.960 --> 00:28:42.279
And if I were using postgress and I wanted to
415
00:28:42.559 --> 00:28:45.240
keep everything simple, like not bring in Redish just for
416
00:28:45.279 --> 00:28:50.599
that use, I would one hundred percent use the action
417
00:28:50.759 --> 00:28:56.519
cable postgress adapter and not solid cable that is better
418
00:28:56.559 --> 00:28:59.839
tuned to that architecture. Now, if I'm all in on
419
00:28:59.880 --> 00:29:02.440
my SQL already and I don't want to bring in
420
00:29:03.799 --> 00:29:06.519
the reddest dependency just yet, and I know that I
421
00:29:06.559 --> 00:29:10.440
don't have like a ton of web socket traffic right
422
00:29:10.440 --> 00:29:12.799
like I'm just doing a few basic things pushing down
423
00:29:13.559 --> 00:29:16.319
alerts when background jobs finishes, and it's like I only
424
00:29:16.319 --> 00:29:18.200
got six background of stuff. Like It's going to be
425
00:29:18.319 --> 00:29:22.079
totally fine, but you should go into it being aware
426
00:29:22.799 --> 00:29:26.960
that you are going to have some different pain points,
427
00:29:27.039 --> 00:29:31.079
the biggest one being that, of course when you're dealing
428
00:29:31.079 --> 00:29:37.359
with the client server architecture, your primary bottleneck is the
429
00:29:37.400 --> 00:29:43.240
network latency. Like every database is fast, really fast, Like
430
00:29:43.319 --> 00:29:47.039
databases are really magical pieces of technology.
431
00:29:48.720 --> 00:29:52.559
I want to just interject, So not twenty twenty four,
432
00:29:52.559 --> 00:29:56.160
but twenty twenty three when David or DHH gave his
433
00:29:56.759 --> 00:29:59.559
keynote talk, he talked a lot about why the databases
434
00:29:59.559 --> 00:30:02.400
are fast or and how the disks have gotten faster.
435
00:30:02.599 --> 00:30:05.279
And you know, you were talking about memory capacity. That's
436
00:30:05.279 --> 00:30:07.799
another thing that's expanded, and you've alluded to that. So
437
00:30:09.000 --> 00:30:11.720
I love all of the allusions to you know, modern
438
00:30:11.759 --> 00:30:15.039
computing and just what the capabilities are. But anyway, go ahead.
439
00:30:17.000 --> 00:30:23.319
Yeah, And so you are penalized in a client server
440
00:30:23.480 --> 00:30:28.279
architecture for making a large number of small queries. And
441
00:30:28.359 --> 00:30:31.000
this is one of the biggest mindset shifts you need
442
00:30:31.039 --> 00:30:33.200
to make if you want to embrace sequlite, because it's
443
00:30:33.279 --> 00:30:38.839
really deep in rails developers like limpic system, Right, I
444
00:30:38.920 --> 00:30:44.720
want to prefer a small number of large queries, but
445
00:30:44.880 --> 00:30:47.920
when you're working with SQL, you actually want to flip that.
446
00:30:48.480 --> 00:30:52.240
You are going to get a better performance profile by
447
00:30:52.279 --> 00:30:59.039
having a large number of small queries. And so that's
448
00:30:59.119 --> 00:31:01.160
like one consideration. And so if we think about something
449
00:31:01.200 --> 00:31:03.960
like action cable, the nature of action cable is like
450
00:31:04.359 --> 00:31:08.079
it's you can't aggregate those queries, right, Like if you're
451
00:31:08.119 --> 00:31:12.920
just doing the polling, you're just making that get well,
452
00:31:13.079 --> 00:31:16.160
I'm using an HTTP verb there, but you know we're
453
00:31:16.160 --> 00:31:22.160
making that reade. Yeah, and you have to eat the
454
00:31:22.160 --> 00:31:27.440
network latency. You just have to. So for something like
455
00:31:27.720 --> 00:31:32.720
a solid cable, it's really well tuned to seqlite. The
456
00:31:32.759 --> 00:31:37.319
way that seqlite is built and the fact that we
457
00:31:37.680 --> 00:31:40.519
isolated in a separate database like means you don't even
458
00:31:40.519 --> 00:31:43.480
have to worry about concurrency. Of course, if base Camp
459
00:31:43.519 --> 00:31:46.279
were to use this like, they would experience very real
460
00:31:46.319 --> 00:31:52.079
pain because now polling for action cable messages is contending
461
00:31:52.240 --> 00:31:56.960
with serving all of their other application database needs. Right.
462
00:31:57.000 --> 00:31:58.720
So if they just use a single big bee feed
463
00:31:58.799 --> 00:32:02.279
my sql in since and they're doing all of their
464
00:32:02.319 --> 00:32:05.240
application reading and writing as well as this website of
465
00:32:05.319 --> 00:32:10.279
pulling like now their connection pool, the resources of that
466
00:32:10.359 --> 00:32:14.680
instance are all like being used up for both of
467
00:32:14.720 --> 00:32:19.400
these use cases. And so you do need to be
468
00:32:19.839 --> 00:32:25.880
more thoughtful, especially if you're unsure about how your usage
469
00:32:25.960 --> 00:32:29.160
might scale over time. If you're using something like postgress
470
00:32:29.160 --> 00:32:33.000
in my squel especially around actually cable for something like
471
00:32:33.039 --> 00:32:37.559
the cash. I actually think that there are real advantages
472
00:32:37.759 --> 00:32:41.160
to using a database back cash, regardless of the database
473
00:32:41.640 --> 00:32:45.680
engine you are using. At the scale of base Camp.
474
00:32:46.599 --> 00:32:50.240
They through experience and they talk about this in their
475
00:32:50.279 --> 00:32:56.079
Raales World talks from last year introducing solid cash. Came
476
00:32:56.119 --> 00:32:59.160
to see, Yeah, that contention is too high, So they
477
00:32:59.279 --> 00:33:03.880
split out separate my SQL database for their cash, and
478
00:33:05.240 --> 00:33:10.559
that I think is going to be the case at
479
00:33:10.839 --> 00:33:14.039
really sort of ity decent size for your cash. But
480
00:33:15.279 --> 00:33:17.160
the point that they make, which is like when you
481
00:33:17.200 --> 00:33:22.039
just massively expand the size of your cash, I can
482
00:33:22.119 --> 00:33:25.119
just keep way more things in the cash, and I
483
00:33:25.160 --> 00:33:28.559
can keep them in the cash for longer. Then your
484
00:33:28.559 --> 00:33:31.759
cash becomes more useful because you just have more hits. Naturally,
485
00:33:31.759 --> 00:33:34.640
you just there's more things to hit against, and that's
486
00:33:34.720 --> 00:33:38.480
what the database offers you. And because of modern computing,
487
00:33:38.559 --> 00:33:43.200
because like the file systems are faster, the RAM is faster,
488
00:33:43.599 --> 00:33:48.160
the networks are faster, the difference that you have between
489
00:33:48.359 --> 00:33:50.480
even some of your cash hits, like needing to go
490
00:33:50.519 --> 00:33:53.039
to disc to fetch the data to send it back,
491
00:33:54.119 --> 00:33:56.920
is much smaller than it used to be, So I
492
00:33:56.920 --> 00:34:01.880
would recommend looking at solid cash for any application, regardless
493
00:34:01.880 --> 00:34:07.279
of the database engineer using for solid q. If you
494
00:34:07.359 --> 00:34:10.360
are all in on postgress. This is another case where honestly,
495
00:34:12.360 --> 00:34:18.119
probably good job is a better fit, like it's explicitly
496
00:34:18.239 --> 00:34:23.639
tuned to postgress and using its features. The really nice
497
00:34:23.639 --> 00:34:26.840
thing that makes solid q quite competitive is that just
498
00:34:26.920 --> 00:34:30.039
it has more useful features out of the box, like
499
00:34:30.119 --> 00:34:33.599
recurring jobs. That's just that they are, and they have
500
00:34:33.719 --> 00:34:36.079
done a ton of work to make it super performant
501
00:34:36.719 --> 00:34:41.360
on postgress in my SQEL. Obviously, base Camp and thirty
502
00:34:41.400 --> 00:34:44.519
seven Signals use my SQL, and they are using solid
503
00:34:44.599 --> 00:34:47.400
q in production doing like tens of millions of jobs,
504
00:34:49.079 --> 00:34:54.719
so it's totally production ready at really large skills as well.
505
00:34:55.199 --> 00:34:57.840
So I would also feel very comfortable using solid q
506
00:34:58.480 --> 00:35:03.239
with my SQEL or postgress at larger use cases. But
507
00:35:03.480 --> 00:35:06.880
in general, all three of these gems, like lean on
508
00:35:07.159 --> 00:35:12.719
pulling and sequel, it's just the best sort of perfect
509
00:35:12.719 --> 00:35:15.800
fit if you want to use a database engine and
510
00:35:15.840 --> 00:35:18.440
you just want to pull all the time, because those
511
00:35:18.559 --> 00:35:23.039
reads are effectively free, I mean, they're so fast and
512
00:35:23.039 --> 00:35:29.679
there's really no contention or penalty, So it's it's like, yeah,
513
00:35:29.800 --> 00:35:32.880
just pull all the time, like you basically have real time.
514
00:35:32.960 --> 00:35:36.199
There's not a ton of complexity and there's not really
515
00:35:36.239 --> 00:35:39.599
a ton of trade offs.
516
00:35:41.280 --> 00:35:46.159
So one thing that I'm wondering then, is because I
517
00:35:46.280 --> 00:35:49.679
just switched over to solid que and it has a
518
00:35:49.719 --> 00:35:52.239
plug in that like you fire up Puma and it
519
00:35:52.320 --> 00:35:56.360
runs all your So everything just runs on the same
520
00:35:56.400 --> 00:35:59.079
server essentially, right, you just tell it run the server
521
00:35:59.159 --> 00:36:01.360
and runs the server. It's the workers and it does everything.
522
00:36:03.000 --> 00:36:05.639
So what you're telling me is is if I'm still
523
00:36:05.679 --> 00:36:08.880
really in love with PostgreSQL, I can do that for
524
00:36:09.039 --> 00:36:13.400
all of my I guess primary level data, and then
525
00:36:13.480 --> 00:36:16.960
for the queue and the cash and the cable stuff,
526
00:36:17.320 --> 00:36:20.119
I can still use squeal light and just avoid all
527
00:36:20.159 --> 00:36:24.039
of that network latency on these tiny polling queries. Is
528
00:36:24.119 --> 00:36:24.800
that what I'm hearing?
529
00:36:25.519 --> 00:36:28.320
Yeah, So the if you just run rails new with defaults,
530
00:36:28.480 --> 00:36:30.800
you're going to get seqlite everything. You're going to get
531
00:36:30.920 --> 00:36:33.320
all the solid gems. And the way that it will
532
00:36:33.360 --> 00:36:38.840
be structured is that Rails will create four different database files.
533
00:36:39.960 --> 00:36:43.440
It'll create a we'll just talk about development. A development
534
00:36:43.440 --> 00:36:46.800
that's sqlight three, A cash, A queue, and a cable
535
00:36:47.320 --> 00:36:48.280
that sqlight three.
536
00:36:48.440 --> 00:36:50.760
So yeah, four different I find that I was using
537
00:36:50.800 --> 00:36:54.000
Postgress and it's still created the other three with SQLite,
538
00:36:55.280 --> 00:36:56.079
I believe.
539
00:36:57.159 --> 00:37:01.320
Yeah, that might be true. I honestly haven't spent a
540
00:37:01.400 --> 00:37:05.239
ton of time initializing rails applications with anything about postpress
541
00:37:05.679 --> 00:37:11.800
those details as well. But so what that means is
542
00:37:11.840 --> 00:37:17.079
that a so SQA has this constraint of linear rights.
543
00:37:18.079 --> 00:37:22.119
That constraint is at the file level, So you can
544
00:37:23.079 --> 00:37:27.079
right to your primary database, add a job to the queue,
545
00:37:27.360 --> 00:37:31.679
add a message to the backlog for cable messages, and
546
00:37:32.039 --> 00:37:35.159
add a cash entry in parallel. You can do all
547
00:37:35.159 --> 00:37:39.280
four of those operations at exactly the same time, so
548
00:37:40.039 --> 00:37:45.679
there's no contention across the different services. And all three
549
00:37:45.719 --> 00:37:51.079
of those additional services will run directly inside of PUMA
550
00:37:51.119 --> 00:37:55.000
if they need a separate process, like solid q does,
551
00:37:56.199 --> 00:38:00.079
and so you don't have to set anything up and
552
00:38:00.119 --> 00:38:02.760
you don't need anything more than that one single machine.
553
00:38:02.760 --> 00:38:06.039
So like I build and deploy my applications on hatchbox
554
00:38:06.079 --> 00:38:11.840
on a single Digital Ocean droplet, and I have. I
555
00:38:11.880 --> 00:38:15.360
had the big first app that I really went all
556
00:38:15.400 --> 00:38:17.400
in on seqy with and led to a lot of
557
00:38:17.880 --> 00:38:22.000
these additions to rails. It's still running in production and
558
00:38:22.039 --> 00:38:25.639
it is on a twenty dollars a month droplet, and
559
00:38:25.719 --> 00:38:29.719
that app has made multiple millions of dollars in revenue
560
00:38:29.960 --> 00:38:35.760
in the last four years. It's it's it's really kind
561
00:38:35.760 --> 00:38:41.960
of beautiful how cheap you can run these applications like this,
562
00:38:42.079 --> 00:38:45.239
Like this is a targeted B to B application that
563
00:38:45.400 --> 00:38:49.159
just isn't going to have ever more than a thousand
564
00:38:49.239 --> 00:38:54.840
concurrent users. And so I just I don't need a
565
00:38:54.880 --> 00:38:57.000
huge box. I don't need to run a whole bunch
566
00:38:57.039 --> 00:39:01.920
of things. I can. I was a single dev I
567
00:39:01.960 --> 00:39:07.280
built it and maintained it for years. It was super straightforward, lean,
568
00:39:07.559 --> 00:39:11.519
easy to operate, easy to deploy, easy to evolve, super
569
00:39:11.559 --> 00:39:15.760
fast for those kinds of applications. So I think, like
570
00:39:15.840 --> 00:39:18.960
particularly B to B applications are like a great fit.
571
00:39:19.079 --> 00:39:23.079
Like you have a certain kind you typically have a
572
00:39:23.239 --> 00:39:28.079
sort of geographic boundary, right, Like you're building a business
573
00:39:28.079 --> 00:39:30.320
in the UK, You're mostly going to have UK customers.
574
00:39:30.360 --> 00:39:32.800
You're building a business in Europe, you'll mostly of European customers.
575
00:39:32.840 --> 00:39:34.400
You're building a business in the US, you'll mostly of
576
00:39:34.480 --> 00:39:39.960
US customers. You're going totally virally successful. It means like
577
00:39:40.000 --> 00:39:42.559
maybe you have ten thousand customers, right, Like, it's just
578
00:39:42.639 --> 00:39:46.280
totally different than trying to build the next TikTok. SQL
579
00:39:46.320 --> 00:39:51.960
Light and vertical scaling are such high leverage decisions for
580
00:39:52.039 --> 00:39:55.239
those kinds of businesses. You can operate your business so
581
00:39:56.119 --> 00:40:01.079
efficiently and add so many high power features to your
582
00:40:01.079 --> 00:40:07.039
application so quickly and so cheaply. That's where I get like,
583
00:40:07.199 --> 00:40:14.039
particularly passionate. It's just like that more people appreciate that. Yeah, objectively,
584
00:40:14.119 --> 00:40:17.199
Postgress is a great database. I have nothing negative to
585
00:40:17.199 --> 00:40:20.039
say about it, but the idea that it is the
586
00:40:20.039 --> 00:40:23.519
only viable option for a database in twenty twenty five
587
00:40:23.559 --> 00:40:29.519
and production is objectively laughable. Sequelight, in my Sequel and
588
00:40:29.599 --> 00:40:33.960
Postgress are all really great databases that there are very
589
00:40:34.079 --> 00:40:37.039
legitimate use cases for in twenty twenty five.
590
00:40:38.000 --> 00:40:42.760
Yeah, that's quite a pitch, and I'm not surprised to
591
00:40:42.840 --> 00:40:45.679
yet that from you, But how passionately you've been kind
592
00:40:45.679 --> 00:40:48.079
of waving this flag for the last couple of years,
593
00:40:48.079 --> 00:40:52.960
and I'm giving sequel Light a go on a side
594
00:40:52.960 --> 00:40:59.079
project I've got just started looking forward to diving into that.
595
00:41:00.199 --> 00:41:03.199
How do you feel about the architecture that Chuck briefly
596
00:41:03.239 --> 00:41:07.719
mentioned by a primary database is Postgress, but you're using
597
00:41:08.800 --> 00:41:12.760
sequel Light for jobs and cashing and solid cable or whatever.
598
00:41:13.360 --> 00:41:15.159
How do you feel about that kind of architecture.
599
00:41:17.760 --> 00:41:20.880
I think that it makes sense and it can work,
600
00:41:21.000 --> 00:41:25.320
especially if you have an existing application and you want
601
00:41:25.360 --> 00:41:28.800
to start exploring some of these things. I think that
602
00:41:29.119 --> 00:41:33.559
solid cash driven by Sequelite is a really great place
603
00:41:33.639 --> 00:41:38.119
to start. Like, take an existing application, leave your Postgress Heroku,
604
00:41:38.559 --> 00:41:41.199
well not on Heroku. Sequel doesn't work on Heroku, so
605
00:41:41.400 --> 00:41:44.559
not on Heroku. If you're using Heroku, just hug Reticent
606
00:41:44.639 --> 00:41:50.159
Postgress and you'll be fine. But in a context where
607
00:41:50.159 --> 00:41:54.920
you're not using Heroku. Yeah, just like spin up a
608
00:41:54.960 --> 00:42:01.360
cash database with seqle it can get really huge, not
609
00:42:01.400 --> 00:42:05.880
can have any problems. There's zero need or reason to
610
00:42:06.199 --> 00:42:10.719
go all in on one immediately. It's totally fine to
611
00:42:10.760 --> 00:42:19.239
mix and match the For me, one of the primary
612
00:42:19.320 --> 00:42:26.119
benefits of sqlit is the opportunity to have a maximally
613
00:42:26.440 --> 00:42:31.000
operationally simple application. So for me, I start all new
614
00:42:31.000 --> 00:42:33.480
applications going all in on sqlight because it's like I
615
00:42:33.519 --> 00:42:36.559
don't want to run two machines. I don't want to
616
00:42:36.599 --> 00:42:40.280
figure out how to secure and network two machines. I
617
00:42:40.320 --> 00:42:42.719
don't want to, Okay, I'll just do one machine. And
618
00:42:42.760 --> 00:42:45.360
it's like, let me make sure I'm running the two processes.
619
00:42:45.440 --> 00:42:47.320
Let me make sure I'm doing that resiliently. Like what
620
00:42:47.440 --> 00:42:51.920
happens if my postcrest process goes down? How do I
621
00:42:51.960 --> 00:42:54.760
have my app process handle that gracefully? How I need
622
00:42:54.760 --> 00:42:57.679
to have another process running that watches the postcress process
623
00:42:57.679 --> 00:42:59.519
to bring it back up when it goes down. Like
624
00:42:59.800 --> 00:43:03.719
it just it gets a little bit out of hand
625
00:43:04.519 --> 00:43:09.039
rather quickly. But if you have an existing application, this
626
00:43:09.199 --> 00:43:12.440
is not in any way and all or nothing. I
627
00:43:12.480 --> 00:43:17.239
think cash is a great place to start. Cable can
628
00:43:17.559 --> 00:43:19.239
also be a good place to start, though, if you're
629
00:43:19.280 --> 00:43:24.360
already on postgress the action cable postgress adapters, I would
630
00:43:24.440 --> 00:43:27.519
start there. You still don't have to reach for rettis.
631
00:43:29.079 --> 00:43:33.239
But yeah, mixing and matching is one of the core
632
00:43:33.280 --> 00:43:35.920
philosophies of rails, right, This is O my case, at
633
00:43:35.920 --> 00:43:41.519
its best. So I do embrace it and encourage it,
634
00:43:41.679 --> 00:43:45.840
particularly for migrations. But I will beat the dead horse
635
00:43:45.880 --> 00:43:48.639
once again if you're starting something, if you're running rails new,
636
00:43:48.840 --> 00:43:52.519
like just give it one month, give it one month.
637
00:43:52.639 --> 00:43:54.440
Just try it for one month, That's all I ask.
638
00:43:55.679 --> 00:43:57.400
And if you say, you know what, I just really
639
00:43:57.400 --> 00:44:01.760
really love postgress, awesome, go use post But if Mordv's
640
00:44:01.840 --> 00:44:09.280
just tried, I think we would see a real change
641
00:44:09.800 --> 00:44:14.320
in the makeup of RAILS applications in production, because there
642
00:44:14.360 --> 00:44:20.360
are very real benefits for bootstrap businesses to lean on
643
00:44:20.599 --> 00:44:23.440
this technology.
644
00:44:23.559 --> 00:44:25.840
So I got a couple of questions here. One is
645
00:44:25.840 --> 00:44:28.199
is you've talked about some of the optimizations that you
646
00:44:28.199 --> 00:44:32.679
can use in postgress. You know, whether you're enforcing foreign
647
00:44:32.760 --> 00:44:36.159
keys or you know, some of the other pragmas that
648
00:44:36.199 --> 00:44:38.559
you can turn on to kind of tune your database.
649
00:44:39.199 --> 00:44:45.320
And it sounded like Rails does some of this for you. You
650
00:44:44.239 --> 00:44:48.199
can correct me if I'm wrong there, but I guess
651
00:44:48.239 --> 00:44:52.599
that's my question is do I effectively get all the
652
00:44:52.639 --> 00:44:55.199
goodies for free or do I have to go in
653
00:44:55.360 --> 00:44:58.800
and set up seqlight and or rails in a particular
654
00:44:58.840 --> 00:45:03.639
way to get any you know, to optimize for whatever
655
00:45:03.679 --> 00:45:04.400
I'm doing with it.
656
00:45:05.599 --> 00:45:11.079
You get all of the essential goodies for free for
657
00:45:11.440 --> 00:45:15.079
having a production ready experience on day one. All of
658
00:45:15.119 --> 00:45:20.559
that is by default. You don't have to touch anything. But
659
00:45:21.639 --> 00:45:24.679
like with most of Rails, you have levers available to you.
660
00:45:24.840 --> 00:45:30.599
So for example, there are dozens of sequel like pragmas.
661
00:45:30.719 --> 00:45:34.039
You can configure all kinds of things about seql Light
662
00:45:34.119 --> 00:45:40.440
runs the database EAMO file accept a pragma's key and
663
00:45:40.480 --> 00:45:44.880
you pass a nested hash and you can just set
664
00:45:44.920 --> 00:45:48.480
whatever pragmas you want in your database YAMO. So that's
665
00:45:48.519 --> 00:45:51.199
built into rails now. So you can add configuration. You
666
00:45:51.239 --> 00:45:53.920
can overwrite configuration. If you're like, hey, I've got I'm
667
00:45:53.920 --> 00:45:56.880
looking at my VPS, I know the services I'm running
668
00:45:56.920 --> 00:45:59.599
on it. I can commit way more memory to my
669
00:45:59.639 --> 00:46:03.119
squel database. I know I have four I know I'm
670
00:46:03.159 --> 00:46:05.320
going to do five connections each. I know I've got
671
00:46:05.360 --> 00:46:07.400
this much RAYM, I know all the other services. Like,
672
00:46:07.679 --> 00:46:12.119
let's give every single connection like one gig. I'm on
673
00:46:12.199 --> 00:46:14.079
a big beef server. Let's give them all one gig.
674
00:46:14.159 --> 00:46:18.599
Overwrite that I'm huge memory addresses for every connection. You
675
00:46:18.639 --> 00:46:25.119
can do that, but there is not a single required
676
00:46:27.440 --> 00:46:30.840
action that you need to take. You can literally run
677
00:46:30.960 --> 00:46:37.320
RAILS new and put it on a production server and
678
00:46:37.360 --> 00:46:41.119
everything will be good. So it's just just write your
679
00:46:41.119 --> 00:46:46.079
business logic. But you can do extra things. So you
680
00:46:46.079 --> 00:46:49.679
can also pull in. I have a jem to provide
681
00:46:49.719 --> 00:46:54.320
access to the SQLite extension ecosystem and make that easy
682
00:46:54.320 --> 00:46:55.960
to bring into your application. So you can bring in
683
00:46:56.000 --> 00:47:01.360
Squalite extensions like the vector search extension and integrate that
684
00:47:01.440 --> 00:47:05.679
into your application very easily doing backups. That's not something
685
00:47:05.679 --> 00:47:08.119
that is like a Rails default, but there is a
686
00:47:08.159 --> 00:47:12.599
gem to do that easily. So if you really like
687
00:47:12.760 --> 00:47:15.360
take it seriously, like if I was building a business,
688
00:47:16.000 --> 00:47:18.079
there are a few extra steps that I would take.
689
00:47:18.119 --> 00:47:20.400
But that's true of every Rails application right Like you,
690
00:47:21.440 --> 00:47:25.199
there's like the Rails plus subset of like gems that
691
00:47:25.480 --> 00:47:30.719
everyone turns to to really solidify an application for showtime.
692
00:47:31.639 --> 00:47:37.880
But the core fundamentals with Rails eight, every single essential
693
00:47:37.960 --> 00:47:44.559
detail is packaged into Rails New and requires no additional
694
00:47:44.599 --> 00:47:48.559
work to get a production ready experience.
695
00:47:51.000 --> 00:47:54.519
Makes sense. The other question I have is I'm just imagining.
696
00:47:55.119 --> 00:47:56.559
I'm trying to think through some of the apps that
697
00:47:56.599 --> 00:47:59.079
I'm either built or in building or things like that.
698
00:48:01.159 --> 00:48:04.639
I mean, one of them you mentioned that, I guess
699
00:48:04.639 --> 00:48:07.960
I'm worried about workload, right, and I know that we've
700
00:48:07.960 --> 00:48:10.239
talked in the past about hey, it can get you know,
701
00:48:10.280 --> 00:48:14.000
you can get pretty large workload. But let's say I
702
00:48:14.039 --> 00:48:16.280
do hit some kind of limitation, right, I don't know
703
00:48:16.280 --> 00:48:18.280
how likely that is, but let's say that I do
704
00:48:18.360 --> 00:48:20.639
hit some kind of limitation. So I want to shard
705
00:48:20.639 --> 00:48:24.599
my database, or you know, maybe I want to break
706
00:48:24.599 --> 00:48:26.639
it up by tenant because I'm doing a multi tenant
707
00:48:26.719 --> 00:48:29.920
database or something like that, or a multi tenant app.
708
00:48:31.000 --> 00:48:32.639
Like how hard is it to do that kind of
709
00:48:32.639 --> 00:48:35.159
a thing and how likely am I to even need
710
00:48:35.199 --> 00:48:35.840
to consider it?
711
00:48:37.280 --> 00:48:39.360
Yeah, I'm going to start with the second question first.
712
00:48:40.000 --> 00:48:43.840
So the team at thirty seven Signals did a lot
713
00:48:43.880 --> 00:48:51.519
of different load testing for the Campfire application, and the
714
00:48:51.599 --> 00:48:54.760
camp Fire application uses Squi for the primary database and
715
00:48:55.239 --> 00:48:59.280
reddis for all of the accessories. This is because they
716
00:48:59.280 --> 00:49:02.320
built it in reales seven one days and it's harder
717
00:49:02.360 --> 00:49:05.280
to all the sologems didn't exist and as various reasons,
718
00:49:05.280 --> 00:49:08.280
but I took that code base, swapped out all the
719
00:49:08.280 --> 00:49:13.079
reddis for SQL. I ran my own benchmarks, and like
720
00:49:13.440 --> 00:49:17.320
everything was at least as fast, if not faster in
721
00:49:17.400 --> 00:49:23.519
their benchmarking in a chat application using the primary chat
722
00:49:23.559 --> 00:49:27.639
sending messages and getting the responses back, so right, heavy, right,
723
00:49:27.719 --> 00:49:31.920
not like a typical eighty twenty read write when they
724
00:49:31.960 --> 00:49:36.679
got the when they benchmark their beefiest server. In their benchmark,
725
00:49:37.000 --> 00:49:39.239
you can go and find their posts to get the
726
00:49:39.280 --> 00:49:41.039
details of that machine. I don't recall at the top
727
00:49:41.039 --> 00:49:49.079
of my head. They supported fifty thousand concurrent chat participants. Wow,
728
00:49:49.599 --> 00:49:53.360
So imagine that's basically like a fifty to fifty read
729
00:49:53.360 --> 00:49:55.920
write split and that's fifty thousand. So you go eighty twenty,
730
00:49:57.000 --> 00:50:00.960
you can get to one hundred thousand concurrent users. I
731
00:50:01.000 --> 00:50:04.559
am one hundred percent confident and application can support one
732
00:50:04.599 --> 00:50:09.480
hundred thousand concurrent users on SQL eight. So just I
733
00:50:09.519 --> 00:50:11.519
want to be very clear about like where the limit is,
734
00:50:11.559 --> 00:50:14.320
because like I know, I understand it. I have the
735
00:50:14.320 --> 00:50:16.519
impulse too. I had the impulse for a long time.
736
00:50:16.559 --> 00:50:20.599
It's like, ah, like maybe just maybe I'm building the
737
00:50:20.639 --> 00:50:23.559
next Facebook, you know, and then what am I going
738
00:50:23.639 --> 00:50:26.880
to do if I have like a billion users? It's like, okay, yes,
739
00:50:27.199 --> 00:50:31.599
but I promise you you aren't, Like I can mathematically
740
00:50:31.639 --> 00:50:33.719
guarantee you you're not going to win the lottery and
741
00:50:33.760 --> 00:50:37.400
you're not building the next Facebook. And the odds of
742
00:50:37.440 --> 00:50:43.000
your application having one hundred thousand concurrent users is astronomical.
743
00:50:43.079 --> 00:50:47.559
So just that's step one to really state very clearly
744
00:50:48.599 --> 00:50:51.960
you aren't going to hit a performance limit. You won't.
745
00:50:52.519 --> 00:50:55.639
Now that being said, let's imagine you did accidentally build
746
00:50:55.639 --> 00:50:57.519
the next Facebook on top of a sequel add on
747
00:50:57.559 --> 00:51:04.400
rails application. I'm welcome to you do. Then well, I
748
00:51:04.440 --> 00:51:08.559
think that sharting is indeed like the natural place to go.
749
00:51:09.320 --> 00:51:14.679
And if you will permit me a bit of gentle teasing,
750
00:51:15.719 --> 00:51:19.199
I happen to know that there is a project underway
751
00:51:19.559 --> 00:51:25.679
right now that will make the sequelite sharting story much better.
752
00:51:27.519 --> 00:51:31.360
Right now, it would be a fair bit of manual
753
00:51:32.239 --> 00:51:36.639
futzing about to get it set up right, and at
754
00:51:36.679 --> 00:51:40.280
some point in twenty twenty five this project will be
755
00:51:40.360 --> 00:51:45.480
released and it will be way simpler. So as long
756
00:51:45.519 --> 00:51:48.199
as you don't become the next Facebook, as long as
757
00:51:48.199 --> 00:51:51.039
you don't have more than one hundred thousand concurrent users
758
00:51:51.159 --> 00:51:55.360
in an eighty twenty read write crowd application before the
759
00:51:55.480 --> 00:51:58.719
end of twenty twenty five, you have nothing to worry about,
760
00:51:58.760 --> 00:52:04.199
because it will become really straightforward to sharred to the
761
00:52:04.320 --> 00:52:08.239
level of giving every individual tenet of your application their
762
00:52:08.320 --> 00:52:15.840
own Seqlite database. So the sky really is the limit
763
00:52:16.519 --> 00:52:18.599
and by the end of twenty twenty five, I think
764
00:52:18.639 --> 00:52:24.480
there will basically be no performance problems aside from hitting
765
00:52:24.519 --> 00:52:28.920
the bottleneck of I have vertically scaled to the biggest
766
00:52:28.920 --> 00:52:33.079
possible machine that exists, and I still have too much traffic.
767
00:52:33.159 --> 00:52:37.000
Like I'm Facebook going from one billion to two billion
768
00:52:37.039 --> 00:52:41.639
people now, and now I need to horizontally shard my application.
769
00:52:43.400 --> 00:52:46.280
But there are a number of tools in that space.
770
00:52:46.519 --> 00:52:52.400
They're all necessarily younger than tools in the Postgress and
771
00:52:52.880 --> 00:52:55.760
my squel space, and that we're like built in that
772
00:52:55.800 --> 00:52:59.440
world from day one, but they do exist. They are
773
00:52:59.440 --> 00:53:02.480
getting more more battle tested each year, so there is
774
00:53:02.519 --> 00:53:05.639
a story available to you to horizontally shard. Whether that's
775
00:53:05.760 --> 00:53:08.880
using lib sql, which now has a Ruby adapter and
776
00:53:09.000 --> 00:53:12.280
an active record adapter, whether that is waiting for the
777
00:53:12.320 --> 00:53:15.159
Limbo project, which the Turso team, you know, they are
778
00:53:15.159 --> 00:53:21.320
basically rewriting sequalite in Rust with acinc built in from
779
00:53:21.320 --> 00:53:25.400
the ground up, waiting for that to fully come to maturity,
780
00:53:25.440 --> 00:53:32.039
whether it's using light fs to get console driven cluster
781
00:53:32.760 --> 00:53:36.840
multi node cluster management for seqoid databases. There are a
782
00:53:36.920 --> 00:53:40.199
number of tools here in this space, all sort of
783
00:53:40.239 --> 00:53:44.639
maturing at their own sort of pace. With their own communities.
784
00:53:45.159 --> 00:53:47.719
So there is a story to be told for horizontal
785
00:53:47.719 --> 00:53:52.320
sharding as well. So short summary of the answer, you
786
00:53:52.400 --> 00:53:56.400
aren't going to hit computational limits on the sequlite statistically,
787
00:53:56.400 --> 00:53:59.559
I can say that almost with a guarantee. If you did,
788
00:54:01.079 --> 00:54:05.159
there are paths available to you, whether that is sharding
789
00:54:05.159 --> 00:54:07.199
your database so that you can deal with more right
790
00:54:07.280 --> 00:54:10.480
throughput and you can get a BEEF server and the
791
00:54:10.559 --> 00:54:13.719
BF server will handle your users just fine. Or you
792
00:54:13.760 --> 00:54:18.360
go super nova and you have a million concurrent users
793
00:54:19.039 --> 00:54:22.800
and you start horizontally sharding your application and you pick
794
00:54:22.880 --> 00:54:25.639
up a tool like light a fasts and bring a
795
00:54:26.239 --> 00:54:31.320
distributed sequel architecture into the mix. There are tools and
796
00:54:31.400 --> 00:54:36.199
communities and solutions on all of these points that are
797
00:54:36.960 --> 00:54:40.199
present and open source and available to us all. So
798
00:54:40.719 --> 00:54:43.760
there really is not an actual ceiling.
799
00:54:46.119 --> 00:54:49.760
So one thing where I would still be a little reticent,
800
00:54:50.079 --> 00:54:51.639
and you can tell me if I'm wrong. And it's
801
00:54:51.719 --> 00:54:56.159
not necessarily for my own projects, because I completely get
802
00:54:56.199 --> 00:54:57.679
where you're coming from that I'm not going to be
803
00:54:57.679 --> 00:55:02.119
building the next Facebook. But if you're using sequel light,
804
00:55:02.239 --> 00:55:04.440
then I think I would I be right in saying
805
00:55:04.480 --> 00:55:09.920
that schedule downtime is something you kind of need to
806
00:55:09.960 --> 00:55:15.800
just accept if you're making big changes, because like hundred
807
00:55:15.800 --> 00:55:17.400
percent up time is not a thing, but if you
808
00:55:17.519 --> 00:55:20.679
need to be as close to that as you can
809
00:55:20.840 --> 00:55:26.159
possibly be, when you kind of need to do bluegreen
810
00:55:26.199 --> 00:55:30.480
de plost, bringing another sever up, making sure it's accepting requests,
811
00:55:30.480 --> 00:55:32.480
and then taking the old one down just to make
812
00:55:32.519 --> 00:55:37.360
sure you don't go down. That kind of system is
813
00:55:37.480 --> 00:55:39.800
not great for sequel light. Would I be right in
814
00:55:39.840 --> 00:55:40.519
saying that.
815
00:55:43.320 --> 00:55:47.320
A little bit? Not quite in the generic way that
816
00:55:47.360 --> 00:55:49.960
you said, but you're you're getting to a core point.
817
00:55:50.719 --> 00:55:54.280
So at the generic point, like I use hatchbox from
818
00:55:54.320 --> 00:55:59.519
all of my hatchbox uses the old Capistrano style deployment
819
00:55:59.639 --> 00:56:05.079
where you just add another directory to the releases directory
820
00:56:05.599 --> 00:56:10.679
and your rails server process is running and you just
821
00:56:10.719 --> 00:56:15.159
swap out the sibling and that one hundred percent works.
822
00:56:15.199 --> 00:56:19.199
Like I have zero downtime deployments. I've not you know,
823
00:56:20.039 --> 00:56:21.679
oh I'm deploying. I have to bring it down for
824
00:56:21.719 --> 00:56:28.559
five minutes. So that is totally fine. When you embrace
825
00:56:28.679 --> 00:56:33.440
vertical scaling. If your machine dies and you need to recover,
826
00:56:34.039 --> 00:56:36.960
then yeah, you have downtime because your machine went down
827
00:56:37.000 --> 00:56:40.800
and you need to recover. So it's not so much
828
00:56:40.800 --> 00:56:43.679
around deployment as it is around disaster recovery, where you
829
00:56:43.760 --> 00:56:46.440
will have downtime. But that's much more a factor of
830
00:56:46.440 --> 00:56:49.199
embracing vertical scaling than it is a factor of sequelite.
831
00:56:49.239 --> 00:56:49.440
Though.
832
00:56:49.960 --> 00:56:53.119
If you embrace equal, I recommend you embrace vertical scaling
833
00:56:53.159 --> 00:56:53.519
as well.
834
00:56:54.400 --> 00:56:57.679
When you say vertical scaling, you're saying you add memory
835
00:56:57.880 --> 00:57:02.239
ad disc Yeah, you on your application everything.
836
00:57:02.320 --> 00:57:05.480
They're getting a bigger bucks as you need it. Yeah, exactly.
837
00:57:07.039 --> 00:57:11.920
But on the deployment front, it is worth being explicit
838
00:57:12.239 --> 00:57:19.000
about the fact that if you have an old successful
839
00:57:19.440 --> 00:57:26.440
application with Sqli that has some very large tables and
840
00:57:26.599 --> 00:57:31.119
you make certain kinds of schema migrations to those tables,
841
00:57:32.440 --> 00:57:37.920
you're gonna have some pain because if you if your
842
00:57:37.920 --> 00:57:43.400
migration requires making rights, the migration rights contend with the
843
00:57:43.440 --> 00:57:46.320
application rights. Seql I only allows one right operation at
844
00:57:46.320 --> 00:57:50.960
a time, and the way that we typically write these migrations,
845
00:57:51.039 --> 00:57:53.599
they're very, very greedy, so they will tend to beat
846
00:57:53.599 --> 00:57:57.239
out your application rights, and so your application will suffer
847
00:57:57.320 --> 00:58:01.840
and very likely experience some form of downtime. So there
848
00:58:01.920 --> 00:58:07.400
are migration situations in which the simplest way to avoid
849
00:58:07.519 --> 00:58:12.320
any angry pain from customers is to do schedule downtime.
850
00:58:14.880 --> 00:58:22.840
But that that situation is doing rights against a very
851
00:58:22.920 --> 00:58:26.239
large table, and by very large, I mean like over
852
00:58:26.280 --> 00:58:29.400
a million rows let's say, which, of course it is
853
00:58:29.599 --> 00:58:30.599
not crazy to get to.
854
00:58:30.719 --> 00:58:34.639
But adding an index or calculating a value on a
855
00:58:34.679 --> 00:58:37.000
new column for every row.
856
00:58:37.400 --> 00:58:39.559
Yeah, so yeah, adding an index would be the most
857
00:58:39.559 --> 00:58:46.519
common one. Supplying like a new default value, backfilling data
858
00:58:46.840 --> 00:58:52.039
to a column that you added work like that, Yeah, exactly,
859
00:58:54.880 --> 00:58:56.679
So it's a little bit more nuanced. But it is
860
00:58:56.760 --> 00:59:03.519
true that vertical scaling means if you need five nines
861
00:59:03.559 --> 00:59:08.800
of uptime a you don't. I should start there, like,
862
00:59:08.920 --> 00:59:11.639
if you need five lines of uptime, you have been
863
00:59:11.920 --> 00:59:17.679
sold marketing from AWS. That is a lie. Literally, AWS
864
00:59:17.760 --> 00:59:19.840
is the only service that needs five mines of uptime,
865
00:59:19.840 --> 00:59:23.679
and they lie in order to keep it. So even
866
00:59:23.719 --> 00:59:27.000
AWS doesn't have five nines of uptime, it doesn't exist,
867
00:59:27.079 --> 00:59:29.360
so you don't really need it. But let's say you actually,
868
00:59:29.440 --> 00:59:34.280
like truly need three nines of uptime, if that is
869
00:59:34.400 --> 00:59:39.360
essential to your business, I would not recommend a vertical
870
00:59:39.400 --> 00:59:42.880
scaling approach. I would recommend going in the complete opposite
871
00:59:42.880 --> 00:59:49.039
direction and distributing your compute and your persistent data to
872
00:59:49.119 --> 00:59:52.840
the edge, and really recognizing like for you to be
873
00:59:52.840 --> 00:59:54.800
successful in your business, you're going to need to solve
874
00:59:54.800 --> 00:59:58.719
the distributed systems problem. I will make the note though,
875
00:59:59.159 --> 01:00:05.360
that seqlight is particularly good if you need to push
876
01:00:05.400 --> 01:00:08.920
compute and data to the edge. It's not going to
877
01:00:08.960 --> 01:00:11.800
help you any if you put one big postcress server
878
01:00:11.880 --> 01:00:16.360
in Virginia and you put ten application instances in every continent.
879
01:00:17.559 --> 01:00:20.159
In fact, it's going to kill you because you're going
880
01:00:20.239 --> 01:00:23.559
to make one HDTP. Your customers will make one HDTP
881
01:00:23.679 --> 01:00:27.239
request from their laptop to your service which is right
882
01:00:27.280 --> 01:00:30.480
next to them, and then your application will make probably
883
01:00:30.519 --> 01:00:33.599
on average, ten HTTP requests to your server for all
884
01:00:33.599 --> 01:00:37.760
the different queries. And it would have been much better
885
01:00:38.039 --> 01:00:42.960
for your customer to make one long network request to
886
01:00:43.079 --> 01:00:45.760
Virginia and then your application be right next to the
887
01:00:45.800 --> 01:00:48.480
database and do ten really short queries than the opposite
888
01:00:48.519 --> 01:00:51.440
way around. But if you're going to the edge, you
889
01:00:51.559 --> 01:01:02.360
have computational constraints, and it's really perfect actually to just
890
01:01:02.400 --> 01:01:08.079
have a file on disc And so just as one example,
891
01:01:08.159 --> 01:01:09.760
we can find the blog post and add it to
892
01:01:09.800 --> 01:01:13.119
the show notes. There was a company that's doing authentication
893
01:01:13.239 --> 01:01:17.320
as a service. They're called OsO and as a part
894
01:01:17.360 --> 01:01:20.519
of their business success, like A, we need three nines
895
01:01:20.519 --> 01:01:24.760
of uptime and B because our customers will be hitting
896
01:01:24.800 --> 01:01:27.960
our API for every one of their customers web requests,
897
01:01:28.559 --> 01:01:32.079
our P ninety five needs to be ten milliseconds. So
898
01:01:32.199 --> 01:01:34.679
how do we do that? Like, well, the only way
899
01:01:34.679 --> 01:01:37.159
that we do that is we put ten application instances
900
01:01:37.199 --> 01:01:41.800
around the globe, So we just have to do distributed service.
901
01:01:43.039 --> 01:01:45.320
So then the question becomes, well, how do we do that? Well,
902
01:01:45.440 --> 01:01:47.559
obviously I don't want to put one database in Virginia
903
01:01:47.679 --> 01:01:50.360
that'll destroy my There's no way I have a P
904
01:01:50.519 --> 01:01:51.039
ninety five to.
905
01:01:51.079 --> 01:01:53.880
Ten milli seconds. You can't make that round trip that fast, right.
906
01:01:54.079 --> 01:01:55.960
And you're going to make ten of them. So I
907
01:01:56.039 --> 01:01:58.320
have to put my data right next to my application.
908
01:01:58.320 --> 01:02:00.199
Now I'm going to have ten instances of my application.
909
01:02:01.559 --> 01:02:05.000
So what is going to allow me to have all
910
01:02:05.039 --> 01:02:07.440
of the sort of data access patterns I need from
911
01:02:07.440 --> 01:02:12.920
my application but will work in the constrained environment of
912
01:02:13.599 --> 01:02:16.719
the edge instances I'm using SQL? It will be great.
913
01:02:16.719 --> 01:02:19.840
So then how do I solve the problem of distributed Sequlite.
914
01:02:19.960 --> 01:02:23.519
They did a home grown solution, so they build on
915
01:02:23.639 --> 01:02:28.840
top of Kafka, and so they distribute the Sequelite data
916
01:02:29.719 --> 01:02:33.239
and get to eventual consistency across all of their instances
917
01:02:33.360 --> 01:02:40.079
through COFCA. So anyway, all that to say, I really
918
01:02:40.159 --> 01:02:43.840
think that the world is moving towards the edges and
919
01:02:43.880 --> 01:02:48.800
that the sanest thing to do moving forward is to
920
01:02:48.880 --> 01:02:53.159
either fully embrace vertical scaling, like I'm building a business
921
01:02:53.199 --> 01:02:56.199
in Germany for Germans, I don't need I just need
922
01:02:56.199 --> 01:02:58.559
the one server in Frankfort and it'll it's going to
923
01:02:58.559 --> 01:03:02.320
be great, or embracing the other extreme and saying like
924
01:03:02.480 --> 01:03:06.440
I'm building a global business, I'm going to have customers
925
01:03:06.480 --> 01:03:08.400
that are as valuable to me in Japan and in
926
01:03:08.559 --> 01:03:13.480
South Palo and so I just need a fully distributed thing.
927
01:03:13.960 --> 01:03:20.159
And I think that sequel I is fairly uniquely positioned
928
01:03:20.679 --> 01:03:24.000
with the set of trade offs and value propositions that
929
01:03:24.039 --> 01:03:27.039
it makes to be a really solid choice in those
930
01:03:27.039 --> 01:03:31.840
two extreme scenarios and post press in my sequel we're
931
01:03:32.000 --> 01:03:34.719
really well positioned for the middle case, where you're trying
932
01:03:34.719 --> 01:03:37.559
to have as much of both as you can. But
933
01:03:38.119 --> 01:03:39.880
my sense is that the world is moving away from
934
01:03:39.920 --> 01:03:42.960
the middle case and embracing the edges and the extremes
935
01:03:43.119 --> 01:03:46.199
is going to be likely a better business decision, and
936
01:03:46.239 --> 01:03:49.199
so really thinking about the kind of application you're building
937
01:03:49.679 --> 01:03:54.840
and what kind of architecture makes most sense. But seql
938
01:03:54.840 --> 01:04:01.280
I is kind of weirdly perfect at the extremes. Nice.
939
01:04:01.880 --> 01:04:06.840
I have one last burning question regarding database backups. And
940
01:04:06.920 --> 01:04:09.920
I know, if I remember correctly, the last time you
941
01:04:10.000 --> 01:04:12.119
were on the show, it was before my time, but
942
01:04:12.159 --> 01:04:14.239
I remember watching the video. I think you deleted a
943
01:04:14.280 --> 01:04:18.719
production seql light database by accident. If I recall correctly,
944
01:04:20.000 --> 01:04:23.760
I think with sequel Light it's easier to do that
945
01:04:24.119 --> 01:04:26.960
than with any other database system because it's just a
946
01:04:27.000 --> 01:04:30.199
file and you can quite easily delete it. I know,
947
01:04:30.559 --> 01:04:34.960
I know light stream is a thing. But let's say
948
01:04:36.119 --> 01:04:39.559
you're me and allergic to dependencies. Because I hate putting
949
01:04:39.559 --> 01:04:43.079
stuff in my gem file and I want to do
950
01:04:43.480 --> 01:04:46.159
I'm like, it's just a file. All I need to
951
01:04:46.199 --> 01:04:49.159
do is make copies of this file to do backups.
952
01:04:49.559 --> 01:04:53.400
I'm sure I can write something myself. What's the recommended
953
01:04:53.519 --> 01:04:56.079
way to do seql light background You just copy paste
954
01:04:56.079 --> 01:04:58.760
the file out or is there some kind of native thing.
955
01:05:00.079 --> 01:05:03.360
To do backups the fantom ability CP.
956
01:05:04.639 --> 01:05:08.960
Exactly. So I just I just provided a link here
957
01:05:09.000 --> 01:05:10.800
in our little chat. We can add it to the
958
01:05:10.800 --> 01:05:14.679
showtes as well. But one of the other great sequeled
959
01:05:14.719 --> 01:05:18.519
enthusiasts in the Ruby community old mo As. He goes
960
01:05:18.719 --> 01:05:26.280
by on Gethue. So he has a great blog post
961
01:05:26.400 --> 01:05:30.719
on different backup strategies for sequel I, and he he
962
01:05:30.800 --> 01:05:35.800
really covers them very thoroughly, but just to just to
963
01:05:35.880 --> 01:05:37.960
hit you with a few things. So sequel like comes
964
01:05:37.960 --> 01:05:44.800
with three different backup strategies you can call backup, you
965
01:05:44.840 --> 01:05:52.880
can call dump. These are slightly slightly different. They also
966
01:05:53.000 --> 01:05:57.960
have like the Seql command vacuum into. So he breaks
967
01:05:58.000 --> 01:06:00.840
down those differences. I won't reatrat of that right now,
968
01:06:01.239 --> 01:06:05.320
but indeed genuinely you can call CP. And one of
969
01:06:05.360 --> 01:06:08.199
the points he makes, which is I will reiterate here,
970
01:06:08.440 --> 01:06:16.159
is that most production Linux servers are running file systems
971
01:06:16.400 --> 01:06:21.519
that have copy on right functionality available, and so if
972
01:06:21.559 --> 01:06:29.360
you do CP with the opten ref link always you
973
01:06:29.400 --> 01:06:36.519
get actually like a really really fast, very very space
974
01:06:36.679 --> 01:06:42.840
efficient backup of your database. And then of course you
975
01:06:43.119 --> 01:06:45.639
do have something like light stream. So the benefit of
976
01:06:45.679 --> 01:06:49.360
light stream is that you can get really fine grained backup,
977
01:06:49.440 --> 01:06:54.039
so you get point in time recovery, but it does
978
01:06:54.039 --> 01:06:55.559
come at the cost of you have to pay for
979
01:06:55.719 --> 01:06:58.400
the S three compatible bucket stores service that you're going
980
01:06:58.480 --> 01:07:01.840
to use. Pay for that storage, you have to run
981
01:07:01.880 --> 01:07:05.679
another process. So if a light stream process to copy
982
01:07:05.719 --> 01:07:12.639
over the data. If you don't need like resolution to
983
01:07:12.719 --> 01:07:15.639
the second level for backups and you just want to
984
01:07:15.639 --> 01:07:19.519
set up a crime job, you absolutely can use one
985
01:07:19.559 --> 01:07:23.519
of the other four methods. And his blog post really
986
01:07:23.559 --> 01:07:25.920
breaks down the pros and cons, like which ones are
987
01:07:25.960 --> 01:07:29.719
more space efficient, which ones have faster restore performance, what's
988
01:07:29.760 --> 01:07:32.599
the complexity to use them, what's the level of durability
989
01:07:32.599 --> 01:07:35.239
that you get. So I would just read through that
990
01:07:35.320 --> 01:07:38.400
article and think about like your use cases. But if
991
01:07:38.440 --> 01:07:41.360
you don't want to bring in a dependency, whether at
992
01:07:41.360 --> 01:07:43.320
the file system or at the SQUL light level, you
993
01:07:43.360 --> 01:07:46.280
have four different options for generating a backup that have
994
01:07:46.320 --> 01:07:51.559
different pros and cons around durability, restoration speed, and space efficiency.
995
01:07:53.079 --> 01:07:57.119
Yeah, for me, the thing that I'm concerned about, and
996
01:07:57.199 --> 01:07:59.719
granted what my first job out of college, I was
997
01:07:59.719 --> 01:08:03.119
working for an online backup company, right it back up
998
01:08:03.159 --> 01:08:05.320
all of your pictures of your kids, the cloud kind
999
01:08:05.360 --> 01:08:05.760
of thing.
1000
01:08:06.440 --> 01:08:06.840
And.
1001
01:08:09.199 --> 01:08:12.400
So I'm imagining, I, you know, if a meteor hits
1002
01:08:12.440 --> 01:08:14.719
the data center and I want to deplay it somewhere else,
1003
01:08:14.760 --> 01:08:17.680
then I need that file in the cloud somewhere. I
1004
01:08:17.680 --> 01:08:19.520
guess I could just have it on my local machine,
1005
01:08:19.520 --> 01:08:21.920
but anyway, I need it somewhere that's not the data center,
1006
01:08:22.479 --> 01:08:25.359
and so that's that's what I'm imagining. But yeah, I mean,
1007
01:08:25.359 --> 01:08:28.239
this gets me a file that I can push to
1008
01:08:28.359 --> 01:08:32.720
the cloud. So do you just I guess I'm wondering
1009
01:08:32.800 --> 01:08:35.039
what those options are. Do you just extract the file
1010
01:08:35.199 --> 01:08:37.880
as long as you have whatever time granularity you like,
1011
01:08:37.960 --> 01:08:40.239
and then figure out how to get it up int aws.
1012
01:08:41.239 --> 01:08:45.239
Yeah, So if you just wanted to use one of
1013
01:08:45.279 --> 01:08:50.279
these sort of simpler built in services and you want
1014
01:08:50.520 --> 01:08:54.800
different levels of durability, the basics that you have is like,
1015
01:08:55.760 --> 01:08:58.520
let me put it somewhere else on this machine. So
1016
01:08:58.560 --> 01:09:00.119
it's in the totally different director. So the way in
1017
01:09:00.119 --> 01:09:03.680
which I accidentally deleted my directory, there was a feature
1018
01:09:03.680 --> 01:09:07.720
in hatchbox which has been changed by the way, where
1019
01:09:08.640 --> 01:09:11.479
in the WebUI you can rename an application. And so
1020
01:09:11.520 --> 01:09:13.239
I was just fuzzing about. It's like I'm going to
1021
01:09:13.279 --> 01:09:18.479
standardize all my applications because I'm OCD, not clinically I
1022
01:09:18.520 --> 01:09:23.199
should say, that's a colloquialism. But so I just changed
1023
01:09:23.199 --> 01:09:25.680
the name of the application. And the way in which
1024
01:09:26.039 --> 01:09:30.079
they pushed that change to the servers they ran RMRF
1025
01:09:30.399 --> 01:09:33.359
on the application directory and then ran a new deploy
1026
01:09:33.920 --> 01:09:35.880
And so the fact that my sequel like directory lived
1027
01:09:35.880 --> 01:09:38.920
in the shared subdirectory did not help me at all.
1028
01:09:40.159 --> 01:09:42.359
So I could have a backup to just some other
1029
01:09:42.720 --> 01:09:46.600
place that's not likely to be accidentally RMRF. But then
1030
01:09:46.600 --> 01:09:48.239
it's like, well, if my machine goes down, that's not
1031
01:09:48.279 --> 01:09:49.760
so great, so I want to put it somewhere else.
1032
01:09:49.840 --> 01:09:53.520
You can use network attached storage. Now, like all of
1033
01:09:53.560 --> 01:09:57.359
these services provide volume storage. So the other thing you
1034
01:09:57.359 --> 01:10:00.000
can do is you just CP it onto the volume storage.
1035
01:10:00.239 --> 01:10:02.359
So now it's on a different machine, but it's in
1036
01:10:02.399 --> 01:10:04.520
the same data center. So if the data center goes down,
1037
01:10:05.199 --> 01:10:07.359
it's not so great. So then you could set up
1038
01:10:07.399 --> 01:10:10.319
a third layer to pick up those files and put
1039
01:10:10.359 --> 01:10:13.359
them on some other service. Right, you could just set
1040
01:10:13.399 --> 01:10:16.800
up back blaze probably on those levels machines however you
1041
01:10:16.800 --> 01:10:23.840
wanted to do that, or you could also just you're like, well,
1042
01:10:23.880 --> 01:10:29.039
I really want the durability, then I would embrace the
1043
01:10:29.079 --> 01:10:34.520
dependency of light stream, like it's a really low overhead dependency,
1044
01:10:34.920 --> 01:10:37.880
and so now you can get your backups a fine
1045
01:10:37.920 --> 01:10:41.800
grained but most importantly be in a completely other service,
1046
01:10:41.880 --> 01:10:46.680
in a completely other data center that has no relationship
1047
01:10:46.760 --> 01:10:52.039
to your application service, data center or machine. But generally
1048
01:10:52.079 --> 01:10:55.960
that's like the layers of a durability that you have
1049
01:10:56.000 --> 01:10:58.119
available to you, and you can you know, however far
1050
01:10:58.239 --> 01:11:00.000
you want to go for your use cases, you can
1051
01:11:00.039 --> 01:11:02.279
work your way out without a ton of effort.
1052
01:11:03.600 --> 01:11:07.560
Cool. We've spoken quite a lot today about like you
1053
01:11:07.640 --> 01:11:10.359
won't need this, like you don't need three nines of
1054
01:11:10.359 --> 01:11:12.920
our time and stuff. So from my point of view,
1055
01:11:13.600 --> 01:11:16.239
when I was thinking about this for one of my
1056
01:11:16.319 --> 01:11:18.920
own projects, I was like, all I need is a
1057
01:11:18.960 --> 01:11:21.880
head snow box with an attached volume, and I'll just
1058
01:11:21.960 --> 01:11:25.000
ride backups to the attached volume of course the data center.
1059
01:11:25.840 --> 01:11:29.479
But for my small scale I think that's fine, Like
1060
01:11:29.960 --> 01:11:30.840
would you agree with that?
1061
01:11:32.880 --> 01:11:38.279
Yeah, I mean so for most things, I think that
1062
01:11:38.359 --> 01:11:42.600
the trade off for if the if everything goes to shit,
1063
01:11:43.159 --> 01:11:47.720
everything has gone to shit, right, it's like just presuming
1064
01:11:47.760 --> 01:11:50.680
things don't go to shit and not take like you know,
1065
01:11:50.720 --> 01:11:53.600
it's it's the old adage that it's the last twenty
1066
01:11:53.640 --> 01:11:56.800
percent to finish something that takes ninety percent of the time.
1067
01:11:58.359 --> 01:12:02.760
That really is true. And most services do not require
1068
01:12:02.920 --> 01:12:07.880
the level of resiliency that would take the next ninety
1069
01:12:07.880 --> 01:12:10.560
percent of effort. And so just saying, like, you know,
1070
01:12:10.680 --> 01:12:17.840
how actually critical is this. And so for anything where
1071
01:12:17.880 --> 01:12:22.119
I'm not trying to make money, you know, it exists
1072
01:12:22.359 --> 01:12:25.159
until it doesn't, right, And so I'm not going to
1073
01:12:25.159 --> 01:12:27.800
put a ton of extra time and effort and money
1074
01:12:28.520 --> 01:12:30.800
for things that are trying to make money. Then you
1075
01:12:30.840 --> 01:12:33.840
do need some durability because like you have to keep
1076
01:12:33.880 --> 01:12:37.800
making money if something bad happens. But that's basically it.
1077
01:12:38.600 --> 01:12:40.680
That has been my own rule of thumb. So every
1078
01:12:40.720 --> 01:12:45.680
single non money making application I have, I don't do
1079
01:12:46.079 --> 01:12:51.039
fancy live stream backups for all of those applications, I
1080
01:12:51.079 --> 01:12:53.159
actually don't do any backups, to be honest with you,
1081
01:12:53.600 --> 01:12:56.399
because I know the one way in which it failed,
1082
01:12:56.399 --> 01:12:59.359
and I talked with Chris Oliver and he changed it.
1083
01:12:59.439 --> 01:13:05.159
And so unless I manually go in there and r MRF,
1084
01:13:05.359 --> 01:13:07.880
like my data is fine, it's all set up correctly.
1085
01:13:09.640 --> 01:13:14.239
So I don't even run CP or anything for anything else.
1086
01:13:14.840 --> 01:13:17.359
I personally just jumped to live stream. But like network
1087
01:13:17.399 --> 01:13:22.000
attacks attached storage with CP great. You know it's a
1088
01:13:22.079 --> 01:13:22.840
nice metal ground.
1089
01:13:23.960 --> 01:13:26.399
So is light stream a paid service or is it just.
1090
01:13:26.920 --> 01:13:29.800
No, it's an open source utility. You have to pay,
1091
01:13:29.920 --> 01:13:34.920
so it pipes your data to some bucket storage service
1092
01:13:35.279 --> 01:13:39.319
like S three R two Digital Lotion spaces, so you
1093
01:13:39.399 --> 01:13:42.239
have pay for that storage, but the tool itself is
1094
01:13:42.279 --> 01:13:44.159
a MIT open source utility.
1095
01:13:45.279 --> 01:13:47.560
Awesome. One other thing I just want to throw in,
1096
01:13:47.600 --> 01:13:51.000
because we've mentioned it a few times now. I did
1097
01:13:51.159 --> 01:13:57.399
a quick Google search on like how many minutes or
1098
01:13:57.399 --> 01:14:01.359
how many hours of downtime per year is five nines?
1099
01:14:01.399 --> 01:14:08.039
So five nines is about six minutes of downtime per year,
1100
01:14:09.960 --> 01:14:13.079
and yeah, I mean no, nobody's gonna notice. I think
1101
01:14:13.079 --> 01:14:17.520
if it's six hours, which three nines is about what
1102
01:14:17.720 --> 01:14:21.640
eight and three quarters hours per year? So you know,
1103
01:14:21.800 --> 01:14:24.560
if you have an outage here and an outage there, right,
1104
01:14:25.119 --> 01:14:29.640
you know for fifteen minutes or twenty minutes, you're going
1105
01:14:29.720 --> 01:14:31.960
to be well over the three nines and you're gonna
1106
01:14:31.960 --> 01:14:33.119
be fine.
1107
01:14:33.560 --> 01:14:35.800
Yeah. And I think the other key point that I
1108
01:14:36.560 --> 01:14:41.000
like to preach on is that there is a massive
1109
01:14:41.039 --> 01:14:46.600
difference to the health and sustainability of your business between
1110
01:14:46.640 --> 01:14:54.520
scheduled and unscheduled downtime. Right, true, scheduled downtime is a beautiful, simple,
1111
01:14:54.680 --> 01:14:58.239
powerful tool in your business is tool belt. It is
1112
01:14:58.319 --> 01:15:02.840
not very difficult to commune, nicate to your users in
1113
01:15:02.880 --> 01:15:08.600
a way that makes them feel respected and appreciated to
1114
01:15:09.920 --> 01:15:14.359
take thirty minutes of downtime to deal with a complicated
1115
01:15:14.439 --> 01:15:19.720
database migration, or to vertically scale your single machine up
1116
01:15:19.760 --> 01:15:24.600
to a bigger instance. And so I feel like we
1117
01:15:24.720 --> 01:15:27.960
sort of in a lot of ways, the conversation around
1118
01:15:28.239 --> 01:15:30.159
web development has gotten to a place where we have
1119
01:15:30.239 --> 01:15:32.560
sort of simplified things and we just talk about downtime
1120
01:15:32.600 --> 01:15:36.119
as if it's all the same, and it isn't. And
1121
01:15:36.520 --> 01:15:40.880
attempting to again like swallow all of the complexity to
1122
01:15:41.079 --> 01:15:45.720
have a system that theoretically never ever ever goes down
1123
01:15:46.479 --> 01:15:51.239
is a really deep rabbit hole of complexity. And we
1124
01:15:51.279 --> 01:15:55.640
can win back so much time and so much mental
1125
01:15:55.680 --> 01:16:00.199
space if we just accept the leverage available to us
1126
01:16:00.239 --> 01:16:03.079
to say I am going to control my application goes down.
1127
01:16:03.079 --> 01:16:07.560
And it's true, like a vast majority of applications have
1128
01:16:07.960 --> 01:16:11.239
really regular usage patterns and as a part of their
1129
01:16:11.319 --> 01:16:15.800
usage pattern, have multiple hours where no one is using
1130
01:16:15.800 --> 01:16:19.239
the application, like you have time available to you to
1131
01:16:19.399 --> 01:16:21.720
just like go down and literally no one will ever notice.
1132
01:16:22.880 --> 01:16:25.319
And even if you have people using it all the time,
1133
01:16:25.479 --> 01:16:27.199
like you're going to have low periods and you can
1134
01:16:27.359 --> 01:16:29.560
just let people know, Hey, I'm going to be down
1135
01:16:29.600 --> 01:16:33.079
for thirty minutes two weeks from now, and everything will
1136
01:16:33.079 --> 01:16:34.880
be fine, like you can continue making money.
1137
01:16:36.000 --> 01:16:39.960
Yeah, well, it also in my opinion, I mean, you know,
1138
01:16:40.359 --> 01:16:42.000
I don't remember if it was you or I. Usually
1139
01:16:42.000 --> 01:16:46.880
that brought up five nines, but the diminishing returns, right,
1140
01:16:47.319 --> 01:16:49.880
I mean, because you go from what six minutes to
1141
01:16:50.119 --> 01:16:54.560
almost an hour for four nines. It's just you're optimizing
1142
01:16:54.560 --> 01:16:59.600
away just a tiny bit of unplanned outage. And you know,
1143
01:17:00.199 --> 01:17:02.920
I don't know, I feel what you're saying there. Yeah,
1144
01:17:03.000 --> 01:17:05.640
if you know that you might cause it, then you
1145
01:17:05.720 --> 01:17:09.000
just let people know, Hey, you know, we're doing a
1146
01:17:09.039 --> 01:17:12.359
scheduled maintenance and hey, maybe you get lucky and your
1147
01:17:12.359 --> 01:17:15.439
index doesn't take more than a few minutes to build anyway,
1148
01:17:16.079 --> 01:17:19.079
but just letting people know it might be slow or
1149
01:17:19.159 --> 01:17:22.920
go down. Yeah, you get a ton of grace out
1150
01:17:22.920 --> 01:17:27.119
of that. One other thing I wanted to ask you about.
1151
01:17:27.199 --> 01:17:30.279
So the word is is that you're building a course
1152
01:17:30.560 --> 01:17:33.600
or you've built a course about some of this stuff.
1153
01:17:33.640 --> 01:17:34.920
Do you want to tell us about that?
1154
01:17:36.159 --> 01:17:42.079
Yeah, so you might have been able to tell But
1155
01:17:42.319 --> 01:17:49.319
I'm pretty passionate about the general concept of giving individuals,
1156
01:17:49.359 --> 01:17:56.359
giving small teams leverage. And I personally think that RAILS
1157
01:17:56.399 --> 01:17:59.960
eight and sequel Light, when paired together, are a really
1158
01:18:00.880 --> 01:18:08.319
powerful lever for making things on the Internet. And so
1159
01:18:08.600 --> 01:18:14.119
I partnered with the Internet's favorite dad, Aaron Francis, and
1160
01:18:14.520 --> 01:18:20.119
I recorded at the end of last year about sixty
1161
01:18:20.720 --> 01:18:25.520
video video course called High Leverage Rails. So you can
1162
01:18:25.560 --> 01:18:28.319
find it at high Leverage Rails dot com and it
1163
01:18:28.359 --> 01:18:32.640
will be coming out in a couple of weeks. And
1164
01:18:32.760 --> 01:18:38.199
it is a course to walk you through everything that
1165
01:18:38.239 --> 01:18:41.600
you need to know, but nothing you don't. So try
1166
01:18:41.640 --> 01:18:44.439
to keep it as like lean and focus as possible
1167
01:18:45.079 --> 01:18:53.399
to build applications with RAILS and SEQLI. So we walk
1168
01:18:53.439 --> 01:18:58.039
through like an introductory sort of module onlike all right,
1169
01:18:58.159 --> 01:19:02.520
active record and SQLA, what is really happening here under
1170
01:19:02.520 --> 01:19:05.880
the hood. Feel comfortable with how Squalite is working and
1171
01:19:05.960 --> 01:19:09.479
sort of some of these differences around preferring a large
1172
01:19:09.560 --> 01:19:12.159
number of small queries to a small number of large queries.
1173
01:19:13.000 --> 01:19:16.279
And then we just start building a simple application to
1174
01:19:16.359 --> 01:19:20.239
go through all the details, like to really understand what
1175
01:19:20.399 --> 01:19:23.239
is solid Q, what is solid cable, what is solid cash,
1176
01:19:23.640 --> 01:19:27.680
how are they set up with squal light, what are
1177
01:19:27.680 --> 01:19:30.680
the trade offs? What are we doing? So we just
1178
01:19:30.880 --> 01:19:35.439
we step through the whole process of building a basic application,
1179
01:19:35.880 --> 01:19:40.239
focusing on giving people the knowledge and the tools and
1180
01:19:40.279 --> 01:19:43.680
the confidence for them to then go and do this themselves,
1181
01:19:44.479 --> 01:19:46.319
and we go all the way to shipping it to production.
1182
01:19:46.800 --> 01:19:52.600
So we use hatchbox is the tool that I use
1183
01:19:52.800 --> 01:19:58.479
most and think is perfectly well suited to sqalide on
1184
01:19:58.560 --> 01:20:03.680
rails applications, and Chris at gorails has actually sponsored the course,
1185
01:20:04.199 --> 01:20:09.840
which is exciting. So from from rails new to running
1186
01:20:09.840 --> 01:20:16.439
and production, we build a whole host of different features
1187
01:20:16.520 --> 01:20:20.600
to really exercise all of the sort of functionality available
1188
01:20:20.600 --> 01:20:23.199
to us in the rails application. So we set up jobs,
1189
01:20:23.239 --> 01:20:27.079
we have a cash, we send action cable messages, We
1190
01:20:27.199 --> 01:20:33.279
do Jason some fancy Jason stuff. We do full text search.
1191
01:20:33.319 --> 01:20:36.399
We take advantage of sequalize full text search engine dig
1192
01:20:36.399 --> 01:20:40.840
into all these different details. I think it's going to
1193
01:20:40.840 --> 01:20:44.800
be a really excellent resource, particularly for people who are
1194
01:20:45.439 --> 01:20:50.319
interested in a lot of these kinds of philosophies. I've
1195
01:20:49.800 --> 01:20:52.600
been harpening on here on this call around, like I
1196
01:20:52.640 --> 01:20:57.319
want to find leverage, I want to generate value. I
1197
01:20:57.319 --> 01:21:00.760
want to move fast, but I care about quality. I
1198
01:21:00.800 --> 01:21:04.000
care about understanding what I'm doing, right, Like, sure, AI
1199
01:21:04.119 --> 01:21:06.159
can help you move really fast, but in two months,
1200
01:21:06.199 --> 01:21:07.840
when you need to make a change or there's a
1201
01:21:07.880 --> 01:21:11.760
bug and you're wading through the mountain of slop that
1202
01:21:11.880 --> 01:21:13.840
makes no sense to you, and you really don't know
1203
01:21:13.880 --> 01:21:16.479
any of these tools, you're going to be in for
1204
01:21:16.520 --> 01:21:19.439
a world of pain. So really, I really want to
1205
01:21:20.760 --> 01:21:25.439
push this idea that you can just master two very approachable,
1206
01:21:25.520 --> 01:21:30.479
simple tools. Right, It's super easy to get started. There's
1207
01:21:31.720 --> 01:21:36.199
it's straightforward to really genuinely master these tools, and then
1208
01:21:36.399 --> 01:21:39.319
you have the world at your fingertips. Because not only
1209
01:21:39.359 --> 01:21:43.000
can you move fast, can you leverage AI with tools
1210
01:21:43.000 --> 01:21:45.880
that have been around for decades, right, Like there's all
1211
01:21:45.920 --> 01:21:48.239
of these ais have been trained with like millions and
1212
01:21:48.239 --> 01:21:52.039
millions and millions of tokens on these topics, these tools,
1213
01:21:53.119 --> 01:21:54.880
But you can be a true craftsman. You can have
1214
01:21:54.920 --> 01:21:57.840
full control over what you're doing, such that your system
1215
01:21:57.880 --> 01:22:02.039
and your your business can evolve over time without you
1216
01:22:02.159 --> 01:22:04.600
feeling anxious about like what the hell's going on. So
1217
01:22:04.680 --> 01:22:08.680
this middle ground of like finding your leverage, mastering your tools,
1218
01:22:08.680 --> 01:22:12.439
but still moving fast, being responsible and adapted to the market.
1219
01:22:12.479 --> 01:22:15.720
I think this is my sense for like where the
1220
01:22:15.760 --> 01:22:19.479
future of the sort of web economy is going, and
1221
01:22:19.840 --> 01:22:22.000
this is my attempt to sort of help as many
1222
01:22:22.039 --> 01:22:24.479
people as possible to get their foot in the door
1223
01:22:24.880 --> 01:22:28.560
and start making noise on the internet for the betterment
1224
01:22:28.640 --> 01:22:28.880
of us.
1225
01:22:28.920 --> 01:22:33.479
All awesome. So where do you get the course?
1226
01:22:35.439 --> 01:22:38.199
Yep, check it out at high Leverage rails dot com.
1227
01:22:38.640 --> 01:22:40.159
You can sign up for the waitlist. We're going to
1228
01:22:40.239 --> 01:22:43.600
have a early bird discount, so get on the waitlist.
1229
01:22:44.159 --> 01:22:48.279
Planned release is February eighteenth. All the videos have been recorded,
1230
01:22:49.159 --> 01:22:51.600
we've done the first passive editing, so we're just doing
1231
01:22:51.600 --> 01:22:55.000
our final passive editing right now, so I expect that
1232
01:22:55.039 --> 01:22:58.000
it'll come out on February eighteenth, and if you are
1233
01:22:58.000 --> 01:23:01.239
on the waitlist, you get a nice discount. So high
1234
01:23:01.279 --> 01:23:05.720
leverdrails dot com check it out, and I look forward
1235
01:23:05.760 --> 01:23:07.560
to hearing everyone's feedback.
1236
01:23:09.520 --> 01:23:15.159
Awesome. All right, Well I was going to go to picks,
1237
01:23:15.199 --> 01:23:17.600
but you mentioned that you're going to be speaking at
1238
01:23:17.600 --> 01:23:20.600
some conferences in April before the show, so you want
1239
01:23:20.600 --> 01:23:22.119
to just let people know where those are. Two and
1240
01:23:22.159 --> 01:23:23.239
then we'll do our picks.
1241
01:23:24.640 --> 01:23:28.319
Yeah. I literally just before the show found out the
1242
01:23:29.039 --> 01:23:33.880
third one. So in three straight weeks in April, I'm
1243
01:23:33.920 --> 01:23:37.960
going to be bouncing from South Paolo, Brazil. We're obviously
1244
01:23:37.960 --> 01:23:42.079
speaking at Tropical on Rails, then hopping back over the
1245
01:23:42.119 --> 01:23:48.079
ocean to Poland to speak at Fratslavarb and then the
1246
01:23:48.119 --> 01:23:51.000
week after that I have to fly over to Japan.
1247
01:23:51.079 --> 01:23:54.359
I'll be at Ruby Kaigi this year and speaking there.
1248
01:23:54.439 --> 01:23:58.760
So three straight weeks, three different continents. It's going to
1249
01:23:58.800 --> 01:24:02.760
be a fun and take a month. Awesome.
1250
01:24:04.359 --> 01:24:08.199
Well, yeah, if you are going to be at any
1251
01:24:08.239 --> 01:24:12.159
of those conferences, definitely look for Steven. Let's go ahead
1252
01:24:12.159 --> 01:24:14.560
and do some pics. Are you have some picks?
1253
01:24:15.079 --> 01:24:18.119
Yeah? I do. It's funny. Last week I was so
1254
01:24:18.199 --> 01:24:21.439
distracted for about half this podcast because I broke production
1255
01:24:21.560 --> 01:24:22.600
for my client.
1256
01:24:22.760 --> 01:24:26.319
And when you didn't, the database did delete the database
1257
01:24:26.880 --> 01:24:32.760
just causing some mile errors, not the best, and I
1258
01:24:32.760 --> 01:24:35.119
did a pick for my app which I relaunched called
1259
01:24:35.159 --> 01:24:36.560
scatterg on that email.
1260
01:24:36.279 --> 01:24:39.000
And neglected to say what it was about what it
1261
01:24:39.039 --> 01:24:41.840
actually does. So I'm just gonna go ahead and repick
1262
01:24:42.319 --> 01:24:45.159
scatter Gun that email, which is a mailing list platform
1263
01:24:45.199 --> 01:24:49.239
with a focus on privacy and simplicity. So it's like
1264
01:24:50.079 --> 01:24:53.079
my take on stuff like mail Jim, but without all
1265
01:24:53.159 --> 01:24:56.479
the extra marketing stuff. Is like, literally, all you want
1266
01:24:56.560 --> 01:24:59.800
to do is drop a subscription box on your website
1267
01:24:59.840 --> 01:25:03.800
and and email that list. That's kind of what skadagun does.
1268
01:25:03.800 --> 01:25:05.840
So you can just drop a web component into your
1269
01:25:05.840 --> 01:25:08.840
website to collect email addresses, and then scadagun will give
1270
01:25:08.880 --> 01:25:12.119
you an email alis and just fire up your favorite
1271
01:25:12.560 --> 01:25:16.239
email editor and send an email to that alis. My
1272
01:25:16.359 --> 01:25:18.720
app will receive it. It will tack on an unsubscribed
1273
01:25:18.720 --> 01:25:21.039
link at the bottom and send it out your list.
1274
01:25:21.560 --> 01:25:24.680
That's literally all it does. To be honest, it's fairly simple.
1275
01:25:24.720 --> 01:25:27.840
It's always going to be fairly simple. It's never going
1276
01:25:27.920 --> 01:25:30.640
to have fancy marketing features. But if that's the kind
1277
01:25:30.680 --> 01:25:33.439
of thing that you fancy, then give that a go.
1278
01:25:33.640 --> 01:25:36.000
So I've done the proper sales which this time, so
1279
01:25:36.000 --> 01:25:37.680
I'm going to leave that to the side. For next
1280
01:25:37.800 --> 01:25:41.000
picks because last time it's just like after I got
1281
01:25:41.079 --> 01:25:43.359
the college, like I mentioned it and then I forgot
1282
01:25:43.359 --> 01:25:46.680
to say what it actually did. Yeah, that's what breaking
1283
01:25:46.720 --> 01:25:51.560
production will do to your brain. And I'm going to
1284
01:25:51.560 --> 01:25:54.359
do another couple of picks. I'll do a music pick.
1285
01:25:54.479 --> 01:25:56.640
So I think I might have mentioned them on this
1286
01:25:56.680 --> 01:25:59.960
podcast before, but I love a band called Solstice. There
1287
01:26:00.560 --> 01:26:03.640
really grassroots level band here in the UK, and they
1288
01:26:03.680 --> 01:26:07.159
do kind of folk kind of music mixed with rock,
1289
01:26:07.239 --> 01:26:10.560
mixed with progressive rock, weird shit. They've got a new
1290
01:26:10.600 --> 01:26:12.680
album coming out in a couple of months and I've
1291
01:26:12.680 --> 01:26:14.960
got early access to it and i've pretty much I
1292
01:26:15.039 --> 01:26:17.239
got it two days ago and i've had it on
1293
01:26:17.399 --> 01:26:21.199
loop pretty much ever since. There's a track on there
1294
01:26:21.239 --> 01:26:26.039
called twin Peak is the most hauntingly moving piece of
1295
01:26:26.119 --> 01:26:30.439
music I've heard in many, many years. So if that
1296
01:26:30.560 --> 01:26:34.159
sounds vaguely interesting, maybe go check out the last album,
1297
01:26:34.199 --> 01:26:37.960
which is Light Up. And their upcoming album is called Clan,
1298
01:26:38.359 --> 01:26:43.399
which will be out in early April, I believe. And
1299
01:26:44.239 --> 01:26:46.680
I've been going for quite a lot of comedy gigs
1300
01:26:46.680 --> 01:26:50.119
in the last week. So there's a comedian called Robotton
1301
01:26:50.800 --> 01:26:54.319
who I really like He's got a YouTube, an older
1302
01:26:54.359 --> 01:26:57.600
special effice on YouTube for free. So if you like
1303
01:26:58.079 --> 01:27:01.600
we had stand up comedy, then go check out Robot
1304
01:27:01.680 --> 01:27:06.279
and the Time Show. It's on YouTube for free. So, yeah,
1305
01:27:06.319 --> 01:27:08.399
those are my picks.
1306
01:27:10.720 --> 01:27:14.079
Awesome. I'm gonna throw out a few picks here. So
1307
01:27:14.319 --> 01:27:21.840
I've gotten really deep into the launch for Ruby Geniuses
1308
01:27:21.920 --> 01:27:24.840
and JavaScript Geniuses, so just to give you a quick
1309
01:27:24.880 --> 01:27:26.800
rundown on those, and then I'll do my board game
1310
01:27:26.840 --> 01:27:31.520
pick and stuff like that. So and I'll try not
1311
01:27:31.600 --> 01:27:34.000
to be too long winded. When I got into programming
1312
01:27:34.199 --> 01:27:36.640
and the times where I felt like I was really progressing,
1313
01:27:36.640 --> 01:27:38.520
I had a handful of things really going for me.
1314
01:27:39.800 --> 01:27:41.920
I had meetups that I could go to and people
1315
01:27:41.920 --> 01:27:44.479
I could talk to with the meetups. I had a
1316
01:27:44.520 --> 01:27:50.119
mentor that I was working with at the time, I
1317
01:27:50.239 --> 01:27:53.960
was watching rails casts. Anyway, there are a bunch of
1318
01:27:54.000 --> 01:27:57.439
things that I wanted. We kind of did book clubs,
1319
01:27:58.279 --> 01:28:00.479
I got some mentorship from the guy I was doing
1320
01:28:00.560 --> 01:28:03.520
Ruby Rogues with way back in the day. There were
1321
01:28:03.520 --> 01:28:09.000
conferences and anyway, So what I'm looking to provide to
1322
01:28:09.039 --> 01:28:13.359
people is that kind of experience because a lot of
1323
01:28:13.359 --> 01:28:16.479
these things have kind of gone away or I've just
1324
01:28:16.560 --> 01:28:18.399
kind of advanced to the point where I'm not working
1325
01:28:18.399 --> 01:28:21.840
with anybody who really pushes me on tech or anything
1326
01:28:22.000 --> 01:28:26.479
Ruby or anything like that. And so you know, I
1327
01:28:26.479 --> 01:28:28.479
get a certain level of that from this show. You know,
1328
01:28:28.520 --> 01:28:31.239
because we're talking to Steven. Stephen knows way more about
1329
01:28:32.279 --> 01:28:34.359
Sequelight and some of this stuff that I do, right,
1330
01:28:34.479 --> 01:28:37.039
so I can ask them all my questions. But what
1331
01:28:37.119 --> 01:28:39.079
I wanted to do was just put together this system.
1332
01:28:39.119 --> 01:28:43.079
And so I'm scheduling meetups right now. We'll do a
1333
01:28:43.079 --> 01:28:46.560
meetup every week. One of the meetups every month is
1334
01:28:46.600 --> 01:28:50.199
going to be something about AI because that seems to
1335
01:28:50.199 --> 01:28:54.159
be where people are at. And then we're going to
1336
01:28:54.199 --> 01:28:55.920
be I'm going to be putting out videos on how
1337
01:28:55.960 --> 01:28:59.640
to do stuff, primarily with rails. Like I said, I'm
1338
01:28:59.640 --> 01:29:01.439
also doing in the cell on the JavaScript end, and
1339
01:29:01.479 --> 01:29:08.279
so that'll probably focus more on React. But yeah, we're
1340
01:29:08.319 --> 01:29:10.840
going to have a place where you can come and chat.
1341
01:29:12.399 --> 01:29:17.359
There are going to be premium podcast episodes, and you'll
1342
01:29:17.359 --> 01:29:19.319
get discounts to any of the summits that I put out.
1343
01:29:19.359 --> 01:29:21.479
So last year I did a summit on just where
1344
01:29:21.520 --> 01:29:24.640
Ruby was headed, and so this year I'm looking at
1345
01:29:24.680 --> 01:29:26.479
doing some more of that stuff, probably in the summer,
1346
01:29:27.640 --> 01:29:30.000
and you'll also get discounts on retreats. I want to
1347
01:29:30.000 --> 01:29:32.680
start doing retreats where you know, we kind of get
1348
01:29:32.880 --> 01:29:36.119
like a big giant Airbnb that has a room that
1349
01:29:36.159 --> 01:29:39.520
can hold twenty or thirty people and invite you know,
1350
01:29:39.560 --> 01:29:42.000
a handful of people, so you know, maybe Steven or
1351
01:29:42.039 --> 01:29:44.560
maybe a USh or you know, if we can get
1352
01:29:44.640 --> 01:29:50.680
Uncle Bob Martin or DHH or Aaron Patterson or some
1353
01:29:50.720 --> 01:29:53.159
of these folks that you all know, right, so we
1354
01:29:53.199 --> 01:29:56.199
all get into a room and just talk about how
1355
01:29:56.239 --> 01:29:58.079
we can program better. But you kind of get that
1356
01:29:58.119 --> 01:30:01.760
FaceTime with somebody who's I guess sort of at that
1357
01:30:01.880 --> 01:30:05.960
higher level of access, and also you know somebody that
1358
01:30:06.000 --> 01:30:11.680
you want to sit and pick their brain. So anyway,
1359
01:30:11.960 --> 01:30:14.479
that's that's the deal. You can go to rubygeniuses dot
1360
01:30:14.520 --> 01:30:17.760
com and sign up. I've also been sending out emails
1361
01:30:17.760 --> 01:30:21.520
to my email list. I signed up for the system
1362
01:30:21.520 --> 01:30:24.119
I'm using before I knew about scattergun. Sorry I usual,
1363
01:30:25.600 --> 01:30:29.399
but anyway, so just go to rubygeniuses dot com if
1364
01:30:29.439 --> 01:30:34.479
you sign up before Valentine's Day February fourteenth. I'm also
1365
01:30:34.560 --> 01:30:38.279
throwing in a one hour coaching session, and my coaching
1366
01:30:38.319 --> 01:30:41.840
rates about two hundred dollars an hour, and so that's
1367
01:30:41.880 --> 01:30:45.159
a two hundred dollars value right there. And then the
1368
01:30:45.199 --> 01:30:48.960
other thing I'm throwing in is either over WhatsApp or
1369
01:30:49.039 --> 01:30:52.680
something similar, I'm offering text and voice coaching. So you'll
1370
01:30:52.720 --> 01:30:56.319
send me a message, hey, I got stuck doing this right,
1371
01:30:56.359 --> 01:30:58.199
and then I'll get back to you and say, Okay,
1372
01:30:59.159 --> 01:31:01.319
this is how I would do it, or you know,
1373
01:31:01.399 --> 01:31:03.319
these are the things that I would look at or
1374
01:31:03.359 --> 01:31:06.000
you know whatever, right anywhere up and down the chain
1375
01:31:06.000 --> 01:31:10.880
from performance to how do I do this particular thing?
1376
01:31:11.600 --> 01:31:13.840
And if I don't have the answer, the nice thing
1377
01:31:13.920 --> 01:31:16.159
is is I've been doing things like Ruby rogues long
1378
01:31:16.279 --> 01:31:19.199
enough to where I probably know somebody who knows the answer,
1379
01:31:19.239 --> 01:31:20.880
and so I should be able to track it down.
1380
01:31:21.600 --> 01:31:24.319
So anyway, that's what that's what we're putting together with
1381
01:31:24.439 --> 01:31:26.960
rubygeniuses dot com. You can sign up for a month,
1382
01:31:27.039 --> 01:31:28.359
you can sign up for a year, or you can
1383
01:31:28.439 --> 01:31:31.720
sign up forever. Those are the prices, So go to
1384
01:31:31.760 --> 01:31:35.840
the website check that out. And yeah, I'm super excited
1385
01:31:35.840 --> 01:31:38.800
to be collaborating with people I did leave off. We're
1386
01:31:38.800 --> 01:31:40.439
going to be doing a monthly book club as well,
1387
01:31:40.439 --> 01:31:43.960
So we're going to be picking up something like Pragmatic
1388
01:31:44.000 --> 01:31:48.960
Programmers or my book, yeah, or your book. We might
1389
01:31:49.000 --> 01:31:52.680
do that, right, and so then it's you know, you
1390
01:31:52.800 --> 01:31:56.520
show up, so you've read the book, probably worked through it,
1391
01:31:56.560 --> 01:31:59.800
and then you know, we show up. Usually the authors
1392
01:31:59.800 --> 01:32:01.680
there because I know a lot of them too, and
1393
01:32:01.720 --> 01:32:04.439
we'll do that so that that's all part of the deal.
1394
01:32:04.760 --> 01:32:07.119
Some of them might be combined with the JavaScript group
1395
01:32:07.239 --> 01:32:12.840
if if the book is more generic programming. But I'm
1396
01:32:12.880 --> 01:32:15.600
not going to shy away from picking you know, a
1397
01:32:15.720 --> 01:32:18.960
solid Ruby book or a solid JavaScript book for the
1398
01:32:18.960 --> 01:32:20.720
other group, and we'll probably do both of those in
1399
01:32:20.760 --> 01:32:24.319
the same month so that we can have separate meetings.
1400
01:32:24.319 --> 01:32:29.159
But anyway, that's what we're pulling together. I just people
1401
01:32:29.239 --> 01:32:32.239
are looking to connect. A lot of times people find
1402
01:32:32.319 --> 01:32:35.239
jobs through these things. There's going to be some level
1403
01:32:35.319 --> 01:32:39.880
of career growth, job hunt content to it, just because
1404
01:32:39.880 --> 01:32:41.319
I've talked to a whole bunch of people are looking
1405
01:32:41.359 --> 01:32:44.439
for that. But that's what we're putting together. So Rubygeniuses
1406
01:32:44.479 --> 01:32:51.319
dot Com. Now, as far as board games go, I'm
1407
01:32:51.319 --> 01:32:55.720
trying to remember I think last week, No, I don't
1408
01:32:55.720 --> 01:32:57.319
think I did pick it last week, so I'll pick
1409
01:32:57.319 --> 01:33:00.560
it this week. I played a game with my my friends.
1410
01:33:00.560 --> 01:33:03.880
So one of my friends is teaching the hot games
1411
01:33:03.960 --> 01:33:08.800
at SaltCON, which is a conference in March in Layton, Utah,
1412
01:33:09.279 --> 01:33:11.960
north of Salt Lake where you show up and you
1413
01:33:12.119 --> 01:33:14.119
play games with people. I went last year. I'm probably
1414
01:33:14.119 --> 01:33:18.279
not gonna be able to go this year. But anyway,
1415
01:33:18.279 --> 01:33:20.199
he's teaching the game, so he's trying to learn the games.
1416
01:33:20.920 --> 01:33:24.479
And we played a game last week called Cascadero, and
1417
01:33:24.840 --> 01:33:31.239
Cascadero is you're effectively placing a little horseman and they're
1418
01:33:31.399 --> 01:33:36.880
carrying word about the new king's ascension to the throne.
1419
01:33:37.079 --> 01:33:39.159
Not that that has any bearing on anything, it's just
1420
01:33:39.239 --> 01:33:46.199
kind of the premise of the game. And then yeah,
1421
01:33:46.239 --> 01:33:50.960
so when you place your piece next to a city,
1422
01:33:51.840 --> 01:33:55.600
then if it's part of a group, so if it's
1423
01:33:55.640 --> 01:33:59.039
your second piece in the group or higher, then you
1424
01:33:59.119 --> 01:34:03.520
get to move move up the progress track. And as
1425
01:34:03.520 --> 01:34:07.239
you move up the progress track, you get points and
1426
01:34:07.680 --> 01:34:09.560
you win at the end of the game if you
1427
01:34:09.600 --> 01:34:12.640
have the most points and you've gotten your color of
1428
01:34:12.680 --> 01:34:14.439
your marker all the way to the end of the
1429
01:34:14.439 --> 01:34:17.720
board board game. Geek weights at at two point five
1430
01:34:17.760 --> 01:34:20.000
to three. I liked it. I don't know if I
1431
01:34:20.000 --> 01:34:22.119
would go buy it, but I did like it. It was fun.
1432
01:34:22.840 --> 01:34:25.239
I kind of want to play it again, just because
1433
01:34:26.760 --> 01:34:28.800
some of the mechanics were a little bit odd to
1434
01:34:28.840 --> 01:34:32.039
pick up, and now that I understand them, right, I'd
1435
01:34:32.079 --> 01:34:33.720
like to play it again and see if I can
1436
01:34:34.399 --> 01:34:39.800
do better. It says it's age is fourteen plus. The
1437
01:34:39.840 --> 01:34:42.439
community says ten plus can play it. That's probably fair.
1438
01:34:43.119 --> 01:34:45.319
We played with three players. It's two to four players,
1439
01:34:45.880 --> 01:34:49.760
and anyway, it was kind of a fun game. It's done.
1440
01:34:50.319 --> 01:34:54.880
It's made by what's his name, Rainer Kenitsia. I'm sure
1441
01:34:54.920 --> 01:34:58.159
I'm destroying his name. I think he's German or something,
1442
01:34:59.359 --> 01:35:04.520
but anyway, his he's done a bunch of other games
1443
01:35:04.520 --> 01:35:07.720
that you've probably played if you're in the board game.
1444
01:35:10.319 --> 01:35:10.720
Area.
1445
01:35:11.319 --> 01:35:15.560
So another one that I picked was lem. Anyway, he's
1446
01:35:15.600 --> 01:35:21.279
put out dozens and dozens of games, and so Lost
1447
01:35:21.279 --> 01:35:26.560
Cities I think I've picked on here before. So anyway, yeah,
1448
01:35:26.640 --> 01:35:29.680
go check it out. I'll put a link into board
1449
01:35:29.720 --> 01:35:32.840
game Geek where they have that rating, and then I'll
1450
01:35:32.880 --> 01:35:35.199
also put a link in for Amazon so you can
1451
01:35:35.199 --> 01:35:38.359
go buy it on my affiliate which doesn't count cost
1452
01:35:38.399 --> 01:35:41.199
you anymore, but I get a kick back if you
1453
01:35:41.279 --> 01:35:44.920
use my link. So anyway, I've gone on way longer
1454
01:35:44.920 --> 01:35:47.600
than I wanted to, so I will let Steven do
1455
01:35:47.680 --> 01:35:48.319
some picks.
1456
01:35:50.720 --> 01:35:55.000
Yeah, so my first pick a little bit of a
1457
01:35:56.000 --> 01:35:58.680
I don't know play, I guess on the word. But
1458
01:35:58.760 --> 01:36:01.439
my first pick is going to be the city of Philadelphia,
1459
01:36:03.000 --> 01:36:07.000
and there are two reasons why. The first reason is
1460
01:36:07.039 --> 01:36:11.640
that I am picking the Philadelphia Eagles to become Super
1461
01:36:11.640 --> 01:36:17.880
Bowl champions this Sunday in Super Bowl fifty nine. Just
1462
01:36:17.960 --> 01:36:22.479
for a quick bit of context, Born and raised in Louisiana,
1463
01:36:23.439 --> 01:36:26.600
but lived in Philadelphia for about eight years before I
1464
01:36:26.680 --> 01:36:32.560
moved to Berlin six years ago, and I just can't
1465
01:36:33.239 --> 01:36:35.560
I don't know how anybody lives in Philadelphia and doesn't
1466
01:36:35.600 --> 01:36:38.000
become an Eagles fans. I think they put something in
1467
01:36:38.039 --> 01:36:44.079
the water. So I'm a big Eagles fan. But the
1468
01:36:44.159 --> 01:36:47.359
second reason that Philadelphia is my pick is that I
1469
01:36:47.439 --> 01:36:51.720
am super excited for Rails Comp this year. For anyone
1470
01:36:51.760 --> 01:36:55.800
who doesn't know, this is the final Rails comp. Ruby
1471
01:36:55.800 --> 01:36:59.079
Central has been putting on Rails Comp for over a
1472
01:36:59.119 --> 01:37:02.720
decade now, I don't even know how many years, and
1473
01:37:03.600 --> 01:37:06.439
they have decided to focus their efforts on a single
1474
01:37:06.479 --> 01:37:09.760
conference each year, and so this year they are doing
1475
01:37:09.880 --> 01:37:12.279
Rails Comp but not Ruby Comp, and then moving forward
1476
01:37:12.279 --> 01:37:16.199
they will only be doing Ruby Comp. And so the
1477
01:37:16.319 --> 01:37:22.880
very last Rails Comp ever, is being hosted in Philadelphia, Pennsylvania,
1478
01:37:23.720 --> 01:37:27.119
which will be the only time that a Ruby Central
1479
01:37:27.119 --> 01:37:32.960
conference has ever been in Philadelphia. I know the hotel
1480
01:37:33.159 --> 01:37:35.880
that the conferences is happening at, I know the area
1481
01:37:35.920 --> 01:37:40.840
where the conference is happening. There's like really great food options,
1482
01:37:41.079 --> 01:37:44.720
really great experience. It's going to be an absolute blast.
1483
01:37:46.079 --> 01:37:49.319
I am really looking forward to that. The call for
1484
01:37:49.359 --> 01:37:51.399
papers just opened up and that's going to be open
1485
01:37:51.479 --> 01:37:56.439
for all of February. I recommend everyone to take the
1486
01:37:56.520 --> 01:38:00.399
time to like sit down and gather your thoughts, try
1487
01:38:00.439 --> 01:38:07.439
to present a interesting, cogent narrative for something that you
1488
01:38:07.760 --> 01:38:10.960
have thought about or done recently. Like all of us
1489
01:38:10.960 --> 01:38:15.479
have done interesting things, all of us have had interesting experiences,
1490
01:38:16.039 --> 01:38:20.920
and there is so much value in even just preparing
1491
01:38:20.920 --> 01:38:25.159
a proposal, let alone you getting get accepted and then
1492
01:38:25.159 --> 01:38:29.279
preparing a talk and getting involved in that side of
1493
01:38:30.680 --> 01:38:35.920
the conference experience. So big year for Philadelphia. Let's go
1494
01:38:36.199 --> 01:38:43.279
Eagles and hopefully I'm not too sad on Monday, but
1495
01:38:43.319 --> 01:38:47.239
that's going to be my first pick. Just for a
1496
01:38:47.279 --> 01:38:56.880
technology related pick. I have been using a couple of
1497
01:38:56.960 --> 01:39:02.039
gems from my friend and elaborator in the open source world,
1498
01:39:02.159 --> 01:39:06.840
Joel Drapper for some of my more recent projects, and
1499
01:39:06.840 --> 01:39:08.319
I just want to shout them out because I've really
1500
01:39:08.399 --> 01:39:10.800
enjoyed using them so too. In particular, for those who
1501
01:39:10.840 --> 01:39:14.319
don't know, Joel Drappers, the creator of flex. Flex is
1502
01:39:14.520 --> 01:39:21.039
the view library in Ruby that allows you to write
1503
01:39:21.199 --> 01:39:28.520
HTML with pure Ruby, so no templating string, templating syntax ERB,
1504
01:39:28.680 --> 01:39:32.199
you just write normal Ruby. That's a great project. But
1505
01:39:32.239 --> 01:39:36.720
he also has a library called Literal, which we're going
1506
01:39:36.800 --> 01:39:39.960
to be talking more about at Bratslav in Poland in April.
1507
01:39:40.840 --> 01:39:45.600
It's called Literal and it's a really really fascinating project.
1508
01:39:46.600 --> 01:39:48.720
So I'm just going to tease more of the details
1509
01:39:48.720 --> 01:39:50.439
of our talk there. The name of our talk is
1510
01:39:50.479 --> 01:39:54.760
going to be Ruby has always had types, and for
1511
01:39:54.800 --> 01:39:58.119
those of you who don't know, Ruby has always had types, uh,
1512
01:39:58.439 --> 01:40:04.720
and Literal sort of finishes the type system that has
1513
01:40:04.760 --> 01:40:06.640
existed in Ruby for a long time and the way
1514
01:40:06.640 --> 01:40:09.079
that it does that and the way that Ruby has types.
1515
01:40:09.119 --> 01:40:12.039
Is that there is a run time type system. So
1516
01:40:13.159 --> 01:40:15.039
what does that mean? What can you do with that?
1517
01:40:15.319 --> 01:40:18.840
How is that different from a statically typed language, a
1518
01:40:18.880 --> 01:40:22.359
compiled time typing system. There's a lot of interesting stuff
1519
01:40:22.399 --> 01:40:24.560
to stay there, and I'll just leave you teas. Come
1520
01:40:24.640 --> 01:40:27.640
check it out at bratslall RB in April, or just
1521
01:40:28.399 --> 01:40:31.720
go check out literal dot fund read the documentation. That
1522
01:40:31.840 --> 01:40:36.399
is a really excellent library. And then a newer library
1523
01:40:36.439 --> 01:40:38.800
that is still sort of in beta but is really
1524
01:40:38.880 --> 01:40:43.199
exciting is called quick draw. That is a testing framework
1525
01:40:43.239 --> 01:40:47.920
and test runner, so basically a direct competitor to urspec
1526
01:40:47.960 --> 01:40:51.920
and mini test. And the big selling point is that
1527
01:40:52.079 --> 01:40:56.680
it is built from the ground up specifically for our
1528
01:40:56.760 --> 01:41:03.760
modern multicore environment, and so it has a highly optimized
1529
01:41:03.800 --> 01:41:10.199
handwritten test runner that distributes your tests as efficiently as
1530
01:41:10.880 --> 01:41:15.319
possible across all of your computer's cores, and so it
1531
01:41:15.479 --> 01:41:17.880
runs at you know, if you have like a ten
1532
01:41:17.920 --> 01:41:24.039
core machine, about ten times faster than the equivalent in
1533
01:41:24.119 --> 01:41:30.720
ourspec and it is just incredibly fast. It is so cool.
1534
01:41:30.720 --> 01:41:33.840
It has a watch mode and you can just like
1535
01:41:33.920 --> 01:41:36.399
leave it on and you're just saving your files and
1536
01:41:37.600 --> 01:41:40.960
it just finishes. I am working on a sequel like parser,
1537
01:41:41.640 --> 01:41:44.119
and I have a test suite currently at about thirty
1538
01:41:44.159 --> 01:41:48.319
five thousand assertions and I just leave watch mode and
1539
01:41:48.439 --> 01:41:51.159
it runs in like on my M one Mac in
1540
01:41:51.239 --> 01:41:56.039
like seven hundred milliseconds, so I just am refactoring away
1541
01:41:56.079 --> 01:41:59.880
and letting it sit in watch mode. So really, really
1542
01:42:01.119 --> 01:42:04.079
awesome projects. I've I've really enjoyed working on them with
1543
01:42:04.159 --> 01:42:06.239
him and using them in my work for the last
1544
01:42:06.279 --> 01:42:08.279
few so I definitely recommend people check those out.
1545
01:42:10.479 --> 01:42:14.560
Very cool. Yeah, a lot of stuff to check out there. Well,
1546
01:42:14.560 --> 01:42:19.159
thanks for coming. I'm not gonna talk about the Super
1547
01:42:19.199 --> 01:42:21.359
Bowl because I like both teams. I'm trying to decide
1548
01:42:21.359 --> 01:42:26.399
which way I want to go, So anyway, thanks for coming, Steven,
1549
01:42:27.279 --> 01:42:27.760
thank you so.
1550
01:42:27.760 --> 01:42:29.479
Much for having me. It was a blast to talk
1551
01:42:29.520 --> 01:42:31.399
about all this stuff with y'all. Yeah.
1552
01:42:31.439 --> 01:42:34.760
Absolutely, Well, let's wrap up until next time, folks. Max
1553
01:42:34.800 --> 01:42:35.000
out