TRASCRIZIONE 🔗
WEBVTT
1
00:00:04.679 --> 00:00:07.240
Hey, folks, welcome back to another episode of the Ruby
2
00:00:07.320 --> 00:00:10.400
Rogues podcast. I'm your host this week, Charles max Wood,
3
00:00:10.400 --> 00:00:12.880
and I am here with Carmine Paulino.
4
00:00:14.279 --> 00:00:18.120
And you said you're from Berlin, from Italy, Southern Italy.
5
00:00:18.519 --> 00:00:20.440
I always get excited when I have Italian song because
6
00:00:20.440 --> 00:00:21.359
I lived in Italy.
7
00:00:21.120 --> 00:00:23.239
For a couple of years, and I'm like, oh, all
8
00:00:23.280 --> 00:00:26.039
the good memories anyway, and the good memories are the
9
00:00:26.079 --> 00:00:28.960
good food and good people anyway. But yeah, you live
10
00:00:29.000 --> 00:00:32.560
in Berlin and you wrote the Ruby llm jem. I'm
11
00:00:32.600 --> 00:00:35.479
sure there's a lot more to know about you. What
12
00:00:35.520 --> 00:00:36.320
else should people know?
13
00:00:36.399 --> 00:00:38.320
And go, oh, Carmine is so cool.
14
00:00:39.880 --> 00:00:42.280
Well, thank you very much for the introduction, Charles. It's
15
00:00:42.280 --> 00:00:46.280
a pleasure to be here. So I started using technology
16
00:00:46.280 --> 00:00:49.200
when I was really, really young. So at the age
17
00:00:49.200 --> 00:00:51.560
of five, I started to write my first batch scripts
18
00:00:52.000 --> 00:00:57.320
to automates launching my favorite games, and damn, moment of
19
00:00:57.359 --> 00:01:00.799
making a computer do exactly what I wanted felt like magic.
20
00:01:01.159 --> 00:01:06.040
So that sparked lifelong fascination with technology. And at the
21
00:01:06.079 --> 00:01:10.400
moment I am building like AI tools, you know, I'm
22
00:01:10.400 --> 00:01:13.840
building chat with work, which is this retrieval advantage generation
23
00:01:14.040 --> 00:01:17.239
app that you can connect to your Google Drive and
24
00:01:17.359 --> 00:01:24.760
Slack Notion accounts essentially chat with all your documents, just
25
00:01:24.799 --> 00:01:27.239
as if Chgpt or Claud.
26
00:01:27.000 --> 00:01:28.239
Would know anything about them.
27
00:01:29.120 --> 00:01:32.040
I also co founded fresh Flow, which is also an
28
00:01:32.079 --> 00:01:35.680
AI based startup that helps supermarkets reduce food waste and
29
00:01:35.719 --> 00:01:40.079
maximize their profits. And yeah, I've done a bunch of things.
30
00:01:40.640 --> 00:01:41.120
Essentially.
31
00:01:42.159 --> 00:01:46.439
I graduated in computer science from University of Bologna, then
32
00:01:46.480 --> 00:01:51.640
did my master in AI from Free University of Amsterdam,
33
00:01:51.799 --> 00:01:54.319
and then I've been a data scientist ever since. But
34
00:01:54.680 --> 00:01:58.400
I think relevant to this podcast is that my last
35
00:01:58.680 --> 00:02:02.319
project at the Universe Still Bologna, which was for Internet
36
00:02:02.400 --> 00:02:06.879
and Web Technologies, was actually in Ruby. And the story
37
00:02:06.920 --> 00:02:10.400
goes that, you know, it was supposed to be in
38
00:02:10.400 --> 00:02:16.439
PHP and at the time, mind you, yeah, at the time,
39
00:02:16.560 --> 00:02:18.280
I was already.
40
00:02:20.039 --> 00:02:20.840
It's a great.
41
00:02:20.639 --> 00:02:26.639
Story because I was already looking at acer News and
42
00:02:27.280 --> 00:02:29.680
at the time, all my team, you know, we were
43
00:02:29.680 --> 00:02:34.639
four people, they all knew PHP and I didn't, right,
44
00:02:35.039 --> 00:02:37.520
But I followed after News and I saw Ruby on
45
00:02:37.599 --> 00:02:41.120
rails and this was, I believe, if I remember correctly,
46
00:02:41.280 --> 00:02:46.159
was twenty ten, so you know the era, right, And
47
00:02:46.800 --> 00:02:49.599
I was like, so intrigued by Ruby on Rails at
48
00:02:49.639 --> 00:02:54.000
the time that I convinced my entire team to switch
49
00:02:54.039 --> 00:02:56.840
to Ruby and rails and we all had so much
50
00:02:56.879 --> 00:02:59.759
fun that actually some of us went to work for
51
00:03:00.319 --> 00:03:02.639
other companies that use Ruby and rails. In fact, one
52
00:03:02.639 --> 00:03:05.400
of my friends I went to work for SoundCloud here
53
00:03:05.439 --> 00:03:08.120
in Berlin. I even thought about doing that, I even
54
00:03:08.159 --> 00:03:10.520
went through the application, but then I thought, hey, I
55
00:03:10.560 --> 00:03:13.759
want to have a master degree and you know, maybe
56
00:03:13.759 --> 00:03:17.560
look at something else as well. And that's where I
57
00:03:17.560 --> 00:03:21.680
started looking at AI. And I was a data scientist
58
00:03:21.800 --> 00:03:24.719
for many years. Whenever I could do anything that was
59
00:03:24.759 --> 00:03:26.680
related to the web, it was Ruby on rails anyway,
60
00:03:27.800 --> 00:03:33.039
but mostly Python, mostly machine learning engineering, you know, trying
61
00:03:33.080 --> 00:03:37.719
to scale also systems, so between you know, the DevOps
62
00:03:37.759 --> 00:03:40.439
to the modeling level, right, So it was kind of
63
00:03:40.439 --> 00:03:44.400
a full stack, let's say, machine learning person for a
64
00:03:44.439 --> 00:03:47.879
long time. And then well I decided to go back
65
00:03:47.919 --> 00:03:51.520
a little bit to academia. So I went to the
66
00:03:51.560 --> 00:03:55.400
volunteering Institute in the UK, mostly helping out with an
67
00:03:55.479 --> 00:04:00.960
organization called that as Signs for Social Good. And then
68
00:04:01.560 --> 00:04:05.080
I received a call from entrepreneur first and they told
69
00:04:05.080 --> 00:04:07.400
me like, hey, you have a profile of a potential founder.
70
00:04:07.960 --> 00:04:10.800
I was like, really, because that's that was my dream
71
00:04:10.919 --> 00:04:13.719
and I never thought I could do that. So I
72
00:04:13.800 --> 00:04:15.960
went for it and then I found it fresh Flow
73
00:04:17.000 --> 00:04:18.920
and now I'm actually.
Load full transcript
74
00:04:18.560 --> 00:04:19.240
Doing my own thing.
75
00:04:19.240 --> 00:04:23.399
I'm a slopreneur and I'm building chat with work and
76
00:04:23.480 --> 00:04:27.079
so to connect back to rubylm since now I'm completely
77
00:04:27.079 --> 00:04:30.399
free and I'm doing you know, like a web app essentially,
78
00:04:31.279 --> 00:04:35.639
it was obviously done in Ruby and rails and at
79
00:04:35.680 --> 00:04:39.560
the time and maybe I'm jumping too far, but at
80
00:04:39.600 --> 00:04:41.720
the time, I wanted to do it because.
81
00:04:44.279 --> 00:04:45.199
It's all connected.
82
00:04:46.720 --> 00:04:52.160
I was searching for like libraries for building chat with
83
00:04:52.319 --> 00:04:56.639
work and I saw some of the libraries out there
84
00:04:56.759 --> 00:04:59.720
and I wasn't so satisfied with them, so I decided
85
00:04:59.720 --> 00:05:00.639
to build my own.
86
00:05:02.240 --> 00:05:04.639
Right Yeah.
87
00:05:05.000 --> 00:05:07.560
So yeah, it's it's interesting kind of getting into that
88
00:05:07.680 --> 00:05:10.360
a bit. And some of this reflects, you know, with
89
00:05:10.399 --> 00:05:13.279
my own background, right, you know, with you know, I
90
00:05:13.319 --> 00:05:15.000
was kind of a solopreneur. I had a couple of
91
00:05:15.040 --> 00:05:17.959
virtual assistants, but I mostly just you know, ran things
92
00:05:18.000 --> 00:05:21.480
myself with the podcast network for I don't know, like
93
00:05:22.040 --> 00:05:25.120
eight years, and you know, I built most of the
94
00:05:25.120 --> 00:05:30.079
software that the podcast run on and you know, just
95
00:05:30.079 --> 00:05:32.839
just on and on and on and on. I'm not
96
00:05:32.839 --> 00:05:36.439
going to get into kind of the emotional wreckage that
97
00:05:36.480 --> 00:05:41.160
I went through the last few years, especially with you know, now,
98
00:05:41.199 --> 00:05:43.480
I'm in a full time job and I love the
99
00:05:43.480 --> 00:05:45.120
people I work with, and I love the projects I
100
00:05:45.199 --> 00:05:48.439
work on. But there's just something about working on your
101
00:05:48.480 --> 00:05:51.079
own thing and having it, you know, come together in
102
00:05:51.120 --> 00:05:51.439
that way.
103
00:05:51.519 --> 00:05:54.680
So yeah, yeah, I really identify with a lot of
104
00:05:54.680 --> 00:05:58.120
what you're talking about. But yeah, I wanted to.
105
00:05:59.240 --> 00:06:01.240
I wanted to jump in and just start talking about
106
00:06:01.279 --> 00:06:04.079
Ruby LM, and I know that the like the AI
107
00:06:04.199 --> 00:06:07.240
space and the LLM space is pretty hot right now.
108
00:06:08.759 --> 00:06:10.680
And that's one of the things that I've been wanting
109
00:06:10.680 --> 00:06:15.240
to put together for a while. Is specifically you said, hey,
110
00:06:15.399 --> 00:06:17.439
you know, you can get in, you can chat with.
111
00:06:17.399 --> 00:06:19.639
Your dropbox or your Google Driver or whatever.
112
00:06:20.160 --> 00:06:22.839
And for me, I want the same thing, but I
113
00:06:22.879 --> 00:06:24.639
want to be able to do it with my podcasts.
114
00:06:24.720 --> 00:06:28.319
Right It's like, hey, you know, change the host out right.
115
00:06:28.439 --> 00:06:31.600
So like on Javascriptjbver, we just had somebody, you know,
116
00:06:31.600 --> 00:06:33.439
he's like, hey, I've got to take some time away
117
00:06:33.560 --> 00:06:36.639
and write, so things like that. You know, he's the
118
00:06:36.720 --> 00:06:40.519
last guy that's one of the originals besides me, Right,
119
00:06:40.639 --> 00:06:42.759
so chat with it about that, or Hey, how's the
120
00:06:42.800 --> 00:06:47.360
traffic look, or you know, publish this episode on this
121
00:06:47.439 --> 00:06:49.120
date instead of that date and that kind of a thing.
122
00:06:49.160 --> 00:06:52.920
And so yeah, I got excited about it.
123
00:06:53.000 --> 00:06:57.519
And then the local meetup the last like three months
124
00:06:57.519 --> 00:07:01.560
have all had presentations on how to through API's hit.
125
00:07:03.360 --> 00:07:05.199
LLM end points basically.
126
00:07:05.560 --> 00:07:09.079
And you know so, and I've been playing with some
127
00:07:09.160 --> 00:07:11.480
of that, and we've had Obi Fernandez on to talk
128
00:07:11.519 --> 00:07:15.199
about Ray and so I got everything hooked up and
129
00:07:16.800 --> 00:07:19.480
was figuring all that stuff out. Ray is our AIX
130
00:07:19.519 --> 00:07:21.560
for the people who are listening and trying to go
131
00:07:21.600 --> 00:07:26.040
find it, and he talked about it before, but I
132
00:07:26.399 --> 00:07:29.399
found that the documentation didn't give me clear direction on
133
00:07:29.680 --> 00:07:33.040
how to get everything together the way I wanted. And
134
00:07:33.319 --> 00:07:36.160
he kind of implies that you can stream the response back,
135
00:07:37.480 --> 00:07:41.959
but he basically says it's in my book, and I went,
136
00:07:42.000 --> 00:07:43.279
I have the book, and I went looked in the
137
00:07:43.279 --> 00:07:46.800
book and I got lost. So but but I did
138
00:07:46.879 --> 00:07:49.560
have a fundamental thing working to do the presentation at
139
00:07:49.560 --> 00:07:52.120
the meet up last night, and I was doing a
140
00:07:52.199 --> 00:07:54.839
chat bot and I just hooked it up to OpenWeatherMap
141
00:07:55.160 --> 00:07:57.480
dot org and so it was just you know so.
142
00:07:57.959 --> 00:07:59.600
But then I was showing him, Look, you don't just
143
00:07:59.639 --> 00:08:02.040
ask it, hey, what's the weather. Here's the latitude longitude.
144
00:08:02.279 --> 00:08:05.120
But I put in, is a good is it a
145
00:08:05.120 --> 00:08:07.079
good time right now to go have a picnic?
146
00:08:07.120 --> 00:08:07.560
And right?
147
00:08:07.600 --> 00:08:12.199
It came back and said the weather is right. So
148
00:08:12.279 --> 00:08:16.279
it figured out my
cities at this latitude longitude and right.
149
00:08:16.319 --> 00:08:18.639
And this is the power of the AI, right, is
150
00:08:18.680 --> 00:08:21.240
that it's smart enough to figure out some of the context.
151
00:08:21.399 --> 00:08:24.439
And I'm being really long winded and I'm sorry, but anyway,
152
00:08:24.480 --> 00:08:27.079
what happened was, for some reason, the open router gem
153
00:08:28.040 --> 00:08:33.120
it was working Tuesday night, and then Wednesday morning when
154
00:08:33.159 --> 00:08:35.919
I was fiddling with it, it just wouldn't connect and
155
00:08:35.919 --> 00:08:36.519
it was given.
156
00:08:36.320 --> 00:08:37.320
Me a four or four error.
157
00:08:37.639 --> 00:08:41.000
Of course, the error also said that my JWT token
158
00:08:41.200 --> 00:08:44.440
was wrong, but I was just using an apikey and
159
00:08:44.519 --> 00:08:47.200
so I think something must have changed on their end
160
00:08:47.600 --> 00:08:51.240
or there was something funky going on between me and them,
161
00:08:51.240 --> 00:08:53.360
because it worked fine one day and was broken the next.
162
00:08:53.399 --> 00:08:56.960
I'm like, I really want to show those guys something working.
163
00:08:57.320 --> 00:09:00.240
And then I was like, but I'm talking to Carmine tomorrow,
164
00:09:01.360 --> 00:09:03.080
and so I just went and grabbed it and within
165
00:09:03.120 --> 00:09:04.799
an hour and a half I had the exact same
166
00:09:04.840 --> 00:09:09.480
thing working. And so, anyway, this is cool stuff, cool
167
00:09:09.519 --> 00:09:13.679
cool gem. Anyway, So I'm just curious. I mean, if
168
00:09:13.720 --> 00:09:15.799
you had any other people come to you and basically
169
00:09:15.799 --> 00:09:20.720
say this stuff is great, save my bacon. I don't
170
00:09:20.759 --> 00:09:25.480
know what's been your feedback on Ruby LLM.
171
00:09:26.039 --> 00:09:31.799
It has been absolutely overwhelmingly positive. I couldn't have imagined
172
00:09:31.879 --> 00:09:35.639
how good this would have been. So when I released
173
00:09:35.639 --> 00:09:38.080
the gem, I was like sitting on it for like
174
00:09:38.120 --> 00:09:42.799
two weeks and on you know zero point one point
175
00:09:42.879 --> 00:09:45.440
zero pre forty nine or something like that.
176
00:09:45.519 --> 00:09:47.639
Right, I had to release a lot of pre release.
177
00:09:47.440 --> 00:09:50.879
Versions for myself, and I thought, hey, maybe I should
178
00:09:50.919 --> 00:09:53.399
release it right now instead of waiting until I get
179
00:09:53.480 --> 00:09:57.080
everything downe with chatwood work and you know, doing it
180
00:09:57.200 --> 00:10:00.559
let's say the DHH way where you released it after base, Right,
181
00:10:01.039 --> 00:10:03.600
that was my original plan. I was like, well, what
182
00:10:03.679 --> 00:10:06.159
can go wrong? And so I released it and I
183
00:10:06.240 --> 00:10:11.519
made some write posts and immediately in two days we
184
00:10:11.559 --> 00:10:14.120
got to two hundred and sixty style and my goal
185
00:10:14.399 --> 00:10:20.360
was to get to three hundred, so already kind of
186
00:10:20.360 --> 00:10:23.759
achieved my goal in no time at all, and then
187
00:10:24.080 --> 00:10:28.399
someone posted it on Accre News and I remember that
188
00:10:28.639 --> 00:10:31.039
night I was playing in a club.
189
00:10:31.080 --> 00:10:34.399
So I'm also a DJ onlands, you know, my free time.
190
00:10:36.120 --> 00:10:40.360
And yeah, and I come back home, I go to sleep,
191
00:10:41.519 --> 00:10:44.559
and around like six am in the morning, I get
192
00:10:45.320 --> 00:10:48.600
so many notifications of discord, which for some reason I
193
00:10:48.600 --> 00:10:53.080
could hear. I didn't like mute my computer and I
194
00:10:53.200 --> 00:10:55.919
go and check and people were saying like, oh my god,
195
00:10:55.960 --> 00:10:57.960
you made it to a front page of Accre News.
196
00:10:58.039 --> 00:11:01.360
How amazing. And in one we got to one point
197
00:11:01.399 --> 00:11:07.720
seven thousand stars. Wow, which was absolutely overwhelming. Yeah. So
198
00:11:07.799 --> 00:11:12.960
the reception has been fantastic. I've made new friends. There's
199
00:11:13.000 --> 00:11:15.200
some people that I talked to now kind of regularly.
200
00:11:16.159 --> 00:11:18.360
I have an audience on Twitter, which I didn't really
201
00:11:18.399 --> 00:11:23.080
have before. So it has been absolutely fantastic. Just mostly
202
00:11:23.799 --> 00:11:25.960
I would say ninety nine point nine nine nine percent
203
00:11:25.960 --> 00:11:27.879
and everything has been super positive.
204
00:11:28.080 --> 00:11:29.879
Very cool, very very cool.
205
00:11:30.519 --> 00:11:34.399
Yeah, I have to say, yeah. I mean my feedback
206
00:11:34.480 --> 00:11:38.240
is is that it's really simple to use. I think
207
00:11:38.279 --> 00:11:41.200
a lot of people get worried that oh it's AI
208
00:11:41.399 --> 00:11:43.320
and I have to know all these things and.
209
00:11:44.799 --> 00:11:47.039
No, you just tell it. I want to chat, you know,
210
00:11:47.480 --> 00:11:48.440
you have to do a little.
211
00:11:48.240 --> 00:11:49.919
Bit of setup so that it knows how to connect
212
00:11:49.960 --> 00:11:53.200
open ai or something. But that was for me at least, literally,
213
00:11:53.840 --> 00:12:00.559
here's my API key go crazy so.
214
00:12:00.559 --> 00:12:00.840
Great.
215
00:12:01.000 --> 00:12:04.480
Yeah, that was exactly the type of design that I
216
00:12:04.519 --> 00:12:09.159
wanted to have. So basically the reason why I made
217
00:12:10.000 --> 00:12:13.879
Ruby l LAMB is because when I looked at the
218
00:12:13.879 --> 00:12:18.600
other libraries, they essentially fit in two different categories. Right,
219
00:12:18.679 --> 00:12:21.279
So you have the libraries that are mostly done by
220
00:12:21.320 --> 00:12:24.039
Alex Rudahl. Shout out to him, Oh.
221
00:12:24.000 --> 00:12:33.759
Basic, great guy, that his library is amazing. It does everything, yes, exactly.
222
00:12:34.919 --> 00:12:38.600
So his library is his main library is Ruby, open ai, right,
223
00:12:38.679 --> 00:12:42.759
and then he has also the Entropic gem. They are
224
00:12:43.320 --> 00:12:46.679
lightweight layers on top of the HDCP calls, right. So
225
00:12:47.000 --> 00:12:49.320
what that means is that it's really easy to support
226
00:12:49.480 --> 00:12:52.840
every single function from the l lamps, but it's also
227
00:12:53.000 --> 00:12:55.720
really hard to then switch between different l lamps.
228
00:12:55.559 --> 00:12:56.279
If you have to switch.
229
00:12:56.360 --> 00:12:59.279
And one of my use cases for chat with work.
230
00:13:01.399 --> 00:13:04.840
Exactly, and all of the response objects, right, they're all different.
231
00:13:05.519 --> 00:13:09.000
So when I was designing Chat with Work, I wanted
232
00:13:09.039 --> 00:13:12.480
to have something that could make it easy to switch
233
00:13:12.559 --> 00:13:18.279
between models and I was thinking either mid conversation as
234
00:13:18.320 --> 00:13:22.159
a you know, north star, and so I needed to
235
00:13:22.200 --> 00:13:24.879
have something that was a unified interface.
236
00:13:25.960 --> 00:13:27.360
So I looked at the other.
237
00:13:27.480 --> 00:13:31.600
Type of libraries, you know, so like lang chain RB specifically,
238
00:13:32.360 --> 00:13:37.240
and I saw that I wanted to have some other features.
239
00:13:37.440 --> 00:13:40.600
The design wasn't exactly what I would have done for
240
00:13:40.639 --> 00:13:45.200
a library, but I implemented it. But then I wanted
241
00:13:45.200 --> 00:13:50.679
to have the pricing information, the model information, and listing
242
00:13:50.720 --> 00:13:54.279
of the models. So I started modifying it, and I
243
00:13:54.320 --> 00:13:57.159
started to go into the code of lang chain and
244
00:13:58.120 --> 00:14:00.600
it is quite a big jem. It's not as it
245
00:14:00.600 --> 00:14:03.840
has a lot of code. Yeah, so already you know,
246
00:14:03.879 --> 00:14:07.759
it was quite difficult to orienter yourself around, and it
247
00:14:07.840 --> 00:14:10.279
is quite I would I want to say, it's a
248
00:14:10.320 --> 00:14:14.600
little over engineered for my taste, right, Okay, But I
249
00:14:14.759 --> 00:14:19.000
started modifying it, and also like, it depends on the
250
00:14:19.879 --> 00:14:23.840
Alex's gems, right, so in order to have that same feature,
251
00:14:24.120 --> 00:14:26.960
I need to modify the entropic jam. Then also lun
252
00:14:27.080 --> 00:14:30.159
chain and also lun chain or be rails right, because
253
00:14:30.200 --> 00:14:32.120
there's a separate JEM for the rails integration.
254
00:14:32.440 --> 00:14:34.080
I was like, is this really what I want?
255
00:14:34.519 --> 00:14:37.600
Also, I noticed other things that I didn't like about it,
256
00:14:37.639 --> 00:14:42.399
which is that it put the providers first. And I
257
00:14:42.440 --> 00:14:47.919
think philosophically speaking, we don't really care about providers, right,
258
00:14:48.039 --> 00:14:49.399
people care about models.
259
00:14:50.080 --> 00:14:51.399
Where do they come from?
260
00:14:51.960 --> 00:14:55.360
Maybe it's interesting in terms of pricing, But if you
261
00:14:55.399 --> 00:14:57.360
want to just chat with a model, you want to
262
00:14:57.399 --> 00:15:00.840
just go like Ruby lam or Chat model and just
263
00:15:00.960 --> 00:15:04.200
go with it, right. I think that's the that's the
264
00:15:04.240 --> 00:15:09.240
beauty and why people have been finding Ruby LM so interesting.
265
00:15:10.039 --> 00:15:13.799
So one question I have puts the models first?
266
00:15:15.480 --> 00:15:16.440
Yeah, one question I.
267
00:15:16.440 --> 00:15:19.960
Have on that real quick is so initially I was,
268
00:15:20.080 --> 00:15:22.960
like I said, I was using open router and open
269
00:15:23.039 --> 00:15:25.519
router it kind of does the you can switch models
270
00:15:26.000 --> 00:15:29.399
on their service, right, so you're you can pick them,
271
00:15:29.399 --> 00:15:31.200
but you don't have to set up all the different
272
00:15:31.440 --> 00:15:37.960
service providers. So why not go with something like that?
273
00:15:37.960 --> 00:15:38.720
That's a good question.
274
00:15:39.240 --> 00:15:44.799
Well, the open router basically locks you in into that environment,
275
00:15:45.279 --> 00:15:48.240
so you need to use open router. What if open
276
00:15:48.320 --> 00:15:52.679
router dies or if providers like alge my Little Story
277
00:15:53.440 --> 00:15:58.440
or you four four errors right exactly. So I didn't
278
00:15:58.440 --> 00:16:01.639
want to be locked in into anything. I wanted to
279
00:16:01.720 --> 00:16:05.759
use truly open source software just like you know, Ruby
280
00:16:05.759 --> 00:16:10.279
and rails are and that's why I decided, you know what,
281
00:16:10.399 --> 00:16:13.240
this is probably a good idea to do. And yeah,
282
00:16:13.279 --> 00:16:17.240
there were other reasons actually why I thought that the
283
00:16:17.360 --> 00:16:22.720
existing gems were not of my taste. For example, streaming,
284
00:16:22.799 --> 00:16:25.519
even though like line chain or b has, you know,
285
00:16:25.519 --> 00:16:29.559
I unified the interface for most things. Streaming is where
286
00:16:29.559 --> 00:16:33.799
it gets complicated. So whenever you get a chunk, you
287
00:16:33.879 --> 00:16:38.279
actually get a chunk that is exactly the object that
288
00:16:38.600 --> 00:16:42.639
is retrieved from the API itself, So you need to
289
00:16:43.720 --> 00:16:46.519
parse the chunks in a different way. If you have
290
00:16:46.559 --> 00:16:49.759
an entropic model, then you have an opening eye model.
291
00:16:50.759 --> 00:16:56.720
And so all of these problems combined made it
so
292
00:16:56.759 --> 00:16:59.360
that I wanted to build my own.
293
00:17:00.440 --> 00:17:04.160
Nice Well I'm glad you did because it's it's freaking awesome.
294
00:17:04.759 --> 00:17:09.519
So yeah, let's let's dive in a little bit. So
295
00:17:09.640 --> 00:17:13.160
you mentioned models, and if you get into like the
296
00:17:13.200 --> 00:17:16.680
getting started, I mean, I only hooked up open AI,
297
00:17:16.960 --> 00:17:20.079
but it sounds like you want people to be able
298
00:17:20.160 --> 00:17:23.240
to switch seamlessly. So if I put in an open
299
00:17:23.279 --> 00:17:28.640
aike and an Anthropic key, and one or two other providers,
300
00:17:28.640 --> 00:17:32.519
maybe a Gemini key, I can switch between those seamlessly.
301
00:17:35.400 --> 00:17:37.039
That's right. That's right.
302
00:17:37.920 --> 00:17:41.640
And the way that this works is that Ruby ALAM
303
00:17:41.720 --> 00:17:45.359
essentially has two different parts of the code. So you
304
00:17:45.440 --> 00:17:49.759
have the clean interface with the user, right, So you
305
00:17:49.880 --> 00:17:53.599
have the chat object, you have the message object, you
306
00:17:53.720 --> 00:17:58.640
have the content object, et cetera, et cetera, and then
307
00:17:58.680 --> 00:18:01.720
you have the mess part of the code, which is
308
00:18:01.759 --> 00:18:05.799
the provider code, right, and what the provider code does
309
00:18:05.880 --> 00:18:10.400
I'm saying messy, but I think it's decent. But it
310
00:18:10.400 --> 00:18:15.000
has all the messy parts of actually communicating with the
311
00:18:15.000 --> 00:18:19.559
provider interfaces and translating between you know, the Ruby LM
312
00:18:19.640 --> 00:18:26.319
internal representation and the provider representation. It sounds pretty straightforward,
313
00:18:26.759 --> 00:18:29.160
and so I implemented it that way so that you
314
00:18:29.319 --> 00:18:35.160
can have a very simple interface but actually change between models,
315
00:18:35.839 --> 00:18:39.119
because then it's just a question of parsing and retrieving
316
00:18:39.200 --> 00:18:44.559
data and formatting it for their for the format not
317
00:18:44.720 --> 00:18:47.839
really about you know, a specific representation that only works
318
00:18:48.240 --> 00:18:50.599
for the specific provider. I definitely didn't want to do that,
319
00:18:51.119 --> 00:18:54.359
and the way to achieve that was to essentially study
320
00:18:54.839 --> 00:18:59.240
the the APIs from you know, open Ai and Entropic,
321
00:19:00.319 --> 00:19:04.160
because that's where I started, and figure out what is
322
00:19:04.200 --> 00:19:08.319
the best way that they implement things, and how would
323
00:19:08.319 --> 00:19:11.920
I implement it in terms of the objects, the Ruby objects.
324
00:19:11.519 --> 00:19:13.000
That I want to have in my library.
325
00:19:13.480 --> 00:19:15.680
And so I picked and choose, you know, how to
326
00:19:15.720 -->
00:19:21.079
represent tool calls for example, and that's always now represented RUBYLM.
327
00:19:21.160 --> 00:19:22.599
And then if the auto.
328
00:19:22.359 --> 00:19:26.680
Provider that doesn't represent it this way, you know, will
329
00:19:26.720 --> 00:19:32.559
get a different format method in their modules in order
330
00:19:32.559 --> 00:19:35.039
to do that.
331
00:19:35.039 --> 00:19:35.720
That's awesome.
332
00:19:35.920 --> 00:19:38.680
So one thing that I'm wondering about is do the
333
00:19:38.720 --> 00:19:42.640
different models have different capabilities that you have to adjust
334
00:19:42.720 --> 00:19:44.920
for or let people know, Hey, you can do this
335
00:19:45.079 --> 00:19:48.759
with I don't know, GPT and Anthropic, but not with
336
00:19:48.880 --> 00:19:52.559
Gemini or something like that.
337
00:19:52.559 --> 00:19:55.680
That's right, Yeah, that's a great question. So I implemented
338
00:19:55.680 --> 00:19:58.799
them all the registry. That's actually where this whole thing
339
00:19:58.839 --> 00:20:01.359
about RUBYLM started right when I wanted to implement the
340
00:20:01.599 --> 00:20:07.039
model registry for for line chain. And it basically has
341
00:20:07.720 --> 00:20:12.559
a bunch of different capabilities per each model. So for example,
342
00:20:13.960 --> 00:20:17.799
it would know if a model has a function support,
343
00:20:18.440 --> 00:20:23.119
if we would have a Jason mode soon when we
344
00:20:23.160 --> 00:20:26.359
will actually have structured output support, if it will not,
345
00:20:26.440 --> 00:20:30.359
if we will have structured output support, what is the
346
00:20:30.400 --> 00:20:33.799
context length? What is the maximum amount of tokens for
347
00:20:34.000 --> 00:20:37.119
the output, you know, and all that information is put
348
00:20:37.119 --> 00:20:41.799
together in adjacent file that we distribute directly with the library,
349
00:20:41.839 --> 00:20:46.640
because one of the design principles is to be a
350
00:20:46.680 --> 00:20:50.160
turnkey solution. So you just as you as you did,
351
00:20:50.200 --> 00:20:52.920
like you installed it, you put your opening ike and
352
00:20:52.960 --> 00:20:55.559
there you go. You have a chat interface if you
353
00:20:55.599 --> 00:20:58.319
want to write at least in the common line. And
354
00:20:58.839 --> 00:21:00.839
this is exactly the experience that I wanted to have
355
00:21:00.920 --> 00:21:04.400
and I want people to have. So if there is
356
00:21:04.440 --> 00:21:07.880
a model that doesn't have let's say, function calling support,
357
00:21:08.880 --> 00:21:11.880
we will throw an error because we will know it
358
00:21:12.000 --> 00:21:16.160
from the from the models Dore Jason. And the way
359
00:21:16.200 --> 00:21:18.799
that at the moment we make the models Dore Jason
360
00:21:19.119 --> 00:21:24.480
is to query the APIs from the listing model APIs
361
00:21:24.519 --> 00:21:28.680
from open EI, from Entropic, Gemini, et cetera, and then
362
00:21:28.720 --> 00:21:35.240
augmented with our own code that figures out based on
363
00:21:35.279 --> 00:21:38.559
the model family, like that's the model family has function calling,
364
00:21:38.920 --> 00:21:43.640
that's the small family has you know, Jason mode, what's
365
00:21:43.680 --> 00:21:45.160
the context length, et cetera, et cetera.
366
00:21:45.279 --> 00:21:47.119
And you will find that in the code in the.
367
00:21:47.039 --> 00:21:51.880
Capabilities dot r B files in the module in the
368
00:21:51.920 --> 00:21:59.359
model sorry, in the provider modules. Now, however, since I've
369
00:21:59.359 --> 00:22:01.839
grown a little bit tired of you know, having to
370
00:22:01.960 --> 00:22:09.200
maintain that code and have you know, also just parsing
371
00:22:09.559 --> 00:22:14.640
the provider's web pages, which is actually kind of costly,
372
00:22:14.720 --> 00:22:16.400
especially when you do it with lams. And there is
373
00:22:16.440 --> 00:22:20.400
some code in Ruby LAM to do that in the tasks,
374
00:22:21.039 --> 00:22:23.200
but I remember the last time I used it, I
375
00:22:23.240 --> 00:22:26.519
spent twenty bucks just to parse all of the new
376
00:22:26.799 --> 00:22:31.920
open Ai model pages. Well, since I grown tired of
377
00:22:31.960 --> 00:22:36.680
doing that, I actually contacted an ex employee of mine
378
00:22:36.680 --> 00:22:40.440
from fresh Flow who now is also an entrepreneur, and
379
00:22:40.480 --> 00:22:43.680
he has a company called Parsera and they're basically doing
380
00:22:44.559 --> 00:22:49.200
scraping of web pages using l lams and then they
381
00:22:49.240 --> 00:22:52.640
returned that as data in Jason and I thought that
382
00:22:52.640 --> 00:22:54.400
that would have been the perfect use case in order
383
00:22:54.480 --> 00:22:57.799
to showcase their product, and we would get like an
384
00:22:57.799 --> 00:23:01.359
API for free in order to just you know, query
385
00:23:01.599 --> 00:23:07.559
about the capabilities of any LM in existence, since unfortunately
386
00:23:07.599 --> 00:23:09.720
the providers they don't provide that at the moment. Right,
387
00:23:09.799 --> 00:23:14.039
Like when you get a list model endpoint response, you
388
00:23:14.200 --> 00:23:17.160
just have most of the time just the model ID
389
00:23:17.640 --> 00:23:22.720
and sometimes the capabilities. I think Google does this and
390
00:23:23.039 --> 00:23:26.359
some other small things, but not the full you know,
391
00:23:26.559 --> 00:23:31.160
model info object that we have inside Ruby ELM. So
392
00:23:31.920 --> 00:23:35.359
basically what Persera is now building and they are like,
393
00:23:35.440 --> 00:23:40.359
I wouldn't say ninety percent. There is this API that
394
00:23:40.440 --> 00:23:43.079
you can just with a get request query and you
395
00:23:43.200 --> 00:23:49.920
have a beautiful interface, you know, a beautiful Jason object
396
00:23:50.480 --> 00:23:56.160
with all of the different capabilities, all the prices, all
397
00:23:56.359 --> 00:24:01.119
well formatted for all the major you know, for providers
398
00:24:01.119 --> 00:24:04.200
that we have in Ruby l M. And so I
399
00:24:04.200 --> 00:24:08.319
will scrap that code. I'm really excited when I remove code.
400
00:24:09.039 --> 00:24:12.599
So I'll scrap that code and simply like called that
401
00:24:12.599 --> 00:24:14.480
that API from from then on.
402
00:24:14.680 --> 00:24:16.119
So I'm really excited for that.
403
00:24:17.119 --> 00:24:20.319
So you're not going to have a Jason in your
404
00:24:20.960 --> 00:24:23.359
in the gym anymore. It's going to be an API
405
00:24:23.480 --> 00:24:26.519
call or the APA call. API call will generate the Jason.
406
00:24:29.839 --> 00:24:30.599
Rather the second.
407
00:24:30.720 --> 00:24:32.960
Yeah, I still want to have the turnkey solution so
408
00:24:33.000 --> 00:24:37.279
that anybody that installs it, maybe doesn't have Internet, I
409
00:24:37.279 --> 00:24:40.039
don't know, doesn't want to do that API call of
410
00:24:40.039 --> 00:24:44.279
course the Internet, then it can still actually have that
411
00:24:45.599 --> 00:24:49.039
installed and without having an extra API call every time.
412
00:24:49.160 --> 00:24:51.000
And it's refreshed models for example.
413
00:24:51.359 --> 00:24:51.640
Yep.
414
00:24:53.640 --> 00:24:57.759
So yeah, let's dive into what Ruby LLM actually provides,
415
00:24:58.000 --> 00:25:01.119
because I think people can now kind of be comfortable
416
00:25:01.119 --> 00:25:04.119
with Hey, look, it does all these models. It knows
417
00:25:04.160 --> 00:25:06.839
which models will do which things, and so if you
418
00:25:06.880 --> 00:25:09.920
try and do something with one model that it just doesn't,
419
00:25:10.200 --> 00:25:12.680
you know, it's not compatible with the model, It'll tell
420
00:25:12.680 --> 00:25:14.000
you and then you can do something different.
421
00:25:15.960 --> 00:25:17.880
So let's start with.
422
00:25:17.839 --> 00:25:21.359
The chat because that's where I started, and I was
423
00:25:21.400 --> 00:25:23.720
just blown away that it was that easy, because you
424
00:25:23.839 --> 00:25:29.160
literally just do you instantiate a chat object and then
425
00:25:29.200 --> 00:25:32.359
you do it dot ask and then you get content
426
00:25:32.440 --> 00:25:35.839
back and you can post it. And what I found
427
00:25:35.960 --> 00:25:38.240
was that I had to pull in the common mark
428
00:25:38.319 --> 00:25:41.039
gem and tell common Mark to convert it to HTML
429
00:25:41.359 --> 00:25:48.279
and I was set. That was like, it is there
430
00:25:48.319 --> 00:25:49.640
anything more to it?
431
00:25:49.680 --> 00:25:51.680
Am I missing it? I mean, we get into tools
432
00:25:51.720 --> 00:25:52.160
in a minute.
433
00:25:52.160 --> 00:25:59.599
But so in the chat you can also specify contents, right,
434
00:25:59.680 --> 00:26:02.559
so you can send an image with it, you can
435
00:26:02.599 --> 00:26:04.880
send the audio file with it, you can send a
436
00:26:04.920 --> 00:26:05.720
PDF with it.
437
00:26:06.839 --> 00:26:08.519
And also there I tried.
438
00:26:08.359 --> 00:26:12.160
To make the design of the library feel super easy
439
00:26:12.799 --> 00:26:14.839
so that if you need to analyze an image it's
440
00:26:14.920 --> 00:26:18.519
just a chat dot ask what's in this image? Come
441
00:26:18.640 --> 00:26:23.240
up with colon open curly braces image and then the
442
00:26:23.279 --> 00:26:27.359
path through your image or the HCP link to your image.
443
00:26:28.680 --> 00:26:32.839
And that also I think follows the same philosophy or
444
00:26:32.880 --> 00:26:36.279
rubylm of convention over configuration and you know, just trying
445
00:26:36.279 --> 00:26:41.160
to make people's life easier. Then you can also stream
446
00:26:41.200 --> 00:26:44.640
the responses in real time, right, and you can simply
447
00:26:44.720 --> 00:26:47.759
use Ruby blocks to do that. So it's super simple.
448
00:26:49.119 --> 00:26:51.920
It's just a chat dot ask whatever you want to ask,
449
00:26:52.200 --> 00:26:56.559
do chunk and then print chann dot content and the
450
00:26:56.680 --> 00:26:59.759
chunk is a chunk object and it looks like a message.
451
00:27:00.119 --> 00:27:01.880
In fact, if you look at the code is pretty
452
00:27:01.920 --> 00:27:04.599
much exactly like a message. I think it's just inherits
453
00:27:04.960 --> 00:27:10.319
from message and that's it. And it will if you
454
00:27:10.440 --> 00:27:14.319
just print the contents, it will simply print that chunk.
455
00:27:14.720 --> 00:27:17.839
But then in the background we're actually accumulating the response
456
00:27:18.400 --> 00:27:22.680
and so the result of that chat dot ask would
457
00:27:22.680 --> 00:27:26.759
actually be a full, fully blown message and it will
458
00:27:26.799 --> 00:27:29.880
be also persistent in the background in the in the
459
00:27:29.920 --> 00:27:35.319
memory that Ruby LM do chat has about messages, Yeah,
460
00:27:35.559 --> 00:27:40.039
what are capabilities that have? So it can also generate images.
461
00:27:40.799 --> 00:27:45.839
Because I mean you have so you have the capability.
462
00:27:46.039 --> 00:27:47.759
Yeah, and it just does it in a do loop
463
00:27:47.880 --> 00:27:51.000
and yeah, it has the chunk content and it has
464
00:27:51.119 --> 00:27:53.400
like a print or puts.
465
00:27:54.200 --> 00:27:56.960
In kind of your just general Ruby application.
466
00:27:57.559 --> 00:28:01.759
But I don't know, people who listen to this show
467
00:28:01.839 --> 00:28:03.799
know that I'm totally in love with Turbo.
468
00:28:04.480 --> 00:28:06.319
It was like, where have you been on my life Turbo?
469
00:28:07.799 --> 00:28:10.640
But you have examples on how to push the stuff
470
00:28:10.640 -->
00:28:17.440
out to Turbo as well, right, So one one question
471
00:28:17.480 --> 00:28:19.880
that I have with this because it shows a job
472
00:28:20.720 --> 00:28:24.839
and the job uses the Turbo streams channel broadcast right
473
00:28:25.480 --> 00:28:29.880
to update the message, which is you know what you want?
474
00:28:31.880 --> 00:28:34.079
I'm just I'm trying to figure out exactly where this
475
00:28:35.720 --> 00:28:36.799
gets called from.
476
00:28:38.359 --> 00:28:43.839
Does it just run periodically? And then so.
477
00:28:47.400 --> 00:28:51.279
How it works in chatwood work is that there is
478
00:28:51.319 --> 00:28:54.839
a job. It's not called chat stream job, it's called
479
00:28:54.880 --> 00:28:59.039
slightly differently, but there is that job similar to the
480
00:28:59.160 --> 00:29:03.640
job in the guides, and what it does is to
481
00:29:03.640 --> 00:29:10.319
stream via Turbo, essentially appending the chunk content to a
482
00:29:10.440 --> 00:29:15.119
div that is hidden, and then I use mark dot
483
00:29:15.160 --> 00:29:19.440
j as in order to doing the rendering of the
484
00:29:20.440 --> 00:29:23.400
of the actual message. The reason why I do this
485
00:29:24.000 --> 00:29:28.920
is because I used to have everything server side and
486
00:29:29.640 --> 00:29:33.079
do all of the rendering of markdown server side and
487
00:29:33.119 --> 00:29:37.759
then send you know, the HTML over But the problem
488
00:29:37.839 --> 00:29:41.359
is that when you get to really big messages is
489
00:29:41.400 --> 00:29:44.599
starting to stream every single time you have a new chunk,
490
00:29:45.039 --> 00:29:48.880
which may be even every single word, or you know,
491
00:29:49.240 --> 00:29:55.079
even sometimes less than one word, And you can imagine
492
00:29:55.079 --> 00:30:00.839
that refreshing the message so fast and and doing it
493
00:30:00.920 --> 00:30:05.519
so many times it actually bogs down your connection. So
494
00:30:05.680 --> 00:30:08.319
at some point I noticed that my connection was downloading
495
00:30:08.359 --> 00:30:13.279
like multiple megabytes per second right of HTML. So it
496
00:30:13.279 --> 00:30:15.319
didn't make any sense to me. So what I'm doing
497
00:30:15.400 --> 00:30:18.759
is to simply append the chunk to that specific dive
498
00:30:19.880 --> 00:30:22.319
and then marked will look at that div and it
499
00:30:22.359 --> 00:30:25.240
would actually do the rendering in an udative.
500
00:30:24.960 --> 00:30:26.279
That I actually displayed to the user.
501
00:30:26.839 --> 00:30:29.279
And that also gives me the possibility to do a
502
00:30:29.319 --> 00:30:35.079
copy functionality in which the when you copy the message,
503
00:30:35.640 --> 00:30:37.880
it would actually copy the markdown version of the message,
504
00:30:37.880 --> 00:30:40.599
which I think is so much more useful than getting
505
00:30:40.640 --> 00:30:46.440
the full blown HTML from the page. And
so how
506
00:30:46.440 --> 00:30:48.960
do I call that job? Well, I call it from
507
00:30:49.200 --> 00:30:57.839
the the actual controller. So whenever the user asks something,
508
00:30:58.119 --> 00:31:00.880
you know the controller, which is fire that job in
509
00:31:00.920 --> 00:31:05.640
the background, and the job will simply process whatever comes
510
00:31:05.640 --> 00:31:06.319
from the OLM.
511
00:31:06.480 --> 00:31:13.279
Okay, so in the controller then I have something that effectively,
512
00:31:13.960 --> 00:31:18.319
I mean, do I tell it I'm streaming it? I
513
00:31:18.359 --> 00:31:20.480
guess I just have the do block and so then
514
00:31:20.720 --> 00:31:26.519
every time gets executive executed, sorry, it calls the job
515
00:31:26.599 --> 00:31:29.880
or how I'm just trying to wrap my brain around this.
516
00:31:34.759 --> 00:31:39.000
Right, So basically the controller would simply you know, maybe
517
00:31:39.039 --> 00:31:40.799
you want to create a message in the controller, do
518
00:31:40.880 --> 00:31:43.480
whatever you want, but in the end it needs to
519
00:31:43.680 --> 00:31:48.160
call that chat stream job so that you can process
520
00:31:49.079 --> 00:31:54.000
the streaming in the background because you can't have that
521
00:31:54.119 --> 00:31:56.920
running in the controller because it's such a long running thing.
522
00:31:58.759 --> 00:32:02.559
And that's exactly how I do it in chatwood work.
523
00:32:04.000 --> 00:32:06.000
Okay.
524
00:32:06.039 --> 00:32:09.400
So basically what you've got going on is, yeah, you've
525
00:32:09.400 --> 00:32:14.079
got your and it shows it in the chat right,
526
00:32:14.119 --> 00:32:16.079
So you do the chat dot ask with the do
527
00:32:16.839 --> 00:32:19.799
inside the do you call the job the job is
528
00:32:19.839 --> 00:32:24.079
streaming stuff out through turbo and then you have marked
529
00:32:24.119 --> 00:32:30.359
basically putting it in. Yeah, that's cool, that's correct. When
530
00:32:30.400 --> 00:32:34.680
your controller action finishes, then it renders a turbo stream whatever.
531
00:32:38.359 --> 00:32:42.640
Yeah, so if you set up your turbus, your broadcasts,
532
00:32:42.680 --> 00:32:48.599
you know, inside your application record, you know, on your models. Basically,
533
00:32:49.039 --> 00:32:52.000
it would also update the whole message when it's finished,
534
00:32:53.200 --> 00:32:55.200
and you can also tell it to you know, update
535
00:32:55.319 --> 00:32:59.039
of typing indicators and perhaps the button you know that
536
00:32:59.279 --> 00:33:02.880
has to change from a send to a stop. All
537
00:33:02.920 --> 00:33:06.240
of that you can do actually really really easily with Rails,
538
00:33:07.519 --> 00:33:12.319
and no wonder because Rails is so fantastic for these things.
539
00:33:13.279 --> 00:33:16.559
One thing that I think is interesting about the jobs,
540
00:33:16.640 --> 00:33:22.720
if I may, is that what I
noticed, especially using
541
00:33:23.359 --> 00:33:28.599
things like solid que which use threads, is that you
542
00:33:28.640 --> 00:33:34.039
have a certain number of threads that you can use maximum, right,
543
00:33:34.079 --> 00:33:37.799
So you need to set it up and you have
544
00:33:39.279 --> 00:33:42.559
one worker per process as far as I remember, and
545
00:33:42.599 --> 00:33:45.119
then you need to set up how many threads per
546
00:33:45.160 --> 00:33:47.640
process you have as workers.
547
00:33:47.799 --> 00:33:48.000
Right.
548
00:33:48.119 --> 00:33:52.400
So even if you set it up in a way
549
00:33:52.440 --> 00:33:56.400
that it will process many many connections at the same time,
550
00:33:56.440 --> 00:33:59.119
many many chat streaming jobs at the same time, you
551
00:33:59.160 --> 00:34:02.920
will always be limited by that. And the limitation is
552
00:34:02.920 --> 00:34:08.400
that every single thread in Ruby is eight megabytes. And
553
00:34:09.679 --> 00:34:12.599
also you're occupying a whole slot, right, So you can
554
00:34:12.639 --> 00:34:17.480
imagine an application like chat with work it has not
555
00:34:17.519 --> 00:34:19.559
only the chat streaming job, but it also has the
556
00:34:19.639 --> 00:34:23.719
process document jobs, right, and the discovery document jobs and
557
00:34:24.039 --> 00:34:26.880
all of these and there's many, many jobs running at
558
00:34:26.920 --> 00:34:29.440
the same time, but they're all very lightweight, right that
559
00:34:29.519 --> 00:34:34.159
they're not doing much CPU wise, they're mostly like sending
560
00:34:34.239 --> 00:34:38.760
things back and forth through APIs and then you know,
561
00:34:38.880 --> 00:34:42.480
the grant of the work is probably done in postcris,
562
00:34:42.519 --> 00:34:47.280
you know, because I use postgris also as a vector storage.
563
00:34:47.719 --> 00:34:52.199
So they're all io bound, not CPU bound, and they're
564
00:34:52.280 --> 00:34:56.199
consuming one thread every single time there is a job
565
00:34:56.280 --> 00:35:01.280
running at the same time, and so that was not ideal.
566
00:35:01.400 --> 00:35:05.360
And in the Python world and in the JavaScript world,
567
00:35:05.639 --> 00:35:09.599
you know, for io bound operations, everybody would use a
568
00:35:09.719 --> 00:35:12.360
syn ko for example in Python, right, or a sync
569
00:35:12.480 --> 00:35:15.599
wait in JavaScript. And so what I noticed is that
570
00:35:16.239 --> 00:35:19.079
now I came back to Ruby, there's a bit of
571
00:35:19.119 --> 00:35:24.800
a lack of understanding or you know, knowledge about using
572
00:35:25.840 --> 00:35:32.480
concurrency instead of using parallelism, well fake parallelism with threats,
573
00:35:33.880 --> 00:35:36.599
And so I delved deeper into it. And actually there
574
00:35:36.679 --> 00:35:42.079
is a beautiful ecosystem of asynchronous gems such as a sync,
575
00:35:42.840 --> 00:35:47.400
a sink cable, ACNK job, anything that is built by
576
00:35:48.000 --> 00:35:53.480
Samuel Williams I aquatics, you know, also called socketry, is
577
00:35:54.039 --> 00:35:59.159
absolutely fantastic. And what I also noticed is that Ruby
578
00:35:59.280 --> 00:36:04.400
is probably the best language to do concurrency because in
579
00:36:04.440 --> 00:36:08.800
most other languages you need to uppend these or prepense
580
00:36:10.840 --> 00:36:14.239
these keywords like a sync await every time you need
581
00:36:14.280 --> 00:36:17.320
to do such an operation, But in Ruby you don't
582
00:36:17.360 --> 00:36:22.119
need to because if the C level operation actually is
583
00:36:22.360 --> 00:36:26.800
fiber aware, then you get fiber awareness for free. So
584
00:36:26.920 --> 00:36:32.480
you just need to essentially use or wrap whatever code
585
00:36:32.559 --> 00:36:36.320
you have into let's say on a sync block, and
586
00:36:36.960 --> 00:36:39.960
if you do it right, it will work in parallel
587
00:36:40.199 --> 00:36:41.039
in concurrent.
588
00:36:41.360 --> 00:36:42.840
Actually awesome.
589
00:36:43.039 --> 00:36:46.320
So one question I have on that is, because I've
590
00:36:46.400 --> 00:36:50.239
never even used this before, can you run this and
591
00:36:51.360 --> 00:36:53.800
solid que side by side and then just throw the
592
00:36:53.840 --> 00:36:57.280
acinc jobs at one and any irregular stuff at the
593
00:36:57.280 --> 00:36:59.360
other or do you just put everything into an acin job.
594
00:37:02.000 --> 00:37:05.000
At the moment, how I'm using it is everything is
595
00:37:05.119 --> 00:37:09.639
in a saying job, but you can totally right random
596
00:37:09.679 --> 00:37:13.159
side side by side. I think there is a Q
597
00:37:13.400 --> 00:37:17.679
adapter that you can use in each single job, so
598
00:37:17.760 --> 00:37:20.440
you just you know, you have the que as the
599
00:37:20.480 --> 00:37:23.000
fault for example, right, so you can have a Q
600
00:37:23.119 --> 00:37:28.320
adapter and you just say solid Q for certain jobs,
601
00:37:28.440 --> 00:37:31.960
and you can do a saying job for your jobs.
602
00:37:32.920 --> 00:37:34.920
That's a good tip because yeah, if I'm going to
603
00:37:34.960 --> 00:37:39.480
build a podcast assistant, I'm probably gonna run into the
604
00:37:39.519 --> 00:37:42.880
same constraints, and so that is really helpful.
605
00:37:43.320 --> 00:37:50.840
Right, Yeah, Yeah, I'm no, it's just saying that I
606
00:37:51.440 --> 00:37:56.800
basically discovered this this fly, and I think that it's
607
00:37:56.840 --> 00:38:00.320
really a matter of communication to the Ruby community out
608
00:38:00.519 --> 00:38:03.599
saying like, hey, there's not just this way of doing
609
00:38:03.679 --> 00:38:06.320
concurrency in paralyism that we're all familiar with, you know,
610
00:38:06.599 --> 00:38:10.480
like threads, but there's also fibers, and you know, there's
611
00:38:11.360 --> 00:38:14.320
other software that you could use that even though you
612
00:38:14.400 --> 00:38:18.480
don't have io bound operations, you can simply replace Puma
613
00:38:18.480 --> 00:38:22.119
with Falcon and have you know, five x speed ups.
614
00:38:22.159 --> 00:38:25.840
Sometimes it's really so much faster it's noticeable. I didn't
615
00:38:25.840 --> 00:38:28.000
do any benchmarks, but it's really Yeah.
616
00:38:28.039 --> 00:38:30.079
I've talked to Samuel before and I got the impression
617
00:38:30.119 --> 00:38:33.599
that some workloads really lend themselves to it and some
618
00:38:33.760 --> 00:38:34.519
don't as much.
619
00:38:34.559 --> 00:38:37.679
But yeah, Falcon is really cool.
620
00:38:40.440 --> 00:38:43.280
One other thing that I'm curious about here, So when
621
00:38:43.280 --> 00:38:46.000
I built my app the other night, with the chat.
622
00:38:46.440 --> 00:38:49.159
I realized that in my controller, I was instantiating a
623
00:38:49.199 --> 00:38:54.000
new chat every time writers doing RUBLLM dot chat. And
624
00:38:54.440 --> 00:38:58.000
what I want is I want an ongoing conversation with
625
00:38:59.280 --> 00:39:01.760
you know, so that it kind of maintains the context
626
00:39:01.840 --> 00:39:07.480
and things like that. And there's an example in here
627
00:39:07.519 --> 00:39:10.239
where you're basically what I would do, like in a
628
00:39:10.320 --> 00:39:15.159
RAILS console or IRB or something where you know, it's
629
00:39:15.199 --> 00:39:18.039
like chat dot ask and then you send it another
630
00:39:18.079 --> 00:39:21.199
message chat dot ask and it looks like that maintains
631
00:39:21.239 --> 00:39:24.519
that context. But how do I do that between requests
632
00:39:25.159 --> 00:39:28.960
when I'm hitting the controller anew every time? But I
633
00:39:29.000 --> 00:39:30.679
still want it to be part of that same.
634
00:39:30.559 --> 00:39:36.599
Chat, right, I think this is really important to say.
635
00:39:36.679 --> 00:39:44.480
So the RAILS integration essentially persistent messages, Yeah, persistent messages
636
00:39:44.599 --> 00:39:47.400
inside active record models.
637
00:39:47.480 --> 00:39:54.679
So we have essentially some helpers which are called.
638
00:39:54.480 --> 00:40:00.559
Acts as chat, acts as message access tool call that
639
00:40:00.639 --> 00:40:03.800
you can drop in into your models. You know, if
640
00:40:03.840 --> 00:40:07.480
it's called chat the model for chats, then you just
641
00:40:08.280 --> 00:40:14.840
say class you know chat inheriting from application record and
642
00:40:14.920 --> 00:40:19.320
then just say access chat and you have essentially all
643
00:40:19.360 --> 00:40:24.719
the connection to also the message model. You have persistence
644
00:40:25.480 --> 00:40:29.960
and every single method that we have in the classic
645
00:40:30.039 --> 00:40:33.880
chat interface, you know in plain old Ruby objects.
646
00:40:34.400 --> 00:40:36.400
You would have it also in rails.
647
00:40:36.559 --> 00:40:40.199
So if you do chat dot ask, it will first
648
00:40:40.360 --> 00:40:46.039
create a message with the with the user role and
649
00:40:46.559 --> 00:40:53.920
save it and then also call the API for whatever
650
00:40:54.519 --> 00:40:58.920
LLLM you're using, retrieve that and then put it inside
651
00:40:58.920 --> 00:41:02.119
a new message anderson is that and then return that
652
00:41:02.320 --> 00:41:06.280
message at the end. So it looked like exactly the
653
00:41:06.320 --> 00:41:09.719
same interface for PLANEOAR Ruby objects, but you have the
654
00:41:09.760 --> 00:41:12.800
persistence done not just in memory but also in your database.
655
00:41:13.159 --> 00:41:16.679
Right, what if I'm not using rails or not using
656
00:41:16.840 --> 00:41:19.239
active record, is there a way to do this like
657
00:41:19.320 --> 00:41:22.360
in Sinatra or something like that and that anyway?
658
00:41:24.239 --> 00:41:28.639
Right, I haven't used Sinatra recently. The last time I
659
00:41:28.719 --> 00:41:36.239
used it was more than ten years ago. Maybe it's right,
660
00:41:37.440 --> 00:41:41.119
So I wouldn't know how to actually build that out,
661
00:41:42.159 --> 00:41:45.199
but it's possible to. If it's possible to make an
662
00:41:45.239 --> 00:41:50.679
adapter for whatever type of you know, storage you have,
663
00:41:51.519 --> 00:41:53.639
it should be pretty easy and people can just look
664
00:41:53.679 --> 00:41:56.639
at the access code and see how we've done it.
665
00:41:56.760 --> 00:41:59.119
Right, Yeah, that makes sense.
666
00:41:59.159 --> 00:42:00.880
You can't just look at his chat and then see
667
00:42:00.920 --> 00:42:06.960
what she did. Very cool let's talk about tools for
668
00:42:07.000 --> 00:42:08.920
a minute, because this was one of the other things
669
00:42:08.920 --> 00:42:11.159
I showed off the other night was essentially I created
670
00:42:11.159 --> 00:42:16.599
a tool that just checked the weather, and yeah, I
671
00:42:16.639 --> 00:42:18.679
didn't use the same service you did, but it was
672
00:42:18.760 --> 00:42:22.920
basically the same thing. I used Open weather Map instead
673
00:42:22.960 --> 00:42:26.840
of Medio or whatever you used, but yeah, it was
674
00:42:26.880 --> 00:42:28.480
the same deal. Do you want to explain to people
675
00:42:28.480 --> 00:42:30.000
what tools are and how they work?
676
00:42:34.559 --> 00:42:34.880
Right?
677
00:42:34.960 --> 00:42:38.199
So, I think tools, especially with MCP, they became the
678
00:42:38.239 --> 00:42:42.400
most hot thing talked about at the moment in AI. Right,
679
00:42:44.199 --> 00:42:49.440
So for people who are not familiar essentially with LLM models,
680
00:42:49.800 --> 00:42:54.039
you can create a tool in your own language and
681
00:42:54.079 --> 00:42:58.159
then tell them like, hey, this is the way that
682
00:42:58.239 --> 00:43:03.119
this tool works. So
this is the description of the tool,
683
00:43:03.599 --> 00:43:05.760
and these are all the parameters that we need. These
684
00:43:05.760 --> 00:43:08.679
are the parameters that are optional, and these are all
685
00:43:08.719 --> 00:43:12.280
the descriptions for every single parameters. And then the LM
686
00:43:12.320 --> 00:43:15.199
will decide when to use that tool.
687
00:43:15.400 --> 00:43:15.599
Right.
688
00:43:15.679 --> 00:43:20.639
So let's say you created this tool for weather, but
689
00:43:21.239 --> 00:43:23.920
then the user talks about something else, wants to talk
690
00:43:23.960 --> 00:43:27.920
about MacBooks, right, the LLLM will not actually call the
691
00:43:27.960 --> 00:43:32.320
tool every single time. And I think the fundamental difference
692
00:43:33.079 --> 00:43:37.880
between tools and the classical RAG for example that we
693
00:43:38.039 --> 00:43:40.719
used to have, you know, even just a year ago
694
00:43:41.159 --> 00:43:46.159
at this point is that you don't always do retrieval
695
00:43:46.199 --> 00:43:50.480
advantage generation. So basically you don't always search, you know,
696
00:43:50.559 --> 00:43:52.960
the documents and then put them in the context of
697
00:43:53.039 --> 00:43:57.719
the message every single time you have a new message. Basically,
698
00:43:57.840 --> 00:44:03.639
tools enable agentic aid I because the AI can be
699
00:44:03.679 --> 00:44:06.440
an agent and really figure out if it needs to
700
00:44:06.519 --> 00:44:12.400
use this tool or not. And they are now like
701
00:44:12.480 --> 00:44:15.840
all the age with MCP, So MCP, for it's not familiar,
702
00:44:16.119 --> 00:44:21.119
is model context protocol, and it's a way to bunch
703
00:44:21.320 --> 00:44:27.280
up a bunch, you know, a bunch of tools into
704
00:44:27.400 --> 00:44:31.119
one single interface and give that interface to the l
705
00:44:31.199 --> 00:44:32.920
l M so that the l M can figure out
706
00:44:33.719 --> 00:44:36.320
you know, hey, I can now access the file system,
707
00:44:36.440 --> 00:44:40.800
I can now use VIM, I can now use vs code,
708
00:44:40.920 --> 00:44:46.119
I can now use I don't know whatever API. And
709
00:44:46.480 --> 00:44:49.519
it's essentially an implementation of tools in a way, right,
710
00:44:49.559 --> 00:44:52.079
but just in a different API.
711
00:44:57.079 --> 00:44:57.280
Yeah.
712
00:44:57.280 --> 00:45:01.280
Absolutely, So this is where like with my my thing
713
00:45:01.360 --> 00:45:04.760
with my podcast stuff is you know, it's not just
714
00:45:05.440 --> 00:45:08.800
which episodes were most popular or what other thing do
715
00:45:08.880 --> 00:45:11.320
I want you to look up, but it's hey, reschedule this,
716
00:45:11.480 --> 00:45:13.920
or hey, invite this other person or do this other thing,
717
00:45:14.599 --> 00:45:17.960
and it can through the API go and talk to
718
00:45:18.000 --> 00:45:19.800
the other system and make it do something
for me.
719
00:45:21.599 --> 00:45:22.159
That's right.
720
00:45:22.599 --> 00:45:26.199
So it opens up a lot of possibilities for using
721
00:45:26.360 --> 00:45:29.639
lms for all kinds of disparate features.
722
00:45:29.719 --> 00:45:29.880
Right.
723
00:45:30.880 --> 00:45:35.000
You can save things, you can retrieve things from the Internet.
724
00:45:35.599 --> 00:45:40.000
You can imagine for example Google search or like document search,
725
00:45:41.079 --> 00:45:47.360
or creating new blog posts. It's just I think the
726
00:45:47.400 --> 00:45:50.119
possibilities are endless because you're letting.
727
00:45:49.840 --> 00:45:51.480
The AI use your code.
728
00:45:52.119 --> 00:45:57.039
Of course, you need to be aware that also ais
729
00:45:57.079 --> 00:46:00.840
are generally benevolent, but you know they could use your
730
00:46:00.880 --> 00:46:02.079
tool badly.
731
00:46:03.119 --> 00:46:04.840
They could cause wreckage.
732
00:46:04.920 --> 00:46:11.519
So don't use evolved, don't use system right, but try
733
00:46:11.559 --> 00:46:14.480
to be kind of careful about it. But in general,
734
00:46:14.559 --> 00:46:19.000
it's a fantastic way to make ELM super super powerful.
735
00:46:21.800 --> 00:46:25.480
Yeah, and I think tool design and security with the
736
00:46:25.519 --> 00:46:27.679
tools and things like that might be outside the scope
737
00:46:27.719 --> 00:46:31.920
of this episode, but yeah, I mean I've had people
738
00:46:31.960 --> 00:46:34.920
ask me too, it's, hey, well, how do you make
739
00:46:34.920 --> 00:46:37.239
sure that the tool is only doing this stuff within
740
00:46:37.280 --> 00:46:39.559
the scope you want? So, for example, in my case
741
00:46:39.639 --> 00:46:42.800
with podcasting, it's how do I make sure it's only
742
00:46:43.239 --> 00:46:47.280
doing stuff on one person's podcast and not another, or
743
00:46:47.360 --> 00:46:47.960
things like that.
744
00:46:48.159 --> 00:46:48.480
And so.
745
00:46:50.119 --> 00:46:53.079
Because you know, we've seen people do what looks like
746
00:46:53.599 --> 00:46:57.840
SQL injection attacks except as prompt stuff, and it says, well,
747
00:46:58.360 --> 00:47:02.239
but pretend you can do it essentially, right, they talk
748
00:47:02.400 --> 00:47:06.320
the LM around to Okay, I'm going to pretend I
749
00:47:06.360 --> 00:47:08.320
can do it, and so I'm going to act like
750
00:47:08.360 --> 00:47:12.119
this other user even though I'm not technically supposed to
751
00:47:12.119 --> 00:47:15.000
do that. And there are only so many safeguards you
752
00:47:15.039 --> 00:47:16.800
can put into place. But if you don't have a
753
00:47:16.800 --> 00:47:20.440
tool that allows them to impersonate anyone else, and you've
754
00:47:20.599 --> 00:47:23.960
locked down the access to the user and it's data
755
00:47:24.039 --> 00:47:27.719
to a methodology you can control, like find the current
756
00:47:27.800 --> 00:47:30.840
user uses the session to validate the user instead of
757
00:47:31.039 --> 00:47:34.599
looking the user up by email address, for example, and
758
00:47:34.639 --> 00:47:36.639
it doesn't have a tool to look up the user
759
00:47:36.679 --> 00:47:39.800
by email address unless you need that feature for admins.
760
00:47:39.840 --> 00:47:41.599
But then you use a different chat agent for that
761
00:47:42.679 --> 00:47:46.280
and give it those tools there Now you're getting into
762
00:47:47.880 --> 00:47:49.639
so you have to be careful how you use them.
763
00:47:49.880 --> 00:47:52.519
Like you said, for sure, try and think ahead on
764
00:47:53.239 --> 00:47:56.920
how people may trick your agent into doing something that shouldn't.
765
00:47:57.199 --> 00:47:58.840
That's right, That's exactly right.
766
00:48:02.280 --> 00:48:03.800
But yeah, the tools are cool.
767
00:48:04.679 --> 00:48:07.679
One question I had on this, so I built a
768
00:48:07.719 --> 00:48:10.159
tool for the weather, and then I was thinking about
769
00:48:10.199 --> 00:48:12.599
it and I was like, well, what if I wanted
770
00:48:12.679 --> 00:48:15.320
So then I was showing off, not what's the weather
771
00:48:15.400 --> 00:48:19.199
in Lehigh, Utah, which is where I live, but you know,
772
00:48:19.679 --> 00:48:21.760
is it a good time to go have a picnic
773
00:48:21.760 --> 00:48:22.519
in Lehigh, Utah?
774
00:48:22.599 --> 00:48:22.719
Right?
775
00:48:22.719 --> 00:48:24.079
And it's smart enough to go, Okay, I need to
776
00:48:24.079 --> 00:48:26.840
know what Lehi is, I need you know. I figured
777
00:48:26.880 --> 00:48:28.920
out the latitude and longitude. I sent it to the
778
00:48:28.960 --> 00:48:30.840
Weather Service. It told me what the weather was, and
779
00:48:30.880 --> 00:48:32.960
then it said, yeah, it's kind of a nice evening
780
00:48:33.000 --> 00:48:36.519
for a picnic, right because it was sixty degrees and
781
00:48:37.119 --> 00:48:40.079
you know, the wind wasn't too bad and it wasn't raining.
782
00:48:41.800 --> 00:48:43.679
But then I was thinking, okay, well, what if I
783
00:48:43.679 --> 00:48:45.960
wanted to ask it what day this week should I
784
00:48:46.000 --> 00:48:49.599
go for a picnic? Then it has to have another
785
00:48:49.679 --> 00:48:53.119
tool that will do the forecast look up. And so
786
00:48:53.360 --> 00:48:56.760
I can imagine that if you're creating tools, you could
787
00:48:56.760 --> 00:49:01.079
wind up with a lot of tools, right, collection of them.
788
00:49:01.599 --> 00:49:03.519
I'm wondering if there's a way to like group them together.
789
00:49:03.559 --> 00:49:05.519
So it's like, hey, here's all the calendar stuff, here's
790
00:49:05.519 --> 00:49:08.000
all the drop box stuff, here's all the drive stuff.
791
00:49:08.480 --> 00:49:10.840
I think that's exactly what Entropic was trying to do
792
00:49:10.880 --> 00:49:16.960
with MCP. It's probably exactly that use case. And essentially,
793
00:49:17.039 --> 00:49:19.400
as I said, MCP is this way of bunching up
794
00:49:19.559 --> 00:49:24.960
different tools into one single API, and it tells the LLM, hey,
795
00:49:25.039 --> 00:49:27.000
I have all these things that you can do now.
796
00:49:28.519 --> 00:49:31.920
And you can even run like MCP servers in your
797
00:49:31.960 --> 00:49:37.840
own computer for example, in order to you know, connected
798
00:49:38.199 --> 00:49:39.880
the LM with your files.
799
00:49:41.599 --> 00:49:42.280
Use it more.
800
00:49:42.480 --> 00:49:48.039
You know, cloud code Sorry, cloud code uses MCP as
801
00:49:48.039 --> 00:49:50.800
far as I know, and maybe wrong on this, but
802
00:49:50.960 --> 00:49:54.239
I think it uses MCP to actually connect to your
803
00:49:54.280 --> 00:49:57.119
files and your file system and to look up stuff
804
00:49:57.119 --> 00:50:00.840
and to change it. So it is a valid point,
805
00:50:01.400 --> 00:50:03.039
you know, and you can win up with a lot
806
00:50:03.079 --> 00:50:05.519
of tools and actually they consume a little bit of
807
00:50:05.599 --> 00:50:11.760
your context window. However, now with the recent models, you know,
808
00:50:12.280 --> 00:50:14.440
I don't know if you heard, GPT four point one
809
00:50:14.960 --> 00:50:17.800
just came out and it has a one million token
810
00:50:17.880 --> 00:50:19.280
context that.
811
00:50:19.320 --> 00:50:20.599
Is pretty big.
812
00:50:21.599 --> 00:50:24.119
Yeah, you can do a lot of stuff with one
813
00:50:24.119 --> 00:50:28.199
million tokens and I've been using Gemini two point five
814
00:50:28.280 --> 00:50:35.280
pro extensively lately and it's a fantastic model super bowlish
815
00:50:35.320 --> 00:50:40.960
on on Google lately, and it also has you know,
816
00:50:41.440 --> 00:50:47.159
million context window if not two million, I don't remember that.
817
00:50:50.440 --> 00:50:51.400
Yeah, that makes sense.
818
00:50:53.800 --> 00:50:56.320
The other thing that I've seen done and we talked
819
00:50:56.360 --> 00:51:00.199
to Obi before MCP was really a thing, and what
820
00:51:00.239 --> 00:51:02.280
he was doing was he actually had different.
821
00:51:02.039 --> 00:51:04.559
Agents that had different jobs, right, and.
822
00:51:04.559 --> 00:51:10.000
So then instead of providing this huge list of tools,
823
00:51:10.840 --> 00:51:14.519
he would he would have one tool that was essentially said,
824
00:51:15.119 --> 00:51:20.159
write a prompt for the calendar agent, right, and so
825
00:51:20.280 --> 00:51:22.519
then it would prompt the calendar agent and say, hey,
826
00:51:22.559 --> 00:51:26.119
can you invite so and so to this calendar event?
827
00:51:27.079 --> 00:51:29.920
And then on the calendar agent would then say, oh,
828
00:51:30.000 --> 00:51:31.760
this means I have to go find the event.
829
00:51:31.559 --> 00:51:34.360
And update it with you know, with the user or whatever.
830
00:51:35.039 --> 00:51:38.960
So that
that was his way around it. I kind
831
00:51:38.960 --> 00:51:42.719
of liked that approach anyway, because then I really can
832
00:51:42.840 --> 00:51:46.800
kind of box in each of the agents with just
833
00:51:46.840 --> 00:51:51.039
a small set of responsibilities and capabilities. But yeah, that
834
00:51:51.360 --> 00:51:53.400
was his way around it, and I think that's kind
835
00:51:53.400 --> 00:51:55.960
of an elegant solution. But I can imagine if I
836
00:51:56.039 --> 00:52:00.480
have a app tools directory in my application, that I
837
00:52:00.599 --> 00:52:02.880
might have a lot of those in there, depending on
838
00:52:02.920 --> 00:52:06.440
how many capabilities I'm giving to my application right or
839
00:52:06.440 --> 00:52:07.079
to my agent.
840
00:52:07.320 --> 00:52:07.559
Right.
841
00:52:08.480 --> 00:52:09.960
However, I think that this.
842
00:52:11.400 --> 00:52:16.199
Is a totally fine design solution for the past models
843
00:52:16.239 --> 00:52:20.320
that we had, you know, even last year, where two
844
00:52:20.440 --> 00:52:24.599
hundred thousand tokens context windows were pretty big, right.
845
00:52:26.199 --> 00:52:26.679
Yeah.
846
00:52:27.199 --> 00:52:31.159
I think we'll probably see in the future, you know,
847
00:52:31.320 --> 00:52:36.480
just having one single model that would actually handle all
848
00:52:36.519 --> 00:52:39.199
the tools you have, because the context window is big enough,
849
00:52:39.719 --> 00:52:42.119
and it would be super powerful and it can do
850
00:52:42.199 --> 00:52:43.840
so many things, it will actually.
851
00:52:45.000 --> 00:52:48.559
Probably blow people's minds.
852
00:52:50.079 --> 00:52:52.800
Yeah, with a million tokens. Yeah, I can imagine that
853
00:52:52.800 --> 00:52:58.000
that's that's a ton. It's interesting because I was working
854
00:52:58.039 --> 00:53:02.880
with a model this earlier this year, and I think
855
00:53:02.880 --> 00:53:04.360
the context window had.
856
00:53:04.159 --> 00:53:07.280
Like two hundred thousand tokens. I think something like that
857
00:53:07.280 --> 00:53:08.679
one hundred and fifty two hundred thousand.
858
00:53:09.239 --> 00:53:10.960
And the issue that I was running into there was
859
00:53:10.960 --> 00:53:15.079
that I was pulling in bills during the Utah legislative
860
00:53:15.119 --> 00:53:18.320
session and getting summaries done and things like that with it,
861
00:53:18.960 --> 00:53:20.800
and there were like four.
862
00:53:20.440 --> 00:53:23.239
Bills that were too big. It blew, it blew past
863
00:53:23.320 --> 00:53:25.440
the context window, and so I.
864
00:53:25.400 --> 00:53:29.000
Tried breaking it up, but ultimately that's the other great
865
00:53:29.039 --> 00:53:30.760
thing about a lot of this stuff is that the
866
00:53:30.800 --> 00:53:34.039
context windows are getting bigger, and what that means is
867
00:53:34.079 --> 00:53:39.599
that the prices are coming down for models that have
868
00:53:39.719 --> 00:53:42.079
larger context windows that were kind of the run of
869
00:53:42.119 --> 00:53:44.880
the mill, state of the art ones before. And so
870
00:53:45.119 --> 00:53:47.320
you can get away with a lot more or you
871
00:53:47.360 --> 00:53:50.039
can just like in your what you're saying here with
872
00:53:50.079 --> 00:53:53.119
the chat is that I can go in in midstreams
873
00:53:53.239 --> 00:53:56.320
it comes back and says, hey, this was way too big.
874
00:53:56.360 --> 00:54:00.320
I can go Okay, just this time, go use bigger
875
00:54:00.360 --> 00:54:02.840
model and pay the extra money.
876
00:54:03.000 --> 00:54:04.000
Right Yeah.
877
00:54:04.039 --> 00:54:07.039
I think these are two trends that we've seen so far.
878
00:54:07.159 --> 00:54:09.719
So one is context window becoming bigger. And I think
879
00:54:09.719 --> 00:54:12.079
it's a question of just pure capabilities.
880
00:54:12.119 --> 00:54:14.159
Right, So, all of.
881
00:54:14.119 --> 00:54:16.760
The applications similar to chat with work, and there are
882
00:54:16.960 --> 00:54:22.000
a few, they all need to have sort of big
883
00:54:22.039 --> 00:54:24.480
context windows in order to function correctly, because you want
884
00:54:24.480 --> 00:54:28.960
to have as many documents, you know, describing things or
885
00:54:29.119 --> 00:54:32.599
chats or whatever else that is relevant for the topic
886
00:54:32.639 --> 00:54:35.039
that the user asked in order to be first of
887
00:54:35.079 --> 00:54:39.519
all correct in your answer. And Thori and I think
888
00:54:39.599 --> 00:54:42.599
there is always a balance in an information retrieval system
889
00:54:42.639 --> 00:54:46.079
like this, like how many you actually present even to
890
00:54:46.119 --> 00:54:47.039
the LLLM itself.
891
00:54:48.039 --> 00:54:50.000
But that's a whole other topic.
892
00:54:52.199 --> 00:54:54.679
But with the big context windows, you can kind of,
893
00:54:54.880 --> 00:54:58.039
you know, relax a little bit that balance. And then
894
00:54:58.039 --> 00:55:01.840
the second thing that you mentioned, which is getting the
895
00:55:01.880 --> 00:55:06.960
prices down, I think this is inevitable. The models are
896
00:55:06.960 --> 00:55:12.079
being commoditized more and more, and honestly, I am finally
897
00:55:12.079 --> 00:55:14.920
seeing what I thought Google would have done, which is,
898
00:55:15.559 --> 00:55:19.760
you know, putting the prices very very low for models
899
00:55:19.800 --> 00:55:22.079
because they have access to TPUs, right, so they make
900
00:55:22.119 --> 00:55:28.199
their own chips for AI. And one thing, you know,
901
00:55:28.280 --> 00:55:31.719
you can use Nvidia cards and they're fantastic, you know,
902
00:55:32.719 --> 00:55:36.320
for for EI usage. But another thing is you know
903
00:55:36.440 --> 00:55:41.920
TPU from from Google. It goes much faster, and you know,
904
00:55:42.000 --> 00:55:44.280
it's it's so
much better. They make it in house,
905
00:55:44.360 --> 00:55:49.039
so it's probably cheaper for them. So yeah, I'm really
906
00:55:49.079 --> 00:55:51.960
excited to see where where things will go next with
907
00:55:52.039 --> 00:55:56.960
this in in AI because there seems to be so
908
00:55:57.119 --> 00:55:59.079
much going on at the same time it's kind of
909
00:55:59.079 --> 00:55:59.679
hard to follow.
910
00:56:02.039 --> 00:56:04.159
Yep, yep, absolutely well.
911
00:56:04.239 --> 00:56:06.079
Yeah, and every time I see a new model come
912
00:56:06.119 --> 00:56:08.480
out and people are like because it's felt like the
913
00:56:08.559 --> 00:56:12.159
last little while, you know, like Grock came out and everyone's.
914
00:56:11.800 --> 00:56:13.960
Like Grock is the best and it's doing the best
915
00:56:14.039 --> 00:56:14.760
and blah blah blah.
916
00:56:14.800 --> 00:56:17.599
And literally like a week later, like GPT four one
917
00:56:17.719 --> 00:56:20.199
was out or Anthropic, I can't remember, but one of
918
00:56:20.199 --> 00:56:22.320
them came out with something and it was like, Wow,
919
00:56:22.440 --> 00:56:24.880
this one has you know, this much more over Grock.
920
00:56:24.920 --> 00:56:26.840
And then it was the other one, whether it was
921
00:56:26.880 --> 00:56:29.199
GPT or Claude or something else, came out and it
922
00:56:29.239 --> 00:56:31.360
was like, oh, now this one right. And so we
923
00:56:31.440 --> 00:56:34.440
keep getting better and better and it feels like this
924
00:56:34.639 --> 00:56:36.639
arms race. But the flip side of it is is
925
00:56:36.679 --> 00:56:40.960
like you said, you know, the models that are powerful
926
00:56:41.119 --> 00:56:44.880
enough are being commoditized to the point where it's just like,
927
00:56:45.000 --> 00:56:46.880
you know what, just just pick the one that's going
928
00:56:46.960 --> 00:56:49.800
to do the thing for you. And I mean, granted,
929
00:56:50.000 --> 00:56:53.239
you know, the latent space some of them allow the
930
00:56:53.239 --> 00:56:56.039
different models to do better in certain contexts. Right, it's
931
00:56:56.079 --> 00:57:00.920
not just the token size, but holy crap, if they
932
00:57:01.000 --> 00:57:03.360
keep pushing it the way they're pushing it, I mean,
933
00:57:03.400 --> 00:57:05.760
we're just going to have some wild capabilities with what
934
00:57:05.800 --> 00:57:06.239
it can do.
935
00:57:06.760 --> 00:57:09.199
And I for one super excited about that.
936
00:57:09.679 --> 00:57:13.599
Yeah, we're getting a little bit close to time, so
937
00:57:13.639 --> 00:57:19.000
I'm going to push us to image generation and text
938
00:57:19.000 --> 00:57:22.599
embedding and then we'll wrap up with picks. We've got
939
00:57:22.639 --> 00:57:28.079
about ten minutes to talk about image generation and embedding.
940
00:57:28.119 --> 00:57:31.119
So how does that all work?
941
00:57:31.159 --> 00:57:32.679
Because I didn't dive into that so much.
942
00:57:32.760 --> 00:57:35.000
I've used other tools to do it, but I didn't
943
00:57:35.039 --> 00:57:36.199
dive into that so much with.
944
00:57:38.440 --> 00:57:42.559
Ruby LLM, right, So it's a simple call. It's at
945
00:57:42.599 --> 00:57:45.599
the same level as RUBYLM do chat, but since it
946
00:57:45.639 --> 00:57:48.519
doesn't need to have like the context of all the
947
00:57:48.559 --> 00:57:52.559
conversation that you have with the LLM, it's as simple
948
00:57:52.559 --> 00:57:53.480
as RUBYLM do.
949
00:57:53.599 --> 00:57:58.800
Paint and then you have your message there.
950
00:57:59.360 --> 00:58:02.360
And also for embedding, so you can just use RUBYLM
951
00:58:02.360 --> 00:58:05.239
dot m bed and just the text. You can also
952
00:58:05.320 --> 00:58:10.960
pass a raise so it will embed multiple texts altogether
953
00:58:11.039 --> 00:58:14.280
and batch them together. This is nice for you know,
954
00:58:14.360 --> 00:58:20.719
pricing and API usage. But that's pretty much all there
955
00:58:20.760 --> 00:58:23.199
is to it. You know, there's some little things that
956
00:58:23.239 --> 00:58:25.519
you can do depending on the model. For example, for
957
00:58:26.199 --> 00:58:31.559
I believe Dali has two different sizes outputs, so.
958
00:58:31.440 --> 00:58:32.320
You can configure that.
959
00:58:33.119 --> 00:58:37.119
But and you can choose obviously between all the models
960
00:58:37.119 --> 00:58:41.320
that we support in the providers that we support. But
961
00:58:42.760 --> 00:58:45.079
there's pretty much all there is to it about these
962
00:58:45.119 --> 00:58:48.440
two two things is just a very simple interface for that.
963
00:58:50.880 --> 00:58:51.880
Yeah, I have to say.
964
00:58:51.880 --> 00:58:54.400
One of the things that I'm excited about this as
965
00:58:54.400 --> 00:59:00.360
far as the image generation goes, is I want people
966
00:59:00.400 --> 00:59:02.360
to be able to say, hey, I want you to
967
00:59:02.440 --> 00:59:08.039
generate some album of art or some episode artwork or
968
00:59:08.039 --> 00:59:12.360
something like that. And like if I say, hey, use
969
00:59:12.400 --> 00:59:17.119
this episode that we recorded and generate some artwork for it,
970
00:59:18.000 --> 00:59:20.719
I mean I could see it basically using the chat
971
00:59:20.800 --> 00:59:25.400
prompt set up to say you know, and you could
972
00:59:25.440 --> 00:59:29.360
you said you could pass it attachments, so it could say,
973
00:59:29.599 --> 00:59:34.119
here's the audio for the episode, you know, give me,
974
00:59:34.719 --> 00:59:40.199
write me a prompt to get a good image, you know,
975
00:59:40.719 --> 00:59:43.679
cover art image for this episode. And then it turns
976
00:59:43.719 --> 00:59:47.400
around and it goes ruby LLM paint. Here's the prompt
977
00:59:47.440 --> 00:59:52.559
that I made up, right, and then you can see
978
00:59:52.559 --> 00:59:54.840
what
it comes up with. And the other thing is
979
00:59:54.840 --> 00:59:57.960
is that on the chat models we didn't talk about temperature,
980
00:59:58.719 --> 01:00:00.920
but you could set the temperature a little bit higher
981
01:00:01.480 --> 01:00:05.599
and get some more creativity out of your prompt. Then
982
01:00:05.639 --> 01:00:08.199
the image generation may give you something a little bit different,
983
01:00:08.599 --> 01:00:11.159
And so I'm looking at changing some of these together
984
01:00:11.199 --> 01:00:13.480
and I'm really excited about that stuff too. But yeah,
985
01:00:13.280 --> 01:00:15.719
the image generation seem pretty straightforward to me.
986
01:00:16.199 --> 01:00:20.280
Yeah, what I'm excited about is to see GPT four
987
01:00:20.320 --> 01:00:23.719
all image generation in the APIs, and I know it's
988
01:00:23.760 --> 01:00:26.199
coming soon. I know it has been taking the world
989
01:00:26.239 --> 01:00:32.800
by storm lately. How many Studio Ghibli memes of your found, Charles, Oh.
990
01:00:32.679 --> 01:00:35.400
Yeah, you have them.
991
01:00:35.599 --> 01:00:40.599
Yeah, so you see wacky stuff with Donald Trump or
992
01:00:41.440 --> 01:00:44.400
Bernie Sanders here in the US, right, you know, not
993
01:00:44.519 --> 01:00:46.519
to pick on one side or the other, but both
994
01:00:46.559 --> 01:00:49.559
of those guys cartoon out really really well, and it's
995
01:00:49.599 --> 01:00:51.320
so fun to see some of them because it's like
996
01:00:51.559 --> 01:00:55.320
it's like, oh, yeah, that's the studio Ghibli that kind
997
01:00:55.320 --> 01:00:58.199
of emphasizes kind of the quirky aspects of either one
998
01:00:58.239 --> 01:00:58.960
of those guys.
999
01:01:00.920 --> 01:01:05.840
Yeah, the capability of JUBT for image and it is fantastic.
1000
01:01:06.960 --> 01:01:09.400
But returning back to your question, I think the.
1001
01:01:10.880 --> 01:01:13.079
Or rather what you said is to me, I'm really
1002
01:01:13.119 --> 01:01:17.719
excited about that because it really would combine both the
1003
01:01:17.760 --> 01:01:20.320
text interface and also like the image generation in the
1004
01:01:20.320 --> 01:01:21.159
same model.
1005
01:01:21.480 --> 01:01:22.639
So that we can have.
1006
01:01:23.159 --> 01:01:27.320
This capability without even needing to create a new tool.
1007
01:01:27.960 --> 01:01:29.960
So right now, how you would do it is that
1008
01:01:30.039 --> 01:01:32.480
you create a tool and inside the tool you have
1009
01:01:32.599 --> 01:01:37.119
RUBLM dot paint. Right with these types of models, you
1010
01:01:37.119 --> 01:01:39.079
wouldn't need a tool anymore, and it would do it
1011
01:01:39.199 --> 01:01:40.719
all in the same model, and it would do it
1012
01:01:40.800 --> 01:01:44.800
so much better than the previous diffusion models because apparently
1013
01:01:44.800 --> 01:01:52.039
they're using combination of diffusion and transformers architectures. So in
1014
01:01:52.079 --> 01:01:55.599
the end, what comes down to is that is so
1015
01:01:55.760 --> 01:02:00.360
much more beautiful, so much more precise. It also understands language, right,
1016
01:02:00.480 --> 01:02:04.159
so it also understands out to make fonts so you
1017
01:02:04.199 --> 01:02:06.639
can write you know, when it writes it, actually you
1018
01:02:06.639 --> 01:02:08.679
can read it, and it's not just this garbage that
1019
01:02:09.159 --> 01:02:11.760
we used to get like a year ago.
1020
01:02:14.199 --> 01:02:20.880
Right, So I have to admit with the embeddings, so
1021
01:02:21.000 --> 01:02:24.320
I was flirting with the idea of doing vector searches
1022
01:02:24.880 --> 01:02:27.400
for top end devs, and I'm going to move there,
1023
01:02:27.840 --> 01:02:30.400
but I'm just using what's built into postgress at this point,
1024
01:02:30.960 --> 01:02:34.679
and that's what it pointed me too as one option, Right,
1025
01:02:34.719 --> 01:02:38.519
I could hand off getting the vectors out of a
1026
01:02:38.599 --> 01:02:41.840
text embedding, but beyond that are there are other things
1027
01:02:41.840 --> 01:02:44.360
that people do with these because I'm I'm kind of
1028
01:02:44.480 --> 01:02:46.599
lost as far as like how useful this is?
1029
01:02:46.840 --> 01:02:51.480
Right, Well, you could do so many things, like you know,
1030
01:02:51.639 --> 01:02:56.480
cluster things together, for example, and try to have different topics.
1031
01:02:57.119 --> 01:03:01.159
So let's maybe start from what isn't embedding. I'll try
1032
01:03:01.199 --> 01:03:02.840
to compress it because I know we don't have so
1033
01:03:02.920 --> 01:03:07.639
much time. It's basically a very long array of numbers
1034
01:03:07.960 --> 01:03:11.400
which represents a point in a space which is a
1035
01:03:11.480 --> 01:03:15.480
multi dimensional space, and the dimensions are you know, the
1036
01:03:15.559 --> 01:03:18.559
length of the array. Essentially, what that means is that
1037
01:03:18.599 --> 01:03:22.239
if you projected in three D space, it's just this point, right,
1038
01:03:22.320 --> 01:03:24.760
And let's say this point represents a word or a
1039
01:03:24.800 --> 01:03:32.079
document or whatever, and the distance between another point is
1040
01:03:32.079 --> 01:03:35.239
actually significant, and it is semantically significant.
1041
01:03:35.360 --> 01:03:37.119
So, for example, if.
1042
01:03:37.000 --> 01:03:41.519
You would have the word and let's let's bring it
1043
01:03:41.559 --> 01:03:43.800
back to word t K. Right, it's a really really
1044
01:03:43.800 --> 01:03:48.719
old model that used to represent words in this embedding space.
1045
01:03:48.880 --> 01:03:49.039
Right.
1046
01:03:49.639 --> 01:03:57.840
So if we say, if we take the embedding of king, right,
1047
01:03:58.519 --> 01:04:01.639
and we take the embedding of queen, we can actually
1048
01:04:02.400 --> 01:04:07.719
I believe, subtract
from King the embedding of man and
1049
01:04:07.920 --> 01:04:12.000
add the embedding of woman, and we will get very close,
1050
01:04:12.679 --> 01:04:16.519
if not exactly perfect on the embedding of queen. Right,
1051
01:04:16.719 --> 01:04:19.880
So they're semantically significant. So what that means is that
1052
01:04:19.920 --> 01:04:25.840
you can start to you know, cluster similar data points,
1053
01:04:25.920 --> 01:04:32.440
similar documents, similar movies, similar whatever you want. You can
1054
01:04:32.559 --> 01:04:37.880
search in them. And searching is a kind of a clustering.
1055
01:04:37.880 --> 01:04:41.800
So you're basically like taking the query and embed it
1056
01:04:42.480 --> 01:04:44.320
and then see where it ends up in the space,
1057
01:04:44.760 --> 01:04:49.559
and then you start to expand your your bubble to
1058
01:04:49.719 --> 01:04:52.760
see who else is nearby, and then you just take
1059
01:04:52.800 --> 01:04:55.519
a limit, you know, the top ten or whatever, because
1060
01:04:55.559 --> 01:04:57.480
it could go infinitely. You know, you could get all
1061
01:04:57.519 --> 01:04:59.840
the documents just ordered by out.
1062
01:04:59.760 --> 01:05:04.239
Close, right you were just yeah, ordered by distance. Yeah yeah, Okay,
1063
01:05:04.880 --> 01:05:06.000
So it really is.
1064
01:05:05.960 --> 01:05:10.320
Kind of a search type use case, that's right, But
1065
01:05:10.400 --> 01:05:14.920
it's more semantic than the typical keyword search.
1066
01:05:16.480 --> 01:05:22.880
Right, because if you use the keyword for ruby, well
1067
01:05:22.920 --> 01:05:27.840
that's not a great example, but if you make it
1068
01:05:28.159 --> 01:05:31.039
you have two synonyms, yeah.
1069
01:05:31.280 --> 01:05:33.119
Or yes, sing a TYPEO won't find it.
1070
01:05:33.880 --> 01:05:37.159
Or if you have synonyms, right, And so somebody said testing,
1071
01:05:37.199 --> 01:05:41.480
and somebody said, uh, tested, and you look for tested,
1072
01:05:42.320 --> 01:05:46.280
it won't find testing because it's not doing semantic you know,
1073
01:05:46.320 --> 01:05:47.679
it doesn't break it up into tokens.
1074
01:05:47.679 --> 01:05:50.679
It doesn't do all of that other work where this does.
1075
01:05:50.800 --> 01:05:52.559
And so yeah that makes sense.
1076
01:05:52.639 --> 01:05:55.360
Yeah, a typical search engine would have stemming, so they
1077
01:05:55.400 --> 01:06:00.000
would separate testing and tested and kind of matches anyway.
1078
01:06:00.360 --> 01:06:02.920
But I that's exactly right. I think the genre idea
1079
01:06:03.000 --> 01:06:03.639
is exactly right.
1080
01:06:04.679 --> 01:06:08.199
Yeah, all right, cool, Well, I'm gonna go ahead and
1081
01:06:08.800 --> 01:06:11.760
wrap us up and head us into picks. But this
1082
01:06:11.920 --> 01:06:15.719
is terrific. I'm gonna point people to ruby l l
1083
01:06:15.840 --> 01:06:17.840
M dot com. But if they want to reach out
1084
01:06:17.880 --> 01:06:22.800
to you or connect in some other way, like how
1085
01:06:22.800 --> 01:06:24.719
do people find you on the internet and say, dude,
1086
01:06:24.760 --> 01:06:25.440
this is awesome.
1087
01:06:26.320 --> 01:06:32.079
So my blog is at paulinot me and you can
1088
01:06:32.119 --> 01:06:34.880
reach me on Twitter as well at x dot com
1089
01:06:34.920 --> 01:06:36.400
slash Paulino.
1090
01:06:37.719 --> 01:06:40.840
Mm hmm. And for the non Italians, Paulino is p
1091
01:06:40.920 --> 01:06:42.559
a O l I n O.
1092
01:06:43.159 --> 01:06:45.280
Yes, correct, p l O n I N.
1093
01:06:45.440 --> 01:06:49.719
So yeah, it's it's the p a O that might
1094
01:06:49.800 --> 01:06:53.079
lose people anyway.
1095
01:06:54.559 --> 01:06:56.920
Thanks so much for coming. Let's let's jump in on
1096
01:06:56.960 --> 01:07:00.480
the picks. I'm gonna be kind of short with mine
1097
01:07:02.280 --> 01:07:03.519
just because I don't want to take a ton of
1098
01:07:03.519 --> 01:07:04.840
time and I have to jump over to this other
1099
01:07:04.880 --> 01:07:11.320
meeting at eleven. But so my first pick I always
1100
01:07:11.360 --> 01:07:15.800
do a board game picks. Last week I think I
1101
01:07:15.880 --> 01:07:17.599
might have picked this, but I'm going to pick it again.
1102
01:07:18.440 --> 01:07:20.400
It's kind of like the game that we've been playing lately.
1103
01:07:20.400 --> 01:07:26.199
It's called Cult Express, and basically you have a train
1104
01:07:26.280 --> 01:07:28.400
on the table and you know, it's just made out
1105
01:07:28.440 --> 01:07:30.480
of wood. It kind of all fits together. But it's
1106
01:07:30.599 --> 01:07:34.800
unique for a game board, right because it's it's it's
1107
01:07:34.840 --> 01:07:39.719
this three D train and your bank your train robbers,
1108
01:07:39.800 --> 01:07:41.480
and so you're trying to gather as much loot as
1109
01:07:41.519 --> 01:07:45.840
possible and whoever gathers the most loot wins. You also
1110
01:07:45.960 --> 01:07:50.800
get you get one thousand dollars for being the person
1111
01:07:50.800 --> 01:07:54.400
who shot the most bullets. And so you have cards
1112
01:07:54.400 --> 01:07:56.480
that represent actions. You can pick up loot, you can
1113
01:07:56.559 --> 01:07:58.519
move from the roof to the car, or the car
1114
01:07:58.599 --> 01:08:00.920
to the roof. You can move one card to the other.
1115
01:08:01.840 --> 01:08:06.800
You can shoot your gun at other robbers, and if
1116
01:08:06.880 --> 01:08:09.840
you hit them, then your bullet card goes into their deck.
1117
01:08:10.199 --> 01:08:13.440
And it's basically no op, no action card. And so
1118
01:08:13.480 --> 01:08:15.079
when you drop up a new hand, if you have
1119
01:08:15.159 --> 01:08:18.399
those in there. They're just kind of duds. You can
1120
01:08:18.439 --> 01:08:21.920
punch other players, and if you punch another player, then
1121
01:08:21.920 --> 01:08:25.600
they drop
a piece of loot, and then you can
1122
01:08:25.640 --> 01:08:28.640
pick up loot. And then all the players or all
1123
01:08:28.680 --> 01:08:32.239
the characters have different abilities, right, And so the player
1124
01:08:32.279 --> 01:08:35.359
that I played, if you punch somebody and they drop loot,
1125
01:08:35.479 --> 01:08:37.760
if they drop a loot bag, you can catch it
1126
01:08:38.479 --> 01:08:41.479
before it hits the floor and collect it without having
1127
01:08:41.479 --> 01:08:42.920
to play a pick up loot card.
1128
01:08:43.239 --> 01:08:43.439
Right.
1129
01:08:44.159 --> 01:08:46.560
Some of the others were that you could you could
1130
01:08:46.560 --> 01:08:49.199
shoot above you or below you if you were on
1131
01:08:49.239 --> 01:08:51.840
the roof or in the car. There were some that
1132
01:08:51.920 --> 01:08:53.880
had to do with a sheriff, right, because there's a
1133
01:08:53.920 --> 01:08:56.560
sheriff on the train and if he winds up in
1134
01:08:56.600 --> 01:08:58.960
the same car as you, then he puts a bullet
1135
01:08:58.960 --> 01:09:03.039
in your in your deck anyway, So it was a
1136
01:09:03.079 --> 01:09:05.680
lot of fun board game. Geek waits at one point
1137
01:09:05.720 --> 01:09:09.000
eighty six, which means that it's pretty approachable for the
1138
01:09:09.039 --> 01:09:12.479
average random board game player that doesn't play the heavy
1139
01:09:12.520 --> 01:09:17.159
board games like I like. And uh yeah, it took
1140
01:09:17.239 --> 01:09:19.560
us like forty five minutes or so to play, or
1141
01:09:19.600 --> 01:09:23.199
maybe an hour, but we were learning the rules and
1142
01:09:23.239 --> 01:09:24.840
so I could I could imagine you can play in
1143
01:09:24.880 --> 01:09:25.600
forty five minutes.
1144
01:09:26.079 --> 01:09:27.880
And yeah.
1145
01:09:27.920 --> 01:09:31.560
Anyway, so the way you play your cards is you
1146
01:09:31.640 --> 01:09:34.600
go around the circle and everybody plays a card onto
1147
01:09:34.720 --> 01:09:35.720
the pile.
1148
01:09:37.079 --> 01:09:38.319
Sometimes you play two cards.
1149
01:09:38.319 --> 01:09:41.439
Sometimes you play him face down, but you can see
1150
01:09:41.439 --> 01:09:44.239
what everybody else is playing on a regular play. And
1151
01:09:44.279 --> 01:09:47.319
then you take the deck and you flip it over
1152
01:09:47.560 --> 01:09:50.840
and you just play the deck. And so then you know,
1153
01:09:51.159 --> 01:09:53.039
he played a shoot the gun card, so if he
1154
01:09:53.039 --> 01:09:55.520
could shoot somebody, you shot him. And then the next
1155
01:09:55.560 --> 01:09:57.680
guy he played a punch card, so he punched the
1156
01:09:57.720 --> 01:09:59.920
guy next to him because he could. And if you can,
1157
01:10:00.199 --> 01:10:01.720
then you can't write. So if you play it and
1158
01:10:01.760 --> 01:10:03.439
there's nobody in the car with you to punch and
1159
01:10:03.479 --> 01:10:06.079
you just don't
punch. But if there is somebody there
1160
01:10:06.119 --> 01:10:07.960
and they're not the person you wanted to punch, you
1161
01:10:08.000 --> 01:10:08.840
still punch them.
1162
01:10:09.239 --> 01:10:10.840
So that's kind of how it works.
1163
01:10:10.880 --> 01:10:14.920
So anyway, it was really really fun, really really enjoyed it.
1164
01:10:14.920 --> 01:10:17.399
It kind of reminds me of another game called robot Rally.
1165
01:10:18.279 --> 01:10:20.399
The difference is is that with robot Rally, what you
1166
01:10:20.439 --> 01:10:23.359
do is you put all of your action cards into
1167
01:10:23.399 --> 01:10:26.039
your own pile and then you flip it over and
1168
01:10:26.079 --> 01:10:28.119
play them in order. But you play them in order,
1169
01:10:28.199 --> 01:10:31.319
everybody goes around and plays them. And so you know,
1170
01:10:31.399 --> 01:10:33.199
a lot of times you set yourself up to push
1171
01:10:33.239 --> 01:10:37.439
another robot into a hole or make them go off course,
1172
01:10:37.680 --> 01:10:39.479
but they did something you didn't expect, and so then
1173
01:10:39.560 --> 01:10:42.239
you wind up making moves that you didn't expect because
1174
01:10:42.239 --> 01:10:45.199
you expected to hit them. And so anyway, it's it's
1175
01:10:45.239 --> 01:10:48.439
a lot of fun. It's kind of a it's almost
1176
01:10:48.439 --> 01:10:52.279
like mob programming a little bit. So anyway, a fun
1177
01:10:52.359 --> 01:10:58.600
cult express it. It's really fun. Besides that, at work,
1178
01:10:59.039 --> 01:11:01.760
we're doing a Ruby book club and we're reading Sandy
1179
01:11:01.800 --> 01:11:12.159
Metz's oh what is it called. It's the Pewter Book,
1180
01:11:12.239 --> 01:11:19.760
The Practical Object Oriented Design in Ruby, and it's a
1181
01:11:19.840 --> 01:11:22.520
terrific book. I don't agree with everything that Sandy says
1182
01:11:22.520 --> 01:11:24.279
in it, but a lot of it I do, and
1183
01:11:24.640 --> 01:11:26.199
the rest of it every time I read it and
1184
01:11:26.279 --> 01:11:28.000
I think about it for a while ago.
1185
01:11:29.319 --> 01:11:31.920
Okay, yes, but with caveats.
1186
01:11:31.840 --> 01:11:34.760
Right, so right, anyway, if you haven't read it, it
1187
01:11:34.800 --> 01:11:37.239
was written in twenty eighteen, the second edition, but it's
1188
01:11:37.239 --> 01:11:38.119
still pretty down good.
1189
01:11:38.159 --> 01:11:39.680
So I'm going to pick that.
1190
01:11:39.960 --> 01:11:43.560
And yeah, in a couple of weeks, we're going to
1191
01:11:43.600 --> 01:11:47.399
be talking to the guys that built Ruby events dot org.
1192
01:11:48.239 --> 01:11:51.840
It was originally Ruby videos dot dev and so it
1193
01:11:51.880 --> 01:11:53.920
had all the conference videos. They were trying to collect
1194
01:11:53.920 --> 01:11:58.079
all of those. But yeah, now they're pulling in like
1195
01:11:58.119 --> 01:12:00.640
all the upcoming conferences and stuff like
that, making it
1196
01:12:00.680 --> 01:12:01.960
easy to figure all that stuff out.
1197
01:12:02.000 --> 01:12:05.800
So I'm going to pick that too, Carmine, what are
1198
01:12:05.840 --> 01:12:06.319
your picks?
1199
01:12:06.880 --> 01:12:08.439
My picks? Can I go with music?
1200
01:12:09.880 --> 01:12:10.119
Yeah?
1201
01:12:10.199 --> 01:12:12.760
Yeah, yeah, yeah, all right cool. So I run a
1202
01:12:12.840 --> 01:12:16.640
disco collective here in Berlin. It's called Floppy Disco Fun
1203
01:12:16.960 --> 01:12:19.680
and what women trying to do is to bring back
1204
01:12:19.720 --> 01:12:21.159
the funk a little bit in Berlin. As you know,
1205
01:12:21.199 --> 01:12:24.159
it's probably as you probably know, it's a very much
1206
01:12:24.199 --> 01:12:28.520
of a techno city. And I want to pick some
1207
01:12:28.760 --> 01:12:31.039
friends that have been playing with us that have been
1208
01:12:31.079 --> 01:12:37.319
absolutely fantastic. Cody Curry, he is a jazz house master.
1209
01:12:37.479 --> 01:12:41.800
We play with him actually this Saturday, and he has
1210
01:12:42.239 --> 01:12:46.399
releases on Spotify that you know, they have three million plays,
1211
01:12:46.479 --> 01:12:50.920
So definitely hitting a spot there with people. Very jazzy,
1212
01:12:51.399 --> 01:12:56.399
very groovy, very soft jazz kind of style, but in
1213
01:12:56.520 --> 01:13:01.359
house music, so really really cool stuff, really chill guy,
1214
01:13:01.399 --> 01:13:02.880
also really cool.
1215
01:13:03.520 --> 01:13:03.640
RP.
1216
01:13:03.800 --> 01:13:06.840
Brown from the same label. Toy Toonics is the label
1217
01:13:06.880 --> 01:13:09.359
by the way, it's an Italian German label by an
1218
01:13:09.359 --> 01:13:10.239
Italian German guy.
1219
01:13:11.640 --> 01:13:12.560
And r P.
1220
01:13:12.760 --> 01:13:13.720
Brown is.
1221
01:13:16.399 --> 01:13:20.239
More on the UK side of the spectrum. I want
1222
01:13:20.239 --> 01:13:24.479
to say, so he combines you know, disco with UK
1223
01:13:24.680 --> 01:13:29.000
influences because he lived there for a little bit, uh
1224
01:13:29.039 --> 01:13:33.800
and Delphonic different label. He posts on a bunch of
1225
01:13:33.840 --> 01:13:38.239
labels actually, and he has been I think one of
1226
01:13:38.279 --> 01:13:45.560
the biggest disco influencers here in Berlin and DJs and
1227
01:13:45.600 --> 01:13:47.319
he plays a lot and he's uh.
1228
01:13:47.880 --> 01:13:50.880
Just also just such a nice guy. So I want
1229
01:13:50.920 --> 01:13:51.640
to pick these three.
1230
01:13:54.079 --> 01:13:58.600
And in terms of books, uh, the Hot Wire book
1231
01:13:58.920 --> 01:14:03.399
by Joe Oh, fantastic, fantastic book.
1232
01:14:04.319 --> 01:14:06.760
Been reading this and now you're really enjoying it.
1233
01:14:08.239 --> 01:14:08.920
Which one is that?
1234
01:14:09.760 --> 01:14:15.800
I think it's called All to Our tur Ball? I sorry, Joe.
1235
01:14:16.760 --> 01:14:20.119
Why are Native for rails developers?
1236
01:14:20.159 --> 01:14:20.319
Oh?
1237
01:14:20.319 --> 01:14:26.239
Hot Wire Native Yes by Joe mast Yeah, yeah, yeah, yeah,
1238
01:14:26.239 --> 01:14:29.399
that's a terrific book on practprog. We talked to him
1239
01:14:29.399 --> 01:14:31.640
a little bit ago and we're going to have him
1240
01:14:31.640 --> 01:14:32.119
come back on.
1241
01:14:32.079 --> 01:14:34.479
Because there was just more to talk about. So I'll
1242
01:14:34.479 --> 01:14:35.720
look for that a month or so.
1243
01:14:36.439 --> 01:14:36.840
Awesome.
1244
01:14:39.159 --> 01:14:42.119
All right, good deal. Well, thanks for coming, Thank you
1245
01:14:42.159 --> 01:14:42.720
for having me.
1246
01:14:43.239 --> 01:14:44.079
It's really a pleasure.
1247
01:14:44.199 --> 01:14:46.920
This was so cool. I may not sleep for a
1248
01:14:46.920 --> 01:14:48.359
week because I'm playing with this stuff.
1249
01:14:50.319 --> 01:14:52.119
I'll feel bad and happy at the same time.
1250
01:14:53.680 --> 01:14:56.800
Yeah, all right, Well I'm gonna go ahead and push stop.
1251
01:14:56.880 --> 01:14:58.479
Stick around because we got to make sure we get
1252
01:14:58.520 --> 01:15:02.239
all your stuff uploaded. But to the listener until next time,
1253
01:15:02.319 --> 01:15:07.159
max out. M m mm hmmm.