Navigating Compatibility Challenges: Practical Steps to Implement Bzlmod in Bazel Projects

1
00:00:07,440 --> 00:00:10,820
Hello, everyone. Welcome back to Aspect Insights.
2
00:00:11,360 --> 00:00:14,740
This is episode 13, of your favorite
3
00:00:15,135 --> 00:00:18,815
livestream and podcast about Bazel and its ecosystem. My name is
4
00:00:18,815 --> 00:00:22,275
Alex. I'm the cofounder of Aspect Build. We make a monorepo
5
00:00:22,415 --> 00:00:26,175
developer platform based on Bazel. And today, I'm very excited to
6
00:00:26,175 --> 00:00:29,940
have with me a a a Googler from a long time ago,
7
00:00:29,940 --> 00:00:33,700
Mike Bland, who is, his official title is a
8
00:00:33,700 --> 00:00:37,000
platform architect of developer experience at inflow, although,
9
00:00:37,380 --> 00:00:41,005
he sometimes likes to be called an instigator instead. Today, we're
10
00:00:41,005 --> 00:00:44,685
gonna be talking about Bizzle mod. We did talk about this before on the
11
00:00:44,685 --> 00:00:47,265
show, so if you wanna scroll back to episode 7,
12
00:00:48,125 --> 00:00:51,725
Fabian Rummertime, talked about some of the implementation bits of Bizzle mod.
13
00:00:51,725 --> 00:00:54,925
And today, what we're hoping to do is get a little bit more practical and
14
00:00:54,925 --> 00:00:58,720
help you to do the adoption because Bazel 8 will be released very
15
00:00:58,720 --> 00:01:02,559
soon now, probably in the next couple weeks, and, the workspace
16
00:01:02,559 --> 00:01:05,680
support is disabled by default. So it's going to be the first time that you're
17
00:01:05,680 --> 00:01:09,060
really, prompted pretty hard to go through with this migration.
18
00:01:09,865 --> 00:01:13,485
So, with that, I'm really excited to welcome Mike to the show. Hello.
19
00:01:13,865 --> 00:01:17,625
Howdy. Nice to be here. Thanks, Alex. Oh, it's it's a thrill
20
00:01:17,625 --> 00:01:21,465
to have, someone, with your level of background and experience. I you were at
21
00:01:21,465 --> 00:01:24,745
Google before me, and so I would love to, you know, maybe we can start
22
00:01:24,745 --> 00:01:27,470
off. You tell us a little bit more. Give give yourself more of an intro
23
00:01:27,470 --> 00:01:31,229
than I'm qualified to do. Let's see. I was
24
00:01:31,229 --> 00:01:35,070
there from 2,005 to 2011. I was there
25
00:01:35,070 --> 00:01:38,530
for the beginning of what was Blaze. I didn't work on it directly,
26
00:01:38,590 --> 00:01:42,055
but, I was the first guy Johannes
27
00:01:42,055 --> 00:01:45,814
Hinkle told that he got something to build with, Blaze after pulling
28
00:01:45,814 --> 00:01:49,655
an all niner niner. So I've got that going. But, yeah, I was there till
29
00:01:49,655 --> 00:01:53,020
2011. Also helped launch, the
30
00:01:53,020 --> 00:01:56,700
internal remote caching execution platform there, integrated
31
00:01:56,700 --> 00:02:00,000
with Blaze. And I did some other stuff,
32
00:02:00,540 --> 00:02:04,380
here and there, and, I was at a company called, Cvent for
33
00:02:04,380 --> 00:02:07,605
a while here in Northern Virginia. Then I went to Apple for about 4 years,
34
00:02:07,765 --> 00:02:11,285
2018 to 2022. And I've been at,
35
00:02:11,685 --> 00:02:15,445
Enslow since March 2024. Yeah. It's awesome
36
00:02:15,445 --> 00:02:19,285
that you're back in the Bazel ecosystem. After having been there, it's for the
37
00:02:19,285 --> 00:02:22,980
first commit, the first all nighter commit. I would love I mean, there's
38
00:02:22,980 --> 00:02:25,700
so many other topics we could probably talk about, but the first I heard of
39
00:02:25,700 --> 00:02:29,540
you when I started at Google was this testing grouplet. And a
40
00:02:29,540 --> 00:02:33,300
lot of companies maybe today, most of us think that testing is automated
41
00:02:33,300 --> 00:02:36,645
testing is obvious. I don't know if engineers actually feel that way, but back then
42
00:02:36,645 --> 00:02:40,365
it was not obvious. And and convincing Googlers to do anything, you know,
43
00:02:40,365 --> 00:02:43,805
back then it was a very bottoms up kind of culture. And so it wasn't
44
00:02:43,805 --> 00:02:47,645
that, like, Larry and Sergei could say, write tests. It had to be you. Yeah.
45
00:02:47,805 --> 00:02:51,245
Well, interesting turn of phrase because, you know, one of our biggest claims to fame
46
00:02:51,245 --> 00:02:54,010
was testing on the toilet, which is what you do when you get desperate to
47
00:02:54,010 --> 00:02:57,709
get people's attention. Yeah. Do something that now seems so,
48
00:02:57,769 --> 00:03:01,610
you know, widespread industry practice. Oh, you got when I got one of these old
49
00:03:01,610 --> 00:03:05,254
toilets. Do you have one of these too? I did. But
50
00:03:05,575 --> 00:03:09,294
Cool. I lost it in a wildfire in California. But
51
00:03:09,415 --> 00:03:13,015
Oh, that's terrible. I'm so sorry. Sorry. I wasn't gonna bring that up.
52
00:03:13,015 --> 00:03:16,710
But that's, yeah. Okay. Well, I I what
53
00:03:16,710 --> 00:03:19,950
happens when I go, deviate from the script and ask ask other questions. Okay. Let
54
00:03:20,000 --> 00:03:22,550
let's get to our topic for today then. So Mhmm. I think it's probably worth
55
00:03:22,550 --> 00:03:25,990
spending a couple of minutes introducing Bizzlemod again just in case, you know, this is
56
00:03:26,150 --> 00:03:28,330
in in case some listeners don't already know.
57
00:03:33,325 --> 00:03:36,285
And I feel like the the most interesting question, especially since you're talking about the
58
00:03:36,285 --> 00:03:40,065
inception of Blaze is, like, why is Bizelmata a new thing if
59
00:03:40,445 --> 00:03:44,125
Blaze was around since 2,000, you know, 6 or
60
00:03:44,125 --> 00:03:47,849
7? Yeah. Well, I I
61
00:03:47,849 --> 00:03:51,209
figured maybe you would, know about this more than me because, you know, you were
62
00:03:51,209 --> 00:03:54,709
there longer, for sort of the introduction of,
63
00:03:55,689 --> 00:03:59,390
what would become Bazel, you know, the open source version. My understanding
64
00:03:59,450 --> 00:04:03,155
is, you know, when you have an open source product that
65
00:04:03,155 --> 00:04:06,995
needs to integrate with other, you know, repositories and things, and Google
66
00:04:06,995 --> 00:04:10,835
just didn't have that. They had the monorepo where they checked everything into
67
00:04:10,835 --> 00:04:14,435
their own third party directory. So, you know, there there was
68
00:04:14,435 --> 00:04:17,620
no, I guess, complexity
69
00:04:19,360 --> 00:04:22,340
around trying to play NICE with other
70
00:04:22,960 --> 00:04:26,560
external packages and however they were configured and the tool chains they were compatible with
71
00:04:26,560 --> 00:04:30,365
and all that kind of stuff. So I think
72
00:04:30,365 --> 00:04:34,205
that's why, you know, when it got open sourced and we wanted it or
73
00:04:34,205 --> 00:04:38,044
we. I still say we. And, you know, the Bazel project wanted it
74
00:04:38,044 --> 00:04:41,565
to be, I guess, more generally useful to a broader
75
00:04:41,565 --> 00:04:45,210
audience. They had to get something working, I presume.
76
00:04:45,210 --> 00:04:48,910
I I wasn't there for it, but that's my understanding
77
00:04:48,970 --> 00:04:51,470
is where version 1.0 Workspace,
78
00:04:52,650 --> 00:04:56,330
came into the picture. And then, of course, the second system effect sets in, and
79
00:04:56,330 --> 00:05:00,025
that's by the way, I've I've always been pronouncing it Bazelmod. That's
80
00:05:00,025 --> 00:05:03,544
how I've heard it. Just so I'm not trying to, like, enforce the
81
00:05:03,544 --> 00:05:06,925
pronunciation. But if I say it that way, I haven't heard Bazelmod
82
00:05:06,985 --> 00:05:10,685
yet. Okay. Well, this is a a great reason for us to have a
83
00:05:10,860 --> 00:05:14,060
a show with some audio and not just typing because we can we can find
84
00:05:14,060 --> 00:05:17,900
out different pronunciations. And, yeah, some people have asked me, is it pronounced Basil
85
00:05:17,900 --> 00:05:20,939
or Basil? And I just try to say, like, you you pronounce it, like, we
86
00:05:20,939 --> 00:05:24,675
understand what you mean. Mhmm. It's you
87
00:05:24,675 --> 00:05:28,515
you mentioned, vendoring into Google's monorepo and,
88
00:05:28,515 --> 00:05:32,195
yeah, that's the biggest, like, wow, I think, for everyone. And you mentioned tool
89
00:05:32,195 --> 00:05:35,635
chains also. Like, it's not just that they check-in their node modules and their Python
90
00:05:35,635 --> 00:05:39,450
packages and whatever they use from from from, you know, package managers. They
91
00:05:39,450 --> 00:05:43,130
check-in, like, the GCC source code. Right? Like, it's the whole thing. It's
92
00:05:43,130 --> 00:05:46,890
insane. Yeah. And I think, you know, the way
93
00:05:46,890 --> 00:05:50,730
Google does monorepo, sounds terrifying to people because it is pretty
94
00:05:50,730 --> 00:05:54,545
terrifying, I think. Uh-huh. Unless you have enough engine you know, the world's smartest
95
00:05:54,545 --> 00:05:58,305
engineers or whatever, Googlers claim to be. And
96
00:05:58,305 --> 00:06:01,105
and so yeah. This is this is not the system everybody else wanted and what
97
00:06:01,105 --> 00:06:04,860
they want is hopefully this new Bazel mod.
98
00:06:05,160 --> 00:06:08,840
So yeah. Let's let's jump into, like, what's good about it, why they would
99
00:06:08,840 --> 00:06:12,539
want it, like, what makes it easier to use? Mhmm.
100
00:06:13,479 --> 00:06:17,319
Yeah. So and by the way, I'll preface this by saying I hadn't really
101
00:06:17,319 --> 00:06:21,155
done that much Bazel until, from the time I left, you know, Google
102
00:06:21,155 --> 00:06:24,915
in 2011 till I joined inflow. I did play with it a little
103
00:06:24,915 --> 00:06:28,755
bit at at Apple, you know, because Simon Stewart was there and, you know, he
104
00:06:28,835 --> 00:06:32,580
you know, I played with a little bit with him. But the real deep
105
00:06:32,580 --> 00:06:36,180
dive I've done has been since, you know, March, April of this
106
00:06:36,180 --> 00:06:39,800
year. But my understanding is,
107
00:06:41,220 --> 00:06:44,900
you know, workspace, I think, was a valiant attempt to get
108
00:06:44,900 --> 00:06:48,165
something out the door that started working. And there are still some workspace
109
00:06:49,745 --> 00:06:53,425
abstractions that remain in Bazelmod, like
110
00:06:53,425 --> 00:06:57,265
repository rules and HTTP archive. Like, those aren't going
111
00:06:57,265 --> 00:07:00,885
away. They're still part of Bazelmod. And with the right design,
112
00:07:01,810 --> 00:07:05,490
you can actually share some of these implementations. It's not like, you know, if you
113
00:07:05,490 --> 00:07:09,330
maintain a rule set or anything like that. You have to throw everything away
114
00:07:09,330 --> 00:07:13,030
and start from scratch. With Basel mod, you just might have some compatibility,
115
00:07:13,730 --> 00:07:17,544
work to do. Right. And I think it made that easier. Right? Because, like, HTTP
116
00:07:17,544 --> 00:07:21,145
archive, you can still just use that, and it's, like, added this
117
00:07:21,145 --> 00:07:24,925
capability to make it easier to reuse the workspace file, concepts.
118
00:07:25,305 --> 00:07:29,070
Yeah. But I think what you know, and it's probably familiar
119
00:07:29,070 --> 00:07:32,750
to everyone that part of what made Workspace a problem
120
00:07:32,750 --> 00:07:36,290
after a while is that it just executed everything sequentially.
121
00:07:36,430 --> 00:07:40,030
You can load things at any point. You can
122
00:07:40,030 --> 00:07:43,875
load something that something later on in the file depends upon. And
123
00:07:43,955 --> 00:07:47,735
and so I think, with that, you got into
124
00:07:48,195 --> 00:07:51,475
ordering constraints. You know, you had to be careful of the order in which you
125
00:07:51,475 --> 00:07:55,315
you did things. And it was still a little bit spooky action at
126
00:07:55,315 --> 00:07:58,835
a distance. All the repositories were imported into this global
127
00:07:58,835 --> 00:08:02,650
namespace. So, you know, things could get stomped on, workspace wouldn't tell you
128
00:08:02,650 --> 00:08:06,250
about it. There was just a lot of ambiguity that people
129
00:08:06,250 --> 00:08:09,930
discovered in practice. Yeah. And And then yeah. The rule set
130
00:08:09,930 --> 00:08:13,625
authors, that had an implication also because rules docker or rules
131
00:08:13,625 --> 00:08:17,465
proto, for example, have dependencies on a bunch of different languages. And so with the
132
00:08:17,465 --> 00:08:21,305
ruleset author there, you you're trying to be careful not to break people, but
133
00:08:21,305 --> 00:08:25,065
anytime you bump a dependency version, if somebody's listed you first in their workspace
134
00:08:25,065 --> 00:08:28,590
file, you could break them. Yeah. Oh, and I'm gonna
135
00:08:28,590 --> 00:08:31,890
circle back to why that problem still exists to some degree
136
00:08:32,190 --> 00:08:35,870
in Bazelmod. But in
137
00:08:35,870 --> 00:08:39,375
general, like, the more I started learning about
138
00:08:39,455 --> 00:08:43,075
Bazelmod and its, design and constraints,
139
00:08:43,775 --> 00:08:46,915
the more I was like, oh, this is this is really pretty cool. And
140
00:08:47,615 --> 00:08:51,215
a big thing for me and and big part of, why I was
141
00:08:51,215 --> 00:08:54,940
compelled to help launch the, you know, the forge, the remote
142
00:08:54,940 --> 00:08:58,320
caching execution platform back in the day at the old company,
143
00:08:59,260 --> 00:09:03,100
is in order to make use of it, you had to clean your stuff
144
00:09:03,100 --> 00:09:06,720
up. Right? And so this is sort of a case,
145
00:09:06,845 --> 00:09:10,065
a similar case that I see where Basel mod has many advantages.
146
00:09:11,005 --> 00:09:14,205
But you might have some work to do before you can even turn it on
147
00:09:14,205 --> 00:09:17,965
and start using it. But once you've made those changes, you are
148
00:09:17,965 --> 00:09:21,690
definitely in a much better state and it's gonna be more difficult to sort
149
00:09:21,690 --> 00:09:24,030
of backslide out of that.
150
00:09:25,530 --> 00:09:29,130
So that's part of, you know, why I'm particularly, you
151
00:09:29,130 --> 00:09:32,110
know, engaged by the the work of Bazelmod migration.
152
00:09:33,295 --> 00:09:36,895
Yeah. And it's not just that it helps you with your
153
00:09:36,895 --> 00:09:40,495
dependencies, but also there's, like, tool chain registration that can be tucked into the
154
00:09:40,495 --> 00:09:44,095
module file for one of the libraries you use. So, you know, for
155
00:09:44,095 --> 00:09:47,315
example, the bazellib foundation library
156
00:09:48,580 --> 00:09:52,020
has, like, some some helper utilities that are fetched as tool chains. All that
157
00:09:52,020 --> 00:09:55,540
registration can also be, done conveniently for you and is
158
00:09:55,700 --> 00:09:59,380
participates in this in this Bazel mod thing. Yeah. And and the
159
00:09:59,380 --> 00:10:02,740
burden does, you know, land on the rules authors to do that. And, you know,
160
00:10:02,740 --> 00:10:05,855
we'll talk about a little bit later. That's kind of what I've been doing with
161
00:10:05,855 --> 00:10:09,455
rule Scala is, doing some of that, tool
162
00:10:09,455 --> 00:10:13,214
chain, abstraction. Because I think, you
163
00:10:13,214 --> 00:10:16,920
know, something else about, workspace is
164
00:10:16,920 --> 00:10:20,519
the burden was on the consumer. Like, oh, here's some macros. Make sure you call
165
00:10:20,519 --> 00:10:24,120
these to configure our, you know,
166
00:10:24,120 --> 00:10:27,879
our, repo that you're importing. Whereas
167
00:10:27,879 --> 00:10:31,485
with bazelmod, the, you know, the module dot
168
00:10:31,485 --> 00:10:34,845
bazel from that module, that repo can do a lot of the
169
00:10:34,845 --> 00:10:38,065
configuration for you and then expose just some basic,
170
00:10:39,485 --> 00:10:43,165
API for, you know, configuring some key parameters. And so I think
171
00:10:43,165 --> 00:10:46,690
that's also a really interesting
172
00:10:46,750 --> 00:10:50,130
development. It it really does become easier for the user.
173
00:10:51,150 --> 00:10:54,830
And and because the, the model under Bazel
174
00:10:54,830 --> 00:10:58,450
mod is to, like, collect all this metadata from the entire entire entire
175
00:10:58,590 --> 00:11:02,315
dependency graph to say who needs, you know, what configured which
176
00:11:02,315 --> 00:11:05,695
way. And then it will execute module extensions
177
00:11:05,995 --> 00:11:09,835
that will instantiate repositories in a separate namespace that you
178
00:11:09,835 --> 00:11:13,035
don't need to know about, mulp pollute your other modules or the rest of your
179
00:11:13,035 --> 00:11:16,180
project. I'm, you know, it's just a really cool,
180
00:11:16,880 --> 00:11:20,320
design, I think, and interesting to to see it
181
00:11:20,320 --> 00:11:24,100
happen. Yep. Yep. I agree. So
182
00:11:24,880 --> 00:11:28,640
maybe let's talk about another the next topic, which
183
00:11:28,640 --> 00:11:32,375
is how how Bazel mod interacts with package
184
00:11:32,675 --> 00:11:36,515
managers. Of course, it has its own it's you can
185
00:11:36,515 --> 00:11:40,035
consider it to be a package manager for Starlark modules in a sense. And so
186
00:11:40,035 --> 00:11:43,415
it has its own things that package managers have, like a registry,
187
00:11:44,090 --> 00:11:47,610
a public registry where everyone can contribute these. You can also make your own private
188
00:11:47,610 --> 00:11:51,130
registries. So let's just remind
189
00:11:51,130 --> 00:11:54,970
everybody how that interacts with the language specific package managers they had already. Right?
190
00:11:54,970 --> 00:11:58,365
Because the goal is, for the most part, we're not trying to replace those. Right.
191
00:11:58,365 --> 00:12:01,725
As a matter of fact, it's a case of, you know, an extra level of
192
00:12:01,725 --> 00:12:05,325
abstraction solving all problems. Right? Like, so if you're
193
00:12:05,325 --> 00:12:08,385
configuring a build, you're, you know, configuring,
194
00:12:09,725 --> 00:12:13,290
modules with Starlark code. Under the hood, the rules
195
00:12:13,290 --> 00:12:17,050
authors providing those modules have done the work of, you
196
00:12:17,050 --> 00:12:20,810
know, trying to invoke NPM, mypy, you
197
00:12:20,810 --> 00:12:24,570
know, go get whatever. That,
198
00:12:24,570 --> 00:12:28,214
you know, those rules modules will do the translations to the other
199
00:12:28,214 --> 00:12:30,795
package managers. And there's some complexity there.
200
00:12:32,295 --> 00:12:35,895
But at least, you know, you've you know, the way the
201
00:12:35,895 --> 00:12:39,595
ecosystem is supposed to work is, like, trust the people providing
202
00:12:39,895 --> 00:12:43,579
the Starlark abstraction for, like, the Python
203
00:12:43,579 --> 00:12:47,420
registry to get it right. And then you as a
204
00:12:47,420 --> 00:12:51,180
consumer do not have to fiddle with, you know, all the
205
00:12:51,180 --> 00:12:54,699
requirements files and different configuration files and things like
206
00:12:54,699 --> 00:12:58,225
that. Yeah. Alright.
207
00:12:58,285 --> 00:13:02,125
So, I mean, maybe what what are some of the
208
00:13:02,125 --> 00:13:05,964
things about the Bazel central registry that are useful for people to know,
209
00:13:05,964 --> 00:13:09,060
like, how how it works or, like, how you interact with it?
210
00:13:09,779 --> 00:13:12,900
Yeah. I mean, I think one of the neat things is there's really not a
211
00:13:12,900 --> 00:13:16,740
lot to it. Like, you go to registry dot bazel, dot build, I believe
212
00:13:16,740 --> 00:13:20,580
it is. And, you search for your module and say,
213
00:13:20,580 --> 00:13:23,955
what's what version is out there, and it has a little snippet for you to
214
00:13:23,955 --> 00:13:26,595
copy and paste the Bazel depth into your Bazel mod. And it also has a
215
00:13:26,595 --> 00:13:30,435
convenient link to the repo if you wanna, you know, go further into
216
00:13:30,435 --> 00:13:34,115
the documentation for that package. It's pretty lightweight. And I also
217
00:13:34,115 --> 00:13:37,939
think it's neat that it's essentially if I understand it,
218
00:13:37,939 --> 00:13:41,699
like, just a static, you know, JSON file or collection of
219
00:13:41,699 --> 00:13:45,540
JSON files. And the spec is in, you know, bazel dot build. The
220
00:13:45,540 --> 00:13:49,365
documentation site tells you what the spec is. So if you wanted
221
00:13:49,365 --> 00:13:53,145
to do your own version, if you needed to be, you know, if you
222
00:13:53,205 --> 00:13:56,805
had increased levels of, you know, scrutiny around security
223
00:13:56,805 --> 00:13:59,785
or you're in an air gapped environment or something like that,
224
00:14:00,965 --> 00:14:04,649
you can still use the same essential mechanism just
225
00:14:04,649 --> 00:14:08,329
pointed at a different implementation. I think that's rather powerful as
226
00:14:08,329 --> 00:14:12,089
well. Yep. Yeah. The simplest one is if you make a fork of that GitHub
227
00:14:12,089 --> 00:14:15,930
repo, you can use the dash dash registry flag on the command line
228
00:14:15,930 --> 00:14:19,755
to point to, like, raw.githubusercontent.com/ your
229
00:14:19,755 --> 00:14:23,435
fork, essentially. And GitHub will serve an HTTP, you
230
00:14:23,435 --> 00:14:27,195
know, site that has that structure, and therefore, the Basel
231
00:14:27,195 --> 00:14:30,015
client knows what to do. Mhmm. Yeah.
232
00:14:32,150 --> 00:14:34,570
Yeah. And I know we there there's a couple of things
233
00:14:35,590 --> 00:14:39,350
ongoing worth mentioning. I forgot one other thing. One other nice thing
234
00:14:39,350 --> 00:14:43,190
is, like, when you've got a proper Bazel dependency, you generally don't
235
00:14:43,190 --> 00:14:46,805
have to me go to the repo and get the Stanza with the HTTP archive
236
00:14:46,805 --> 00:14:50,324
and the and the Shawson for this version and then the macros I have to,
237
00:14:50,324 --> 00:14:54,165
you know, order right in my workspace. It's just like, no. Here's the Bazel dep.
238
00:14:54,165 --> 00:14:57,845
Here's the version. Off to the races. So I I really
239
00:14:57,845 --> 00:15:01,240
appreciate, the API they put in place for that. And, of course, with the module
240
00:15:01,240 --> 00:15:05,080
extensions, you do have capability to exert control
241
00:15:05,080 --> 00:15:08,920
over the configuration, but it's not like it's imposed upon you in a
242
00:15:08,920 --> 00:15:12,300
very fragile way like, Workspace unfortunately
243
00:15:12,440 --> 00:15:16,214
had. Yeah. I think it's you mentioned the, like, the SHA 256.
244
00:15:16,514 --> 00:15:19,875
So obviously, something has changed that you no longer have to put that in your
245
00:15:19,875 --> 00:15:23,714
file, and we don't want anybody to worry. There are still guarantees. So one
246
00:15:23,714 --> 00:15:27,430
of the guarantees is that, Bazel will always, unless you
247
00:15:27,430 --> 00:15:30,790
pacify telling it not to, it will create a lock file for everything that it
248
00:15:30,790 --> 00:15:33,990
fetched, which will be sort of trust on first use as this notion of, like,
249
00:15:33,990 --> 00:15:36,550
it'll download it and calculate the shot and then it will make sure that it
250
00:15:36,550 --> 00:15:40,230
stays the same so that nobody no attacker can come and change the packages
251
00:15:40,230 --> 00:15:43,635
after the fact. Also, the Bazel central registry
252
00:15:43,774 --> 00:15:47,214
itself, at least, you know, the primary fork that the that the Bazel team and
253
00:15:47,214 --> 00:15:50,915
some of us in the community are maintaining has some immutability guarantees.
254
00:15:51,295 --> 00:15:54,575
So you know that when something is published, if you point to rules go version
255
00:15:54,575 --> 00:15:58,200
0 45, like, that's not gonna change. And then there's some
256
00:15:58,200 --> 00:16:01,660
work going on. I actually have a meeting, later this morning about,
257
00:16:02,280 --> 00:16:06,060
adding provenance attestations so that when you download
258
00:16:06,280 --> 00:16:09,925
prebuilt, archives of the rule sets that are
259
00:16:09,925 --> 00:16:13,765
pointed to by the Bazel central registry, you're also guaranteed that those were built from
260
00:16:13,765 --> 00:16:17,605
the sources that it says they were. Right. So there's yeah. So
261
00:16:17,605 --> 00:16:20,805
it's a pretty good guarantee that Bazel gave you before and, like, that guarantee is
262
00:16:20,805 --> 00:16:24,470
still there and getting better, I think, is the summary. Yeah. And it's
263
00:16:24,470 --> 00:16:27,770
less work. I mean, less work as a consumer.
264
00:16:29,590 --> 00:16:33,430
Yeah. Alright. Well, let's I I think the the reason that I knew you were
265
00:16:33,430 --> 00:16:36,630
working on this is that you wrote some great blog posts that, we mentioned actually
266
00:16:36,630 --> 00:16:40,285
before on the show, I think. So maybe tell us a little bit about, like,
267
00:16:40,285 --> 00:16:44,045
how you got into working on Bazelmod. You mentioned that, you know, since you
268
00:16:44,045 --> 00:16:46,925
started at Edgeflow, you got the chance to come back into Bazel and learn stuff
269
00:16:46,925 --> 00:16:50,285
about it. So I assume you were doing some internal migrations and and,
270
00:16:50,285 --> 00:16:53,940
like, yeah. I mean, how did you come to be
271
00:16:53,940 --> 00:16:57,779
the the expert on this? Accidentally as
272
00:16:58,180 --> 00:17:01,800
yeah. My my whole career can be described as a series
273
00:17:01,940 --> 00:17:05,380
of accidents. So, yeah, I was I was new at
274
00:17:05,380 --> 00:17:09,135
inflow, and I really like I
275
00:17:09,135 --> 00:17:12,895
said, I hadn't done much with with Bazel at all. You know,
276
00:17:12,895 --> 00:17:16,515
I'd left Google in 2011 and only tinkered with it here and there.
277
00:17:17,855 --> 00:17:21,560
But along the way, I I somehow was reading up on it and
278
00:17:21,560 --> 00:17:24,920
saw, oh, there's this new Bazel mod thing that's gonna replace
279
00:17:24,920 --> 00:17:28,760
workspace. I'm like, oh, that's interesting. Do we have that? You know, because
280
00:17:28,760 --> 00:17:32,300
we have a monorepo. I, me and a couple of colleagues,
281
00:17:32,520 --> 00:17:36,115
Janek and Yuval, just published a repo. It's not Bazel mod
282
00:17:36,115 --> 00:17:39,795
related, but, in that or published a repo. Published a
283
00:17:39,795 --> 00:17:42,375
blog post. God. My head is still in Bazel mod.
284
00:17:43,475 --> 00:17:46,835
About, you know, evaluating different, remote
285
00:17:46,835 --> 00:17:50,580
execution platforms and things. And I've got a script that
286
00:17:50,580 --> 00:17:54,100
says, oh, you know, we ran this and this kinda shows you the scope of
287
00:17:54,100 --> 00:17:57,860
our repo. It has all these different languages and things like that in
288
00:17:57,860 --> 00:18:01,695
it. So I was like, well, it'll take
289
00:18:01,695 --> 00:18:05,075
some time for me to get up to speed on the system and, you know,
290
00:18:05,294 --> 00:18:08,835
do any work of, critical path
291
00:18:08,895 --> 00:18:12,335
substance. But, hey. Nobody's working on this Basel mod
292
00:18:12,335 --> 00:18:15,300
thing. And so I just kinda threw it out in the Slack channel. It's like,
293
00:18:15,380 --> 00:18:19,220
you mind if I take a stab at this? And nobody, you know,
294
00:18:19,220 --> 00:18:22,760
screamed at me not to touch it. So I took that as implicit
295
00:18:22,820 --> 00:18:26,580
permission to go forth. And I did. Man, it was
296
00:18:26,580 --> 00:18:30,174
an adventure. It was a fun event. It's the kind of
297
00:18:30,174 --> 00:18:33,395
adventure I enjoy. I like, you know, taking,
298
00:18:34,495 --> 00:18:38,095
things and figuring out why they're not working and make them better. Especially, sort of
299
00:18:38,095 --> 00:18:41,875
like what I was saying earlier with, you know, when I helped launch remote execution
300
00:18:43,649 --> 00:18:47,250
at Google, the process was, well, these
301
00:18:47,409 --> 00:18:51,169
a lot of these problems are basically where
302
00:18:51,169 --> 00:18:54,850
our build is a little fast and loose with the details. Right? And
303
00:18:54,850 --> 00:18:58,524
we need to, you know, improve how we declare run
304
00:18:58,524 --> 00:19:02,044
files and, how we pass in, you know,
305
00:19:02,044 --> 00:19:05,885
certain target paths and things of that nature. And so it was a
306
00:19:05,885 --> 00:19:09,565
lot of little things that, you know, workspace and
307
00:19:09,565 --> 00:19:13,400
that model didn't enforce. But under Basel mod, you
308
00:19:13,400 --> 00:19:16,760
know, it's sort of like well, you get these guarantees of
309
00:19:16,760 --> 00:19:20,360
correctness and, you know, better performance down the line, and you can reason about
310
00:19:20,360 --> 00:19:23,880
it. But to do that, to take advantage of the new hotness, you
311
00:19:23,880 --> 00:19:27,615
gotta clean up the old and busted stuff. And
312
00:19:27,615 --> 00:19:31,155
so it just turned into, like, this incredible journey of discovery.
313
00:19:32,895 --> 00:19:36,575
And, also, I was like, hey. I'm learning and
314
00:19:36,575 --> 00:19:40,400
fixing all these problems, and these are problems that a bunch of other people at
315
00:19:40,400 --> 00:19:44,240
inflow, many of whom used to work on Bazel or otherwise heavily involved in
316
00:19:44,240 --> 00:19:48,000
the ecosystem, they didn't know about this stuff, or they hadn't already
317
00:19:48,000 --> 00:19:51,299
done anything about it. And I'm like, well, if the migration
318
00:19:51,520 --> 00:19:55,059
is taking this much work, which I don't mind doing, but
319
00:19:55,685 --> 00:19:59,365
how how how are our customers and our prospective customers and
320
00:19:59,365 --> 00:20:03,045
others in the community when they're being told to migrate and
321
00:20:03,045 --> 00:20:06,725
they're hitting all these problems? You know, maybe they don't have the luxury
322
00:20:06,725 --> 00:20:10,485
of, time and familiarity and, or just
323
00:20:10,485 --> 00:20:14,220
pure, you know, intrinsic tenacity when it comes
324
00:20:14,220 --> 00:20:17,020
to these kind of problems to sit down and solve them. And so I was
325
00:20:17,020 --> 00:20:20,320
I was thinking, like, wow. You know, these people are gonna be
326
00:20:20,539 --> 00:20:24,005
locked on, like, Bazel 7 or maybe 8 forever
327
00:20:24,005 --> 00:20:27,445
while we're on Basel 15, you know, because, you know, the
328
00:20:27,445 --> 00:20:31,065
migration for them, was a bit
329
00:20:31,365 --> 00:20:35,065
challenging. So I I figured, you know what? Let me,
330
00:20:36,085 --> 00:20:39,740
let me try to take what I've done and share what I know, and hopefully,
331
00:20:39,740 --> 00:20:42,880
it'll prove useful to the to the broader community.
332
00:20:43,500 --> 00:20:46,160
Yeah. That's awesome. I appreciate you.
333
00:20:48,380 --> 00:20:51,505
Oh, my one of my managers at Google, John Micco, like to say, you know,
334
00:20:51,505 --> 00:20:54,705
you just show an engineer running water, and they'll just naturally start building a dam.
335
00:20:54,705 --> 00:20:58,245
And so, like, you you know, you you saw something that needed to happen,
336
00:20:59,025 --> 00:21:02,405
and and and jumped in to do it. I I agree that, like, there's
337
00:21:03,600 --> 00:21:07,440
this thing about iPhone versus Android where one of the big advantages of
338
00:21:07,440 --> 00:21:10,640
iPhone is that they're able to sort of lockstep move the ecosystem forward. And with
339
00:21:10,640 --> 00:21:13,860
Android, you have all these old phones and carriers who have old versions and, like,
340
00:21:14,480 --> 00:21:18,275
it's a little similar that for for Bazel and for our ecosystem,
341
00:21:18,275 --> 00:21:21,075
if we end up with users who are stuck in Bazel 7 forever, you know,
342
00:21:21,075 --> 00:21:24,915
Bazel 8 with the with the dash dash enable workspace flag, we're gonna
343
00:21:24,915 --> 00:21:28,035
be in a fragmented situation where we have to support users on those old things,
344
00:21:28,035 --> 00:21:31,200
and now they're using versions of rules that are 2 years old, and everything has
345
00:21:31,200 --> 00:21:34,080
changed, and we'll we'll never make sense of it. The cognitive load will be too
346
00:21:34,080 --> 00:21:37,540
high. So as an ecosystem, we have to get everybody past this migration.
347
00:21:38,080 --> 00:21:41,760
Right. Yeah. And I and, you know, the
348
00:21:41,760 --> 00:21:45,585
company was like, hey. That's that's valuable. Do it. And so I have
349
00:21:45,585 --> 00:21:48,705
the good fortune being able to spend a good chunk of time on it. Yeah.
350
00:21:48,705 --> 00:21:52,305
That's that's awesome. It's a great contribution. You know, our
351
00:21:52,305 --> 00:21:56,065
companies have to figure out ways to to, justify investing in things
352
00:21:56,065 --> 00:21:59,630
like that. And, yeah, moving the ecosystem forward is fantastic. Mhmm.
353
00:22:00,890 --> 00:22:04,730
So, I mean, I I guess, the
354
00:22:04,730 --> 00:22:08,250
the blog post have been great. I I probably, many people listening have already read
355
00:22:08,250 --> 00:22:12,090
them. You should read them if you haven't. Do you have more coming there?
356
00:22:12,090 --> 00:22:15,495
I mean, is I do. Okay.
357
00:22:16,835 --> 00:22:20,675
Alright. Yeah. So so, I guess I'll talk a little bit
358
00:22:20,675 --> 00:22:24,515
about sort of my thought process around this series because I let's,
359
00:22:24,835 --> 00:22:27,395
I need I should have it up right here. When was the last time I
360
00:22:27,395 --> 00:22:31,030
published The last one of those came out September 6th.
361
00:22:33,250 --> 00:22:36,310
And so those first few blog posts were
362
00:22:39,650 --> 00:22:43,065
hidden knowledge or other little details that are useful to
363
00:22:43,065 --> 00:22:46,905
know, that you can sort of take into your own hands right away
364
00:22:46,905 --> 00:22:50,585
in your own project and, fix little problems, you
365
00:22:50,585 --> 00:22:54,185
know, things like, oh, the main
366
00:22:54,185 --> 00:22:57,960
repository label is now double at sign instead of single
367
00:22:57,960 --> 00:23:00,380
at sign. You know, little things like that or,
368
00:23:03,240 --> 00:23:07,000
you know, maybe you're depending on a repository name and maybe you see
369
00:23:07,000 --> 00:23:09,660
that you don't have to and you sort of take that out of the code
370
00:23:09,880 --> 00:23:13,654
in some way. And then it moved into, like, run
371
00:23:13,654 --> 00:23:17,495
files libraries and using them right and knowing in in
372
00:23:17,495 --> 00:23:20,934
the Python case which one to use because there's one that ships with
373
00:23:20,934 --> 00:23:24,700
Bazel still that, will break under Bazel mod,
374
00:23:24,700 --> 00:23:27,820
but you should be using the one that rules Python. So I decided, let's do
375
00:23:27,820 --> 00:23:31,500
an episode just on run files libraries. And I tried to make that a
376
00:23:31,500 --> 00:23:35,260
fairly complete technical doc that and that's the other thing. They're blog posts
377
00:23:35,260 --> 00:23:37,865
that they're really I nobody's gonna sit down and nobody's gonna sit down and read
378
00:23:37,865 --> 00:23:41,625
those beginning to end for, you know, pleasure. Right? Like, they're Oh, no. It's it's
379
00:23:41,625 --> 00:23:44,985
more like a knowledge base. Yeah. It makes it because the blogging is an easier,
380
00:23:44,985 --> 00:23:48,184
less committed format where you can put part of it and, you know, the rest
381
00:23:48,184 --> 00:23:51,530
of it, you'll get to it. Yeah. For run files, of course, the thing there
382
00:23:51,530 --> 00:23:55,370
is that you are now forced to have your repository named underscore main instead
383
00:23:55,370 --> 00:23:57,770
of whatever it used to be named. And so that broke some of your hard
384
00:23:57,770 --> 00:24:01,610
coded paths. Right. Those before. So we're yeah. The run files helper
385
00:24:01,610 --> 00:24:04,750
gives you an abstraction to avoid having to know what the thing is named.
386
00:24:05,395 --> 00:24:09,075
Yeah. And I it's funny. In that case, I've done
387
00:24:09,075 --> 00:24:11,715
all this work. I was like, oh, we have to change all these run file
388
00:24:11,715 --> 00:24:15,475
paths to underscore main. And then later, I found I I dug up
389
00:24:15,635 --> 00:24:19,475
maybe I should put this in the notes too. I dug up a a GitHub
390
00:24:19,475 --> 00:24:23,280
issue, where I found Tony Audo, who I still have to
391
00:24:23,280 --> 00:24:26,160
reach out to him. He was one of the testing group at New York members
392
00:24:26,160 --> 00:24:29,920
back in the day. And now he, you know, he maintains rules package
393
00:24:29,920 --> 00:24:33,300
and things like that. And I saw him in a thread with,
394
00:24:34,000 --> 00:24:37,764
Richard, Richard l. I'm I'm afraid
395
00:24:37,764 --> 00:24:41,365
I won't be able to pronounce his, Lavoisier, if I'm in the
396
00:24:41,365 --> 00:24:44,804
ballpark. And in that
397
00:24:44,804 --> 00:24:48,485
conversation, it kinda came up that, oh, no. You should be using the rules Python
398
00:24:48,485 --> 00:24:52,289
run files library. And I'm like, what? And sure enough, I go
399
00:24:52,289 --> 00:24:55,830
back and use the right run files library, and I undid all my
400
00:24:56,289 --> 00:24:59,570
underscore main path updates. So stuff like that, I want to get out there. Like,
401
00:24:59,570 --> 00:25:03,250
so people don't go down a particular path where
402
00:25:03,250 --> 00:25:06,825
they're not wrong about, yeah, oh, you know what the problem is and you're
403
00:25:06,825 --> 00:25:10,265
applying a solution, but really the better solution is this other
404
00:25:10,265 --> 00:25:13,865
thing. Yep. Same thing with and speaking of Tony, I know he maintains rules
405
00:25:13,865 --> 00:25:17,545
package, and, that was the 3rd blog post. And
406
00:25:17,545 --> 00:25:20,125
that was born of the fact that everything was building
407
00:25:21,170 --> 00:25:24,770
fine with Bazelmod, but then we found out that some of
408
00:25:24,770 --> 00:25:28,530
the some of the packages we we repackaged and we're deploying on our systems were
409
00:25:28,530 --> 00:25:31,830
silently broken. And it had to do
410
00:25:32,049 --> 00:25:35,595
with the way, you know, we had our build interacting with rules
411
00:25:35,595 --> 00:25:39,295
package. So I was like, well, we can't be the only ones. And so,
412
00:25:40,075 --> 00:25:43,675
I wanted to share, you know, what we call holding it right. I got that
413
00:25:43,675 --> 00:25:45,375
term from my colleague, Corbin,
414
00:25:47,195 --> 00:25:51,000
where, you know, rules package is perfectly compatible
415
00:25:51,380 --> 00:25:55,060
if you use it this way. You know, if you're relying on what are
416
00:25:55,060 --> 00:25:58,740
actually technically deprecated, features of that package, then that's
417
00:25:58,740 --> 00:25:59,960
where you get into trouble.
418
00:26:02,275 --> 00:26:06,035
Yeah. Alright. Well, what about so there's there's a transitive nature of this. Right?
419
00:26:06,035 --> 00:26:09,655
So there's also sort of you fix your project now,
420
00:26:10,835 --> 00:26:14,675
but other people's projects are not fixed. You know, I mean, a year ago, Bazel
421
00:26:14,675 --> 00:26:18,440
not existed, but it was too early to use largely because the
422
00:26:18,440 --> 00:26:21,900
ecosystem wasn't there. So what do you do about sort of patching versus contributing,
423
00:26:22,840 --> 00:26:26,280
when the problem is not in your code? Oh, yeah. And
424
00:26:26,280 --> 00:26:29,960
that's, yeah. That's what that's what I'm in the
425
00:26:29,960 --> 00:26:33,020
middle of now. So couple things. I should say that, like,
426
00:26:33,615 --> 00:26:37,375
the vast, vast majority of our dependencies were lift
427
00:26:37,375 --> 00:26:41,215
and shift. Like, they were just h t p archive calls or, you know,
428
00:26:41,215 --> 00:26:45,055
they were already, you know, properly designed basal
429
00:26:45,055 --> 00:26:48,740
modules out there and we could just do it. But, you know, 80 20
430
00:26:48,740 --> 00:26:52,179
rules that last 20 or however much it is where you have to dig in
431
00:26:52,179 --> 00:26:54,120
and do the work. And,
432
00:26:56,260 --> 00:26:59,745
and so that's gonna be the subject of the next blog post.
433
00:26:59,825 --> 00:27:03,105
Like, well, the problem is in your dependency. What do you do? And I'm gonna
434
00:27:03,105 --> 00:27:06,865
talk about, 1, writing your own module extension to wrap a
435
00:27:06,865 --> 00:27:10,625
package, and 2, get into situations where you
436
00:27:10,625 --> 00:27:13,425
might have to write a patch and how you would go about doing that and
437
00:27:13,425 --> 00:27:16,590
the kind of problems, examples of the kind of problems you might hit.
438
00:27:17,290 --> 00:27:20,350
And the reason it's been so delayed is,
439
00:27:20,970 --> 00:27:24,110
I've been making a trade off up to up to this point of,
440
00:27:25,210 --> 00:27:29,025
I actually got our projects to work by doing
441
00:27:29,025 --> 00:27:32,785
these things and maintaining patches and whatnot. But there
442
00:27:32,785 --> 00:27:36,385
was always a part of me that was like, well, could I
443
00:27:36,385 --> 00:27:40,065
push this upstream? And, in the
444
00:27:40,065 --> 00:27:43,679
case of RulesScala, there is a discussion on it in Slack one day at
445
00:27:43,679 --> 00:27:47,520
inflow. And again, my colleague Jay, we
446
00:27:47,520 --> 00:27:51,140
got in a copy about, you know, this.
447
00:27:51,600 --> 00:27:54,980
And I said, well, I got ruleskala to work with the inflow
448
00:27:55,039 --> 00:27:58,805
example GitHub project. And, you know, maybe I
449
00:27:58,805 --> 00:28:02,565
could contribute something. And, of course, he appealed to my vanity and said, you might
450
00:28:02,565 --> 00:28:05,205
be the best person in the position to do it. And I'm like, great. That
451
00:28:05,205 --> 00:28:07,785
nerd sniped into this. And so,
452
00:28:09,820 --> 00:28:13,660
I started, you know, looking at the threads. I I saw attempts other
453
00:28:13,660 --> 00:28:16,460
people had made, and then I sort of tried to politely jump in and say,
454
00:28:16,460 --> 00:28:20,220
would it be my mind if I took a shot at it? And, I
455
00:28:20,220 --> 00:28:23,664
also had been experimenting with it, and I got it working
456
00:28:23,885 --> 00:28:27,645
in sort of like this one branch. But, since
457
00:28:27,645 --> 00:28:31,485
then, you know, it's been well, one, it's been a wonderful process. It's
458
00:28:31,485 --> 00:28:34,865
been how open source is supposed to work, you know, because
459
00:28:35,325 --> 00:28:39,100
I'm I basically got a, you know, proof of concept going.
460
00:28:39,480 --> 00:28:43,320
I asked permission. People were receptive, the maintainers, and then I started carving
461
00:28:43,320 --> 00:28:47,000
off pieces of it. And some of the pieces you had mentioned this in
462
00:28:47,000 --> 00:28:50,815
your, interview with Fabienne. We're like, well, we need to get ready
463
00:28:50,815 --> 00:28:54,115
for Bazel 7 because it's still building with Bazel 6.
464
00:28:54,895 --> 00:28:58,654
And I found out that with these little tweaks, you know, you can build
465
00:28:58,654 --> 00:29:02,495
under both. And, I started sending them those updates.
466
00:29:02,495 --> 00:29:04,980
And then other little cleanups, refactorings,
467
00:29:06,240 --> 00:29:10,000
tools, things of that nature. And now we're
468
00:29:10,000 --> 00:29:13,440
getting to the point where the actual substantial design work is going
469
00:29:13,440 --> 00:29:16,875
in. So it it's
470
00:29:16,875 --> 00:29:20,715
it's been a really fun project for me to just kinda
471
00:29:20,715 --> 00:29:23,455
get in there and figure out these problems and solve them and to,
472
00:29:25,514 --> 00:29:29,049
and to work with the maintainers of Rural Scholar. It's I
473
00:29:29,270 --> 00:29:32,250
need to make sure I'm looking at their names so I can pronounce them right.
474
00:29:32,710 --> 00:29:36,150
I should have this up already. But, yeah. They
475
00:29:36,309 --> 00:29:40,070
they've just been wonderful with, not only reviewing
476
00:29:40,070 --> 00:29:43,845
my changes, but also pushing back, when, you know, there's
477
00:29:43,845 --> 00:29:47,605
something they're concerned about. And, or making
478
00:29:47,605 --> 00:29:51,445
suggestions, you know, improvements to things. It's definitely that's how pull requests
479
00:29:51,445 --> 00:29:54,325
are supposed to work. You know, I wasn't gonna take, like, my working branch and
480
00:29:54,325 --> 00:29:58,110
send them, like, 5,000 well, more than 5,000 when you
481
00:29:58,110 --> 00:30:01,549
consider, you know, module lock files. But, you know, I wasn't gonna send them, like,
482
00:30:01,549 --> 00:30:05,070
this one, you know, earth
483
00:30:05,070 --> 00:30:08,669
shattering pull request and say merge me. I wanted to make sure that
484
00:30:08,669 --> 00:30:12,325
these different little pieces got the
485
00:30:12,325 --> 00:30:16,005
attention they deserve, that everybody was aligned. Oh, yep. I understand. This
486
00:30:16,005 --> 00:30:19,445
is what needs to happen. This is why this works. And then to get feedback
487
00:30:19,445 --> 00:30:23,125
on the pieces of it, so that I can make improvements as I go. And
488
00:30:23,125 --> 00:30:26,750
it's been a wonderful process. Yeah. I hope I hope I
489
00:30:26,750 --> 00:30:30,270
hope that we can encourage everyone who contributes to open source to learn those
490
00:30:30,270 --> 00:30:33,950
lessons. It's also partly these those those
491
00:30:33,950 --> 00:30:37,070
contributors are volunteering. They don't have very much time. And if you send them something
492
00:30:37,070 --> 00:30:40,315
big, it'll there's just never a period of time where they have a whole hour
493
00:30:40,315 --> 00:30:44,075
to look at a big PR. Right. Exactly. You
494
00:30:44,075 --> 00:30:47,195
have to somehow fit your work into the time slices that they could that that
495
00:30:47,195 --> 00:30:50,555
you think they can make available. And their time slices are probably smaller than you
496
00:30:50,555 --> 00:30:54,350
think because somebody maintaining something like that. I don't know those folks either, but that's
497
00:30:54,350 --> 00:30:58,130
that's a lot of work. Yeah. Yeah. It's, Simonas and,
498
00:30:59,549 --> 00:31:02,929
and Vaidas. And out of respect,
499
00:31:02,990 --> 00:31:06,350
I'm out of respect, I wanna I wanna say Vaidas,
500
00:31:06,590 --> 00:31:09,705
and Simonas, Tunavicius.
501
00:31:10,325 --> 00:31:14,085
I'm, you know, self conscious, you know, as most Americans are about
502
00:31:14,085 --> 00:31:17,925
mispronouncing, but at least I made it to him. But, yeah, they've been and
503
00:31:17,925 --> 00:31:21,605
then, actually, one of the Scala compiler maintainers, a guy
504
00:31:21,605 --> 00:31:25,120
named Wojciek Mazur, noticed what I was doing. He
505
00:31:25,120 --> 00:31:28,800
started, you know, commenting on my pull request and making some really helpful
506
00:31:28,800 --> 00:31:32,560
suggestions. And then we coordinated, like, things he was trying to land with things that
507
00:31:32,640 --> 00:31:36,480
it it's just, you know, sometimes you think of the open community, you
508
00:31:36,480 --> 00:31:39,635
know, some you know, it can be a harsh place sometimes,
509
00:31:41,295 --> 00:31:45,055
but, man, it's been just a pleasure working on this. And taking the time is
510
00:31:45,055 --> 00:31:48,195
not a problem. I'm not in a rush. I'm not trying to force things through.
511
00:31:48,335 --> 00:31:52,170
I'm not trying to pull a x z situation on
512
00:31:52,950 --> 00:31:55,910
them. You know? And it it in the in the end, I think we're gonna
513
00:31:55,910 --> 00:31:59,430
have, you know, the next major release of this thing, not only be Bazel mod
514
00:31:59,430 --> 00:32:02,250
ready, but just be and it's still gonna be workspace compatible.
515
00:32:03,270 --> 00:32:05,290
And it's gonna be, I think, a great
516
00:32:06,794 --> 00:32:10,174
upgrade to everybody in the community working with
517
00:32:10,395 --> 00:32:14,095
Scala and Bazel. Yeah. That's great. I think
518
00:32:14,715 --> 00:32:18,475
Scala is one of the last ones I know of. I mean, the
519
00:32:18,475 --> 00:32:22,310
Bazel central registry has a ton of modules on it already. We
520
00:32:22,310 --> 00:32:26,150
should point out if you want to add more, there used to be a
521
00:32:26,150 --> 00:32:29,930
a poll that, Yun put up on a on a GitHub discussion, and now that's
522
00:32:30,230 --> 00:32:33,510
that's been closed. And there's simply a process of creating an issue on the Bazel
523
00:32:33,510 --> 00:32:37,304
central registry if there's a module that's missing, and it will add a module wanted
524
00:32:37,304 --> 00:32:41,065
tag to it. You can see the module wanted tags on the registry now.
525
00:32:41,065 --> 00:32:44,905
There are a long tail of those, partly
526
00:32:44,905 --> 00:32:48,745
because of we said earlier how Bazel mod is meant to work with
527
00:32:48,745 --> 00:32:52,420
existing package managers, but then there's c plus plus And
528
00:32:52,420 --> 00:32:56,020
so it kind of accidentally became a package manager for c plus plus where
529
00:32:56,020 --> 00:32:59,800
everybody has put patches that add the build files, that add the cc library targets.
530
00:33:00,580 --> 00:33:03,620
Mhmm. And so a bunch of the module wanted a bunch of these modules really
531
00:33:03,620 --> 00:33:07,395
don't have any Starlark code. They're not rules. They're, you know, just how do I
532
00:33:07,395 --> 00:33:11,015
build this the c plus plus library? Whatever. Unfortunate
533
00:33:11,075 --> 00:33:14,835
accident is probably gonna stay that way. I just complain about it
534
00:33:14,835 --> 00:33:18,355
every chance I get. I'm sorry. I can't help myself. Well, there has to be
535
00:33:18,355 --> 00:33:22,040
a little bit of that. I think there's, you know, and I was
536
00:33:22,040 --> 00:33:25,720
planning on maybe bringing this up closer to the end. But I think, you know,
537
00:33:25,720 --> 00:33:29,400
as someone who, at this point, is obviously deeply invested in
538
00:33:29,400 --> 00:33:33,160
rooting for Basel Mod to succeed and wants to help people get there. You
539
00:33:33,160 --> 00:33:36,965
know, there are still a couple of warts, and we need to
540
00:33:36,965 --> 00:33:40,645
be honest about, what we you know, what those
541
00:33:40,645 --> 00:33:43,845
are and what we need to do to work around them today and what we
542
00:33:43,845 --> 00:33:47,625
might need to do to remove them as blockers
543
00:33:47,765 --> 00:33:51,490
for some portion of the audience. Yeah. Well, let's
544
00:33:51,490 --> 00:33:55,250
talk through some of those. Mhmm. You mentioned the at at
545
00:33:55,250 --> 00:33:58,390
earlier, so there's this concept of canonical and
546
00:33:59,090 --> 00:34:02,850
apparent repository names. So you have to be careful to get
547
00:34:02,850 --> 00:34:06,665
those right. And, of course, the problem with error messages is is, they sort
548
00:34:06,665 --> 00:34:10,264
of get lazy evaluated. So error messages will look like, I couldn't find a label
549
00:34:10,264 --> 00:34:13,945
called open square bracket at at no such thing close square
550
00:34:13,945 --> 00:34:17,645
bracket. And then, like, it's just, it's pretty baffling.
551
00:34:17,705 --> 00:34:20,389
So what do you do about those? Like, do I just add the other at?
552
00:34:20,389 --> 00:34:22,710
Like, I change all my at's to at at? I have done that in a
553
00:34:22,710 --> 00:34:26,549
couple spots. Yeah. Yeah. I mean, in some cases, that's exactly
554
00:34:26,549 --> 00:34:28,309
what you need to do. In some cases
555
00:34:30,324 --> 00:34:33,205
well, I you know, I wrote a whole the last blog post that came out
556
00:34:33,205 --> 00:34:36,965
was all about, like, you know, dealing with, trying not
557
00:34:36,965 --> 00:34:40,405
to depend directly on a repository name because of the
558
00:34:40,405 --> 00:34:44,050
canonical versus apparent, abstraction.
559
00:34:45,070 --> 00:34:48,670
And so I tried to, you know, present, like, if you're in this situation, this
560
00:34:48,670 --> 00:34:51,469
is what you can do or here's a workaround. I mean, the first blog post
561
00:34:51,469 --> 00:34:54,989
had some pretty straightforward things. And the next blog post, I'm actually
562
00:34:55,070 --> 00:34:58,285
there's still a couple more issues I hit in ruleScala,
563
00:35:00,265 --> 00:35:03,865
that, I I actually sort of nerd sniped
564
00:35:03,865 --> 00:35:07,465
Fabian into reviewing some of my changes indirectly for that and
565
00:35:07,465 --> 00:35:11,050
ended up with a better solution as a result. There's still
566
00:35:11,050 --> 00:35:14,650
one corner case where it's like, ah, in this one case, I still wanna parse
567
00:35:14,650 --> 00:35:17,950
out an apparent name from a canonical name. But
568
00:35:18,410 --> 00:35:21,550
other than that, the vast majority of cases, you just need to
569
00:35:22,010 --> 00:35:25,835
know, like, this is how you're supposed to handle these things and it's not that
570
00:35:25,835 --> 00:35:29,675
hard once you see it done. But yeah. If you don't
571
00:35:29,675 --> 00:35:33,195
know about this already and you get that error, you know, that that's why I
572
00:35:33,195 --> 00:35:36,474
also try to keep I put a lot of error messages and things in my
573
00:35:36,474 --> 00:35:40,320
blog post, in my commit messages pull request. So if somebody hits that
574
00:35:40,320 --> 00:35:43,760
and searches for it, they'll get a hit and go, oh, okay. You know,
575
00:35:43,760 --> 00:35:47,540
here's something helpful that will allow me to move on.
576
00:35:48,320 --> 00:35:51,780
Yeah. I feel like maybe another quick mental trick,
577
00:35:53,360 --> 00:35:56,835
useful concept is that labels and strings are different objects and to
578
00:35:56,835 --> 00:36:00,355
convert between them takes a little bit of care and Mhmm. You know,
579
00:36:00,355 --> 00:36:02,915
usually, like, the at sign shows up because you have a string version, but if
580
00:36:02,915 --> 00:36:06,515
you use the label constructor, that changes the context in which the string is
581
00:36:06,515 --> 00:36:10,070
interpreted usually in the right way. Yep. There was
582
00:36:10,070 --> 00:36:13,590
also I'll just, put on the screen for a second. They added an a
583
00:36:13,590 --> 00:36:17,190
new API into the native name
584
00:36:17,190 --> 00:36:20,630
space in Starlark or in Bazel's version of Starlark, which is called package
585
00:36:20,630 --> 00:36:24,165
relative label. Mhmm. And that's another way
586
00:36:24,165 --> 00:36:28,005
that you can, avoid having to reference
587
00:36:28,005 --> 00:36:31,765
the repository name. Right? Right. Yeah. I haven't used that
588
00:36:31,765 --> 00:36:35,460
one in particular, but but I think that one of the
589
00:36:35,460 --> 00:36:39,220
important rules of thumb with label is if you're in a macro
590
00:36:39,220 --> 00:36:42,820
and you're referencing something in your own repository, use a
591
00:36:42,820 --> 00:36:46,420
label. Because that way, the the that
592
00:36:46,420 --> 00:36:50,154
string will be anchored to the repository in which the macro
593
00:36:50,154 --> 00:36:53,755
appears. Otherwise, it will be interpreted as relative to the
594
00:36:53,755 --> 00:36:57,434
repository in which the macro is executed, and that's
595
00:36:57,434 --> 00:37:01,115
not always what you want. Yep. Exactly. That's the one that I think
596
00:37:02,140 --> 00:37:05,100
when you when you think of label equals string, then you will then you'll get
597
00:37:05,100 --> 00:37:08,700
that one wrong, definitely. Yeah. Maybe compatibility
598
00:37:09,100 --> 00:37:12,320
chains. Sorry. Go ahead.
599
00:37:14,235 --> 00:37:17,115
Well, I was gonna say compatibility level is another one. This was just this came
600
00:37:17,115 --> 00:37:20,815
up in the rules author sig meeting last week. Yeah. This this week.
601
00:37:21,515 --> 00:37:25,035
So in theory, modules on the on the
602
00:37:25,035 --> 00:37:28,655
central registry can indicate both the minimum and maximum version of their dependencies.
603
00:37:28,875 --> 00:37:32,130
And so there should be a constraint solution that allows us to get the right
604
00:37:32,130 --> 00:37:35,430
versions of things, but every now and then when there's a breaking change,
605
00:37:35,810 --> 00:37:39,170
someone pulls out this compatibility level flag from Bizzle mod.
606
00:37:39,170 --> 00:37:42,850
But the result of that is that the end user is just told, oh, sorry.
607
00:37:42,850 --> 00:37:45,875
Like, your dependencies are unsolvable. Fix it yourself. Right?
608
00:37:47,055 --> 00:37:50,495
Yeah. I mean, it's definitely a hard problem, but it's, you know, it's where
609
00:37:50,495 --> 00:37:54,095
semantic versioning came from. And, and, of
610
00:37:54,095 --> 00:37:57,475
course, part of whether or not semantic
611
00:37:57,535 --> 00:38:00,930
versioning is effective is whether people honor the
612
00:38:00,930 --> 00:38:04,450
semantic versioning schema where breaking changes result in a
613
00:38:04,450 --> 00:38:07,970
major number bump. And I
614
00:38:08,290 --> 00:38:12,050
you know, in the Bismarck world, you have
615
00:38:12,050 --> 00:38:15,785
that concept of compatibility level. And so when
616
00:38:15,785 --> 00:38:18,525
the dependencies are resolved between modules,
617
00:38:19,944 --> 00:38:23,785
the compatibility level is sort of like a super major number. Right? Like,
618
00:38:23,785 --> 00:38:27,625
so you can have different versions of a module that have different
619
00:38:27,625 --> 00:38:31,360
major simver numbers. But if they're listed compatibility level
620
00:38:31,360 --> 00:38:35,060
1, you're still gonna resolve to the, you know, the highest version of that.
621
00:38:35,600 --> 00:38:39,120
And, I hit that problem with
622
00:38:39,120 --> 00:38:42,820
ruleskalla with regards to, the protobuf
623
00:38:43,200 --> 00:38:46,615
module. And I think that protobuf,
624
00:38:48,595 --> 00:38:51,575
actually does bump its simver major
625
00:38:52,755 --> 00:38:56,515
appropriately. But all of the modules now
626
00:38:56,515 --> 00:39:00,160
available all the way back to, you know, 217 to 28
627
00:39:00,160 --> 00:39:03,940
3 are compatibility level 1. And so
628
00:39:04,320 --> 00:39:07,840
when I was also doing the Bazel mod work on
629
00:39:07,840 --> 00:39:11,600
RealScala, I was like, you know what? People are gonna integrate this. They're probably gonna
630
00:39:11,600 --> 00:39:15,415
have newer versions of protobuf. I wonder what what that will do and it blew
631
00:39:15,415 --> 00:39:19,174
things up. And I have an issue. I linked it in the show notes where
632
00:39:19,174 --> 00:39:22,954
I did this whole investigation. It actually took me, you know, weeks of troubleshooting
633
00:39:23,015 --> 00:39:26,855
and figuring things out, where I
634
00:39:26,855 --> 00:39:30,630
have the constraints. Like, if rulescala does this, we can handle this
635
00:39:30,630 --> 00:39:34,230
version of per the buff. If it goes this, we can go to this version,
636
00:39:34,230 --> 00:39:37,990
but we drop that version. And so that's
637
00:39:37,990 --> 00:39:41,369
where my concern comes in with with compatibility level
638
00:39:42,230 --> 00:39:45,865
is that it's not yet something
639
00:39:46,085 --> 00:39:49,924
that is utilized for its intended purpose
640
00:39:49,924 --> 00:39:53,385
to say and I think it was was it Brentley in the rule SIG meeting
641
00:39:53,605 --> 00:39:57,444
said you should bump it if increasing you know, if updating the
642
00:39:57,444 --> 00:40:00,724
dependency breaks your bill. And it's like, yeah. Exactly. That's what I would expect from
643
00:40:00,724 --> 00:40:04,320
it. I I understand that maybe other problems
644
00:40:04,320 --> 00:40:07,680
happen after, you you know, you try to actually run what you built or
645
00:40:07,680 --> 00:40:11,119
whatever. But if somebody, you know, pulls in
646
00:40:11,119 --> 00:40:14,915
ruleskala and then bumps up their protobuf
647
00:40:14,975 --> 00:40:18,495
to 28.3 and we don't have our
648
00:40:18,495 --> 00:40:22,095
internal jars that can talk to to that version of the
649
00:40:22,095 --> 00:40:25,869
proto compiler, that's a problem. Whereas if we
650
00:40:25,869 --> 00:40:29,490
had, different compatibility levels for those protobuf versions,
651
00:40:29,790 --> 00:40:33,550
it seems like well, rulescala gets up to this version and the
652
00:40:33,550 --> 00:40:37,310
main repo gets this other version and maybe there's still
653
00:40:37,310 --> 00:40:40,690
problems after that. But at least there's some signal that there's an incompatibility
654
00:40:41,070 --> 00:40:44,645
there. Yeah. Let's let's
655
00:40:44,645 --> 00:40:48,405
wrap up in a couple minutes here. Is there any any of these other, warts
656
00:40:48,405 --> 00:40:51,945
or or downsides of Basel mod that you wanna let people know?
657
00:40:54,965 --> 00:40:58,750
Let me see. I'm looking at my notes. I wrote
658
00:40:58,750 --> 00:41:01,390
a whole bunch of stuff in advance, and I know I've kinda been weaving in
659
00:41:01,390 --> 00:41:04,130
and out. Kinda look like this.
660
00:41:05,150 --> 00:41:06,210
I think I think
661
00:41:09,015 --> 00:41:12,615
I think yeah. I I I I'm I'm trying to do what I
662
00:41:12,615 --> 00:41:16,214
can to expose, you know, the
663
00:41:16,214 --> 00:41:20,055
problems people might face, provide solutions they have to do
664
00:41:20,055 --> 00:41:23,810
with their own hands, provide workarounds until things
665
00:41:23,810 --> 00:41:27,570
are fixed upstream. And, of course, in the case of Rulescala, I'm, you know, trying
666
00:41:27,570 --> 00:41:31,250
to do the right thing and actually go upstream and do what I can to
667
00:41:31,250 --> 00:41:34,550
help improve that. And but,
668
00:41:36,744 --> 00:41:40,425
you know, things like compatibility level, and, you
669
00:41:40,425 --> 00:41:44,185
know, repository names, you know, we shouldn't depend on them, but a lot of
670
00:41:44,185 --> 00:41:47,805
people depend on them in a lot of different projects. So until everybody
671
00:41:47,945 --> 00:41:51,420
gets with, the program and either handles them correctly,
672
00:41:52,440 --> 00:41:55,960
or dependencies are upgraded, we just need to be aware that that's going to
673
00:41:55,960 --> 00:41:59,800
be a very, very common, stumbling point for a lot of
674
00:41:59,800 --> 00:42:03,345
people trying to migrate. And then, of
675
00:42:03,345 --> 00:42:06,945
course, module dot bazel dot lock. I really
676
00:42:06,945 --> 00:42:10,545
wanna like it, but it just it just seems to be
677
00:42:10,545 --> 00:42:13,665
begging to get get ignored most of the time up to this point. It it
678
00:42:13,665 --> 00:42:17,500
just creates a lot of, you know, it doesn't seem
679
00:42:17,500 --> 00:42:21,259
to lock anything. Right? Like, you can like, Bazel will change it every time
680
00:42:21,259 --> 00:42:25,099
it runs. And, you know, I understand why it's there
681
00:42:25,099 --> 00:42:28,859
and it what it's supposed to support. But in practice, it really
682
00:42:28,859 --> 00:42:32,595
does just generate, like, monster diffs emerge conflicts.
683
00:42:32,595 --> 00:42:35,975
So I'm really kind of on the fence whether to recommend
684
00:42:36,275 --> 00:42:40,115
it actually get checked in or not. Personally, in practice, I
685
00:42:40,115 --> 00:42:43,715
haven't encountered a problem with get ignoring it, but I don't
686
00:42:43,715 --> 00:42:47,560
wanna do that. Yeah. In in theory, it it provides
687
00:42:47,560 --> 00:42:51,400
some guarantees that get ignoring it will will lose, but I I I
688
00:42:51,400 --> 00:42:54,920
think it's practical for most folks to get ignore it right now. It's still improving.
689
00:42:54,920 --> 00:42:58,215
I think we mentioned that on the episode with Fabian. Yeah.
690
00:42:58,535 --> 00:43:02,375
And yeah. There's you know, and and we mentioned that Bazel mod is
691
00:43:02,375 --> 00:43:05,895
is kind of a, an abstraction that goes on top of native package managers. And
692
00:43:05,895 --> 00:43:09,175
if your native package manager already had a lock file, so you have a a
693
00:43:09,175 --> 00:43:12,700
lock requirements or a, you know, NPM lock file.
694
00:43:14,040 --> 00:43:17,000
I I as I recall, one of the fixes we were still working on was
695
00:43:17,000 --> 00:43:19,400
that there's a thing where Bazel mod can trust that that thing is up to
696
00:43:19,400 --> 00:43:23,180
date, and so it doesn't have to, like, relock more than it needs to.
697
00:43:23,480 --> 00:43:27,135
Yeah. But yeah. I guess, you know, just just sort of wrap up and then
698
00:43:27,135 --> 00:43:30,835
on a positive note. Like I said, I'm fascinated by this work. I think it's
699
00:43:30,974 --> 00:43:34,654
definitely driving in the right direction. I wanna do whatever is within my
700
00:43:34,654 --> 00:43:38,174
power to help everybody make migration and use
701
00:43:38,174 --> 00:43:42,020
this wonderful new system. And to bring it back to my own
702
00:43:42,020 --> 00:43:45,860
personal sense of motivation, I feel like better builds are the
703
00:43:45,860 --> 00:43:49,620
foundation for better code quality, and that's my thing that I really
704
00:43:49,620 --> 00:43:53,325
care about. I got my well, my mug is faded, but somebody got for
705
00:43:53,325 --> 00:43:56,865
my birthday, you know, little 99 bugs in the code,
706
00:43:57,325 --> 00:44:00,845
take one down, patch it around 256 little bugs. Right?
707
00:44:00,845 --> 00:44:04,685
Like, you know, that's the world I live in where I wanna avoid those kind
708
00:44:04,685 --> 00:44:08,369
of problems. And, I see, you know, moving
709
00:44:08,369 --> 00:44:11,970
towards Bazelmod and having cleaner, faster, more efficient
710
00:44:11,970 --> 00:44:13,270
builds is a huge,
711
00:44:16,770 --> 00:44:20,450
important prerequisite for getting people to care about code quality
712
00:44:20,450 --> 00:44:24,155
automated testing. Because, yes, people are now more people are testing,
713
00:44:24,455 --> 00:44:28,235
but how are they testing? Now that's the next phase of the challenge.
714
00:44:28,375 --> 00:44:31,335
Well, maybe that will be the next episode. I would love to talk about that
715
00:44:31,335 --> 00:44:35,070
one too. Yeah. I I I'm also
716
00:44:35,230 --> 00:44:38,270
yeah. I think a lot of us who use Bazel do it because we're passionate
717
00:44:38,270 --> 00:44:42,030
about code quality and getting this next year
718
00:44:42,030 --> 00:44:45,390
from Bazel 8, you know, it's gonna drop soon and getting to Bazel 9 when
719
00:44:45,390 --> 00:44:49,185
Workspace is actually deleted. All that code is gonna go away. We've gotta
720
00:44:49,185 --> 00:44:52,865
get the ecosystem to come with us. So please, Mike, I would beg everyone to
721
00:44:52,865 --> 00:44:56,545
to follow Mike's example and contribute upstream. Contribute
722
00:44:56,545 --> 00:45:00,145
doesn't have to mean you send awesome pull requests. It could just mean well formed
723
00:45:00,145 --> 00:45:03,880
issues, answering other people's issues, and it
724
00:45:03,880 --> 00:45:07,320
can also mean getting your company to donate some money that we have an open
725
00:45:07,320 --> 00:45:11,020
collective that helps at least for the rules that are in the basil contrib org.
726
00:45:12,360 --> 00:45:16,040
Donate some you know, ask your manager, can I donate some time to this project?
727
00:45:16,040 --> 00:45:19,455
Like, let me actually carve out 20% time or give me, like, one day a
728
00:45:19,455 --> 00:45:21,875
week for the next 2 months. I wanna go fix this thing.
729
00:45:24,095 --> 00:45:27,775
And, yes, together we'll do it. We'll fix all the
730
00:45:27,775 --> 00:45:30,175
code. We'll fix all the bugs. You can get a new mug that says they're
731
00:45:30,175 --> 00:45:33,960
all fixed. Yes. Well, yeah. Maybe
732
00:45:33,960 --> 00:45:37,800
that's the trick. It's fading. By the time it it's totally blank, you know, we'll
733
00:45:37,800 --> 00:45:41,560
have solved the problem. Yes. Hopefully. Although AI is writing a lot of new
734
00:45:41,560 --> 00:45:45,175
code with bugs. So that that could be another episode. I'm sorry to end. We're
735
00:45:45,175 --> 00:45:48,855
ending on a positive note. This is great. Thank you so much for coming
736
00:45:48,855 --> 00:45:52,535
on. Really appreciate your your depth of expertise and and getting to chat with
737
00:45:52,535 --> 00:45:56,215
you. Yeah. I really appreciate the opportunity, Alex. Thanks for having me. Okay. Sure
738
00:45:56,215 --> 00:45:57,755
thing. See you all next time.
739
00:46:17,425 --> 00:46:20,245
Okay. And we're out. Sorry. It took me a while to find the