Rethinking Developer Tooling in Monorepos with bazel_env - feat. Fabian Meumertzheim

1
00:00:05,440 --> 00:00:09,040
Hello there. Welcome back to Aspect Insights, your
2
00:00:09,040 --> 00:00:12,639
favorite podcast about Bazel and its ecosystem. My name is
3
00:00:12,639 --> 00:00:16,355
Alex Eagle. I'm your host. I started Aspect Build, which
4
00:00:16,355 --> 00:00:20,115
is a company making a monorepo developer platform based on Bazel, and
5
00:00:20,115 --> 00:00:23,474
boy, do we love it. Today, I'm really excited to speak with
6
00:00:23,474 --> 00:00:27,314
Fabian Murmertheim, who works at BuildBuddy. He is a
7
00:00:27,314 --> 00:00:31,130
a Bazel expert, prolific contributor to the Bazel repo. And today, I'm gonna
8
00:00:31,130 --> 00:00:34,890
talk to him about a Bazel rule that he wrote called
9
00:00:34,890 --> 00:00:38,730
Bazel env that gives you a really convenient way to get the right
10
00:00:38,730 --> 00:00:42,250
tools on your path for your developers to
11
00:00:42,250 --> 00:00:46,065
use while developing. I wanna note that we ran into
12
00:00:46,065 --> 00:00:49,664
an issue in a demo during this episode, and, Fabian has
13
00:00:49,664 --> 00:00:53,425
hopefully added a a message to the tooling that will remind you what to
14
00:00:53,425 --> 00:00:56,945
do, which is that we need to run the rehash
15
00:00:56,945 --> 00:00:59,950
commands, to make the tools appear on the path. So when you see that go
16
00:00:59,950 --> 00:01:03,630
wrong during the demo, please be confident that that's gonna work for you.
17
00:01:03,630 --> 00:01:05,810
And with that, let's get started.
18
00:01:07,710 --> 00:01:11,470
So, so excited to have Fabian back with us on
19
00:01:11,470 --> 00:01:14,755
the podcast. Welcome, Fabian. Hi. Hi, Alex.
20
00:01:15,314 --> 00:01:19,155
Thanks for having me back here. Yeah. I was just checking through through
21
00:01:19,155 --> 00:01:22,674
the history. You were on way back in August of last year, and it's been
22
00:01:22,674 --> 00:01:26,435
a couple months actually since we did a recording. So, I think you're the
23
00:01:26,435 --> 00:01:30,270
the first the first repeat guest. Oh, it's the season two now. Yeah. I'm
24
00:01:30,270 --> 00:01:33,710
happy to be here. Yeah. That's that's actually that's a nice way to describe it.
25
00:01:33,710 --> 00:01:37,150
Let's call this, season two episode one. Although now that I've said that, maybe I'm
26
00:01:37,150 --> 00:01:40,290
actually obligated to change the numbering scheme. Never mind.
27
00:01:41,549 --> 00:01:45,195
That's too hard. So, so
28
00:01:45,195 --> 00:01:48,555
I I asked Fabian to come on because, I have been working on this
29
00:01:48,555 --> 00:01:51,835
problem for off and on for a couple of years that our customers have, and
30
00:01:51,835 --> 00:01:55,435
I'm sure yours do as well. And generally, the
31
00:01:55,435 --> 00:01:59,000
problem is, development environments. And
32
00:01:59,540 --> 00:02:02,820
there's a lot of different approaches for it and a lot of things that companies
33
00:02:02,820 --> 00:02:06,180
are doing. And I had an approach that I had gotten
34
00:02:06,180 --> 00:02:10,020
working. And then I think, the thing that Fabian wrote right after I
35
00:02:10,020 --> 00:02:13,545
I did that is actually, a significant improvement. And
36
00:02:13,545 --> 00:02:17,225
so, I've switched over some stuff and, would love to
37
00:02:17,225 --> 00:02:19,405
talk about sort of the background for it.
38
00:02:21,545 --> 00:02:25,225
I'll share my screen, but if you're just listening on audio, we'll we'll explain everything
39
00:02:25,225 --> 00:02:28,940
that we're looking at here. So so this is this is
40
00:02:28,940 --> 00:02:31,680
the project. It's called Bazel n dot Bzl.
41
00:02:33,660 --> 00:02:37,260
Maybe as a as a as a starting point, actually, because we had a
42
00:02:37,260 --> 00:02:40,955
side conversation about it. What's how how come it's named bazel n dot
43
00:02:40,955 --> 00:02:44,334
b z l? Is it like normally, I don't see, like, a a file extension
44
00:02:44,394 --> 00:02:48,235
as part of a repo name or a bazel module name. Yeah. I think
45
00:02:48,235 --> 00:02:52,050
one I mean, one reason for this is it's more memorable this way because
46
00:02:52,050 --> 00:02:55,250
once you've seen it once, you will you will never forget it. Yeah.
47
00:02:55,570 --> 00:02:59,410
But, yeah, I think that the the real idea was that I was looking for
48
00:02:59,410 --> 00:03:02,930
some kind of a canonical scheme to name Bazel
49
00:03:02,930 --> 00:03:06,705
modules or, generally, Bazel projects that aren't really
50
00:03:06,705 --> 00:03:10,545
rule sets, but, like, helper libraries or something that something
51
00:03:10,545 --> 00:03:14,225
clearly Bazel related that isn't the rule set. Because I didn't
52
00:03:14,225 --> 00:03:18,065
wanna call this rules env because it just contains a single rule
53
00:03:18,065 --> 00:03:21,660
that's called Bazel env, and it's not a rule set. It's just a
54
00:03:21,660 --> 00:03:25,260
tool. So, I was looking for for some other naming convention at
55
00:03:25,260 --> 00:03:28,780
this other project, which I ended up calling with config dot
56
00:03:28,780 --> 00:03:32,540
bezel, which also just consists of the single function. And I
57
00:03:32,540 --> 00:03:35,900
just wanted to make it very easy to load this stuff. So if you go
58
00:03:35,900 --> 00:03:39,655
to yeah. Then you can see here that you will never need to remember from
59
00:03:39,655 --> 00:03:42,635
this day on whether it's bezel end slash
60
00:03:43,255 --> 00:03:47,015
bezel end colon def dot bezel or a colon bezel end
61
00:03:47,015 --> 00:03:50,370
dot bezel or something. You just do add project name and load that. And the
62
00:03:50,370 --> 00:03:54,129
symbol that you load is also, like, the name of the project. So, I felt
63
00:03:54,129 --> 00:03:57,349
that this was, like, this sticks in my hat much better.
64
00:03:57,970 --> 00:04:01,730
Yeah. I love it. I think we don't have a good, language
65
00:04:01,730 --> 00:04:05,545
server protocol for editors to type Starlark files is one issue. So,
66
00:04:05,545 --> 00:04:09,305
like, it's hard to get completion on your load statements. And
67
00:04:09,305 --> 00:04:12,905
and I've also noticed recently there seems like some some rule sets have
68
00:04:12,905 --> 00:04:16,524
started to use many load points rather than one public API.
69
00:04:17,740 --> 00:04:21,099
Like, you know, rules go, you get go library and and go binary from the
70
00:04:21,099 --> 00:04:24,460
same place, but rules shell, you have to load from s h library dot bezel
71
00:04:24,460 --> 00:04:27,500
and s h binary dot bezel. It seems like it's getting actually harder to type
72
00:04:27,500 --> 00:04:31,165
these load statements. Yeah. We need better automation for that.
73
00:04:31,385 --> 00:04:35,065
But, yeah, this this is something that that way I sort of, avoided
74
00:04:35,065 --> 00:04:38,585
with this naming scheme. Yeah. I mean, I I personally found this really
75
00:04:38,585 --> 00:04:42,250
inspirational. I was just working on our, a tar rule recently that's in basal
76
00:04:42,250 --> 00:04:45,550
contrib, and so I borrowed your naming scheme for this tar dot b c l.
77
00:04:46,010 --> 00:04:48,890
You know, similarly because it's just one rule, and it's nice to be able to
78
00:04:48,890 --> 00:04:51,930
say so to make a short load statement and more memorable like you said. And
79
00:04:51,930 --> 00:04:55,130
it's only one rule, so it would be weird to call it rules underscore. Although,
80
00:04:55,130 --> 00:04:57,710
there is actually a rules underscore tar on the it's essential
81
00:05:00,065 --> 00:05:03,905
registry. Well, so that's that's an interesting aside.
82
00:05:03,905 --> 00:05:07,425
Although I'm sure, anybody who's trying this will immediately, you know, like,
83
00:05:07,425 --> 00:05:10,165
users are gonna look at this and be like, wait, you can do that?
84
00:05:11,505 --> 00:05:14,699
So maybe it's worth unpacking for a second. How is this even possible? Like, what
85
00:05:14,699 --> 00:05:18,300
does that mean? There's no slash slash. Yeah. So
86
00:05:18,300 --> 00:05:21,520
Bazel has had the shortcut of
87
00:05:22,219 --> 00:05:25,900
some package. So you can do slash slash some
88
00:05:25,900 --> 00:05:29,745
slash package, no colon. And what that means is that
89
00:05:29,745 --> 00:05:33,585
you're referring to the package target in the some
90
00:05:33,585 --> 00:05:37,345
package package. Yeah. So, like, you can rules go
91
00:05:37,345 --> 00:05:40,545
slash slash go as an example. Right? It's just the go target in the go
92
00:05:40,545 --> 00:05:44,240
top level package in the rules go repo. And exactly the same rule
93
00:05:44,240 --> 00:05:47,920
also applies to repos. So if you use a bare repo
94
00:05:47,920 --> 00:05:51,220
name, what you get is the target of that name
95
00:05:51,760 --> 00:05:55,465
in the top level unnamed package of that repo. So
96
00:05:55,465 --> 00:05:59,145
at bezel env dot bezel is just a shortcut for at bezel
97
00:05:59,145 --> 00:06:02,825
env dot bezel slash slash colon bezel env dot
98
00:06:02,825 --> 00:06:06,605
bezel, which is a mouthful, so this is much nicer to type.
99
00:06:06,985 --> 00:06:10,200
Yeah. That's And it all works in the command line. So for example, there are
100
00:06:10,280 --> 00:06:13,960
there's a module that's called build also. And
101
00:06:13,960 --> 00:06:17,560
because that's a very stupid and simple module, it just has, like, a single
102
00:06:17,560 --> 00:06:20,895
useful target, which is called build also. So if you wanna run it on the
103
00:06:20,895 --> 00:06:24,575
command line, you just do Bazel run at build also. Nothing
104
00:06:24,575 --> 00:06:28,095
else. So that's I think that's sort of the the closest that Bazel can get
105
00:06:28,095 --> 00:06:31,475
to something like NPX, where you just find
106
00:06:31,615 --> 00:06:35,250
it. There I guess it's probably worth noting for anybody who's still on an
107
00:06:35,250 --> 00:06:39,090
older Bazel version. There was a while where you could use this syntax sugar in
108
00:06:39,090 --> 00:06:41,650
a Starlark file, but not on the command line. Right? Yeah.
109
00:06:42,449 --> 00:06:45,410
Yeah. Great, great work by,
110
00:06:45,975 --> 00:06:49,495
Shudong who who fixed that just in time for for this, build all the
111
00:06:49,495 --> 00:06:52,855
modules. I think it's in it's definitely in Bazel seven. It may be in Bazel
112
00:06:52,855 --> 00:06:56,535
six. Great. Alright. Well,
113
00:06:56,535 --> 00:07:00,295
let's let's, I'll try and, motivate the topic at hand a
114
00:07:00,295 --> 00:07:01,035
little bit.
115
00:07:04,180 --> 00:07:07,620
Oh, and I'm only sharing one tab. That's not what I meant to do. That's
116
00:07:07,620 --> 00:07:10,840
like every meeting. Right? You're like, oh, hold on. Let me share something else.
117
00:07:16,945 --> 00:07:20,785
Okay. Let's go find the whole window. So, so
118
00:07:20,785 --> 00:07:24,465
yeah, I wrote I wrote this this blog post about, about what you did.
119
00:07:24,465 --> 00:07:28,100
And I tried to motivate this a little bit with the
120
00:07:28,100 --> 00:07:31,800
idea of, like, what is what is the big problem statement here?
121
00:07:33,220 --> 00:07:37,060
And I've seen this called mobile device management
122
00:07:37,060 --> 00:07:40,660
or just device management. The idea that's like, you know, your hand developers a computer,
123
00:07:40,660 --> 00:07:44,205
and then what's supposed to be on there? And this is their work computer. So,
124
00:07:44,345 --> 00:07:47,065
you know, the main thing most of us are familiar with is that there's some
125
00:07:47,065 --> 00:07:50,825
security software on there. And, you know, from Bazel's point of
126
00:07:50,825 --> 00:07:53,945
view, the main reason you know that is that if your if your security scanner
127
00:07:53,945 --> 00:07:57,005
is looking at your Bazel out folder, you're gonna have a bad time.
128
00:07:57,960 --> 00:08:01,800
Right? Okay. But security software is not
129
00:08:01,800 --> 00:08:04,840
the only thing. You also have, you know, just need to be you need you
130
00:08:04,840 --> 00:08:08,520
need certain developer tools on the machine. And for the most part, I
131
00:08:08,520 --> 00:08:11,960
think companies don't even try to distribute this as part of, like, the machine image.
132
00:08:11,960 --> 00:08:15,567
It's not like you get the laptop and it already has things installed on
133
00:08:15,567 --> 00:08:19,156
it. But some big companies do. And certainly at Google, we had,
134
00:08:19,432 --> 00:08:23,021
a pretty robust system of of delivering software. I mean, like, you know,
135
00:08:23,021 --> 00:08:26,609
developer machines was a deployment target, and there were certain people who worked
136
00:08:26,609 --> 00:08:30,370
on developer tools and that was their deployment target.
137
00:08:32,709 --> 00:08:35,190
So before we dive into how to do this with Bazel, I think it's worth
138
00:08:35,190 --> 00:08:39,029
taking a look at a few alternatives that people use. So one of
139
00:08:39,029 --> 00:08:42,605
them is dev containers and I know this from Versus code but I looked it
140
00:08:42,605 --> 00:08:45,405
up and it's actually you know, this is from Microsoft, so I'm sure the Versus
141
00:08:45,405 --> 00:08:48,605
Code team was involved. But they've made us they've made a standard for what they
142
00:08:48,605 --> 00:08:52,045
call dev containers. And
143
00:08:52,045 --> 00:08:54,705
so they they have a standard JSON file.
144
00:08:56,340 --> 00:08:59,140
And and they've got this image on their site, which is kind of,
145
00:08:59,860 --> 00:09:02,660
illustrates the point. And I think of this a little bit like we're shifting left.
146
00:09:02,660 --> 00:09:06,180
Right? So most of us, let's, you know, starting from production, we know that there's
147
00:09:06,180 --> 00:09:09,915
some sort of environment we're shipping to and that's, you know, we we
148
00:09:09,915 --> 00:09:13,435
tell Bazel to build for the target platform and that's where those binaries
149
00:09:13,435 --> 00:09:13,935
go.
150
00:09:16,795 --> 00:09:20,475
And then there's a step before that which is Bazel needs to actually perform the
151
00:09:20,475 --> 00:09:23,310
build. And so we're all kind of used to using tool chains and various things
152
00:09:23,310 --> 00:09:26,270
to get compilers at the right version. And, you know, one of the selling points
153
00:09:26,270 --> 00:09:29,950
of Bazel is that you can get consistent build environment and, of course,
154
00:09:29,950 --> 00:09:33,790
tests as well. So that's supposed to be hermetic. But then if you shift even
155
00:09:33,790 --> 00:09:37,230
further left, now I just wanna have tools that developers can
156
00:09:37,230 --> 00:09:40,905
run before they do Bazel build or Bazel test
157
00:09:41,525 --> 00:09:44,325
potentially. And of course, the
158
00:09:45,365 --> 00:09:49,205
there's a there's a a requirement that's especially hard to meet here, which is you'd
159
00:09:49,205 --> 00:09:52,850
like to use the same version of that tool for development that Bazel is
160
00:09:52,850 --> 00:09:55,730
gonna run for build and test and, you know, in the case that the tool
161
00:09:55,730 --> 00:09:59,410
is used in both contexts. And
162
00:09:59,410 --> 00:10:03,090
so dev containers is one solution for this, which is saying, oh, well, we'll
163
00:10:03,090 --> 00:10:06,894
just drop into a Docker container to do, you know, you're already using a
164
00:10:06,894 --> 00:10:10,654
container at production. Many many developers are. Maybe
165
00:10:10,654 --> 00:10:14,014
you're using a container for CI. And then so, hey, you may as well use
166
00:10:14,014 --> 00:10:17,535
a container also at at development time. And it's kinda nice because if you choose
167
00:10:17,535 --> 00:10:20,260
the same base image, then you have a little bit of a guarantee that, like,
168
00:10:20,260 --> 00:10:24,100
the libraries that are installed in that dev container, you know, let's say you're doing
169
00:10:24,100 --> 00:10:27,220
c plus plus and you have some system libraries and you just you just get
170
00:10:27,220 --> 00:10:30,500
them from the container environment, you know that there's gonna be the same version of
171
00:10:30,500 --> 00:10:32,760
libc or whatever that you're deploying to in production.
172
00:10:35,675 --> 00:10:38,795
So what's wrong with what's wrong with doing it this way? Like, I mean, this
173
00:10:38,795 --> 00:10:42,315
this sounds kinda convenient and, you know, if you're using the Versus code, it understands
174
00:10:42,315 --> 00:10:46,154
this dev container, JSON format, and and offers to just boot
175
00:10:46,154 --> 00:10:49,730
up your environment inside of the container. I don't know. Have you seen
176
00:10:49,730 --> 00:10:53,569
anybody doing it this way? Yeah. There there was even a PR trying
177
00:10:53,569 --> 00:10:57,029
to add dev containers JSON to the base of project itself.
178
00:10:57,490 --> 00:11:01,089
So, I I personally haven't used it. I think
179
00:11:01,089 --> 00:11:04,845
it's, like I mean, it's a very good approach if sort of
180
00:11:04,845 --> 00:11:08,205
everything you do is, like, Docker based and you're comfortable with this,
181
00:11:08,205 --> 00:11:11,965
then it works. I guess there's the sort of the
182
00:11:11,965 --> 00:11:15,745
usual downside of this. I I think, Benjamin
183
00:11:15,805 --> 00:11:19,620
Peterson had a had a great and very funny, Bazel talk
184
00:11:19,620 --> 00:11:23,460
on hermeticity where he sort of talked about the the pros and cons of
185
00:11:23,460 --> 00:11:27,300
Docker containers, and he essentially called this, hash
186
00:11:27,300 --> 00:11:31,060
and stash hermeticity where you have this giant Docker
187
00:11:31,060 --> 00:11:34,875
image blob. And as long as you, like, hash this and use
188
00:11:34,875 --> 00:11:38,475
this all the time, it's super hermetic and, like, everything everything sort of is
189
00:11:38,475 --> 00:11:41,834
portable to anywhere else. But it's essentially, like, a
190
00:11:41,834 --> 00:11:45,455
complete Linux or whatever system in a box with millions of binaries.
191
00:11:46,154 --> 00:11:49,375
If you ever make a change to it, like, you do app get update,
192
00:11:49,940 --> 00:11:53,779
whatever happens. Like, everything changes. Nothing is
193
00:11:53,779 --> 00:11:57,480
repeatable. You will never reproduce the old result, whatever change.
194
00:11:57,940 --> 00:12:01,720
And I think this kind of, like, approach is just not very granular,
195
00:12:03,265 --> 00:12:07,105
which is probably one of the the major downsides. And then I think one other
196
00:12:07,105 --> 00:12:10,785
downside, which is, something that that Bazel developers will
197
00:12:10,785 --> 00:12:14,465
definitely feel is that you have to prepackage this.
198
00:12:14,465 --> 00:12:18,209
Like, if somebody make wants to make a change to their dev environment, for for
199
00:12:18,209 --> 00:12:21,970
example, because you have some some in repo tools that you ship with us, they
200
00:12:21,970 --> 00:12:25,810
essentially need to rebuild and deploy the the the docker container or,
201
00:12:25,810 --> 00:12:29,569
like, build it locally and then rerun everything in that container. They
202
00:12:29,569 --> 00:12:33,165
can't just say, hey. I want this deploy tool changed. I'm
203
00:12:33,165 --> 00:12:36,845
just, like, build the binary, done, run it. This kind of stuff also
204
00:12:36,845 --> 00:12:40,125
doesn't work. So it's not like, for the for the people that work on the
205
00:12:40,125 --> 00:12:43,500
container, it's not as convenient. Yeah. I
206
00:12:43,500 --> 00:12:47,259
suppose you could take care to always have changes
207
00:12:47,259 --> 00:12:51,100
to the development environment be represented as another layer at
208
00:12:51,100 --> 00:12:54,320
the top of the image so that it's not as slow to rebuild.
209
00:12:54,940 --> 00:12:58,675
Yeah. That's good. But even even if you even if you could
210
00:12:58,675 --> 00:13:02,435
solve the problems you mentioned, which are good ones, you still just have a performance
211
00:13:02,435 --> 00:13:04,695
set of working inside of the container. And,
212
00:13:06,195 --> 00:13:09,720
I know that Mac OS has had different kernel
213
00:13:09,720 --> 00:13:13,320
level affordances for containerization, but, like, the file system has has
214
00:13:13,320 --> 00:13:16,840
never been especially fast. And, of course, Bazel has does a ton of interaction with
215
00:13:16,840 --> 00:13:20,680
the file system. Yeah. And then there is also this issue
216
00:13:20,680 --> 00:13:24,055
with nesting. I'm I'm not entirely sure how what sort sort of the the status
217
00:13:24,055 --> 00:13:27,415
of this is today, but if you wanna run Docker in Docker or you want
218
00:13:27,415 --> 00:13:31,095
to run a sandbox binary in Docker, I'm not sure whether that sort
219
00:13:31,095 --> 00:13:34,855
of always works. I mean, you can probably get it to work, but maybe it's
220
00:13:34,855 --> 00:13:38,660
reliable. Yep. That's very true. I guess it's it is
221
00:13:38,660 --> 00:13:41,800
worth saying what one of the advantages for this under Bazel is that
222
00:13:42,260 --> 00:13:45,800
cross platform can be hard. And if you're on a Mac and targeting
223
00:13:45,860 --> 00:13:49,675
Linux, then you'll need cross compilers involved. So
224
00:13:49,675 --> 00:13:52,555
at least, you know, if you can solve the Docker and Docker problem now if
225
00:13:52,555 --> 00:13:56,394
you're on a Mac, you know, your development environment is now Linux. And so it's
226
00:13:56,394 --> 00:13:59,755
less likely that, you know, certain rule sets like rules Python has never had a
227
00:13:59,755 --> 00:14:03,459
great cross built story at least out of the box. It's getting
228
00:14:03,459 --> 00:14:07,000
better, though. So I I I I wanna put out the claim that
229
00:14:07,140 --> 00:14:10,579
Bazel nine will be the time for for correct cross
230
00:14:10,579 --> 00:14:14,420
build. So there's been a lot of work in in rules Python
231
00:14:14,420 --> 00:14:18,155
on integrating UV and its cross platform capabilities, And
232
00:14:18,155 --> 00:14:21,915
I've been hard at work fixing exit platform confusion bugs in
233
00:14:21,915 --> 00:14:25,755
Bazel, so tests and all this stuff. So, yeah, I I hope
234
00:14:25,755 --> 00:14:29,275
that this will like, this whole situation, including, like, cross
235
00:14:29,275 --> 00:14:33,080
unique Windows builds, will be better by the time Bazel nine hits
236
00:14:33,080 --> 00:14:36,780
or at least, like, in minor releases coming after Bazel nine. So there's hope.
237
00:14:37,320 --> 00:14:40,520
Yeah. Thank you so much for working on that. Yeah. I've I've I've I've run
238
00:14:40,520 --> 00:14:44,334
across the used target platform for tests flag, which doesn't quite do
239
00:14:44,334 --> 00:14:47,695
what you want. Right? Yeah. But there's a fixed version of this
240
00:14:47,695 --> 00:14:50,675
enabled at HAP, so that will come in in Bazel nine.
241
00:14:51,455 --> 00:14:55,135
That's cool. I actually quick aside since the last time we
242
00:14:55,135 --> 00:14:58,730
talked, on the podcast was about Bizzle mod rolling out. Do you know
243
00:14:58,889 --> 00:15:02,329
has Bazel nine dropped the workspace support at this point? Is it is the code
244
00:15:02,329 --> 00:15:05,930
gone? Not the entire code because it's just very, very
245
00:15:05,930 --> 00:15:09,690
painful to delete everything. That will probably take a bit more time, but at least
246
00:15:09,690 --> 00:15:13,355
the the enable workspace flag is sort of hard coded to
247
00:15:13,415 --> 00:15:17,255
false at in at master. And, yeah, I think some of the code is
248
00:15:17,255 --> 00:15:20,615
already gone. Okay. Yeah. Cool. Another
249
00:15:20,615 --> 00:15:24,315
shadow should shouldographer going through all of this.
250
00:15:24,695 --> 00:15:28,320
Oh, yeah. There's there's, anybody who works in developer
251
00:15:28,320 --> 00:15:32,000
infrastructure understands that our work is often not
252
00:15:32,000 --> 00:15:35,360
very visible. And when when we break something, everyone
253
00:15:35,360 --> 00:15:38,560
notices and when we make things work, it just, you know, it's taken it's it
254
00:15:38,560 --> 00:15:41,280
just seems like it's it's taken for granted. It's just meant to work that way.
255
00:15:41,280 --> 00:15:45,075
So thank you, Zhidong. And, yeah, I'm I'm
256
00:15:45,075 --> 00:15:48,355
sure that I know that a lot of the motivation there was just to be
257
00:15:48,355 --> 00:15:52,115
able to start removing tech debt, and so I'm I'm sure they've started deleting the
258
00:15:52,115 --> 00:15:53,735
code that's the most painful first.
259
00:15:56,319 --> 00:16:00,160
Yeah. Okay. Well, so let's come back to, other options for this problem
260
00:16:00,160 --> 00:16:04,000
before we dive into Bizzle, Bizzle env. So another one I wanted to
261
00:16:04,000 --> 00:16:07,279
point out. I met actually, I got to meet the Homebrew maintainer, I think his
262
00:16:07,279 --> 00:16:10,935
name is Mike at at, at the at
263
00:16:10,935 --> 00:16:14,695
GitHub Universe last year. And he was talking about this new work
264
00:16:14,695 --> 00:16:17,335
brew project. So the of course, like, if you're using a Mac, you probably use
265
00:16:17,335 --> 00:16:21,035
Homebrew. It's been an amazing package manager for a very long time.
266
00:16:22,790 --> 00:16:25,910
And the whole idea here is to solve exactly the problem we're discussing, which is,
267
00:16:25,910 --> 00:16:29,670
like, if you're if you if your developers are using Homebrew, and they do
268
00:16:29,670 --> 00:16:33,030
have Homebrew for Linux, although it's probably a lot less popular. But so you can
269
00:16:33,030 --> 00:16:36,790
use this system to, to deploy software to your
270
00:16:36,790 --> 00:16:40,404
developers' machines and and manage them. Right? So this gives you sort of the part
271
00:16:40,404 --> 00:16:42,425
of device management that we're talking about.
272
00:16:44,645 --> 00:16:47,365
And I kinda like the sales pitch here. It's like, well, your developers are probably
273
00:16:47,365 --> 00:16:50,640
using Homebrew anyway. So at least, like, now you
274
00:16:50,940 --> 00:16:54,700
can, you know, man put some controls around what they're doing. Like, are they using
275
00:16:54,700 --> 00:16:58,220
tools that they definitely shouldn't be? Are they, you know,
276
00:16:58,220 --> 00:17:01,980
bringing security vulnerabilities into their local environment that
277
00:17:01,980 --> 00:17:05,224
runs as them? And at least there's so much scary stuff that, like, developer tools
278
00:17:05,224 --> 00:17:08,665
run as the user. Like, post install hooks on NPM, thank goodness, are being turned
279
00:17:08,665 --> 00:17:12,505
off by default, but there's a lot of stuff like this. Probably the number one
280
00:17:12,505 --> 00:17:15,865
best marketing pitch if you can say, if you don't use this, then they'll use
281
00:17:15,865 --> 00:17:19,319
it anyway. That's Yes. Pretty
282
00:17:19,319 --> 00:17:23,000
basic way to be in. Yeah. Enough. Of course, I use Homebrew all the time
283
00:17:23,000 --> 00:17:26,839
too. It's great. And yeah. So, like, you know, they they mentioned
284
00:17:26,839 --> 00:17:29,880
on the Homebrew on this Workbrew page that it is, you know, and that you
285
00:17:29,880 --> 00:17:33,695
can install Workbrew via this MDM system. So everybody's
286
00:17:33,695 --> 00:17:36,735
computer has Workbrew and they kinda can't opt out of it. You know, there's some
287
00:17:36,815 --> 00:17:40,015
you can put policy guideline guidelines around whether you can opt out of it. And
288
00:17:40,015 --> 00:17:43,695
then you can actually prevent misuse of the the
289
00:17:43,695 --> 00:17:47,360
Homebrew tool. And it's kinda cute that it was called
290
00:17:47,360 --> 00:17:50,720
Homebrew for a long time and, like, oh, Workbrew. That's perfect. Like, I love the
291
00:17:50,720 --> 00:17:54,480
naming the naming play. Yeah. That that's great.
292
00:17:54,480 --> 00:17:58,080
Maybe. Have they thought of this opportunity in the beginning? I don't know. But,
293
00:17:58,400 --> 00:18:02,215
probably not. But Yeah. There was a great moment maybe. We
294
00:18:02,215 --> 00:18:04,695
should, I should I should chat with him. Maybe he wants to come on and
295
00:18:04,695 --> 00:18:08,535
tell everybody about it. Right. So, so I
296
00:18:08,535 --> 00:18:10,935
I mean, like, why why not do it this way? And I think,
297
00:18:12,500 --> 00:18:15,860
one of the things that you brought up about the dev containers is also the
298
00:18:15,860 --> 00:18:19,240
case here, which is what about tools that you want to write in your repo?
299
00:18:20,500 --> 00:18:23,620
If and it's not necessarily that you're even writing a tool from scratch, but what
300
00:18:23,620 --> 00:18:26,805
if what about a tool that you just need to patch? So, like, maybe you've
301
00:18:26,965 --> 00:18:30,645
forked it. Please don't, you know, that's the f word. But may maybe you
302
00:18:30,645 --> 00:18:33,685
just have you know, you wrote your own entry points, and then you just do
303
00:18:33,685 --> 00:18:36,805
a couple of different things, and then you call into the you know, you just
304
00:18:36,805 --> 00:18:40,325
peel off one layer of the the the porcelain of the tool that you're running.
305
00:18:40,325 --> 00:18:43,740
You write your own, main, and, you know, it's often something like
306
00:18:43,740 --> 00:18:47,500
setting, build workspace directory so that the thing behaves
307
00:18:47,500 --> 00:18:51,340
properly under Bazel run or something like this. And so now
308
00:18:51,340 --> 00:18:54,299
that's a tool that you deploy within your repo even though you only wrote a
309
00:18:54,299 --> 00:18:56,940
few lines of code and the rest of it you get from the third party
310
00:18:56,940 --> 00:19:00,595
location that that most people would install it from. I guess here,
311
00:19:00,595 --> 00:19:04,054
there's there's one difference to dev containers where your dev container
312
00:19:04,195 --> 00:19:08,034
probably ships the, say, Java tool chain that you use in your Bazel
313
00:19:08,034 --> 00:19:11,715
build. So you probably want if you use containers, you probably wanna configure your Bazel
314
00:19:11,715 --> 00:19:15,169
build so that it picks up the the local JDK installed in your
315
00:19:15,169 --> 00:19:18,950
container. But do you really wanna pick up the local JDK
316
00:19:19,090 --> 00:19:22,929
installed by Brew? I guess that's, like, that's okay. You can do
317
00:19:22,929 --> 00:19:26,565
it, but I think that that approach is just much more rare. So now you
318
00:19:26,565 --> 00:19:30,164
get into this problem where, hey. Everyone has the same JDK
319
00:19:30,164 --> 00:19:33,445
installed. Great. But is it the same one that you use in your build? Is
320
00:19:33,445 --> 00:19:37,125
it also the same one that's used for for exit configure actions
321
00:19:37,125 --> 00:19:40,470
because that's a different setting? Are you sure it's the same one? So I think
322
00:19:40,470 --> 00:19:44,170
that's that's one downside compared to to dev containers. But, of course, I mean, maybe
323
00:19:44,230 --> 00:19:48,070
if if there's really catches on people, ditch containers and they just do work
324
00:19:48,070 --> 00:19:51,610
or install everything in CI, I don't I don't know whether that's a thing
325
00:19:51,830 --> 00:19:55,675
or will be a thing. I I imagine it's possible. And Homebrew
326
00:19:55,735 --> 00:19:59,495
formulas, even though they're written in Ruby, do give you some
327
00:19:59,495 --> 00:20:03,095
degree of reproducibility. For example, I think the Bazel Homebrew
328
00:20:03,095 --> 00:20:06,695
formula pins to a JDK and then builds
329
00:20:06,695 --> 00:20:10,520
Bazel. But, you know, most people opt out of that and just get
330
00:20:10,520 --> 00:20:13,160
a home a a prebuilt binary if they can, and, yeah, then you have no
331
00:20:13,160 --> 00:20:16,840
you have no idea what what what j what JRE is gonna be
332
00:20:16,840 --> 00:20:20,600
used to run Bazel. Sorry. For Bazel, they they statically link a
333
00:20:20,600 --> 00:20:23,340
JRE into the binary, but you have this problem a lot of the places.
334
00:20:27,855 --> 00:20:31,615
Okay. So so yeah. So let's let's talk about, you
335
00:20:31,615 --> 00:20:35,055
know, if you're using Bazel for your project, then clearly Bazel can do this sort
336
00:20:35,055 --> 00:20:38,390
of shift all the way left kind of thing. And to be clear, we're talking
337
00:20:38,390 --> 00:20:42,230
about essentially not Bazel test or Bazel build, but specifically
338
00:20:42,230 --> 00:20:46,070
Bazel run. How do I Bazel run? Because I don't I don't want to declare
339
00:20:46,070 --> 00:20:49,510
the inputs if I'm just running a developer tool on my machine. Right? Like, I
340
00:20:49,510 --> 00:20:52,710
explicitly want it to have side effects. I'm not trying to populate the Bazel out
341
00:20:52,710 --> 00:20:55,715
tree. Tree. I just wanna do Bazel run of basically anything.
342
00:20:59,294 --> 00:21:02,254
And I I I wrote up a couple examples here. We'll we'll do a live
343
00:21:02,254 --> 00:21:06,095
demo in a minute, but there's a few, there's a few different kinds of binaries
344
00:21:06,095 --> 00:21:09,390
that you need. Right? So there's I love the way that your tool,
345
00:21:09,930 --> 00:21:13,610
combines all of these in one interface. So we've been talking a
346
00:21:13,610 --> 00:21:17,230
lot about binary targets for programs that you wrote yourself in the monorepo.
347
00:21:18,570 --> 00:21:21,885
But then you have the toolchain. So what if what if you simply need to
348
00:21:21,885 --> 00:21:25,485
run node or, like, you wanna do go go
349
00:21:25,485 --> 00:21:29,325
go mod something? Go get or yeah. Not go get, but go
350
00:21:29,325 --> 00:21:32,845
mod tidy. Go mod tidy is sort of the the prime example where even if
351
00:21:32,845 --> 00:21:36,670
you use Bazel, you still need the the go mod command to interact with
352
00:21:36,670 --> 00:21:40,190
your dependency log file. I mean, we we didn't reinvent the wheel. We just
353
00:21:40,190 --> 00:21:43,310
reduced the file. You need to manage the file in the same way you do
354
00:21:43,310 --> 00:21:46,830
with a with a native non Bazel Go project. So you need to run go
355
00:21:46,830 --> 00:21:50,530
mod tidy. But, so nothing terrible happens
356
00:21:50,670 --> 00:21:54,385
if you use GoMOT at a different slightly different version. But if
357
00:21:54,385 --> 00:21:57,904
if it's a sort of, like, very outdated version of
358
00:21:57,904 --> 00:22:01,664
GoMOT, then maybe it just doesn't work on your file and then just breaks
359
00:22:01,664 --> 00:22:05,399
while your build still succeeds or the other way around. Or if it's a
360
00:22:05,399 --> 00:22:09,080
much if it's a much newer version, it might try to update the Go version
361
00:22:09,080 --> 00:22:12,360
that appears at the top of the Go mod file. Right? Yeah. So all this
362
00:22:12,360 --> 00:22:15,419
kind of stuff you don't want. So you really want the Go mod that corresponds
363
00:22:15,559 --> 00:22:19,184
to the Go that you actually build your Go with. Yeah. And I think I
364
00:22:19,184 --> 00:22:22,705
think every package manager has exactly the same problem. I know for p n p
365
00:22:22,705 --> 00:22:25,424
m, if you use the wrong version, it'll give you a lock file. And at
366
00:22:25,424 --> 00:22:29,105
the at the top, the lock file is now a different version, you know, the
367
00:22:29,105 --> 00:22:31,905
format of the lock file. And I think for cargo, it's the same story,
368
00:22:33,420 --> 00:22:37,020
every language. And it's worth calling out here,
369
00:22:37,020 --> 00:22:40,780
like, this this was already solvable. All you had to
370
00:22:40,780 --> 00:22:44,620
do was run Bazel run slash at rules go slash slash go
371
00:22:44,620 --> 00:22:47,600
mod tidy instead of running go mod tidy. Why isn't that a solution?
372
00:22:48,255 --> 00:22:51,475
Yeah. Actually, this is how Bazel end was born.
373
00:22:52,095 --> 00:22:55,935
So, this was sort of the the the capstone to a
374
00:22:55,935 --> 00:22:58,595
project that had been in the work for a while where,
375
00:22:59,695 --> 00:23:03,480
when Bazel mod came out, I wrote the Go depths extension
376
00:23:03,480 --> 00:23:07,320
for for Gazelle. I mean, Gazelle is essentially the
377
00:23:07,320 --> 00:23:11,159
the Go package manager for rules go. So they really may from that
378
00:23:11,159 --> 00:23:14,600
perspective, should be one module that they aren't. So there is this extension that
379
00:23:14,600 --> 00:23:18,095
consumes your existing files and without any
380
00:23:18,095 --> 00:23:21,855
manual stop steps, turns them into the Go repositories that
381
00:23:21,855 --> 00:23:25,695
you need in your Bazel build. So that was sort of a developer UX
382
00:23:25,695 --> 00:23:29,455
upgrade where in workspace, you had to run some commands to generate some
383
00:23:29,455 --> 00:23:32,995
files and then do stuff. With BesaMon, you can just consume the file.
384
00:23:33,350 --> 00:23:36,950
Then the next step was to have a tool that allows
385
00:23:36,950 --> 00:23:40,630
you to reuse the Go tool chain to deal with this
386
00:23:40,630 --> 00:23:44,310
file and everything else. So this is how the at rules go slash
387
00:23:44,310 --> 00:23:48,025
slash go target was born. It was popular, but the problem is
388
00:23:48,025 --> 00:23:51,625
you can't wire this up in your IDE. Like, how how does
389
00:23:51,625 --> 00:23:55,305
your like, how do you make this code aware if while your go
390
00:23:55,305 --> 00:23:58,525
binary is actually, like, Bazel run this target,
391
00:23:58,909 --> 00:24:02,429
Maybe it works. Maybe you can put a child command in there, but in many
392
00:24:02,429 --> 00:24:05,950
places, you can't. In many places, you just need to put the place of the
393
00:24:05,950 --> 00:24:09,789
file and executable that it can run. And that's actually
394
00:24:09,789 --> 00:24:13,330
how I, came up with the idea for Bazel env. I wanted a more ergonomic
395
00:24:13,390 --> 00:24:16,275
way to run exactly add rules go go.
396
00:24:17,615 --> 00:24:21,215
Yeah. I I I used to have an alias, alias go equals
397
00:24:21,215 --> 00:24:24,815
Bazel run something something. But then, of course, if I'm in some other
398
00:24:24,815 --> 00:24:28,280
folder now, I just get an error that says, like, there's no workspace
399
00:24:28,340 --> 00:24:31,940
here. Yep. I think there's there's
400
00:24:31,940 --> 00:24:35,620
another but but but why is it not a solution to just say
401
00:24:35,620 --> 00:24:38,580
Bazel run on the command line? Like, I I think you're right for the editor
402
00:24:38,580 --> 00:24:42,345
that, like, that's a problem. What about for developers? Like, why can't they just run
403
00:24:42,345 --> 00:24:46,024
go mod tidy that way? They can, but there are
404
00:24:46,024 --> 00:24:49,865
numerous reasons why they maybe won't. For example, they can
405
00:24:49,865 --> 00:24:53,560
forget about it. They may not know about it. They
406
00:24:53,560 --> 00:24:56,780
may do it sometimes, but not the other time.
407
00:24:57,320 --> 00:25:01,080
Or they, run into this issue where, essentially,
408
00:25:01,080 --> 00:25:04,460
they they get fed up with with typing more than just g o.
409
00:25:04,760 --> 00:25:07,960
So they they set up their own alias, but then that alias breaks when they're
410
00:25:07,960 --> 00:25:11,775
in some other directory. And maybe at some point, somebody
411
00:25:11,775 --> 00:25:15,455
in the company wants to rename rules go to my
412
00:25:15,455 --> 00:25:18,815
rules go, and then, like, your muscle memory is
413
00:25:18,815 --> 00:25:22,655
outdated. So all, like, all these reasons, like, there is
414
00:25:22,655 --> 00:25:26,380
no like like, always with Dev UX, there is no hard reason.
415
00:25:26,380 --> 00:25:30,220
You can just say skill issue, type this long target, do it all
416
00:25:30,220 --> 00:25:33,820
the time, remember to do it. You can, but it's just not
417
00:25:33,820 --> 00:25:37,659
something that people will enjoy doing. Yeah. Yeah. I agree. I
418
00:25:37,659 --> 00:25:41,174
think, like, in practice, developers could, but they won't.
419
00:25:41,315 --> 00:25:44,674
Yep. And even the formulation for
420
00:25:44,674 --> 00:25:48,514
this, like, my my earlier solution to this problem was to do
421
00:25:48,514 --> 00:25:52,134
dot slash tools slash go, which is shorter, but still not good enough because
422
00:25:52,195 --> 00:25:55,980
exactly the same reason. You just won't do it. Yeah. And there's one more problem
423
00:25:55,980 --> 00:25:58,960
with this, which is if you're not in the top level directory,
424
00:25:59,820 --> 00:26:03,660
then it won't work. So, that's also something
425
00:26:03,660 --> 00:26:07,305
that that sort of some some monorepo tool sometimes have issues with that,
426
00:26:07,465 --> 00:26:10,365
essentially, they're expected to be run from the top level directory.
427
00:26:10,985 --> 00:26:14,665
And with Bazel, you usually stay in the top level directory
428
00:26:14,665 --> 00:26:18,185
because everything is sort of prefixed from the root. But when you don't, then you
429
00:26:18,185 --> 00:26:21,840
run into, like, an error message and then you need to spend precious
430
00:26:21,840 --> 00:26:25,679
seconds on figuring out what went wrong. Yeah. I mean, having written
431
00:26:25,679 --> 00:26:28,240
the other version of this tool, there's a lot of effort that just goes into
432
00:26:28,240 --> 00:26:32,080
having the right working directory largely because of a whole series of Bazel bugs
433
00:26:32,080 --> 00:26:35,795
that goes back seven years around, you know, why sometimes Bazel
434
00:26:35,795 --> 00:26:39,175
run will have a working directory off in the exec group, but sometimes it won't.
435
00:26:41,795 --> 00:26:45,475
And and the dash dash run under flag was a workaround, but you have to
436
00:26:45,475 --> 00:26:48,060
say you have to put an awkward value in there. Plus, until recently, when you
437
00:26:52,860 --> 00:26:56,400
discard. Yeah. So these tricks sort of didn't
438
00:26:56,460 --> 00:27:00,060
really work that well, which is also I mean, I I tried this first. So
439
00:27:00,060 --> 00:27:03,674
this this is also part of this developing story where the first thing
440
00:27:03,674 --> 00:27:07,514
I thought was why not just fix the run under analysis cache issue and
441
00:27:07,514 --> 00:27:11,355
just recommend that? And I did fix it because I felt
442
00:27:11,355 --> 00:27:15,054
this was worthwhile doing. But then I
443
00:27:15,115 --> 00:27:17,755
I sort of came up with the idea that the best thing to do would
444
00:27:17,755 --> 00:27:21,150
be to actually create a symlink to the thing you want to
445
00:27:21,150 --> 00:27:24,990
run-in a location that's as stable as possible. So that that
446
00:27:24,990 --> 00:27:28,830
would be sort of the the answer because then I mean, you can
447
00:27:28,990 --> 00:27:32,690
in that location, you can have a script that does all this path setup,
448
00:27:33,335 --> 00:27:37,015
CD ing, whatever is needed to run the tool. You just need it to be
449
00:27:37,015 --> 00:27:40,775
in a predictable location. And that's sort of the
450
00:27:40,775 --> 00:27:44,475
the trick that that is used in Basel. Yeah.
451
00:27:44,535 --> 00:27:48,200
Well, let's, maybe let's, do a quick demo because I think
452
00:27:48,200 --> 00:27:51,960
it's always better to show than tell. Although if you're listening audio
453
00:27:51,960 --> 00:27:55,720
only, we'll have to just, narrate what's going on here. So I'm in my projects
454
00:27:55,720 --> 00:27:59,320
folder on my machine. I've been adopting the new pattern. So I'm gonna go into
455
00:27:59,400 --> 00:28:02,985
we have a pretty big Bazel examples, monorepo that's on
456
00:28:03,605 --> 00:28:07,365
GitHub in the in the aspect build org. And so as soon as I enter
457
00:28:07,365 --> 00:28:11,125
this directory, we see this tool called Duran running. And it says it's
458
00:28:11,125 --> 00:28:14,905
loading a dot n v r c file, and it's exporting till the path.
459
00:28:17,130 --> 00:28:20,970
So I guess the first question is what is durev? Yeah.
460
00:28:20,970 --> 00:28:24,570
Durev so when I when I set up to, build
461
00:28:24,570 --> 00:28:28,205
Bazel env, I knew that I needed a certain primitive.
462
00:28:28,265 --> 00:28:32,105
I needed a way to add something to the path that should
463
00:28:32,105 --> 00:28:34,924
be as low effort and automated as possible.
464
00:28:35,865 --> 00:28:39,705
The first thing I considered is, wait, like, let me just hack something
465
00:28:39,705 --> 00:28:43,280
up. And then I took a step back and Googled for
466
00:28:43,420 --> 00:28:46,960
something like that, and I found that there is this very, very
467
00:28:47,100 --> 00:28:50,880
well scoped, very efficient, very well maintained,
468
00:28:51,500 --> 00:28:55,245
very extensible thing called Duranf, which is a tool that
469
00:28:55,245 --> 00:28:57,825
does I mean, it does many things, but it's literally
470
00:28:58,605 --> 00:29:02,365
engineered for this purpose. It has a file that you commit to your
471
00:29:02,365 --> 00:29:05,965
repo. It is a very small tool that you register with your
472
00:29:05,965 --> 00:29:09,730
shell, and your shell includes something like PowerShell, so it's very portable.
473
00:29:10,750 --> 00:29:14,050
And then whenever you change into a directory
474
00:29:14,910 --> 00:29:18,350
where a parent directory contains a dot n for a c file, that dot n
475
00:29:18,350 --> 00:29:21,890
for a c file is interpreted, can export environment variables,
476
00:29:22,294 --> 00:29:25,895
and those are added to your environment. So you can switch
477
00:29:25,895 --> 00:29:29,575
between directories and we will pick up the right environment for the particular
478
00:29:29,575 --> 00:29:33,335
project that you're in. And it's also reasonably secure,
479
00:29:33,335 --> 00:29:36,990
so it has this sort of trust on first use policy where the
480
00:29:36,990 --> 00:29:40,750
first time you enter a directory, you will see an arrow that says there is
481
00:29:40,750 --> 00:29:44,510
a dot net FERSE file, but I'm ignoring it. If you wanna use it, do
482
00:29:44,510 --> 00:29:48,029
allow this this file. So there is a manual step, but that
483
00:29:48,029 --> 00:29:51,805
manual step coming from an IT security perspective is sort of
484
00:29:51,805 --> 00:29:55,565
very well motivated. Yeah. That's actually I'll I'll let me
485
00:29:55,645 --> 00:29:58,365
well, maybe I'll show that for a second because I'm sure that people are curious
486
00:29:58,365 --> 00:30:01,950
what that actually looks like in practice for their engineers. So
487
00:30:02,110 --> 00:30:05,070
I'm just gonna run aspect in it, which is our little utility to make a
488
00:30:05,070 --> 00:30:08,909
new project. Pick some random language and just, like, skip through all
489
00:30:08,909 --> 00:30:12,750
these prompts. And then when I go into
490
00:30:12,750 --> 00:30:16,475
this folder that it was just created, this is what the error looks
491
00:30:16,475 --> 00:30:20,235
like. So Duran says, this folder is blocked, and it tells you
492
00:30:20,235 --> 00:30:23,115
exactly what to do. Right? So for your engineers, all they have to do is
493
00:30:23,115 --> 00:30:26,875
copy paste. Oops. They didn't do it. Copy paste what the what it says to
494
00:30:26,875 --> 00:30:30,580
do. Right. So now now this file is
495
00:30:30,580 --> 00:30:33,720
approved, and maybe we should take a look at the file for a second.
496
00:30:35,060 --> 00:30:38,340
So there's a few lines here of basically this is like,
497
00:30:38,580 --> 00:30:42,195
Durand inserts some symbols into the shell
498
00:30:42,195 --> 00:30:45,975
that it evaluates this in. So this path add is some function that they declared.
499
00:30:47,154 --> 00:30:50,855
It works in a bunch of different shells. So, like, I'm using ZSH
500
00:30:50,914 --> 00:30:53,875
because I'm using whatever default is on Mac, but it works with bash and fish
501
00:30:53,875 --> 00:30:57,330
and, like, a jillion others. Yeah. And then
502
00:30:57,330 --> 00:31:00,929
basically, you've done this clever thing of, like, making a stable output tree by
503
00:31:00,929 --> 00:31:04,549
using a transition into a different platform, which I thought was really cute.
504
00:31:04,610 --> 00:31:08,390
So this doesn't say k eight or or, you know, Darwin
505
00:31:08,450 --> 00:31:09,990
or anything over here, Mac OS.
506
00:31:12,385 --> 00:31:14,865
And then it does this and then you have this log error line. Right? Which
507
00:31:14,865 --> 00:31:17,665
is why we saw the second thing after I did Duran Valau. It gave me
508
00:31:17,665 --> 00:31:20,225
another command to run. But again, like, for your engineers, all they have to do
509
00:31:20,225 --> 00:31:24,070
is, like, blindly copy paste what it says to do, and that's gonna run
510
00:31:24,070 --> 00:31:27,669
the tool. And after this point, my path will be
511
00:31:27,669 --> 00:31:30,010
correct. So we see this doing a bunch of fetching.
512
00:31:31,429 --> 00:31:35,110
So in this case, I picked JavaScript. So it's already and so so then what
513
00:31:35,110 --> 00:31:38,135
you do, which is which is cute, you give you give some, like, health statistics.
514
00:31:38,195 --> 00:31:40,914
Like, yeah. Okay. Things seem to be set up correctly. And then it lists everything
515
00:31:40,914 --> 00:31:44,674
that it was just added to the path, which is great because now I know
516
00:31:44,674 --> 00:31:47,174
exactly which tools I can run and also where they came from.
517
00:31:50,110 --> 00:31:52,750
Now I did notice that, like, the order of things on the path is not
518
00:31:52,750 --> 00:31:55,710
always what I want. So if I do which node, I'm still getting something that's
519
00:31:55,710 --> 00:31:59,010
installed by NVM on my machine and not not the one here.
520
00:32:00,270 --> 00:32:03,170
Is it possible to do, like, a path prepend instead?
521
00:32:04,225 --> 00:32:08,065
Oh, okay. That's, that didn't
522
00:32:08,065 --> 00:32:11,825
happen when I tried it. Okay. I'm not not sure why. Yeah. This should
523
00:32:11,825 --> 00:32:15,505
not happen. I don't know. That that is the the bug.
524
00:32:15,505 --> 00:32:17,910
Life and camp. Wait. Wait. But Oh, yeah. No. It is it is first on
525
00:32:17,910 --> 00:32:20,490
the path. Okay. Well, we're doing some live debugging here.
526
00:32:23,030 --> 00:32:25,929
Can you k. Well,
527
00:32:27,030 --> 00:32:30,875
maybe it's not executable. I I don't know. That that sounds weird.
528
00:32:31,175 --> 00:32:34,535
Okay. That has never happened to me, something I can claim. I think it is
529
00:32:34,535 --> 00:32:38,375
the current version. Alright. Well, hopefully, that's fixed by the time anybody tries it
530
00:32:38,375 --> 00:32:40,775
or maybe we figure out what I'm doing wrong here. We'll do it. Yeah. We
531
00:32:40,775 --> 00:32:44,540
can do it after the recording. There's a bunch of these are definitely
532
00:32:44,540 --> 00:32:47,660
working. So if I do which p n p m that points into this,
533
00:32:48,060 --> 00:32:51,900
special directory that you've placed under Bazel, a Bazel out tree. And
534
00:32:51,900 --> 00:32:55,740
this is this is the critical one. Right? Because, developers interacting with
535
00:32:55,740 --> 00:32:58,835
the package manager should not have to learn a new Bazel specific tool. I want
536
00:32:58,835 --> 00:33:02,675
them to still run PNPM if they're doing JavaScript. That's how you that's how you
537
00:33:02,675 --> 00:33:05,335
add a package and update the lock files and so on.
538
00:33:06,435 --> 00:33:09,635
So, yeah, that's that's really key. Now I also wanted to mention,
539
00:33:10,770 --> 00:33:13,410
some of these other things in the output, and thank you for jumping on this
540
00:33:13,410 --> 00:33:17,250
rules multi tool thing. So rules multi tool comes from Mark and the team
541
00:33:17,250 --> 00:33:21,090
at Therum LP, and, what that
542
00:33:21,090 --> 00:33:24,690
does is it puts a file it it creates a lock
543
00:33:24,690 --> 00:33:28,255
file, but the lock file is basically a bunch of download,
544
00:33:29,275 --> 00:33:33,115
instructions. So it looks very much like an HTTP archive. Right. So
545
00:33:33,115 --> 00:33:36,475
it says this one's HTTP file. If you come from Bazel and, you know, this
546
00:33:36,475 --> 00:33:39,929
one's an archive. So I can grab Terraform from this location. Here's its
547
00:33:39,929 --> 00:33:43,210
SHA, and this is just wiring directly to the Bazel downloader and then hooks up
548
00:33:43,210 --> 00:33:45,929
a tool chain for each tool so you only have to download the ones you
549
00:33:45,929 --> 00:33:49,529
need. And so what you've done is made it super
550
00:33:49,529 --> 00:33:53,289
easy to, to get just to to read that
551
00:33:53,289 --> 00:33:57,115
whole lock file, and we bring it in to the Bazel
552
00:33:57,115 --> 00:34:00,955
env, Bazel env rule here. Obviously, I didn't I
553
00:34:00,955 --> 00:34:03,755
I skipped running the format around this file which is why there's so much extra
554
00:34:03,755 --> 00:34:07,480
white space. But you can you can
555
00:34:07,480 --> 00:34:11,239
grab the you can grab this dictionary from at multitool, which Fabian
556
00:34:11,239 --> 00:34:14,859
added, and then you just stick that on the end of your Bazel env incantation.
557
00:34:15,080 --> 00:34:18,139
And now all of those tools are gonna be on your path, which is
558
00:34:18,760 --> 00:34:21,260
incredibly ergonomics. So now if I do which Terraform,
559
00:34:22,815 --> 00:34:25,955
I have oop. That's another one that's not quite what we wanted to see.
560
00:34:29,534 --> 00:34:31,074
Which Buildozer.
561
00:34:33,534 --> 00:34:36,335
There we go. So, like, Buildozer is coming from the basal end of the way
562
00:34:36,335 --> 00:34:39,270
we expect. Something's weird with your channel. I don't know.
563
00:34:40,690 --> 00:34:43,549
Yeah. We'll figure it
564
00:34:44,069 --> 00:34:47,750
out. Probably may maybe not right now. Yeah.
565
00:34:47,969 --> 00:34:51,510
We've been we've been chatting for, thirty thirty three minutes
566
00:34:51,650 --> 00:34:55,324
and and don't wanna exhaust the attention span of
567
00:34:55,324 --> 00:34:59,005
our listeners. So, so this is
568
00:34:59,005 --> 00:35:02,845
great. I guess, what else have we not already pointed out about this? I guess,
569
00:35:02,845 --> 00:35:06,090
like, also the fact that the tool chain is at a stable path. So, can
570
00:35:06,170 --> 00:35:08,730
can you explain, like, what how would I how would I make use of this
571
00:35:08,730 --> 00:35:12,250
this bit? Yeah. So the idea is so what what you said before, this
572
00:35:12,250 --> 00:35:15,770
uses, a transition to get you a stable path. And,
573
00:35:16,010 --> 00:35:19,690
the idea is that this path is also stable across platforms. So right
574
00:35:19,690 --> 00:35:23,515
now, Bazelon supports Linux
575
00:35:23,515 --> 00:35:27,275
and macOS. I want to support Windows at some point,
576
00:35:27,275 --> 00:35:30,635
but, like, Windows is always extra special and you need to rewrite all your shell
577
00:35:30,635 --> 00:35:33,835
scripts and bad scripts and nobody wants to do that. So,
578
00:35:34,315 --> 00:35:38,000
we'll see. But when I say cross platform right now, I mean,
579
00:35:38,000 --> 00:35:41,600
Linux and macOS. This path will be stable. And, for example, this is a
580
00:35:41,600 --> 00:35:45,140
path that you can now copy into your Versus code
581
00:35:45,520 --> 00:35:49,280
or other IDE, configuration and use as the
582
00:35:49,280 --> 00:35:52,665
source of truth. So, ideally, you have, like, for example, if you're an
583
00:35:52,665 --> 00:35:56,505
IntelliJ person like I am, then, you have a Bazel plug in and it
584
00:35:56,505 --> 00:36:00,345
always uses the correct toolchain. Great. But, if you're in Versus
585
00:36:00,345 --> 00:36:04,185
Code and you don't have that or, like, you just run some custom rules
586
00:36:04,185 --> 00:36:07,790
or custom things that just don't have first year
587
00:36:07,790 --> 00:36:11,630
Bazel, support, you can just plug in this stable path. It will pick up
588
00:36:11,630 --> 00:36:14,530
the toolchain that Bazel uses. Everything is sort of okay.
589
00:36:15,310 --> 00:36:18,750
That's the the idea there. And,
590
00:36:19,365 --> 00:36:22,905
yeah. This this path will be stable and this is also sort of motivated
591
00:36:22,965 --> 00:36:26,645
by some rules go issues, we had in the past where with
592
00:36:26,645 --> 00:36:30,485
Bazel mod, these paths can get really ugly. Like, if you if you use sort
593
00:36:30,485 --> 00:36:33,960
of the target of these siblings instead of these
594
00:36:33,960 --> 00:36:37,800
beautiful paths, they'll be very unstable and weird. So you wouldn't wanna put that
595
00:36:37,800 --> 00:36:41,640
into a file, like a configuration file you commit to your repo. Yeah. They
596
00:36:41,640 --> 00:36:45,400
they contain canonical repository names and text like tildes that are now
597
00:36:45,400 --> 00:36:48,380
pluses and things like that. Right? Yeah. Yeah. That's why.
598
00:36:51,055 --> 00:36:53,975
So there's a fourth category. Let me just run through. So we first of all,
599
00:36:53,975 --> 00:36:57,175
we talked about, tools that you write in your repo and want to distribute to
600
00:36:57,175 --> 00:37:00,234
your engineers. Tool chains,
601
00:37:01,660 --> 00:37:04,860
things that come from rules multi tool or any other sort of, you know, fetch,
602
00:37:04,860 --> 00:37:08,620
like HTTP archive style thing. There's a fourth category which is tools
603
00:37:08,620 --> 00:37:12,460
that come from a package manager. Let me do let
604
00:37:12,460 --> 00:37:15,340
me run Bazel env over here again one more time because I think we have
605
00:37:15,340 --> 00:37:18,895
a couple of these. So like in this example, I've got
606
00:37:18,895 --> 00:37:22,335
Angular. Right? And Angular comes from from
607
00:37:22,335 --> 00:37:26,015
NPM. And I don't have to do, like and so what I want to
608
00:37:26,015 --> 00:37:29,775
happen is I want Bazel to do its normal thing of fetching
609
00:37:29,775 --> 00:37:33,270
NPM packages and then make this binary available. So if I if I do which
610
00:37:33,270 --> 00:37:37,030
n g, I'm getting the one managed by Bazel n. But then if I look
611
00:37:37,030 --> 00:37:39,770
at the tools, build file,
612
00:37:41,590 --> 00:37:45,285
so I've I've put these into a list called n g, and then, n
613
00:37:45,285 --> 00:37:49,045
g comes from, you know, in this case, we have what rules JS
614
00:37:49,045 --> 00:37:52,805
provides to give you an incantation to reach into the bin field of
615
00:37:52,805 --> 00:37:56,565
each package JSON file. If you're doing Python, you can do the same
616
00:37:56,565 --> 00:38:00,320
thing with the console scripts to grab any console script from a Python
617
00:38:00,320 --> 00:38:03,440
tool. So, like, I did this with copier, for example, which is a sort of
618
00:38:03,440 --> 00:38:05,380
a scaffolding stamp out kind of tool.
619
00:38:07,920 --> 00:38:10,100
And then and then here I just have,
620
00:38:12,045 --> 00:38:15,665
you know, here I'm pointing into those. Right? So slash slash tools colon something,
621
00:38:16,205 --> 00:38:20,045
is gonna get installed into my path by Bazel env. And so, you
622
00:38:20,045 --> 00:38:22,445
know, whatever. I put a little bit of Syntax sugar around it, but I have
623
00:38:22,445 --> 00:38:25,265
a couple of different, binaries that I grabbed from NPM.
624
00:38:26,690 --> 00:38:30,390
I've also got an example of a Go tool, called Scaffold
625
00:38:30,609 --> 00:38:34,289
where we did the same thing. Mhmm. Maybe I'll show that
626
00:38:34,289 --> 00:38:38,130
from our own monorepo. If I run Bazel
627
00:38:38,130 --> 00:38:41,945
env over here, I wonder how fast this will be. So we do see a
628
00:38:41,945 --> 00:38:45,065
bunch of fetches and I know one thing that's up and coming or or when
629
00:38:45,065 --> 00:38:47,625
you find some time, we can make the thing a little bit more lazy and
630
00:38:47,625 --> 00:38:51,305
fetch fewer tools that aren't gonna be needed. Yeah. That's
631
00:38:51,305 --> 00:38:54,900
that's a difficult I mean, I I I had this done
632
00:38:55,360 --> 00:38:58,500
in some way half a year ago or something.
633
00:38:59,840 --> 00:39:03,600
It's actually pretty hard to get a good user experience because as
634
00:39:03,600 --> 00:39:07,435
soon as you say do this lazily, what you really get
635
00:39:07,435 --> 00:39:11,135
yourself into is, say, your Versus
636
00:39:11,275 --> 00:39:15,115
code Go, please, may now trigger Bazel whenever
637
00:39:15,115 --> 00:39:18,875
it invokes the Go binary. Yes. Which is weird because it just takes
638
00:39:18,875 --> 00:39:21,275
a long time, and you're like, how is it possibly taking so long? Because all
639
00:39:21,275 --> 00:39:24,580
of the standard all of the progress messages are hidden. Yeah. And then you blame
640
00:39:24,580 --> 00:39:28,420
your build engineer and the blame build engineer blames me and then I wish I'd
641
00:39:28,420 --> 00:39:32,180
never submitted that PR. Yeah. Just joking, but
642
00:39:32,180 --> 00:39:35,935
it's actually pretty hard and it's not I mean, it doesn't even stop
643
00:39:35,935 --> 00:39:39,694
there. So when you say lazy and you may be okay with this
644
00:39:39,694 --> 00:39:43,155
taking a bit longer the first time you you run a tool, that's okay.
645
00:39:43,454 --> 00:39:47,295
But what can happen is that it runs Bazel with slightly different
646
00:39:47,295 --> 00:39:51,119
command line arguments than you did last time. And now it's
647
00:39:51,119 --> 00:39:54,960
slow even though you ran the tool before. And now you come back to
648
00:39:54,960 --> 00:39:58,800
your Bazel build, and now your Bazel build is also slow simply because
649
00:39:58,800 --> 00:40:02,640
you flip flop between, configuration settings. Yep.
650
00:40:02,640 --> 00:40:06,295
So as part of this follow-up. I should remind everyone to switch your
651
00:40:06,295 --> 00:40:10,055
Bazel RC file to use common rather than build because you
652
00:40:10,055 --> 00:40:13,275
may not realize that build is not inherited by query or fetch.
653
00:40:13,495 --> 00:40:17,175
Yeah. And yeah. If you if you do if you set something
654
00:40:17,175 --> 00:40:20,760
weird on run, you may also get into this issue because then we use
655
00:40:20,760 --> 00:40:24,520
Bazel run to run the tool lazily. Yeah. You may also pick up different
656
00:40:24,520 --> 00:40:28,120
flags. So I have I I do think what what what we currently do here
657
00:40:28,120 --> 00:40:31,800
is is is nicer than it than than those
658
00:40:31,800 --> 00:40:35,160
downsides for sure, which is that, like, by the time I've finished doing this Bazel
659
00:40:35,160 --> 00:40:38,974
run, all these tools are ready to execute, and they will behave similarly to how
660
00:40:38,974 --> 00:40:41,855
they would if I had to solve them natively, which is to say they immediately
661
00:40:41,855 --> 00:40:45,375
start. Like, there's no there's no cold start time. Yeah. So there's just, like, a
662
00:40:45,375 --> 00:40:49,055
very, very, very short shell shell script that ends up with an exec of the
663
00:40:49,055 --> 00:40:52,589
tool. So it's it's literally the same thing as running them basically effectively.
664
00:40:53,450 --> 00:40:56,730
Yeah. And I did I did wanna point out this one. So scaffold, for example,
665
00:40:56,730 --> 00:41:00,170
is just a go package, and it and it exposes a binary. And
666
00:41:00,170 --> 00:41:03,790
because, the Bazel gazelle repo has the go repository
667
00:41:04,010 --> 00:41:07,835
rule that knows how to expose binaries. I have it labeled to refer
668
00:41:07,835 --> 00:41:10,795
to this tool. I don't have to write any special build file syntax. I can
669
00:41:10,795 --> 00:41:13,615
just I can just point directly to this tool that Go is fetching.
670
00:41:14,555 --> 00:41:18,155
So I guess at that point, I should I should briefly mention what this dollar
671
00:41:18,155 --> 00:41:21,920
syntax is you see below. Because one I mean, one
672
00:41:21,920 --> 00:41:24,559
way to add a tool is you just point it to a label of an
673
00:41:24,559 --> 00:41:28,240
executable and you run it. But the other way is you can
674
00:41:28,240 --> 00:41:32,079
refer to so called make variables that are often defined by
675
00:41:32,079 --> 00:41:35,615
tool chains. So if you add a tool chain to the tool chains
676
00:41:35,615 --> 00:41:38,755
attribute, you can then also use its make variables,
677
00:41:39,455 --> 00:41:42,975
and that's necessary to get the path. Like, we have a note tool
678
00:41:42,975 --> 00:41:45,555
chain, and now you can use this dollar
679
00:41:46,500 --> 00:41:50,180
parenthesis note path to actually get the path to the note binary
680
00:41:50,180 --> 00:41:53,780
within that tool chain out of the tool chain. And the nice thing is that
681
00:41:53,780 --> 00:41:57,460
Bazel env automatically figures out which make variable refers to
682
00:41:57,460 --> 00:42:00,740
which tool chain, so we'll actually stage sort of the correct set of files for
683
00:42:00,740 --> 00:42:04,445
this thing to work. It hadn't even occurred to me, but I was just working
684
00:42:04,445 --> 00:42:07,085
on that tar dot b z l that I showed at the beginning, and it
685
00:42:07,085 --> 00:42:10,845
exposes the b s d tar as a make variable. And so now I
686
00:42:10,845 --> 00:42:13,665
could have tar be always the same one that Basil's gonna use.
687
00:42:14,580 --> 00:42:18,340
Yeah. And lot lot basically, any rule set that exposes a tool
688
00:42:18,340 --> 00:42:21,700
chain ought to expose this you have to do this trick where you expose a
689
00:42:21,700 --> 00:42:25,300
resolved version of the tool chain because in a label sent, context like this,
690
00:42:25,300 --> 00:42:29,055
Bazel doesn't bother to run it through the tool chain resolution mechanism. Yeah. I
691
00:42:29,055 --> 00:42:32,575
think the the the there's a single Bazel rule that's sort of hard
692
00:42:32,575 --> 00:42:36,415
coded to allow dynamic tool chains, and that's GAN rule. So GAN rule
693
00:42:36,415 --> 00:42:40,095
has this tool chains attribute where you can sort of by adding
694
00:42:40,095 --> 00:42:43,800
tool chain labels to your build file, you're adding toolchains
695
00:42:43,800 --> 00:42:47,500
to the resolution for the general rule, but that's a, like,
696
00:42:47,880 --> 00:42:51,560
a privileged, native rule, and this is not something that baselines can
697
00:42:51,560 --> 00:42:55,400
do. Yeah. Yeah. General is weird and should
698
00:42:55,400 --> 00:42:58,815
be replaced by run binary, but then you you do lose a lot of those.
699
00:42:59,775 --> 00:43:03,135
Yeah. Yeah. The layering violations are sometimes in your
700
00:43:03,135 --> 00:43:06,895
favor. I guess. I I think here, we should probably have
701
00:43:06,895 --> 00:43:10,734
have a shout out to rules s h, which is
702
00:43:10,734 --> 00:43:14,350
a pretty nice project that gives you a hermetic way
703
00:43:14,350 --> 00:43:18,190
to access tools in gen rules. So it's sort of like an analog
704
00:43:18,190 --> 00:43:21,790
of basal length, if you want, for for gen rules where you get make
705
00:43:21,790 --> 00:43:25,565
variables for common POSIX tools, and then you can reference them, which is
706
00:43:25,565 --> 00:43:29,245
also nice. And there's also the the APE module. I think it's maintained by
707
00:43:29,245 --> 00:43:33,005
ARM, which gives you these if you haven't seen it, it's
708
00:43:33,005 --> 00:43:36,525
sort of mind blowing, but there's this, actually portable
709
00:43:36,525 --> 00:43:40,360
executable format, which allows you to have a binary that literally runs
710
00:43:40,360 --> 00:43:44,040
on any platform. Like, it runs on Windows. The same binary runs on MacOS and
711
00:43:44,040 --> 00:43:47,800
Linux. And APE has a bunch of core utils and other
712
00:43:47,800 --> 00:43:51,560
other, like, common UNIX tools as APE binaries. So if you
713
00:43:51,560 --> 00:43:55,085
want, you can, just add add ape slash slash
714
00:43:55,085 --> 00:43:58,685
colon something to your Bazel end, the Bazel, or you
715
00:43:58,685 --> 00:44:02,145
can rule and and use that tool hermetically, which is kind of nice.
716
00:44:03,325 --> 00:44:07,165
Yep. Yeah. You know, interestingly, the the APE tool chain comes
717
00:44:07,165 --> 00:44:09,960
from this project code called Cosmopolitan. Oopsie.
718
00:44:10,580 --> 00:44:14,360
For an engineer named Justine Tunney, who I used to work with at Google. And,
719
00:44:16,180 --> 00:44:19,940
she wrote a a design doc explaining why we could never make Bazel
720
00:44:19,940 --> 00:44:23,320
work properly with node packages because it was a quadratic behavior.
721
00:44:24,255 --> 00:44:27,855
And I always felt like, that was that
722
00:44:27,855 --> 00:44:31,535
was a a high bar to meet, because she's, like, so
723
00:44:31,535 --> 00:44:35,055
brilliant. And yeah. I mean, like, basically, the things she did there of, like, can
724
00:44:35,055 --> 00:44:38,655
you wedge enough headers at the top of a file such that you can
725
00:44:38,655 --> 00:44:42,320
trick elf binaries and like, you know, you can trick a Linux at run time
726
00:44:42,320 --> 00:44:44,720
into thinking it looks like an elf binary, but you can trick a Mac into
727
00:44:44,720 --> 00:44:46,500
thinking it looks like a Darwin binary.
728
00:44:48,800 --> 00:44:52,628
Which, yeah, takes me back to the JavaScript days. There was something called
729
00:44:52,628 --> 00:44:55,635
UMD, which was universal module definition, which is exactly the same thing. Like, what incantation?
730
00:44:55,775 --> 00:44:58,895
And, like, when you look at it, you're like, this is horrifying. I can't believe
731
00:44:58,895 --> 00:45:02,415
it happens to work. So yeah. Anyway, that's
732
00:45:02,415 --> 00:45:05,455
definitely a a good shout out for that project. And that's useful with with with
733
00:45:05,455 --> 00:45:07,670
Bizzle n because you can,
734
00:45:09,410 --> 00:45:13,250
you can grab a whole bunch of, you know, basic Unix utilities. For the most
735
00:45:13,250 --> 00:45:17,090
part, well, no. I take it back actually. It's it's it's quite common
736
00:45:17,090 --> 00:45:20,875
that, like, you have the wrong, you know, developers are running a shell script
737
00:45:20,875 --> 00:45:23,675
that you checked in and they have a different you know, they're on a Mac
738
00:45:23,675 --> 00:45:27,434
and so sed needs dash I dot back in order to do in place
739
00:45:27,434 --> 00:45:30,714
replacements. But on Linux, you can just do sed dash I. And so somebody checks
740
00:45:30,714 --> 00:45:34,095
in a script with sed dash I and breaks people. And if you were using,
741
00:45:35,000 --> 00:45:37,400
if you were getting said from somewhere like that and putting it in your environment,
742
00:45:37,400 --> 00:45:41,160
you wouldn't have that problem. I guess that's that's one other thing
743
00:45:41,160 --> 00:45:44,839
to mention, which is that the I mean, the the idea
744
00:45:44,839 --> 00:45:48,220
of Bazel or one of the the key strength of Bazel is that
745
00:45:48,805 --> 00:45:52,565
build tools should be part of your repo and they should be built from
746
00:45:52,565 --> 00:45:56,025
source and they should be as easy to use as, like,
747
00:45:56,485 --> 00:46:00,165
precompiled tools. And with Baselines for Bazel, this
748
00:46:00,165 --> 00:46:04,010
sort of extends to your monorepo as well because if you're fed up
749
00:46:04,010 --> 00:46:07,849
with writing your tools as shell scripts because they can
750
00:46:07,849 --> 00:46:11,530
just be run, if you just point baseline
751
00:46:11,530 --> 00:46:15,290
for bezel to some target in your repo, if that thing is a
752
00:46:15,290 --> 00:46:18,925
Go binary that has a Java binary runtime dependency,
753
00:46:19,385 --> 00:46:22,505
it will just work. It will sort of be built when you build the base
754
00:46:22,505 --> 00:46:26,125
length, and then you can just run it from the command line. It works. So
755
00:46:26,185 --> 00:46:29,960
if you are familiar with another language, maybe a compiled
756
00:46:29,960 --> 00:46:33,720
language, it should just be as convenient to use as a
757
00:46:33,720 --> 00:46:37,400
shell script for the user. I mean, for the writer, it's always like that
758
00:46:37,400 --> 00:46:41,160
usually. I mean, I don't know that many people that really like to write long
759
00:46:41,160 --> 00:46:44,935
shell scripts. Maybe they exist, but I'm not one of them.
760
00:46:45,075 --> 00:46:48,355
Okay. Well, there you go. But, yeah, if you write a tool in some other
761
00:46:48,355 --> 00:46:52,035
language you're more comfortable in and that language happens to be compiled, then,
762
00:46:52,355 --> 00:46:56,180
this sort of makes up for this for this downside. Yeah. And it's often the
763
00:46:56,180 --> 00:46:59,460
case that there's some library code that's part of your product, but you also need
764
00:46:59,460 --> 00:47:03,220
to have available when you're doing development work, and it's actually great to be able
765
00:47:03,220 --> 00:47:06,740
to use the language that you ship in to also make a version of that
766
00:47:06,740 --> 00:47:10,180
tool so you can run a main, give your developers more power
767
00:47:10,180 --> 00:47:13,444
tools. I guess there's this one controversial,
768
00:47:13,984 --> 00:47:17,444
but I I don't think this is really controversial and this will remain controversial.
769
00:47:18,865 --> 00:47:22,405
Some people very much like to have a single version of everything
770
00:47:22,625 --> 00:47:26,220
in their repo. So if you have
771
00:47:26,220 --> 00:47:29,980
some, whatever, like, diff library that you use in production, you
772
00:47:29,980 --> 00:47:33,660
also wanna use that exact same version for your tools because then you
773
00:47:33,660 --> 00:47:37,180
only have to worry about a single version or something. But then there is this
774
00:47:37,180 --> 00:47:40,380
other camp of people that says, this is terrible. I want to be able to
775
00:47:40,380 --> 00:47:44,115
be on legacy Java six version of this
776
00:47:44,115 --> 00:47:47,635
dependency for prod because I never wanna update it. Otherwise, everything
777
00:47:47,635 --> 00:47:51,175
breaks. But for development, I want to use this very, very modern version.
778
00:47:51,795 --> 00:47:55,575
So tools and prod dependencies should never share a dependency
779
00:47:55,635 --> 00:47:58,080
tree. And, I mean,
780
00:47:59,260 --> 00:48:02,940
both have arguments on their side. I'm firmly in one camp. I won't say which
781
00:48:02,940 --> 00:48:06,619
one, but, it happens. So if you really want
782
00:48:06,619 --> 00:48:10,415
your separate dependency tree, some rule sets
783
00:48:10,415 --> 00:48:14,195
allow this, some others rely on there is a
784
00:48:14,415 --> 00:48:18,115
a flag in in Bessemer mode called, experimental isolated dependencies,
785
00:48:18,735 --> 00:48:22,495
which sort of allows you to use any kind of model extension to
786
00:48:22,495 --> 00:48:25,880
give you this non shared separate closure of dependencies.
787
00:48:26,500 --> 00:48:30,339
If you want this, because Bazel and just accepts targets, you can do
788
00:48:30,339 --> 00:48:33,300
it in whatever way you you see fit and then fit it in. So it's
789
00:48:33,300 --> 00:48:36,755
not a package manager. It doesn't force you to to follow this style of,
790
00:48:36,995 --> 00:48:40,755
using a shared version of a dependency for everything. Yeah. I
791
00:48:41,475 --> 00:48:45,315
for Go is, for example, there's a nice, bit of documentation on how
792
00:48:45,315 --> 00:48:48,355
to use the the tools pattern to put a you know, you basically put sort
793
00:48:48,355 --> 00:48:52,040
of an empty side effect, Go source file just so that
794
00:48:52,040 --> 00:48:55,560
GoMontidee can find it, and then you have a separate closure for tools that doesn't
795
00:48:55,560 --> 00:48:58,860
have to use the same constraint solution as the rest of the repository.
796
00:48:59,080 --> 00:49:01,980
Yeah. Although and this is sort of the the the fun part.
797
00:49:02,875 --> 00:49:06,635
Go 1.24 has first party Go
798
00:49:06,635 --> 00:49:10,155
tool support. So there is no Go tool command that you can run, and it
799
00:49:10,155 --> 00:49:13,835
runs a tool that you first added to your Gomod file. And this
800
00:49:13,835 --> 00:49:17,530
uses the approach where it unifies dependencies with your prod dependencies, And
801
00:49:17,530 --> 00:49:21,290
there's a very, very long hack and use threat
802
00:49:21,290 --> 00:49:25,130
on this where both camps clash and one are like, this is the right
803
00:49:25,130 --> 00:49:28,010
way to do things and the others are, this is terrible and you should never
804
00:49:28,010 --> 00:49:31,585
do it and well, there's no resolution. But, yeah, What we do works with both
805
00:49:31,585 --> 00:49:35,425
camps of Yes. Yes. I think yeah. The the takeaway is
806
00:49:35,425 --> 00:49:37,765
that for Bazel end, you are free to pick either.
807
00:49:39,745 --> 00:49:43,285
And I I'm glad you mentioned this. We didn't we didn't explicitly
808
00:49:43,425 --> 00:49:47,270
say earlier because it does fetch all the tools eagerly,
809
00:49:47,490 --> 00:49:50,450
that might be slow if you have a bunch of different projects in your repo.
810
00:49:50,450 --> 00:49:54,290
You are free to have multiple, dot e n v r c files and
811
00:49:54,290 --> 00:49:57,650
multiple Bazel env targets. And so you could have each subdirectory in your
812
00:49:57,650 --> 00:50:01,235
monorepo have a different path environment, and that's great. So now you just tell your
813
00:50:01,235 --> 00:50:04,835
engineer CD into your folder. You'll you'll and run the thing. You'll have the tools
814
00:50:04,835 --> 00:50:08,355
set up for your project, and you don't have to agree. You don't even have
815
00:50:08,355 --> 00:50:11,155
to agree on which versions of tools you use with projects that live in other
816
00:50:11,155 --> 00:50:13,980
folders in the repo. You're free to do that. Or, I mean, some some mono
817
00:50:13,980 --> 00:50:17,820
repos have this, this convention where the top level folder is
818
00:50:17,820 --> 00:50:21,500
sort of, bifurcated by language. So you have a top level Java directory, a top
819
00:50:21,500 --> 00:50:25,280
level Kotlin, a top level Go or cc or whatever directory.
820
00:50:25,340 --> 00:50:28,485
It also works pretty nicely with that because then if you if you CC into
821
00:50:28,485 --> 00:50:32,085
the CC or if you CD into the CC directory, you just
822
00:50:32,085 --> 00:50:35,865
get your c plus plus tool chain and these tools. You
823
00:50:35,925 --> 00:50:39,660
don't have to download Node, which Yep. Right. Please Even
824
00:50:39,660 --> 00:50:43,260
even if you do have your monorepo structured by team or by, you
825
00:50:43,260 --> 00:50:47,020
know, organization or project, you still probably use a subset of the languages in your
826
00:50:47,020 --> 00:50:50,000
project, and you can just install that subset instead of everything.
827
00:50:53,305 --> 00:50:56,905
Awesome. Thank you so much for coming to talk about this. I've, that
828
00:50:56,905 --> 00:51:00,045
was that was really very thorough, and, I had a good time
829
00:51:00,425 --> 00:51:04,185
revisiting all of the the the historical design decisions and the constraints around
830
00:51:04,185 --> 00:51:08,010
this. And and it's exciting that you I think you've, like, really found, like, the
831
00:51:08,010 --> 00:51:11,630
right way to do this. I hope. I mean, I don't know. But
832
00:51:12,010 --> 00:51:15,630
at least it's something that, I don't know, I like to use.
833
00:51:16,090 --> 00:51:18,570
Yeah. Well, I like to use it too. So I think everybody else should give
834
00:51:18,570 --> 00:51:22,315
it a try. Yeah. So you just and when you wanna try it I mean,
835
00:51:22,315 --> 00:51:26,075
I showed Aspect in it as one example that you can certainly it's
836
00:51:26,075 --> 00:51:29,595
pretty easy to install, so just just go check out the instructions that Fabian wrote
837
00:51:29,595 --> 00:51:32,015
on Bazelenv.bzlrepo.
838
00:51:33,930 --> 00:51:37,770
And, yeah, thank you for writing it. Yeah. Thanks for having me
839
00:51:37,770 --> 00:51:41,310
on the show. Yeah. Absolutely. Well, I can't wait to have you back again,
840
00:51:41,721 --> 00:51:45,421
Next time we have something awesome like this to talk about. Thanks, Bobby. We'll see.
841
00:51:45,561 --> 00:51:46,061
Yeah.