David Lattimore – Faster Linker, Faster Builds

1
00:00:00,050 --> 00:00:08,060
Tim: Sometimes people stop everything
else and decide to focus on
that one thing that must change.
2
00:00:09,680 --> 00:00:11,599
David Latimore is one of those people.
3
00:00:12,229 --> 00:00:19,759
He's currently on sabbatical working on
a linker that will speed up builds for
every rust project that compiles to Linux.
4
00:00:19,970 --> 00:00:24,560
It's ambitious, it's audacious,
and it is fascinating to listen to.
5
00:00:25,303 --> 00:00:28,873
Oh, and by the way, David is
going to be talking at the Rust
6
00:00:28,873 --> 00:00:32,181
Forge Conference in New Zealand
during the last week of August.
7
00:00:32,991 --> 00:00:35,781
He's gonna be talking
about Wild, his new linker.
8
00:00:36,471 --> 00:00:45,451
And if you would like to attend
with a 15% discount, please
visit rustforgeconf.com/compose.
9
00:00:54,489 --> 00:00:56,969
David: it's, exciting to be here
and I love talking about linkers
10
00:00:57,049 --> 00:00:59,149
I'm sure we can go into some
interesting technical details.
11
00:01:00,359 --> 00:01:01,109
Tim: Yes, please.
12
00:01:01,169 --> 00:01:06,349
No, honestly, there are questions
that have popped into my mind,
ever since I discovered that
13
00:01:06,349 --> 00:01:09,679
there is more to a compiler than
just being quote, a compiler.
14
00:01:10,259 --> 00:01:17,369
Like there are other steps involved,
and this word linker popped up and I
see in Rust, I've got this in my cargo
15
00:01:17,369 --> 00:01:22,999
file, I've got this option to, add, LTO
link time optimization I can enable some
16
00:01:22,999 --> 00:01:25,659
flags and I'm really, I've been really
curious about what's going on there.
17
00:01:26,229 --> 00:01:26,589
David: Sure.
18
00:01:26,829 --> 00:01:32,949
So MyLink is called Wild, which,
the naming comes because Link is
19
00:01:32,949 --> 00:01:35,649
traditionally, at least on, on
Linux, traditionally end in ld.
20
00:01:36,229 --> 00:01:37,579
The original link was called ld.
21
00:01:37,759 --> 00:01:44,869
And so subsequently, 'cause I've ended
in LD like gold and mold and LLD and.
22
00:01:45,214 --> 00:01:50,494
So I started with LD and then I, my plan
is to eventually make it incremental,
although it's not incremental yet.
23
00:01:50,494 --> 00:01:53,344
And so I added an I and then I
just did a dictionary search for
24
00:01:53,344 --> 00:01:56,734
words ending in ILD, and Wild was
the one that popped out at me.
25
00:01:57,479 --> 00:01:58,754
Tim: Yeah it works, doesn't it?
26
00:01:58,804 --> 00:02:02,769
I think like it's clear what I mean,
I think it's clear and interesting.
27
00:02:03,099 --> 00:02:03,339
David: It's an
28
00:02:03,589 --> 00:02:05,809
Tim: it all, yeah, it's good.
29
00:02:05,899 --> 00:02:15,689
I also like that the idea, there's
a kind of a connotation of a project
that's attempting to potentially be,
30
00:02:16,319 --> 00:02:21,339
I guess a challenger to orthodoxy or
potentially, or trying things new.
31
00:02:21,459 --> 00:02:25,779
Is that how you see wild, I guess
with the focus on incrementalism?
32
00:02:26,409 --> 00:02:32,649
Is that an indication that you are looking
to build a tool that works differently
than the others that you mentioned?
33
00:02:33,214 --> 00:02:34,984
David: I'm certainly looking
for opportunities where I can
34
00:02:34,984 --> 00:02:37,864
do things differently whilst
being as compatible as possible.
35
00:02:38,644 --> 00:02:45,854
So I started the linker, I
guess about a year and a half
ago, so maybe September, 2023.
36
00:02:46,874 --> 00:02:49,124
I've been working on it more
or less full time since then.
37
00:02:49,844 --> 00:02:56,954
And yeah it's it's still got,
it's still got a way to go,
it is pretty usable already.
38
00:02:56,954 --> 00:03:01,379
So I've been using it as my
default linker for six months
or so now for all my rust work.
39
00:03:01,379 --> 00:03:01,589
And
40
00:03:02,259 --> 00:03:06,084
Tim: Is it one of those projects which
you think, okay, maybe this is like a six
41
00:03:06,084 --> 00:03:09,654
week kind of thing, and then it's actually
there's everything else to implement.
42
00:03:09,684 --> 00:03:11,904
Is that kind of how it happens?
43
00:03:11,904 --> 00:03:15,864
Or did you know that this was this big
audacious project right from the start?
44
00:03:16,109 --> 00:03:17,969
David: No, I knew it was
going to be a big project.
45
00:03:18,019 --> 00:03:23,059
So I had actually attempted
to write a linker once before,
back maybe around 2017.
46
00:03:23,639 --> 00:03:26,969
And I lost access to the source code to
that when I left my previous employer.
47
00:03:27,359 --> 00:03:31,709
But so I knew that running a
link was a lot of work and yeah.
48
00:03:33,684 --> 00:03:40,144
Tim: So you thought, okay, but
yeah, I, okay, I appreciate that,
and so when, actually, let's start
49
00:03:40,204 --> 00:03:45,699
with the question of, let's say
that I wanted to use something
that's different than the default.
50
00:03:46,179 --> 00:03:48,249
And there, as you mentioned, gold mold.
51
00:03:48,399 --> 00:03:49,299
Now wild.
52
00:03:49,719 --> 00:03:51,999
What do I do inside of
cargo to enable that?
53
00:03:53,459 --> 00:04:02,229
David: So the first thing you need to do
is switch to using clang to link rather
than, I guess it probably uses GCC by
54
00:04:02,229 --> 00:04:09,279
default, but if you switch to using clang,
clang, then you can pass the appropriate
link as to clang to tell it to use.
55
00:04:09,369 --> 00:04:15,049
While as the linker I think it is
possible to use GCC, but it's a little
bit trickier because GCC, the way
56
00:04:15,049 --> 00:04:17,689
that you select the linker, it doesn't
let you pass a path to the linker.
57
00:04:18,169 --> 00:04:22,669
It just has a predetermined set of
linkers that it supports and while
58
00:04:22,669 --> 00:04:25,969
isn't current one of those linkers,
so you've got trick it into using.
59
00:04:26,854 --> 00:04:28,984
And thinking that it's
using LD or something.
60
00:04:29,784 --> 00:04:36,344
Tim: So if, if someone was to create their
hobby linker, they would need to somehow
patch or, get a patch accepted upstream
61
00:04:36,344 --> 00:04:43,184
into G CCC potentially to get that main
line so that downstream users would be
able to invoke it through the compiler.
62
00:04:43,969 --> 00:04:45,829
David: I mean it the
easiest way is to trick it.
63
00:04:45,829 --> 00:04:46,099
So
64
00:04:46,099 --> 00:04:46,759
Tim: yeah, sure.
65
00:04:46,794 --> 00:04:49,524
David: a directory with a file called
LD in it, and it's actually your linker.
66
00:04:49,934 --> 00:04:50,144
Yeah.
67
00:04:50,244 --> 00:04:54,444
Tim: Oh I expected that you'd
say you're just sim link ld,
but that sounds fine too.
68
00:04:56,484 --> 00:05:04,984
The and so that kind of goes to something
which is interesting, that the compiler
is actually invoking other tools.
69
00:05:05,234 --> 00:05:15,194
GCC, the GNU Compiler collection,
or clang, the C Lang compiler as
part of the LLVM project, they're
70
00:05:15,194 --> 00:05:21,014
actually, in some sense modular,
they've, they outsource some of
their work to produce an executable.
71
00:05:21,224 --> 00:05:22,249
And one of those is the linker.
72
00:05:23,134 --> 00:05:23,854
David: Yeah, that's right.
73
00:05:23,914 --> 00:05:27,394
And the other interesting thing
there is that even when you're
compiling a rust program, it's
74
00:05:27,394 --> 00:05:30,604
still invoking the C compiler in
order to actually link your program.
75
00:05:30,694 --> 00:05:33,964
So the rust compiler, rust C
doesn't invoke the linker directly.
76
00:05:33,964 --> 00:05:40,114
In general, it can, but if you're
linking against, if you need to link
against the C runtime, which you do
77
00:05:40,114 --> 00:05:45,364
for most programs, then generally
you need to invoke the C compiler.
78
00:05:45,394 --> 00:05:49,984
Tim: And that c runtime is
bolted into your program.
79
00:05:50,154 --> 00:05:56,794
But if you had a bare metal target,
so if you were genuinely writing a
rust code for some compiler target
80
00:05:56,794 --> 00:05:59,584
with no operating system, then
you're completely on your own.
81
00:05:59,704 --> 00:06:03,454
But if you have essentially any operating
system you're going to bring all along.
82
00:06:04,234 --> 00:06:10,914
The c runtime for the ride, which
in kind of by des by design and
partially, the way that, that, the
83
00:06:10,914 --> 00:06:14,594
way that's created is you essentially
require a c compiler to compile rust.
84
00:06:15,114 --> 00:06:15,574
David: That's right.
85
00:06:15,574 --> 00:06:15,904
Yeah.
86
00:06:16,114 --> 00:06:20,494
Certainly if you're running, if
you're writing a desktop application
for Rust, something that uses the
87
00:06:20,494 --> 00:06:24,154
Rust Standard Library on Linux, for
example, then you need the C runtime.
88
00:06:24,764 --> 00:06:28,304
And so various object files that
get added in, stuff that runs at
89
00:06:28,304 --> 00:06:33,044
startup to initialize that runtime and
figuring out all those flags is tricky.
90
00:06:33,279 --> 00:06:40,494
Tim: I, all right, so this is, so we as
programmers, we write a main function,
but that isn't actually the what?
91
00:06:40,824 --> 00:06:46,014
That is not the function that's
initial, like initially jumped
to by the operating system.
92
00:06:46,074 --> 00:06:52,904
When when the program is loaded, that's
actually looking for a different symbol,
which is underscore start, isn't it?
93
00:06:53,374 --> 00:06:53,974
David: Yeah, that's right.
94
00:06:54,154 --> 00:06:54,364
Yep.
95
00:06:54,754 --> 00:06:54,934
Yeah.
96
00:06:54,934 --> 00:06:57,124
On Linux, the default is under
sourced underscore start.
97
00:06:57,544 --> 00:06:57,814
So there can
98
00:06:57,884 --> 00:06:58,454
Tim: Which.
99
00:06:58,564 --> 00:06:59,884
David: of initialization that happens.
100
00:06:59,884 --> 00:07:03,154
So if your program is dynamically
initialized, then the first thing
101
00:07:03,154 --> 00:07:06,364
it will run will be underscore
start from your dynamic loader.
102
00:07:07,174 --> 00:07:11,454
Will then run and do a bunch of
initialization of loading all
the other parts of your program.
103
00:07:11,874 --> 00:07:14,784
And then it will call underscore
start in your actual binary, which
104
00:07:14,784 --> 00:07:17,604
will then do further initialization,
and then it'll eventually run main.
105
00:07:18,084 --> 00:07:20,494
So there can be quite a lot that
happens before you actually get to main.
106
00:07:20,864 --> 00:07:25,124
Tim: I feel as though we've just
created quite a few like items on a
stack of things we need to explain.
107
00:07:25,414 --> 00:07:29,254
Let's see if we can we, because I
think this is all fascinating, right?
108
00:07:29,284 --> 00:07:33,904
So it turns out that starting a program
itself is quite hard, or at least is
109
00:07:33,904 --> 00:07:36,994
more complicated, and it looks I, I'm
just gotta write a couple of things down.
110
00:07:36,994 --> 00:07:41,254
We've got dynamic linking or dynamic
library loading and so forth.
111
00:07:41,464 --> 00:07:44,434
I was curious though about this.
112
00:07:45,394 --> 00:07:48,889
Notion of that the compiler
itself is tool like is it?
113
00:07:49,219 --> 00:07:58,709
We think of the compilation step as a we,
we push a button, we do car cargo build
or rusty if we invoking it directly.
114
00:07:59,509 --> 00:08:00,979
But there's lots of things happening.
115
00:08:01,429 --> 00:08:04,849
What are some of the other tools
in the list as well as the linker?
116
00:08:08,539 --> 00:08:10,639
Do we need to have an
assembler, for example?
117
00:08:13,914 --> 00:08:15,984
David: I don't think it usually
invokes a separate assembler.
118
00:08:16,014 --> 00:08:23,394
It might in some cases, but for the most
part, like if we were compiling rust
code, the rust compiler embeds, LLVM
119
00:08:23,454 --> 00:08:30,594
and ll VM knows how to convert, take
L-L-V-M-I-R and reduce it to machine
code, which then puts in object files and,
120
00:08:30,949 --> 00:08:31,309
Tim: right?
121
00:08:31,639 --> 00:08:34,309
And so what is the linker actually doing?
122
00:08:35,754 --> 00:08:36,054
David: Okay.
123
00:08:36,054 --> 00:08:44,454
So the linker takes a bunch of object
files and combines them together into
an executable, and the linker needs
124
00:08:44,454 --> 00:08:48,084
to decide where to put the different
sections of those object files.
125
00:08:48,084 --> 00:08:51,054
So each object file we compose
of a whole bunch of sections.
126
00:08:51,774 --> 00:08:54,384
there'll be sections that contain
executable code, sections that
127
00:08:54,384 --> 00:08:57,354
contain read-only data, sections
that contain read, write data.
128
00:08:58,194 --> 00:08:59,514
might be debug info.
129
00:09:00,244 --> 00:09:02,824
There's lots of different sections
in each of these object files.
130
00:09:02,914 --> 00:09:07,474
And the linker takes all of those and
merges them together into an executable.
131
00:09:09,054 --> 00:09:14,639
Tim: I have heard the like
AF files and dwarf files.
132
00:09:15,869 --> 00:09:18,464
Is that's related to
this process, isn't it?
133
00:09:18,534 --> 00:09:18,824
Yeah.
134
00:09:18,904 --> 00:09:23,434
David: on Linux, the executables are elf
files, which is, it's a container format.
135
00:09:23,434 --> 00:09:24,004
It's also used
136
00:09:24,094 --> 00:09:25,834
Other unixes as well.
137
00:09:26,234 --> 00:09:27,734
It's not used on Windows and Mac.
138
00:09:27,764 --> 00:09:29,324
They have their own
formats that are different.
139
00:09:30,074 --> 00:09:32,024
dwarf is the debug info format.
140
00:09:32,114 --> 00:09:35,054
So dwarf is contained
within the alpha usually.
141
00:09:36,084 --> 00:09:45,864
Tim: So if I had a rust project
comprised of let's say 10 dependencies,
which will probably end up mushrooming
142
00:09:45,864 --> 00:09:53,514
into 50 or 500, but let's just
say we've got 10 dependencies are
will the rust compiler create?
143
00:09:53,574 --> 00:10:00,684
So if, will it create one file
for each of those dependencies?
144
00:10:00,864 --> 00:10:04,404
And the linker job is to wire them
together and produce something else.
145
00:10:04,404 --> 00:10:12,024
So when you say an object file, is
that the result of compiling one crate?
146
00:10:13,144 --> 00:10:13,844
David: Sort of.
147
00:10:13,904 --> 00:10:19,544
So by default, rust will, order to speed
up compilation rust, will use multiple
148
00:10:19,544 --> 00:10:22,814
code gen units so that it can have
multiple threads doing work at one time.
149
00:10:23,634 --> 00:10:28,734
And each of those code gen units
goes into a separate object file
rust does then, however, take all
150
00:10:28,734 --> 00:10:31,554
of those object files and combine
them together into an archive file.
151
00:10:32,529 --> 00:10:35,289
Which it calls, which it
then renames as a lib.
152
00:10:36,489 --> 00:10:39,939
And so that lib is an archive file
that contains multiple object files.
153
00:10:40,749 --> 00:10:41,649
from each code gen unit.
154
00:10:42,969 --> 00:10:43,189
Tim: Huh.
155
00:10:43,684 --> 00:10:52,994
And this is one of the reasons why
increasing or, changing that cogen
u unit's number changes like the
156
00:10:52,994 --> 00:10:55,604
behavior of the compiler, it speeds
things up, it decreases things.
157
00:10:55,964 --> 00:10:59,984
So if the trade off there would
be, if I have multiple code, if so,
158
00:11:00,224 --> 00:11:04,124
under the debug mode, by default,
I think it's 256 cogen units.
159
00:11:04,124 --> 00:11:08,144
But you can reduce that down
and release to, let's say, one.
160
00:11:08,999 --> 00:11:13,019
You'd have a slower build that is
somehow magically more performant.
161
00:11:13,019 --> 00:11:13,979
Is that correct?
162
00:11:14,039 --> 00:11:14,519
Or
163
00:11:14,909 --> 00:11:15,199
David: Yeah.
164
00:11:15,204 --> 00:11:20,654
It gives the compiler more opportunity
to optimize because it's got visibility
over more of your code at once.
165
00:11:20,954 --> 00:11:23,594
That said though, if you're using
something like link time optimization,
166
00:11:23,714 --> 00:11:28,154
which I guess you'd like to get into at
some point, then maybe mitigates that
167
00:11:28,204 --> 00:11:29,139
Tim: So actually, yes.
168
00:11:29,169 --> 00:11:34,239
I actually hope that we're gonna
spend lots of time on LinkedIn
optimization because I'm very curious.
169
00:11:34,239 --> 00:11:42,264
I've but I. I just wanted to lay out
the land of, okay, so my compile, like
when I hit, when I run Cargo build,
170
00:11:42,264 --> 00:11:47,394
it's doing a lot cargo, is invoking
rust sea, which itself is doing a
bunch of work for along all of my
171
00:11:47,394 --> 00:11:52,104
dependencies, producing object files,
which are probably going to be wrapped
172
00:11:52,164 --> 00:11:59,994
in a files, which are an archive with
one or more object files at some point.
173
00:12:00,564 --> 00:12:05,484
We now have a bunch of archives, bunch
of libs, and we need to fuse them.
174
00:12:05,484 --> 00:12:10,404
I said wired together before, but
I think that a better analogy would
175
00:12:10,404 --> 00:12:15,204
be like fusing them or like welding
them together to produce one binary.
176
00:12:15,264 --> 00:12:22,304
'cause in rust things are generally
statically linked except for a lip C,
but we can modular that out potentially.
177
00:12:22,664 --> 00:12:23,264
Now.
178
00:12:25,289 --> 00:12:28,559
When, and the linker comes in at
that welding step, or essentially
179
00:12:28,559 --> 00:12:31,774
taking all of those archives and
fusing them together, is that,
180
00:12:32,979 --> 00:12:36,239
David: Yeah, so the linker opens all
of those object open, opens all the
181
00:12:36,239 --> 00:12:40,829
archives, finds all the object files and
checks to see what symbols they all have.
182
00:12:40,979 --> 00:12:43,999
And then it makes various decisions
about which archive entries to load.
183
00:12:44,059 --> 00:12:47,989
Because particularly for things
like libey, we don't always
load all archive entries.
184
00:12:48,019 --> 00:12:52,429
We will only load archive entries
in certain circumstances if they
define symbols that are referenced.
185
00:12:53,239 --> 00:12:55,609
once it's decided which archive
entries to load, then it.
186
00:12:56,659 --> 00:13:02,129
It might do, say, a garbage collection
sweep where it traverses the graph of
symbols in order to remove dead code.
187
00:13:02,849 --> 00:13:05,669
then it works out how much space
it needs in all the sections.
188
00:13:05,669 --> 00:13:07,829
Lays everything out and writes the file.
189
00:13:10,419 --> 00:13:11,529
Tim: what is a symbol?
190
00:13:11,619 --> 00:13:14,679
Is this just a, I assume
just a short string?
191
00:13:15,179 --> 00:13:18,059
David: It's a name, so it's a string
that contains a name of something.
192
00:13:18,239 --> 00:13:21,029
So that name could be a function,
it could be a bit of data.
193
00:13:21,739 --> 00:13:23,089
Tim: is there name spacing involved?
194
00:13:23,089 --> 00:13:30,559
How does a, is this actually part
of the design of the compiler or the
programming language to make sure
195
00:13:30,559 --> 00:13:34,909
that symbols that are generated in
multiple places don't somehow collide?
196
00:13:34,919 --> 00:13:38,369
David: So unless he put no mangle on
your symbol, on your functional variable.
197
00:13:38,819 --> 00:13:45,089
Then it'll get mangled by the compiler,
which means it'll take your name and it'll
take the name of the modules you're in.
198
00:13:45,089 --> 00:13:50,639
It'll take the name of the crate you're
in, and it'll also add of a unique suffix.
199
00:13:50,639 --> 00:13:53,609
So if you have multiple versions
of the same crate, then you
need to make them not collide.
200
00:13:54,089 --> 00:14:00,129
So it gets all of these different bits
and pieces and mans them together in
something that's semi human readable.
201
00:14:00,129 --> 00:14:01,899
But we'll have bits in
it that look a bit funny.
202
00:14:03,219 --> 00:14:12,259
Tim: But if I wanted to somehow destroy
my own compilation process and I did
accidentally, if I put no man on five
203
00:14:12,259 --> 00:14:20,329
functions from different and I tried
to if I tried to push them all together
it's presumably almost non-deterministic,
204
00:14:20,329 --> 00:14:23,899
which function Act actually gets called
or which variable is actually referenced.
205
00:14:24,259 --> 00:14:24,714
Is that correct?
206
00:14:24,909 --> 00:14:26,109
David: So you'll probably
get a link error.
207
00:14:26,139 --> 00:14:26,679
You'll probably get
208
00:14:26,719 --> 00:14:26,989
Tim: Okay.
209
00:14:27,009 --> 00:14:28,029
David: ddu duplicate symbols
210
00:14:28,079 --> 00:14:32,729
Tim: So the link is actually gonna
protect me and be nice and say,
Hey look, this is ridiculous.
211
00:14:32,729 --> 00:14:33,844
You need to figure this out.
212
00:14:34,579 --> 00:14:38,719
David: Yeah, there are circumstances
in which it'll will allow multiple
definitions of the same symbol.
213
00:14:38,749 --> 00:14:43,759
So if the symbol is declared as weak, then
it will just take one of the definitions.
214
00:14:44,719 --> 00:14:45,079
But if
215
00:14:45,179 --> 00:14:45,389
Tim: of the,
216
00:14:45,679 --> 00:14:45,919
David: yeah,
217
00:14:46,589 --> 00:14:53,099
Tim: one of the things that you mentioned
earlier was the L-L-V-M-I-R, this
intermediate representation, and would
218
00:14:53,099 --> 00:15:01,969
that be the place where this wink, this
weak annotation on, let's say a would, is
that where something would be placed or,
219
00:15:02,189 --> 00:15:06,419
I, is that available in the rust language
to be able to, I don't actually know.
220
00:15:07,189 --> 00:15:09,559
David: I like it might
be, but I'm not sure.
221
00:15:09,559 --> 00:15:12,129
So most of the testing of these things,
like when I'm testing the linker,
222
00:15:12,129 --> 00:15:15,219
I write little programs to test out
different aspects of the linker.
223
00:15:15,549 --> 00:15:18,999
But most of my test programs I
write in see, just because it gives
224
00:15:18,999 --> 00:15:23,069
you that lower level access to a
lot of weird compiler features.
225
00:15:23,129 --> 00:15:26,299
And i'm much more familiar with doing
those things in sea than I'm in rust.
226
00:15:26,489 --> 00:15:26,729
But
227
00:15:26,779 --> 00:15:32,819
Tim: Because essentially c does
not provide, rust semantics, right?
228
00:15:32,824 --> 00:15:36,964
It is essentially it's some sort of
higher level assembler to use a cliche.
229
00:15:37,444 --> 00:15:42,604
And if you wanted to, yeah, I
guess really test out the linker.
230
00:15:43,204 --> 00:15:46,834
Yeah, of course you would use the
thing that's like easiest to write in.
231
00:15:47,674 --> 00:15:48,724
It makes me curious.
232
00:15:48,934 --> 00:15:56,824
So the intent, and you actually, you
mentioned this earlier, the intent
is to have a, this is not a unique to
233
00:15:56,824 --> 00:16:04,954
rust problem like any while it should
be usable by any compiler backend for
234
00:16:05,754 --> 00:16:06,054
David: Yeah.
235
00:16:06,064 --> 00:16:09,694
For any compiled language,
natively compiled language.
236
00:16:09,694 --> 00:16:10,744
A Linux ideally, yes.
237
00:16:11,909 --> 00:16:14,969
Tim: But, oh, okay.
238
00:16:14,969 --> 00:16:20,769
So here was the an interesting,
potentially not interesting question
of if I wanted to support cross
239
00:16:20,769 --> 00:16:28,149
compilation, I would need a linker
that also understands the calling
conventions of the target as well.
240
00:16:28,149 --> 00:16:29,019
Is that right?
241
00:16:29,819 --> 00:16:31,709
David: So calling conventions, the linker
242
00:16:31,779 --> 00:16:32,259
Tim: Different.
243
00:16:32,339 --> 00:16:33,689
David: anything with calling conventions.
244
00:16:33,729 --> 00:16:34,269
Tim: Okay.
245
00:16:34,299 --> 00:16:34,389
Sure.
246
00:16:34,829 --> 00:16:37,349
David: it does need to do stuff
with the machine code sometimes.
247
00:16:37,349 --> 00:16:38,369
So there'll be bits of,
248
00:16:40,799 --> 00:16:45,419
machine code that the linker needs
to transform, and so it needs to
understand the instruction set.
249
00:16:45,469 --> 00:16:49,339
There's also a lot of of the
main things the linker deals with
are things called relocations.
250
00:16:49,489 --> 00:16:53,089
So a relocation is an instruction
that's emitted by the compiler to
251
00:16:53,089 --> 00:16:56,029
tell a linker to put a particular
value at a particular location.
252
00:16:56,579 --> 00:17:02,589
For example, when the compiler emits your
fun, your compiler function it generates
the machine code for your function.
253
00:17:02,829 --> 00:17:06,369
But wherever your function, say, calls
another function or references some data.
254
00:17:07,119 --> 00:17:08,919
compiler doesn't know where
that other function is.
255
00:17:08,919 --> 00:17:10,419
It doesn't know where
the data is going to be.
256
00:17:10,599 --> 00:17:12,309
'cause the linker is the one
that decides those things.
257
00:17:12,699 --> 00:17:16,929
And so the compiler will emit a
relocation to say, please put the
258
00:17:16,929 --> 00:17:20,649
address of function fu at this
particular offset in my function.
259
00:17:21,369 --> 00:17:24,999
And so the linker is applying
those relocations and some of those
260
00:17:25,029 --> 00:17:28,119
relocations maybe won't actually
get processed until runtime.
261
00:17:28,809 --> 00:17:31,179
a lot are comp, a lot are
processed at link time.
262
00:17:32,139 --> 00:17:35,229
Even the ones that are processed
at runtime, the linker still needs
to deal with them 'cause it needs
263
00:17:35,229 --> 00:17:39,339
to transform them into a different
platform and put them in different
tables and stuff like that.
264
00:17:41,169 --> 00:17:46,839
and so when the linker is processing those
relocations, sometimes it will transform
265
00:17:46,839 --> 00:17:50,349
the machine code into a more optimal
form, depending on what it's linking.
266
00:17:50,709 --> 00:17:52,479
So the link needs to
understand those things.
267
00:17:53,119 --> 00:17:58,139
You mentioned cross compilation, so you
need to if you wanted to cross compile
268
00:17:58,139 --> 00:18:01,169
to a different architecture, your linker
would need to support that architecture.
269
00:18:01,889 --> 00:18:02,069
Yep.
270
00:18:02,599 --> 00:18:05,509
Tim: In order to be able to
facilitate these relocations because
271
00:18:05,509 --> 00:18:10,219
it needs to be able to basically
generate the correct binary,
272
00:18:10,869 --> 00:18:11,319
David: Yes.
273
00:18:11,324 --> 00:18:14,324
Tim: Let's start to chat
about link time optimization.
274
00:18:14,324 --> 00:18:21,304
Now I have some naive questions
to ask you, so I apologize in
advance, but I I'll start there.
275
00:18:22,594 --> 00:18:24,149
The so the first thing is.
276
00:18:26,914 --> 00:18:28,894
Does the linker decide itself?
277
00:18:29,134 --> 00:18:32,584
Could the linker decide
to inline code itself?
278
00:18:32,974 --> 00:18:36,349
Or do things like loop
unrolling if it wanted to, or?
279
00:18:37,494 --> 00:18:38,544
David: Generally not.
280
00:18:38,664 --> 00:18:43,284
So I guess there's two kinds
of that can happen in linkers.
281
00:18:43,284 --> 00:18:47,244
So there's the transformations that I
mentioned where the compiler has emitted
282
00:18:47,274 --> 00:18:50,424
one kind of relocation, and the linker
transforms that into a different kind.
283
00:18:50,889 --> 00:18:52,419
And these are micro optimizations.
284
00:18:52,419 --> 00:18:54,859
They're much templates, very localized.
285
00:18:54,859 --> 00:18:56,299
They're not crossing function boundaries.
286
00:18:56,299 --> 00:18:59,109
They're just that little bit of machine
code that relocation gets transformed.
287
00:18:59,109 --> 00:19:04,069
So they're a kind of optimization
but usually in linkers,
we'll call those relaxations.
288
00:19:04,279 --> 00:19:07,519
I think technically relaxations are maybe
where they make the code shorter, but
289
00:19:07,749 --> 00:19:11,199
I just refer to them all as relaxations,
whether they make the code shorter or not.
290
00:19:12,069 --> 00:19:17,319
so there's those, then there's full
on link time optimization, which is
291
00:19:17,499 --> 00:19:22,359
where, and that's actually very involve,
very much involving the compiler.
292
00:19:22,929 --> 00:19:25,329
And so you're effectively running
the compiler at link time.
293
00:19:25,779 --> 00:19:28,359
And the, it's not actually the linker
that's doing those optimizations.
294
00:19:28,359 --> 00:19:29,079
It's the compiler.
295
00:19:29,769 --> 00:19:37,119
And so usually the way that will work is
you're compiling your code, instead of
having the compiler emit machine code, it
296
00:19:37,119 --> 00:19:40,769
will just store the, its own intermediate
representation in the object file.
297
00:19:41,999 --> 00:19:43,259
And then when you come to linking.
298
00:19:44,324 --> 00:19:46,994
The compiler will take,
will get invoked somehow.
299
00:19:47,144 --> 00:19:51,494
There's a few different ways that
it can be invoked and it will take
300
00:19:51,494 --> 00:19:54,464
that intermediate representation
that was stored in the object files
301
00:19:56,534 --> 00:19:58,004
do the actual optimizations then.
302
00:19:58,814 --> 00:20:01,304
So it's really the compiler that's
doing the optimizations, not the linker.
303
00:20:02,214 --> 00:20:02,784
Tim: Wow.
304
00:20:02,814 --> 00:20:09,084
So I had this idea of, okay,
let's say have a constant,
which is used in five places.
305
00:20:10,164 --> 00:20:16,704
The linker has somehow given
information, and that constant might
be the same in multiple dependencies.
306
00:20:16,704 --> 00:20:24,744
I just think of this, for some reason
I've defined, pie in multiple places, and
the linker can see that constant is used.
307
00:20:25,884 --> 00:20:31,794
And I wondered if the let's say
a static variable has the same
value in multiple different crates.
308
00:20:32,094 --> 00:20:36,564
It would somehow delete all of
the redundant copies and refer
309
00:20:36,564 --> 00:20:40,734
everyone back to a single canonical
place where this thing is defined.
310
00:20:41,429 --> 00:20:44,159
David: So there, there can be
a little bit of that sort of
stuff happening in linkers.
311
00:20:44,219 --> 00:20:48,359
So you can have merge sections
in particular string merge
312
00:20:48,359 --> 00:20:52,049
sections where you have a section
containing multiple strings.
313
00:20:52,739 --> 00:20:54,479
compiler, sorry.
314
00:20:54,479 --> 00:20:58,859
The linker job there in that
case is to take all those
strings and de-duplicate them.
315
00:20:59,519 --> 00:21:06,089
Similarly, you can have merge sections
where the entire section is a single
and the linker will look for other
316
00:21:06,449 --> 00:21:12,039
objects with the same section name with
identical content and deduplicate them.
317
00:21:13,989 --> 00:21:18,969
So there's a little bit of that happening
there, but it's very much the compiler
needs to tell the linker to do that.
318
00:21:19,059 --> 00:21:22,389
So the compiler needs to set the merge
flag on the section, and it's telling the
319
00:21:22,424 --> 00:21:23,594
Please deduplicate this.
320
00:21:24,244 --> 00:21:28,144
Tim: So the linker is really
thinking about sections of memory.
321
00:21:29,604 --> 00:21:29,814
David: Yep.
322
00:21:29,884 --> 00:21:37,309
Tim: Time in some sense because I
was going to think, the follow on
to the constant or statics was whole
323
00:21:37,309 --> 00:21:41,659
functions of essentially if the
signature matched, or let's say even the
324
00:21:41,689 --> 00:21:47,269
resulting code were to match, could you
essentially deduplicate across crates?
325
00:21:47,269 --> 00:21:54,319
Although, I dunno how much completely
redundant code that any given project
might be, but potentially if I have
326
00:21:54,319 --> 00:22:01,589
multiple versions of a single crate and
there might be just say a patch release,
but for some reason I need two versions
327
00:22:01,589 --> 00:22:06,839
of it, there might be quite a lot of
overlap between between that and maybe
328
00:22:06,879 --> 00:22:11,919
the linker could do a lot of janitorial
work to clean up the code paths.
329
00:22:12,464 --> 00:22:16,454
David: So there is another feature
that Linkers can support called
Identical Code Folding, where it
330
00:22:16,454 --> 00:22:20,359
does look for bits of code that are
identical and deduplicate those.
331
00:22:21,089 --> 00:22:24,919
The caveat there is that kind
of doing that kind of work
in the linker is expensive.
332
00:22:25,119 --> 00:22:28,689
It can be an expensive operation
and whilst priority is very
333
00:22:28,689 --> 00:22:31,929
much to be as fast as possible,
particularly for development use.
334
00:22:32,319 --> 00:22:34,599
And so that kind of feature goes against
335
00:22:35,319 --> 00:22:41,324
Tim: Yeah, because the, I have
noticed that if I, again, we say
336
00:22:41,324 --> 00:22:43,904
we've got a hundred dependencies
and I'm running cargo build.
337
00:22:44,264 --> 00:22:48,344
I get to 99 out of a hundred steps,
and that 100th, the N minus one
338
00:22:48,464 --> 00:22:52,574
step, which presumably is the
link step, takes a long time.
339
00:22:53,384 --> 00:22:56,714
It's frustrating because you're
like, I just wanna get to 100% done.
340
00:22:56,864 --> 00:22:59,054
And the.
341
00:23:00,594 --> 00:23:08,334
Related question is why is it that
linkers have this insatiable amount of
342
00:23:08,874 --> 00:23:15,294
appetite for Bram and like compute, they
just seem to take as much as you've got.
343
00:23:16,074 --> 00:23:22,894
Why do they need so much so much Ram and
for example if you were compiling, let's
say the Linux kernel, it's like hard.
344
00:23:23,344 --> 00:23:24,934
David: Linkers do use
different amounts of memory.
345
00:23:25,324 --> 00:23:29,764
So wild does use less memory
generally when I've profiled
it than the other linkers.
346
00:23:30,394 --> 00:23:35,524
That said, it does need to read all
of your input files and it reads
your input files by end mapping them.
347
00:23:36,469 --> 00:23:42,709
The amount of ram will at least be as
much as the sum of all of your, all of the
object files that are fed into the linker.
348
00:23:43,309 --> 00:23:45,979
It won't necessarily actually
page all of those in.
349
00:23:46,189 --> 00:23:49,969
So if there's large parts of those object
files that are never referenced, then they
350
00:23:49,969 --> 00:23:52,939
effectively don't count towards the memory
usage, even though they're mapped in.
351
00:23:53,179 --> 00:23:58,249
If you never actually touch those
pages of memory, then they won't count
towards the processor's memory usage.
352
00:23:58,489 --> 00:24:02,829
But for every page of the input
files that we need to read, at
353
00:24:02,829 --> 00:24:05,909
some point that will count towards
the memory usage of the linker.
354
00:24:07,664 --> 00:24:12,219
Tim: Do you wanna chat a little bit more
about the, like other poss like there's
355
00:24:12,219 --> 00:24:18,579
probably many parts of this optimization
process that I've not mentioned.
356
00:24:18,729 --> 00:24:22,179
Is there anything else that you
think would be worthwhile discussing?
357
00:24:23,159 --> 00:24:27,209
David: I think one, so my main
interest is in making the linker as
358
00:24:27,209 --> 00:24:31,989
fast as possible making, I'm very
interested in making a fast developer.
359
00:24:32,559 --> 00:24:33,039
Cycle.
360
00:24:33,519 --> 00:24:40,469
And so I, I want the linker to be as
fast as possible and of the designs
of the way object files are set up
361
00:24:40,799 --> 00:24:45,059
in some cases are not ideal towards
making the linker as fast as possible.
362
00:24:45,659 --> 00:24:50,339
So you'll have bits of information that
you don't have at the right time, since
you've got a structure your code in a
363
00:24:50,339 --> 00:24:55,479
weird way in order to figure something
out and then go back and fix it up
once you know the information later on.
364
00:24:55,479 --> 00:24:58,539
So there's a lot of sort of gymnastics
you've gotta do in order to get things
365
00:24:58,809 --> 00:25:01,439
because you don't have the information
you need necessarily upfront.
366
00:25:02,139 --> 00:25:08,079
There's also, so one of the slowest
things in linking is string merging.
367
00:25:08,169 --> 00:25:10,239
So d that deduplication of strings.
368
00:25:11,709 --> 00:25:15,219
that's particularly a problem if
you're linking large c plus programs
369
00:25:15,219 --> 00:25:19,989
with debug info because they have
just so much duplicated debug info
370
00:25:20,529 --> 00:25:20,749
Tim: Huh.
371
00:25:20,779 --> 00:25:23,509
David: all the header files
duplicate the same, in same debug
372
00:25:23,509 --> 00:25:27,794
info over and over again the linker
really has a lot of work to do.
373
00:25:27,824 --> 00:25:32,384
If I'm linking something like clang
with debug info, then there might be
six gigabytes of input to the linker.
374
00:25:32,864 --> 00:25:36,434
And most of that is just
these string sections.
375
00:25:36,464 --> 00:25:39,464
And the linker just has to go through
them and hash them and deduplicate them.
376
00:25:39,794 --> 00:25:41,624
And it's just a lot of
accessing at memory.
377
00:25:41,804 --> 00:25:44,024
And that's, it's
378
00:25:44,109 --> 00:25:44,784
Tim: It is not free.
379
00:25:45,074 --> 00:25:47,714
David: yeah, it's not ideal from a
making a fast linker perspective.
380
00:25:48,684 --> 00:25:48,894
Tim: No.
381
00:25:49,914 --> 00:25:52,764
Do you think that, so there's almost
two kind of spinoff questions.
382
00:25:52,764 --> 00:25:55,974
One is, do you think that
it would ever be feasible?
383
00:25:56,404 --> 00:25:58,384
We finish my sentence.
384
00:25:58,774 --> 00:26:07,204
Do you think it would ever be finished,
feasible to have, in some sense a
developer, like a debug development
385
00:26:07,264 --> 00:26:14,504
time linker and a release linker,
something that is like heavily, that
is optimized for producing very good.
386
00:26:15,039 --> 00:26:19,684
Machine code or like almost a petition
in the projects that would use.
387
00:26:20,279 --> 00:26:22,709
David: To an extent you can get
that just by switching flags.
388
00:26:23,079 --> 00:26:27,219
I think caveat there is that I think
a lot of developers don't necessarily
389
00:26:27,529 --> 00:26:30,619
there's plenty of developers that
don't fine tune their builds.
390
00:26:31,069 --> 00:26:33,799
Like I, I hear of lots of people
who are still using the default
391
00:26:33,799 --> 00:26:38,099
Linux linker on Linux, like
using NU ld and it's really slow.
392
00:26:38,309 --> 00:26:38,869
And and yet.
393
00:26:38,974 --> 00:26:43,114
If people complain about compile
times, then that's like the
first thing you should change.
394
00:26:44,594 --> 00:26:48,224
But certainly there's lots of flags
you can change to adjust that.
395
00:26:48,224 --> 00:26:52,184
Turning off debug info, if you're
not using a debugger, turning
off debug info is helpful.
396
00:26:52,434 --> 00:26:58,004
Or if you are using a debugger using
split debug info so that the debug info
goes and, stays in the original object
397
00:26:58,004 --> 00:27:02,624
files and doesn't need to be processed
by the linker, that kind of thing.
398
00:27:02,879 --> 00:27:03,169
Tim: Okay.
399
00:27:03,474 --> 00:27:04,824
And that.
400
00:27:06,069 --> 00:27:08,589
That's actually how things work
over in Windows land, isn't it?
401
00:27:08,739 --> 00:27:11,439
Where the debug information
is in a separate file.
402
00:27:12,339 --> 00:27:15,549
David: Puts the debug info in a
PDB file or something like that.
403
00:27:15,549 --> 00:27:20,129
I haven't really worked on Windows,
but that's what I've heard that but you
still need to generate that PDB file.
404
00:27:20,129 --> 00:27:24,849
So the other option is split debug
info 'cause unpacked, which leaves
the debug info in the original object
405
00:27:24,849 --> 00:27:28,599
files and it's up to the debugger to go
and find it when it needs it, which I
406
00:27:28,729 --> 00:27:29,019
Tim: yeah.
407
00:27:29,069 --> 00:27:29,969
David: A nice solution.
408
00:27:30,449 --> 00:27:30,739
Tim: Yeah.
409
00:27:30,739 --> 00:27:39,574
That does seem like a relatively
practical middle ground especially
if you are, using a debugger during
410
00:27:39,574 --> 00:27:43,834
development and testing, which presumably
that's why when you have debug symbols
411
00:27:43,954 --> 00:27:48,974
involved at all, 'cause you probably
strip them out at, for the release.
412
00:27:49,144 --> 00:27:50,314
Or the final build anyway.
413
00:27:51,814 --> 00:27:54,569
Yeah, that that's okay, but.
414
00:27:56,099 --> 00:27:56,819
The
415
00:27:57,689 --> 00:28:08,339
the cargo dot tomal file just provides
me with, I've got LT O zero thin and
fat slash one, but those are, that
416
00:28:08,399 --> 00:28:11,699
those would provide the sufficient
specificity of what you are saying.
417
00:28:12,119 --> 00:28:14,129
So I need to go into this case.
418
00:28:14,129 --> 00:28:24,189
It would be clang or which would be cargo
config to be able to really start to,
adjust the a knob, the knobs correctly.
419
00:28:24,279 --> 00:28:24,759
Is that right?
420
00:28:25,399 --> 00:28:30,359
David: No, you can tell in your
cargo, Tom, you can tell it that
you don't want debug info or you can
421
00:28:30,499 --> 00:28:30,829
Tim: Oh yeah.
422
00:28:30,829 --> 00:28:31,129
Sorry.
423
00:28:31,129 --> 00:28:31,909
Yeah, we can turn that off.
424
00:28:31,909 --> 00:28:32,299
Yeah, sure.
425
00:28:32,399 --> 00:28:32,519
David: or
426
00:28:32,629 --> 00:28:32,959
Tim: Of course.
427
00:28:32,969 --> 00:28:35,069
David: yeah, strip the debug info.
428
00:28:35,099 --> 00:28:42,909
And so actually, in terms of linking if I
have input files that have debug info in
them, but you tell the linker strip equals
429
00:28:42,909 --> 00:28:46,589
debug, like that's a flag strip, which
the rust compiler can pass to the linker.
430
00:28:47,909 --> 00:28:51,449
then linking is pretty much as fast as
if you didn't have the debug info there.
431
00:28:51,539 --> 00:28:56,999
So having the debug info present in the
object file is not a problem as long
as you tell the linker to strip it.
432
00:28:57,849 --> 00:29:01,709
At least in the case of wild and I
think mold, I think LLD possibly still
433
00:29:01,709 --> 00:29:05,159
processes the debug info and then just
gets rid of it after it's processed it.
434
00:29:05,339 --> 00:29:06,839
But yeah.
435
00:29:08,449 --> 00:29:08,629
Tim: Yeah.
436
00:29:08,629 --> 00:29:09,859
Which is an interest.
437
00:29:09,859 --> 00:29:10,129
Yeah.
438
00:29:10,129 --> 00:29:17,209
I mean there was a, an side note
there, which around architecture
and one of the yeah, the.
439
00:29:18,274 --> 00:29:26,254
One of the points that you made earlier
was the files don't actually l don't
actually lend themselves to having the
440
00:29:26,254 --> 00:29:35,074
right information at the right place or
the right time to really provide like
optimal performance or optimal throughput.
441
00:29:36,454 --> 00:29:43,804
And presumably that means that there
is a bit of a co-evolution of file
formats versus tools and I wonder when
442
00:29:43,804 --> 00:29:51,034
you're going to call Mutiny on the
ALF file format, for example, and say
443
00:29:51,034 --> 00:29:55,734
that its time is over or something and
propose some radical new solution there.
444
00:29:56,319 --> 00:29:59,049
David: I think to an extent it's not
necessarily even Elf, it's the problem.
445
00:29:59,049 --> 00:30:00,759
It's all the stuff that's
built on top of Elf.
446
00:30:00,939 --> 00:30:03,969
So it's the way ELF is used,
that's often problematic.
447
00:30:04,209 --> 00:30:06,009
But yeah, I mean there's probably
things about Elf as well.
448
00:30:07,209 --> 00:30:07,449
Yeah.
449
00:30:08,274 --> 00:30:11,694
Tim: Do you, I know that you, so your
mo, your primary motivation is speed.
450
00:30:12,609 --> 00:30:13,029
David: Yes.
451
00:30:13,029 --> 00:30:13,269
Yeah.
452
00:30:13,269 --> 00:30:17,879
I want so I spent about six years
as a small talk developer, and I got
453
00:30:17,879 --> 00:30:21,119
very used to having instantaneous
feedback from the compiler.
454
00:30:21,729 --> 00:30:26,439
Being able to edit stuff, edit
code while it's running, and
just see the change immediately.
455
00:30:26,799 --> 00:30:32,759
And I guess I want to regain that
feeling of spontaneity and instantaneous
in a compiled language like rust.
456
00:30:34,449 --> 00:30:36,974
Tim: So I'm going to have
a bit of a meta discussion.
457
00:30:37,034 --> 00:30:41,534
So for people that have not used
small talk when actually no.
458
00:30:41,589 --> 00:30:43,059
David, I'm sure that you
can explain this yourself.
459
00:30:43,089 --> 00:30:48,759
When I am interacting with a
small talk program, the small talk
virtual machine it feels live.
460
00:30:48,819 --> 00:30:50,169
Why does it feel live?
461
00:30:50,169 --> 00:30:54,369
It's this kind of, it feels very
immediate, but what's going on there?
462
00:30:55,394 --> 00:30:59,384
David: I guess it because it is a virtual
machine and because you, the, that
463
00:30:59,384 --> 00:31:03,944
virtual machine allows you to do things
like new things whilst it's running.
464
00:31:04,224 --> 00:31:10,439
It was designed that way from
scratch to be a machine that can be
updated while it's running and, yeah.
465
00:31:11,389 --> 00:31:13,309
Tim: Can you disprove?
466
00:31:13,344 --> 00:31:23,604
And one my memory would, I haven't
really used say the any of, I used
squeak actually for which is like
467
00:31:23,604 --> 00:31:28,294
a educational vision, but the but
you're also interacting with binary
468
00:31:28,294 --> 00:31:31,924
files like the, you're not really,
there's no source code really.
469
00:31:31,984 --> 00:31:34,144
It's like you are actually
in the environment.
470
00:31:34,144 --> 00:31:34,624
It's immersive.
471
00:31:36,024 --> 00:31:38,954
David: Yeah, I guess that comes from
having development environment is running
472
00:31:38,954 --> 00:31:42,104
as part of the same program as your
actual application, which is a bit weird.
473
00:31:42,104 --> 00:31:45,914
And I'm not saying that's a good thing but
it does lead to some interesting outcomes.
474
00:31:46,224 --> 00:31:50,774
It means that you can have, your objects
can override the way that they're
displayed in the development environment.
475
00:31:50,774 --> 00:31:54,784
And very extensible and flexible
and you can make changes to your
development environment while it's
476
00:31:54,784 --> 00:31:58,714
running, which can also break, if you
break things that can be bad, but yeah.
477
00:31:59,689 --> 00:32:00,079
It's fun.
478
00:32:02,444 --> 00:32:06,839
Tim: And the let's say that you have
infinite time and infinite resources.
479
00:32:06,839 --> 00:32:11,709
Are there other parts of rust or
things that where you think, oh gosh,
480
00:32:11,709 --> 00:32:16,329
there is an opportunity to fix that
and make it more, let's say snappier.
481
00:32:17,219 --> 00:32:18,209
David: Yeah, I think there's heaps.
482
00:32:18,239 --> 00:32:21,569
I don't think the linker is necessarily
even the most important thing to work on.
483
00:32:21,999 --> 00:32:26,199
But I guess I gravitate
towards independent projects.
484
00:32:26,199 --> 00:32:31,519
I can just work away on unencumbered and
on by myself even though I should probably
485
00:32:31,519 --> 00:32:34,469
be more social and go and, contribute
more to the actual rust compiler.
486
00:32:34,649 --> 00:32:38,329
But I think there's heaps that can be
changed in the rust compiler and in cargo.
487
00:32:38,329 --> 00:32:44,749
And, to give an example, so in cargo at
the moment if you tell cargo that you
wanna strip your binary, so you wanna
488
00:32:44,749 --> 00:32:49,129
strip debug info from your binary,
then it will go and rebuild everything.
489
00:32:50,044 --> 00:32:52,654
though it really only needs to change
the flags that's passing to the linker
490
00:32:53,384 --> 00:32:57,404
that's an example of a change that, I
should probably go and contribute, but
491
00:32:58,239 --> 00:33:01,209
Tim: Principle should be trivial ish.
492
00:33:01,479 --> 00:33:06,459
But presumably because it was
built that way and it works.
493
00:33:08,589 --> 00:33:10,239
Yeah, no, it would be fascinating.
494
00:33:10,299 --> 00:33:10,539
Yeah.
495
00:33:10,539 --> 00:33:14,229
Maybe it would be an a two hour job to
go and find the right part of the code
496
00:33:14,229 --> 00:33:18,219
and then go and make the change and then
push the PR up and see where it lands.
497
00:33:18,399 --> 00:33:21,999
Yeah, no, that's interesting
because I have also
498
00:33:24,009 --> 00:33:33,789
had a difficulty with contributing to
large projects, which is that I. My time
of the available time that I have as
499
00:33:33,889 --> 00:33:40,189
quite patchy in the sense of I don't have
a lot of free time and I find it really
500
00:33:40,189 --> 00:33:48,649
difficult to commit to providing regular
contributions to an established project.
501
00:33:49,039 --> 00:33:58,939
And doing something by yourself
essentially in isolation is one way
to guard against needing to let anyone
502
00:33:58,939 --> 00:34:06,099
down or it gives you a little bit more
freedom to experiment 'cause you're
not bound by code conventions and
503
00:34:06,099 --> 00:34:10,189
so forth and gaining agreement from
lots of the rest of the community.
504
00:34:11,029 --> 00:34:17,999
So I can see why working on something
off to the side is like a, it's quite
a sound strategy for many people.
505
00:34:18,669 --> 00:34:18,869
David: Yeah.
506
00:34:18,869 --> 00:34:22,759
By the same token I really
like working with other people
and collaborating with people.
507
00:34:22,829 --> 00:34:26,789
And it's been great actually having,
I've got a few very active contributors
508
00:34:26,789 --> 00:34:31,589
to Wild that have been contributing a
lot of great new features which is, and
509
00:34:31,589 --> 00:34:31,889
Tim: all right.
510
00:34:31,889 --> 00:34:33,859
This is it's inverted
now you're in charge.
511
00:34:34,009 --> 00:34:42,974
The, do you find the process of, and
so that's really nice to hear actually
that the process of becoming or being
512
00:34:43,024 --> 00:34:51,434
a maintainer of an established project
is a really positive one, or at least
it has been in the main the, are there
513
00:34:51,434 --> 00:35:00,334
any other sort of reflections on doing
or behaving or I guess open source,
being an open source maintainer is there
514
00:35:00,334 --> 00:35:03,869
anything that has been different than
what you expected about that process?
515
00:35:04,689 --> 00:35:09,789
David: No, I think it's been,
I've had other open source
projects in the, in Rust that I've
516
00:35:09,969 --> 00:35:14,809
On previously, and they've had
contributions, but I guess they
haven't had the same sort of
517
00:35:15,199 --> 00:35:18,399
long term, people were there
actually to work on the project.
518
00:35:18,449 --> 00:35:22,729
My other projects have more been they've
maybe had a few repeat contributors,
but for the most part it's felt that
519
00:35:22,729 --> 00:35:25,759
they're implementing a feature that
they particularly want for their use.
520
00:35:25,759 --> 00:35:28,609
'cause they're a user and they wanna just
implement the features that they want.
521
00:35:29,009 --> 00:35:32,079
Whereas this very much feels contributors
who are there as part of the project
522
00:35:32,079 --> 00:35:35,329
who are there to help drive it
forward and wanna see it succeed.
523
00:35:35,329 --> 00:35:36,914
And yeah, that's it.
524
00:35:36,919 --> 00:35:38,989
It feels like we're a
team working on that.
525
00:35:39,044 --> 00:35:39,644
Tim: That's neat.
526
00:35:39,709 --> 00:35:41,809
David: that's a different feeling
and I quite, I really like that.
527
00:35:42,944 --> 00:35:43,329
Tim: Oh, cool.
528
00:35:44,089 --> 00:35:44,209
I.
529
00:35:46,274 --> 00:35:54,124
Keep thinking about something that
you mentioned about the structure
of of, we were talking about our
530
00:35:54,124 --> 00:35:56,404
files and so forth and things not
being quite in the right place.
531
00:35:56,974 --> 00:36:03,034
But I was thinking about this more
broadly in terms of the way that you've
structured your code and with the time.
532
00:36:03,094 --> 00:36:10,054
I'm curious about whether or not you
invested a lot of time early thinking
about the codes architecture and how
533
00:36:10,054 --> 00:36:19,744
you wanted it to grow over the time,
or did you start with something small
and then just radically hack and change
534
00:36:20,074 --> 00:36:28,474
to make, and then did you adjust the
codes a structure or architecture
according to new needs that popped up?
535
00:36:29,359 --> 00:36:31,479
David: Yeah, it's, it is definitely
gone through a lot of evolution.
536
00:36:31,779 --> 00:36:34,449
And that evolution has to an
extent evolved as I've gained.
537
00:36:35,274 --> 00:36:37,264
understanding of things
I'm trying to solve.
538
00:36:37,674 --> 00:36:39,984
Early on I might have done
something a particular way because
539
00:36:39,984 --> 00:36:42,924
I had a particular model in my
mind as to how that thing worked.
540
00:36:43,344 --> 00:36:45,444
And then I discovered an
exception to that case.
541
00:36:45,444 --> 00:36:51,004
I discovered some inputs to the
linker that meant, oh, actually mental
model of this was actually wrong.
542
00:36:51,064 --> 00:36:53,764
I need to actually, fit a different model.
543
00:36:54,094 --> 00:36:57,444
And so then I have to, make, restructure
my code in order to fit that new
544
00:36:57,444 --> 00:37:00,294
model of how I now understand this
problem actually needs to be solved.
545
00:37:01,134 --> 00:37:03,264
So it's evolved quite a
lot as time has gone by.
546
00:37:04,134 --> 00:37:07,104
But I think Rust makes it, rust,
makes it a lot easier to make those
547
00:37:07,104 --> 00:37:10,574
big refactorings, like you can really
let the compiler drive the change.
548
00:37:10,694 --> 00:37:14,664
And often, once you're done
making the change and everything
compiles, then it, it works.
549
00:37:14,664 --> 00:37:15,604
And, a
550
00:37:15,779 --> 00:37:18,309
Tim: Yeah it's almost a shame that's
a cliche because it's quite true.
551
00:37:19,304 --> 00:37:19,424
David: Yeah.
552
00:37:20,659 --> 00:37:24,409
Tim: The because I wonder about
that from two points of view.
553
00:37:24,409 --> 00:37:28,819
One is you're now, you're a team,
and so you can't really impose
554
00:37:28,819 --> 00:37:34,249
massive radical restructures on
everybody anymore, presumably.
555
00:37:34,789 --> 00:37:37,909
David: Even when I'm doing the code
base is big enough now that even
556
00:37:37,909 --> 00:37:41,539
when I'm doing a big restructuring,
it won't be touching all the files.
557
00:37:41,539 --> 00:37:46,589
Like it might be a massive restructuring
and say the way layout is done or
the way writing is done, or, the
558
00:37:46,589 --> 00:37:49,679
way sim the symbol database is being
built, or that, that kind of thing.
559
00:37:49,709 --> 00:37:53,039
Then and as long as I am aware
of what other people are working
560
00:37:53,039 --> 00:37:56,519
on, then I can schedule at
work to reduce merge conflicts.
561
00:37:57,279 --> 00:38:02,109
There's some changes at the moment that
I wanna make, but I'm waiting for some
other big pull requests to get in before
562
00:38:02,109 --> 00:38:07,389
I look to do those changes because I
know that if I don't, then I'll get
probable merge conflicts or someone will.
563
00:38:09,289 --> 00:38:20,129
Tim: Do you, and you also need to have,
think about things upstream in terms of
you need to be able to be compliant with.
564
00:38:21,014 --> 00:38:28,394
I guess command line flags that
are specified for other linkers,
those must be very stable now.
565
00:38:28,464 --> 00:38:28,614
David: Yeah.
566
00:38:28,614 --> 00:38:33,274
There's certainly, new flags are
added all the time and there's,
and they've been being added for
567
00:38:33,274 --> 00:38:37,974
decades, so there's a massive long
tail of often very obscure flags.
568
00:38:37,974 --> 00:38:43,324
If you look through the Goo LD
man page there's so many flags and
we don't support a lot of them.
569
00:38:43,884 --> 00:38:46,454
You'll look at them and you'll be
just like, okay is anyone using that?
570
00:38:46,454 --> 00:38:46,964
I don't know.
571
00:38:46,964 --> 00:38:50,474
Maybe someone was using it at
the time they implemented it,
but now they've stopped using it.
572
00:38:50,574 --> 00:38:55,689
And so I've let implementation of flags be
driven by us finding real world real world
573
00:38:55,689 --> 00:38:58,719
projects that are actually using the flag
we need to make it work without project.
574
00:38:59,559 --> 00:39:01,599
but new flags are being added and.
575
00:39:01,974 --> 00:39:02,154
Yeah.
576
00:39:04,039 --> 00:39:11,429
Tim: Yeah, and there's another
thing of large projects collapse
on their own weight over time.
577
00:39:11,459 --> 00:39:19,289
You could imagine being a longstanding
project like the GNU linker and
probably started quite simple.
578
00:39:19,949 --> 00:39:25,949
But now you need to support every
architecture and every command
579
00:39:25,949 --> 00:39:32,179
line flag from, the early seventies
until now into the future.
580
00:39:32,239 --> 00:39:37,399
And that weight of expectation
and stability means that.
581
00:39:38,444 --> 00:39:47,204
The sort of the code OSS in some way which
limits your ability to really experiment,
or at least that's my presumption,
582
00:39:47,234 --> 00:39:51,494
which is where the opportunity
for projects like wild come in.
583
00:39:54,159 --> 00:39:55,564
David: Yeah, to an extent.
584
00:39:56,984 --> 00:40:00,729
A lot of the flags don't have, like a
lot of the flags are relatively simple.
585
00:40:01,039 --> 00:40:04,404
There's plenty of flags in there
that, you've got the think the code
to pass the flag, and then you've got
586
00:40:04,404 --> 00:40:08,424
a little bit of code to process the
flag and it's pretty isolated and you
don't have to worry about it too much.
587
00:40:08,424 --> 00:40:12,624
There are other cases where it's
a lot more cross-cutting and you,
the flag really does change a lot.
588
00:40:13,384 --> 00:40:17,434
You can, as long as you've got tests of
the flag, you can restructure the way
589
00:40:17,434 --> 00:40:22,354
things work internally and then verify
that you haven't broken that flag.
590
00:40:22,354 --> 00:40:22,504
So
591
00:40:23,469 --> 00:40:26,319
Tim: yeah, and it sounds like, you write
your little c program that exploits some
592
00:40:26,319 --> 00:40:31,299
little kind of corner case and then see
if it behaves the way that you expect.
593
00:40:31,299 --> 00:40:39,269
And then hello fed magic, the, I
was really curious about whether
or not it's easier to implement
594
00:40:39,719 --> 00:40:44,489
new optimizations or new ideas for
optimizations in a newer code base.
595
00:40:45,209 --> 00:40:56,109
Are there any things in terms of, even
though you're focusing on speed, has
a consequence of trying this project
596
00:40:56,229 --> 00:41:03,549
meant that you are now looking at new
ways to get better output as well.
597
00:41:03,629 --> 00:41:10,759
David: Mostly I'm focused on link, how
fast I can link, and I guess I've written
the linker from the start with the idea
598
00:41:10,759 --> 00:41:15,079
that it's very multi-threaded and I do
everything multi-threaded, and that's
certainly where you get on of speed from.
599
00:41:15,169 --> 00:41:20,639
But also I think my approach to
multi-threading I don't actually know
how the other linkers do that, so I
600
00:41:20,639 --> 00:41:23,549
haven't really looked particularly at
the code of any of the other linkers
601
00:41:23,919 --> 00:41:27,149
because I just wanted to figure out
things myself do things my own way.
602
00:41:27,329 --> 00:41:30,089
And so I'll observe the other
linkers mostly as a black box.
603
00:41:30,089 --> 00:41:34,949
I'll run goo LD and see how it
behaves with particular inputs
and see what outputs it produces.
604
00:41:35,589 --> 00:41:38,079
And that's how I've
mimicked their behavior.
605
00:41:38,979 --> 00:41:40,999
Sorry, I don't, I feel like there's
more to your question than that.
606
00:41:41,269 --> 00:41:41,809
I'm not actually sure
607
00:41:41,834 --> 00:41:47,624
Tim: No, I was wondering whether or
not one of the, there was a, when you
mentioned that you were getting new
608
00:41:47,624 --> 00:41:54,644
contributions, I was wondering whether
or not you had interesting optimization
passes that you had not really thought of
609
00:41:55,184 --> 00:42:05,654
just emerge, and the hidden question is
it easier to write system software in rust
than it is to write system software in C?
610
00:42:06,194 --> 00:42:07,094
David: I think it is.
611
00:42:07,304 --> 00:42:08,324
Yeah, I think it definitely is.
612
00:42:08,384 --> 00:42:12,824
Certainly, having a very
multi-threaded program like Rust
is great for multi-threading.
613
00:42:12,874 --> 00:42:16,919
It, it really if you make mistakes
with the threading, then you
tend to get compile errors.
614
00:42:17,079 --> 00:42:18,249
And scope threads are awesome.
615
00:42:18,729 --> 00:42:20,799
I've made so much use of
effectively scope threads.
616
00:42:20,829 --> 00:42:23,754
I'm using rayon for parallelism,
but the same sort of idea.
617
00:42:24,424 --> 00:42:25,339
And so that lets me have.
618
00:42:26,764 --> 00:42:32,154
Data that's just shared across the threads
without any kind of locking or there's no
619
00:42:32,254 --> 00:42:36,469
mut texts, there's no if it's read only
access, then I don't even need Atomics.
620
00:42:36,469 --> 00:42:38,209
I can just access the data just as is.
621
00:42:38,689 --> 00:42:41,299
And I think that's really great
from a performance perspective.
622
00:42:41,629 --> 00:42:46,879
And Rust makes it easier to, or a
lot easier to get that stuff right.
623
00:42:51,179 --> 00:42:54,209
Tim: Have you, do you feel like
you're a bit Russ programmer now, or
624
00:42:54,259 --> 00:42:58,669
you already were an experienced rust
program before you started, but the.
625
00:42:59,344 --> 00:43:03,339
But it seems like it's reinforced
your belief that this is a language
for doing quite big things.
626
00:43:04,569 --> 00:43:08,919
David: Yeah, I very much fell
in love with Rust from the
moment I first learned about it.
627
00:43:09,179 --> 00:43:11,099
Back around about when 1.0 was released.
628
00:43:11,099 --> 00:43:16,219
I was, when I first heard of Rust and
watched a few videos and I. I could
see, because I've been programming
629
00:43:16,219 --> 00:43:21,029
in CC plus, or c and c plus since
the, since the mid nineties.
630
00:43:21,029 --> 00:43:25,859
So I've been using CNC plus for a long
time, and Russ just solved so many of
631
00:43:25,859 --> 00:43:30,419
the problems that, that I've encountered
over the years in those languages.
632
00:43:31,169 --> 00:43:36,174
And my, my learning of rust is
certainly it's always evolving.
633
00:43:36,234 --> 00:43:42,474
Like even in the time I've been writing
the linker, I've seen my, the way
that I write rust evolve and change.
634
00:43:42,754 --> 00:43:47,874
As a small example the way that I'm using
more vertical white space now than I
used to, so I'm putting blank lines in.
635
00:43:48,774 --> 00:43:55,244
I had a comment early on in, when I'd
started the linker someone who was working
with someone who a screen reader, was
636
00:43:55,244 --> 00:44:01,124
having trouble with reading the code and
they said, they commented on the lack of
blank lines, and I'd previously always
637
00:44:01,124 --> 00:44:06,764
put blank lines in to have meaning,
but then I was never quite sure where
I. Where they needed to have meaning.
638
00:44:06,764 --> 00:44:07,334
And so I didn't put them
639
00:44:07,499 --> 00:44:07,699
Tim: Yeah.
640
00:44:07,844 --> 00:44:09,614
David: Whereas now I'm pretty
much just putting blank lines in.
641
00:44:09,614 --> 00:44:15,124
If there's multiple, if I've got a big
if else, then I'll put a blank line
before it and after it just to makes
642
00:44:15,124 --> 00:44:17,854
it a bit easier to separate those
things out and see them visually.
643
00:44:17,914 --> 00:44:22,654
And that's, I'm still updating the code
as I go, so it's not like I went through
and updated all the code, but, whenever
644
00:44:22,654 --> 00:44:26,829
I'm editing a function anyway, I'm adding
in those blank lines and things out.
645
00:44:27,219 --> 00:44:28,539
Tim: That's fascinating actually.
646
00:44:28,719 --> 00:44:36,309
Could you want do you, did you investigate
why it's easier for non-sighted,
people that are using screen reader
647
00:44:36,309 --> 00:44:46,479
technologies to pause code with blank
lines, presumably because there are
pauses added when it's spoken or.
648
00:44:46,529 --> 00:44:50,079
David: maybe, I'm not sure, and in
fact I, I spoke to to someone at Rust
649
00:44:50,079 --> 00:44:54,579
Week who uses a. A screen reader and
they didn't think it would affect them.
650
00:44:54,629 --> 00:44:56,249
But now that I'm used to
it, I prefer it myself.
651
00:44:56,249 --> 00:45:03,539
Like even as a sighted person, I just
prefer, like, when I look at my old code
from my old style, it looks very squished.
652
00:45:04,554 --> 00:45:05,094
Tim: Yeah no.
653
00:45:05,094 --> 00:45:08,094
I actually quite like a
relaxed style of code as well.
654
00:45:08,394 --> 00:45:16,374
I am continuously fighting one
part of rust, which is its tendency
to have this right wood drift.
655
00:45:16,854 --> 00:45:18,564
Especially when you start to match things.
656
00:45:18,564 --> 00:45:27,084
And then if you have results and objects
and, sorry, result wrapping an option
of something which might be an eno.
657
00:45:27,084 --> 00:45:36,734
And now I've got three layers of match
potentially and I'm still trying to figure
out exactly the optimal way to do that.
658
00:45:37,064 --> 00:45:45,174
I tended to go with if else and sorry if
let and, treat, but actually match on the
exceptional case early and early return.
659
00:45:45,234 --> 00:45:56,034
And but yeah, no the it is interesting
how much style or so presumably like
the compiler doesn't care where your
660
00:45:56,034 --> 00:46:00,414
white space, the compiler doesn't care
the way that things look, and yet it
661
00:46:00,414 --> 00:46:04,484
seems to have this really, we have this
intimate connection with with code.
662
00:46:05,304 --> 00:46:09,864
David: And another place where
my style has changed a little
bit is in regards to panics.
663
00:46:10,524 --> 00:46:13,674
I'd previously very much followed
the same school of Thoughters.
664
00:46:13,754 --> 00:46:19,524
I dunno if you've read Burnt
Sushi's writeup on panics,
and basically the position is.
665
00:46:19,859 --> 00:46:24,049
Panics are fine for bugs, and
if it's not, if it's not a bug,
you should return an error.
666
00:46:24,719 --> 00:46:27,689
And and I very much subscribe to that,
I'm perfectly happy to call unwrap.
667
00:46:28,569 --> 00:46:34,719
If I think that this thing should
not be none, and if it's none, then
that's a bug, then I'll call unwrap
668
00:46:34,719 --> 00:46:37,989
on it because, I'm happy if it's,
if it is none, then that's a bug.
669
00:46:38,019 --> 00:46:40,899
And panicking when there's a bug is
fine in my per, from my perspective.
670
00:46:41,199 --> 00:46:48,189
However, where I've changed slightly
is I've found there's often benefits
to returning a result, even in
671
00:46:48,189 --> 00:46:53,889
cases of a bug, especially if the
bug is non-local and fairly cross.
672
00:46:53,939 --> 00:46:59,399
I've got cases in the linker where, for
example, you I need to allocate space in
one section of the code and then allocate
673
00:46:59,399 --> 00:47:02,519
address in another section of the code,
and then actually write those things
674
00:47:02,519 --> 00:47:06,959
out and yet another section of the code,
and those three things need to be sync.
675
00:47:07,019 --> 00:47:10,559
If I allocate too much or too little
space, then I won't be able to allocate
676
00:47:10,559 --> 00:47:12,869
addresses correctly, or I won't
be able to write it out correctly.
677
00:47:13,274 --> 00:47:19,504
And so if I discover there's
too much space or too little
space, a bug in the linker.
678
00:47:20,344 --> 00:47:24,124
and I could panic, but I found
it's often better to, in those
679
00:47:24,124 --> 00:47:29,254
cases, to return an error I can
attach more semantic information.
680
00:47:30,034 --> 00:47:31,084
I don't really care about the back.
681
00:47:31,504 --> 00:47:31,984
Basically.
682
00:47:32,034 --> 00:47:33,654
I know where I am in the code.
683
00:47:33,714 --> 00:47:35,424
There's only really one path to get there.
684
00:47:35,574 --> 00:47:36,594
The back trace is irrelevant.
685
00:47:36,774 --> 00:47:41,844
What I care about is which symbol was I
processing, which file was I processing,
which section was I processing?
686
00:47:43,084 --> 00:47:45,784
That's the sort of the semantic
information that I really care about.
687
00:47:45,934 --> 00:47:50,344
And I get that much more easily if I
return an error rather than a result.
688
00:47:50,344 --> 00:47:52,204
So that's how I've shifted
a little bit in that
689
00:47:53,064 --> 00:47:59,084
Tim: So you're packing in, making use
of the fact that you can result as
a value and you can pack in whatever
690
00:47:59,084 --> 00:48:02,174
information that you actually need
in order to handle that value.
691
00:48:02,474 --> 00:48:07,184
And yeah, in this case it sounds like
your collecting up information from
692
00:48:07,184 --> 00:48:11,839
higher up in the call stack and then
propagating the result back up to that.
693
00:48:11,839 --> 00:48:17,419
So you can gather all of that
information and then spit it out
there is smarter, or at least it
694
00:48:17,539 --> 00:48:22,139
reduces the debugging time than doing
it just panicking down the bottom.
695
00:48:22,414 --> 00:48:22,654
David: Yep.
696
00:48:22,729 --> 00:48:30,139
Tim: There is actually a. I think
that error handling or idiomatic error
handling is one of these areas where,
697
00:48:32,809 --> 00:48:37,669
I don't know if the Russ community
really has a unified story.
698
00:48:37,969 --> 00:48:43,224
Or at least I think there's
kind of multiple levels of
maturity in every project.
699
00:48:43,489 --> 00:48:48,739
And one of the things that people will
lean on are trait objects quite a lot,
700
00:48:48,829 --> 00:48:54,379
especially when they start to intermix
or intermingle different era types.
701
00:48:54,379 --> 00:48:57,899
And then they have this
problem of, type ratio.
702
00:48:58,319 --> 00:48:58,859
And I
703
00:49:00,899 --> 00:49:09,269
struggle a little bit to trying to give
people generalized advice on how to
handle errors because the tools that
704
00:49:09,269 --> 00:49:16,034
Russ provide you, I. Give you lots of
options for being able to handle your
specific case, but it's not as general
705
00:49:16,034 --> 00:49:19,574
purpose as say, like something in the
JVM where you just get the stack trace
706
00:49:19,574 --> 00:49:23,634
or what have you, and, or it's just an
exception which you then have to handle.
707
00:49:24,024 --> 00:49:31,454
But actually and it strikes me as quite
hard to provide like one canonical
answer that will fit all cases.
708
00:49:31,894 --> 00:49:34,864
David: Yeah, I think it's very
much dependent upon what the
709
00:49:34,864 --> 00:49:37,234
program you're writing is or what
the library you're writing is.
710
00:49:37,264 --> 00:49:42,874
And in the case of a library, it's tricky
because you might have multiple different
users that have different competing needs.
711
00:49:43,124 --> 00:49:46,634
Certainly I've worked, when I've
worked in embedded before we had
712
00:49:46,634 --> 00:49:51,224
to pretty much avoid all code that
even possibly could panic because.
713
00:49:51,554 --> 00:49:57,014
Not so much because we didn't wanna
panic as much as if we had code
that could panic, then that would
714
00:49:57,014 --> 00:49:59,174
pull in the panic machinery, which
would pull in all the formatting
715
00:49:59,174 --> 00:50:02,144
machinery, which was very large, and
it would increase our binary size.
716
00:50:02,174 --> 00:50:03,644
And so from that
717
00:50:03,799 --> 00:50:05,959
Tim: So just, the print
line macro is not free.
718
00:50:05,994 --> 00:50:06,384
David: no.
719
00:50:06,829 --> 00:50:07,049
Yep.
720
00:50:08,179 --> 00:50:13,939
Tim: if you've got format augs and format
augs needs to know how to basically format
everything and convert it to a string.
721
00:50:14,209 --> 00:50:18,159
Turns out that's pretty tricky for
things like floating point numbers
and other, basically everything else.
722
00:50:18,159 --> 00:50:22,089
So yeah, I can imagine why if you
only have a couple of kilobytes of
723
00:50:22,089 --> 00:50:26,569
ram or presumably tens of kilobytes
but that's still not free at all.
724
00:50:26,779 --> 00:50:30,429
That's funny because I always thought that
non panicking, we need panic free code.
725
00:50:30,489 --> 00:50:34,049
Partially be mostly because
we just don't wanna panic.
726
00:50:34,079 --> 00:50:44,759
Like we don't want it to stop, but
it's also, we don't want to include
code that can handle a panic.
727
00:50:45,869 --> 00:50:49,859
Is the way out there just
abort or, and just, or.
728
00:50:51,914 --> 00:50:53,744
Essentially panicking.
729
00:50:53,804 --> 00:50:55,574
Your panic handler is a no op.
730
00:50:55,574 --> 00:51:00,374
You just jump to either an in
bus, busy loop or you just shut
down the device or something.
731
00:51:00,374 --> 00:51:00,644
Or
732
00:51:01,034 --> 00:51:01,574
Can you,
733
00:51:01,594 --> 00:51:02,629
David: case we returned results.
734
00:51:02,819 --> 00:51:03,509
Like even though
735
00:51:03,549 --> 00:51:03,839
Tim: okay.
736
00:51:03,979 --> 00:51:06,749
David: panicking is reasonable, like
returning an error result is also.
737
00:51:07,494 --> 00:51:08,154
think you can do.
738
00:51:08,344 --> 00:51:14,184
I think it comes back to there's
lots of different contexts in which,
and there's no one way of doing
739
00:51:14,184 --> 00:51:19,134
things in rust because it really
much depends on what you're building,
as to what the right approach is.
740
00:51:19,794 --> 00:51:23,894
And so diff different approaches work
better and worse in different contexts.
741
00:51:25,639 --> 00:51:27,024
Tim: And David, I need to let you go
742
00:51:27,614 --> 00:51:27,824
David: Ah,
743
00:51:28,794 --> 00:51:31,584
Tim: because I've
dominated your time so far.
744
00:51:31,634 --> 00:51:37,003
I, I. But was there anything else that
you would like to add to the discussion?
745
00:51:38,023 --> 00:51:38,353
David: I guess
746
00:51:38,418 --> 00:51:38,538
Tim: I.
747
00:51:38,713 --> 00:51:42,153
David: wants to help build a linker, then,
I'm always looking for more contributors.
748
00:51:42,163 --> 00:51:45,073
Come along to and file an issue
or discussion or book some time
749
00:51:45,073 --> 00:51:48,033
in my calendar and we can have
a chat about what you wanna do.
750
00:51:48,033 --> 00:51:50,973
And it'd be great to have more
people come along and join in.
751
00:51:52,213 --> 00:51:52,843
Tim: Superb.
752
00:51:53,563 --> 00:51:55,243
Now are you a social media person?
753
00:51:56,218 --> 00:51:57,418
David: I have discord.
754
00:51:57,598 --> 00:51:58,138
That's about it
755
00:51:58,163 --> 00:52:07,073
Tim: Okay I will make sure that there is,
they will try and make sure that it's very
easy to find wild and very easy to find
756
00:52:07,173 --> 00:52:15,243
how to take you up on the offer because
I strongly recommend to anyone that wants
to learn more and maybe contribute to do
757
00:52:15,293 --> 00:52:22,353
it's a getting this over the line, getting
this sort of stabilized or, it is stable,
but bringing more prominence to this area.
758
00:52:22,383 --> 00:52:33,303
It could revolutionize the experience
of developing with the language a
really strong, just hearing that
759
00:52:34,123 --> 00:52:41,723
this qualitative effect of like near,
instantaneous builds would just be,
would it, would, it would be glorious.
760
00:52:41,903 --> 00:52:42,843
And no.
761
00:52:42,843 --> 00:52:48,963
So thank you so much for dedicating so
much time to it and and for being so
generous with your time now and also
762
00:52:48,963 --> 00:52:51,993
supporting and nurturing other people
who wish to contribute to the project.
763
00:52:52,518 --> 00:52:53,088
David: Yeah, no worries.
764
00:52:53,088 --> 00:52:53,358
Thanks.
765
00:52:53,358 --> 00:52:54,538
It's been a fun conversation.
766
00:52:55,333 --> 00:52:55,723
Tim: Oh, cool.
767
00:52:55,753 --> 00:52:56,113
Okay.
768
00:52:56,113 --> 00:52:56,653
Thanks David.