WEBVTT
1
00:00:18.180 --> 00:00:19.420
Hey, Ben.
2
00:00:19.420 --> 00:00:21.100
Hey, Matt.
3
00:00:21.100 --> 00:00:28.040
I can't believe we're laughing already. Ben, very, very, I don't know what the right word would be, but like, you've been auspiciously auspicious.
4
00:00:28.040 --> 00:00:28.040
Auspiciously?
5
00:00:28.040 --> 00:00:39.990
That's not even, though that's not the right word, but you took your microphone and kind of bent it out of the way so we could have a nice talk, and your headset microphone isn't in the way.
6
00:00:39.990 --> 00:00:40.120
Yeah, right.
7
00:00:40.120 --> 00:00:45.940
Oh gosh, it's too early in the week to be recording a podcast, and yet here we are.
8
00:00:45.940 --> 00:00:47.160
You mean Friday? It's...
9
00:00:47.160 --> 00:00:51.060
Yeah, I guess. I don't know, now everyone knows when we record these.
10
00:00:51.060 --> 00:00:52.660
There's not a lot of time left, my man.
11
00:00:52.660 --> 00:01:00.860
That's true. Somehow we're in May as well, which, this podcast will go out in May as well, because it has to.
12
00:01:00.860 --> 00:01:02.320
Yeah. Yeah, right, right.
13
00:01:02.320 --> 00:01:03.580
Otherwise, we're in trouble.
14
00:01:03.580 --> 00:01:04.720
We've not got any in the bank.
15
00:01:04.720 --> 00:01:05.240
Okay.
16
00:01:05.240 --> 00:01:08.560
Yeah, so what are we talking about today, Ben?
17
00:01:08.560 --> 00:01:32.250
I thought we could do at least one podcast, and maybe we'll do this again, on kind of like how computers actually work. And I think a way in which to cover the extremely broad topic of how computers actually work is to look at some of the maybe non-intuitive behaviors that they have.
18
00:01:32.250 --> 00:01:33.010
I love it.
19
00:01:33.010 --> 00:01:33.260
Oh.
20
00:01:33.260 --> 00:01:38.000
Things that, you know, people kind of take for granted as sort of like an abstraction that you rely on.
21
00:01:38.000 --> 00:01:48.890
And, you know, peel back the covers a little bit and explore how it works. And maybe even, if we get to it, some like non-obvious ways in which it breaks.
22
00:01:48.890 --> 00:02:00.490
Right. And like when it doesn't work, what is actually happening? Because I think that sort of understanding the failure modes of these things is a great way to understand what's actually going on.
23
00:02:00.490 --> 00:02:02.190
So that's my general idea.
24
00:02:02.190 --> 00:02:16.520
For certain, yeah. OK, so we've got like this sort of general idea of the theme of how computers really work, which obviously is very close to my heart as well. So this sounds brilliant.
25
00:02:16.520 --> 00:02:28.160
And then we were discussing some things before we started, and you know, shooting around ideas. And I sort of suggested the, there used to be like an old Google interview question, which was, what happens, you know, you type google.com into a browser, you hit enter, what happens?
26
00:02:28.160 --> 00:02:34.960
And you know, I always liked that because there's almost any number of ways that you can go, but how about we simplify that bit?
27
00:02:34.960 --> 00:02:43.700
What happens if I have curl rather than a whole web browser there, and I curl some simple URL, right?
28
00:02:43.700 --> 00:02:44.580
Right.
29
00:02:44.580 --> 00:02:46.760
Not, well, not JavaScript or anything. Well, I guess it doesn't matter, but yeah.
30
00:02:46.760 --> 00:02:47.600
I mean, could be JavaScript. A document.
31
00:02:47.600 --> 00:02:47.800
I guess so.
32
00:02:47.800 --> 00:02:49.280
You're fetching a document with curl.
33
00:02:49.280 --> 00:02:52.160
It doesn't even matter at that point, because it'll just print it out in front of you.
34
00:02:52.160 --> 00:02:52.710
That's right.
35
00:02:52.710 --> 00:02:54.100
Yeah.
36
00:02:54.100 --> 00:03:01.550
So, so yeah. When should we try something like that? What happens if we curled google.com? In fact, I did this only earlier today as I was playing around with sandboxes.
37
00:03:01.550 --> 00:03:02.010
Yeah.
38
00:03:02.010 --> 00:03:02.940
Oh, perfect.
39
00:03:02.940 --> 00:03:07.710
And I curled exactly HTTPS google.com. And I know exactly what it returns.
40
00:03:07.710 --> 00:03:11.200
Can I, can I offer a guess before you tell me the answer?
41
00:03:11.200 --> 00:03:12.100
You can offer away.
42
00:03:12.100 --> 00:03:19.220
Yes. Does it give you a redirect? Something in the HTTP 300 range would be my guess.
43
00:03:19.220 --> 00:03:26.540
It does. It tells me the document has moved, and it tells me the document has moved to www.google.com, which was like, oh, really?
44
00:03:26.540 --> 00:03:26.860
Yes, okay.
45
00:03:26.860 --> 00:03:29.800
Yeah, I guess because I was very lazy.
46
00:03:29.800 --> 00:03:32.940
Yeah, so how does it tell you that?
47
00:03:32.940 --> 00:03:43.740
Well, in this instance, the curl told me just a document. It printed out a document in front of me that said 300 or whatever it was, moved, document moved.
48
00:03:43.740 --> 00:03:50.380
And then it had a tiny little bit of HTML that came to me, because I didn't have, you know, dash D dash or whatever the thing is to print out all the headers and stuff like that. But no.
49
00:03:50.380 --> 00:04:06.180
Okay. Okay. So you didn't see, so this was going to be my question. My understanding of this is, whenever you request a document over HTTP or HTTPS, you're getting two things back: you're getting the headers and you're getting the document, the document body.
50
00:04:06.180 --> 00:04:06.400
Aha.
51
00:04:06.400 --> 00:04:06.620
Yeah.
52
00:04:06.620 --> 00:04:16.180
Right. And so when it was redirecting you, all you were seeing there was the document body, which was some HTML. Is that correct?
53
00:04:16.180 --> 00:04:25.580
Correct. A very small piece of HTML, which you would normally never see. You know, a browser would render that, really because the real meat of it was in the headers, right?
54
00:04:25.580 --> 00:04:26.420
Mm-hmm. Mm-hmm. It would just redirect you. Yeah.
55
00:04:26.420 --> 00:04:35.100
Yeah. Okay. So then when you do that, and I'm just kind of going through, I'm trying to like answer the interview question here a little bit.
56
00:04:35.100 --> 00:04:36.760
That's right, yeah.
57
00:04:36.760 --> 00:04:38.980
So when you, or maybe I'm interviewing you. I don't even know how this is supposed to work.
58
00:04:38.980 --> 00:04:44.680
I don't know which way around this is anymore, because I thought it was going to go the other way and now we seem to have flipped it. Yeah, so carry on.
59
00:04:44.680 --> 00:04:46.720
I don't know who's, who's, I mean, you know, it's a collaboration.
60
00:04:46.720 --> 00:04:48.020
What even job is this?
61
00:04:48.020 --> 00:04:49.060
We're collaborating here. No one knows.
62
00:04:49.060 --> 00:04:50.780
What are the benefits anymore?
63
00:04:50.780 --> 00:04:51.880
We don't have jobs anymore. They're all taken by robots.
64
00:04:51.880 --> 00:04:53.400
Oh, yeah, that's true.
65
00:04:53.400 --> 00:05:09.460
So when you make this request, you're going to get back headers and you're going to get back the body. Like, how is the request delivered and how exactly is the body returned? Like, what protocol is it using to do that?
66
00:05:09.460 --> 00:05:16.620
Right, right. Well, I mean, I know that the particular command I was using is absolutely HTTP slash 1.1.
67
00:05:16.620 --> 00:05:17.300
Mm-hmm.
68
00:05:17.300 --> 00:05:39.760
And I mean, this is one of those cool things. I know that maybe this is showing our age a little bit, or certainly my age. I think I've got a few years on you. But at university in the late 90s, in the mid 90s, darn, you know, learning that you could Telnet to a port on a machine and just type stuff in.
69
00:05:39.760 --> 00:05:56.000
And get them to do things. So like, you know, the most happy day was to discover how to send emails from other people. I don't recommend that folks try this anymore. I don't think it works anymore, but you used to be able to Telnet to the mail port and give it just raw commands.
70
00:05:56.000 --> 00:05:56.280
Yeah.
71
00:05:56.280 --> 00:05:58.850
And it would be like, very trustingly, say, sure.
72
00:05:58.850 --> 00:06:02.780
Right,
73
00:06:02.780 --> 00:06:22.970
that you're god@universe.com, fine, send your emails please, right? So you kind of got this idea that everything was a text-based thing. And I remember this early web thing, and this was HTTP one, so you would just literally Telnet to port 80 of some machine and type get space slash and hit enter, and then you'd get the document back. No headers, no nothing. It was just, that was the whole thing.
74
00:06:22.970 --> 00:06:23.580
Right. Right.
75
00:06:23.580 --> 00:06:35.280
But nowadays, HTTP 1.1, we have to tell it more information. So you are going to connect to a TCP port on the machine, usually port 80. Sorry, 443 nowadays.
76
00:06:35.280 --> 00:06:45.250
Everything's HTTPS. So for the purposes of talking about the protocol, at least this level of the protocol, HTTP, let's ignore SSL for now.
77
00:06:45.250 --> 00:06:45.370
Yes.
78
00:06:45.370 --> 00:06:45.940
For HTTPS it's 443.
79
00:06:45.940 --> 00:06:47.060
Let's see if we can get to it in time.
80
00:06:47.060 --> 00:06:47.620
Right.
81
00:06:47.620 --> 00:06:47.990
But I don't know.
82
00:06:47.990 --> 00:06:48.200
Right. Like Tom said in his most recent video.
83
00:06:48.200 --> 00:07:06.110
Correct. And I'm so sorry, this is my family coming home in the background, and we thought we'd beat them, but no. So my dog is announcing their arrival. Much as you announce your arrival at the HTTP server by asking it...
84
00:07:06.110 --> 00:07:06.610
Yeah.
85
00:07:06.610 --> 00:07:08.760
Whoa, segue there.
86
00:07:08.760 --> 00:07:09.500
Meta.
87
00:07:09.500 --> 00:07:25.360
So get, it's still a verb, like get or put or post. And most of these things would be like get. Then the URL, rather, the document locator on that machine itself, and I forget the exact terminology for it, but that's the bit after the host name,
88
00:07:25.360 --> 00:07:25.960
Yeah.
89
00:07:25.960 --> 00:07:34.400
but before the hash, if there's one in your URL. So like in Google, it will probably just be slash, forward slash, if it was your Google, because that's the only thing you've got.
90
00:07:34.400 --> 00:07:34.760
Mm-hmm.
91
00:07:34.760 --> 00:07:43.510
If you're going to google.com, then you put a space and then you tell it that it's HTTP slash 1.1, which tells it, hey, I'm talking to you in the new way, the new new way.
92
00:07:43.510 --> 00:07:43.760
Mm-hmm. Mm-hmm.
93
00:07:43.760 --> 00:08:02.660
I mean, there's QUIC and there's HTTP 2 and there's all other things, but like, for simplicity, let's stick with this. Then you're expected to send a bunch of headers. Each header is a bunch of ASCII bytes followed by a colon, followed by a space, I think. Maybe the space is optional. And then some other stuff and the new line.
94
00:08:02.660 --> 00:08:03.780
Yeah.
95
00:08:03.780 --> 00:08:13.800
So it's pretty human readable, as these things go. And the one thing that you will be most required, most required, I don't know if it can be more or less required, but...
96
00:08:13.800 --> 00:08:14.860
The utmost requirement.
97
00:08:14.860 --> 00:08:35.180
The highest requirement is to put host colon and say, actually, what host (again, my dog in the background, you'll have to apologize), but what host I'm actually requesting from. Because this was like the big unlocker, I think, in like the early aughts, whenever this thing came out, which allowed you to have more than one website per IP address.
98
00:08:35.180 --> 00:08:36.620
Right, right.
99
00:08:36.620 --> 00:08:42.420
Because if you just do get slash, you're like, well, which website did you want? I don't know, right? It's whatever.
100
00:08:42.420 --> 00:08:43.140
Mm-hmm. Mm-hmm.
101
00:08:43.140 --> 00:08:50.790
But that was the way to solve it. You know, maybe it would have made more sense to do get HTTP colon blah and just give it the whole URL and say, you work it out. But anyway.
102
00:08:50.790 --> 00:08:51.440
Maybe.
103
00:08:51.440 --> 00:09:11.150
But host colon google.com is what we would have said. And then, I mean, from my own personal experience of Telnetting to google.com to see how it works, I know that all you need to do to get it to work at all is to do get slash HTTP 1.1, host colon google.com return, and then a blank line that says, I'm done with the headers now.
104
00:09:11.150 --> 00:09:11.600
Mm-hmm.
105
00:09:11.600 --> 00:09:17.420
And then you get the document that says, no, you meant, surely you meant www.google.com. You're like, ah,
106
00:09:17.420 --> 00:09:19.120
Right, right.
107
00:09:19.120 --> 00:09:31.050
but nevertheless, connectivity is there. And then it will close the connection, I believe. So that's what I think would happen if I'm doing it manually myself with Netcat or Telnet or something of that nature.
108
00:09:31.050 --> 00:09:34.200
What am I missing? What's next?
109
00:09:34.200 --> 00:09:50.800
No, that's good. And I mean, I love doing that exercise when kind of explaining HTTP to people, because it takes the magic out of it. Now, you introduce a whole lot more magic when you add in like TLS and a bunch of WebSockets and other things like that.
110
00:09:50.800 --> 00:10:00.400
There's a fantastic quote by one of our friends. It was to do with optimization, as it happens, but, you mentioned magic and I wanted to sort of bend it to this, which is that he said, you know, like, HTTP is like magic.
111
00:10:00.400 --> 00:10:05.620
And you're like, well, it isn't. It's just like a bunch of ASCII being sent over it.
112
00:10:05.620 --> 00:10:06.600
It's just text.
113
00:10:06.600 --> 00:10:07.310
Yeah. And then...
114
00:10:07.310 --> 00:10:07.660
It's words.
115
00:10:07.660 --> 00:10:16.740
And that is like magic, because if you actually show people how it's done, it's kind of disappointing afterwards. You're like, oh, it's just, oh, I see. Yeah, there's just two of them. Like, oh, yeah.
116
00:10:16.740 --> 00:10:17.830
The rabbit was already in the hat.
117
00:10:17.830 --> 00:10:17.980
Yeah.
118
00:10:17.980 --> 00:10:19.000
It was there the whole time.
119
00:10:19.000 --> 00:10:19.640
That's right. It was there the whole time.
120
00:10:19.640 --> 00:10:22.290
It's not special.
121
00:10:22.290 --> 00:10:22.420
Oh.
122
00:10:22.420 --> 00:10:23.600
It's not special. It's not a special hat.
123
00:10:23.600 --> 00:10:23.600
It's not special.
124
00:10:23.600 --> 00:10:26.420
It's not a special rabbit. It's just, the rabbit was in there the whole time.
125
00:10:26.420 --> 00:10:32.080
It was there the whole time. Exactly. But I love that feeling of, like, magic is that. That's what magic means, really.
126
00:10:32.080 --> 00:10:33.020
Yeah, yeah, yeah.
127
00:10:33.020 --> 00:10:34.940
It's disappointing when you know how it's done, actually.
128
00:10:34.940 --> 00:10:35.860
Right, right.
129
00:10:35.860 --> 00:10:44.580
But anyway, not disappointing, because now you know how it's done. You can go, oh cool, this is something I can build on and I understand. I can debug. And so on.
130
00:10:44.580 --> 00:10:45.660
Mm-hmm.
131
00:10:45.660 --> 00:10:46.220
But this obviously is not curl.
132
00:10:46.220 --> 00:10:46.420
Mm-hmm.
133
00:10:46.420 --> 00:10:48.720
This is just me doing it manually. And you do it manually.
134
00:10:48.720 --> 00:10:56.700
Right. So when curl is doing this, in the base case, when it just works as expected, it is, yes,
135
00:10:56.700 --> 00:11:00.420
Bass. Not, not that kind of bass.
136
00:11:00.420 --> 00:11:05.650
It is opening a TCP connection to whatever IP address was resolved from the host name that you gave it.
137
00:11:05.650 --> 00:11:05.840
No treble.
138
00:11:05.840 --> 00:11:06.520
Yes.
139
00:11:06.520 --> 00:11:16.820
It is sending those, that initial request with the HTTP verb, as it's sometimes called, the get, the post, the put, whatever it is.
140
00:11:16.820 --> 00:11:27.900
It's sending all the headers that it might send. And curl sends a lot of headers on your behalf, right? You don't, you can tell it to send specific headers, but you can also not. I just want this document, please.
141
00:11:27.900 --> 00:11:34.680
And it will add in whatever headers it thinks are appropriate, including a user agent.
142
00:11:34.680 --> 00:11:37.660
For example, like the user agent, yeah, hey, I'm curl this version and stuff like that.
143
00:11:37.660 --> 00:11:38.480
Yep, yep.
144
00:11:38.480 --> 00:11:41.410
And what type of documents they'll accept, I think, is a list of things.
145
00:11:41.410 --> 00:11:42.200
Mm-hmm.
146
00:11:42.200 --> 00:11:50.720
And then potentially something to do with the connection, in terms of closing the connection or keeping the connection open for further requests, which, yeah.
147
00:11:50.720 --> 00:12:04.510
Mm-hmm. Okay. One, side note here, one of the interesting ways that you can detect which client is actually connecting to you... because the user agent is supposed to say, but obviously that's like kind of voluntary.
148
00:12:04.510 --> 00:12:04.760
You can change that, yeah.
149
00:12:04.760 --> 00:12:15.790
You can put whatever you want in there. But one of the interesting things that you can do is you can look at the order of the headers, because the order of the headers is not specified in the protocol.
150
00:12:15.790 --> 00:12:18.540
So it can be whatever you want it to be.
151
00:12:18.540 --> 00:12:21.660
Oh, but it's a sort of, it's a fingerprint for the client code that connected.
152
00:12:21.660 --> 00:12:30.380
Yes, certain clients tend to put the headers in certain orders, and depending on what client you're using, that can be an indication of what it actually is.
153
00:12:30.380 --> 00:12:32.100
Oh my, that's sneaky.
154
00:12:32.100 --> 00:12:33.320
Yes.
155
00:12:33.320 --> 00:12:34.740
Ben, you are a sneaky man.
156
00:12:34.740 --> 00:12:34.960
Right.
157
00:12:34.960 --> 00:12:44.710
What is, is the sort of like fingerprinting based on the fact that, yeah, well, you know, curl always sends them in this order, and Chrome's fetcher does this and Firefox does this?
158
00:12:44.710 --> 00:12:44.860
Right.
159
00:12:44.860 --> 00:12:50.060
So it doesn't matter how you configure it. It's, you know, to say like, lie, user agent liar, is like, yeah, but we know you're Chrome really.
160
00:12:50.060 --> 00:12:59.940
Yeah, yeah. Similarly with like, you know, as you said, like spaces in between the headers and the values, again, optional, and you can do it either way.
161
00:12:59.940 --> 00:12:59.940
Right.
162
00:12:59.940 --> 00:13:13.960
And that creates, also the casing on the values is another thing that can create sort of a fingerprint here. So curl is choosing to do it the way that curl does it. And even if you change the user agent to be like, oh, no, actually I'm Firefox,
163
00:13:13.960 --> 00:13:26.400
it's possible that somebody else might be able to figure that out. But the header order is not actually really, and I might be lying here, but I think it is specifically not important.
164
00:13:26.400 --> 00:13:27.100
Like, the order is completely unspecified.
165
00:13:27.100 --> 00:13:38.770
I think it is unspecified which sequence they're in. Yeah, I think so. But yeah, reader, dear reader, listener, sorry, not reader, gosh. Listen, but again, it's, it's been a week.
166
00:13:38.770 --> 00:13:40.180
Yeah.
167
00:13:40.180 --> 00:13:45.360
You know, check what we're saying here, but this is just two old farts comparing notes on what they remember about how this works.
168
00:13:45.360 --> 00:14:01.200
Right, right, right. And, well, and as you were saying before, oftentimes web servers are very tolerant of any kind of, you know, sort of garbage they may receive. And I think you said that you literally were just doing get slash on Google earlier today with no protocol.
169
00:14:01.200 --> 00:14:04.930
No, this one had a, yeah, no, this one did have a get slash.
170
00:14:04.930 --> 00:14:05.080
Okay.
171
00:14:05.080 --> 00:14:07.340
No, this was still using curl. I didn't actually Telnet today.
172
00:14:07.340 --> 00:14:07.520
Okay.
173
00:14:07.520 --> 00:14:18.160
But like, I think nowadays, if you do get space slash, it will tell you to go away, because that's HTTP 0.9 or whatever, the one that didn't even have the version in it, pre-versioning.
174
00:14:18.160 --> 00:14:18.160
Yeah.
175
00:14:18.160 --> 00:14:19.990
Yeah, the unversioned version. Yeah, right.
176
00:14:19.990 --> 00:14:23.960
Yeah, something like that.
177
00:14:23.960 --> 00:14:42.660
Exactly. So yeah, so those headers, you're sending those headers whether you know curl is doing it for you or not. You can obviously be explicit about it. But if you don't do anything, it'll do it for you. And then, depending on the type of verb you choose, the request itself may also have a body.
178
00:14:42.660 --> 00:14:45.760
Oh, yes, that's a good point.
179
00:14:45.760 --> 00:14:59.400
And this is particularly important for put and post and probably patch, but you can also do it with get.
180
00:14:59.400 --> 00:14:59.620
And patch, I think, is another one I'm thinking of.
181
00:14:59.620 --> 00:14:59.620
Oh.
182
00:14:59.620 --> 00:15:01.600
You can, I'm pretty sure that you can include a request body with get.
183
00:15:01.600 --> 00:15:07.310
I think one of the headers, one of the header fields, is content length. Is that right? Is that the thing that gates this?
184
00:15:07.310 --> 00:15:07.560
Ooh.
185
00:15:07.560 --> 00:15:15.770
And then you tell it how much you're going to be sending it or receiving. You know, certainly the response has a content length that says, this is how long it's going to be, or it can do.
186
00:15:15.770 --> 00:15:16.080
Yeah, yes.
187
00:15:16.080 --> 00:15:20.980
But if you're going to do anything that requires you uploading, you need to tell it where the end of your thing is, your data.
188
00:15:20.980 --> 00:15:32.070
I think you're right. I think you're right. Which is why, by the way, it is particularly difficult, if not impossible, to stream the body of a request.
189
00:15:32.070 --> 00:15:32.380
Uh-huh.
190
00:15:32.380 --> 00:15:45.520
Like, if you have some input stream that you're reading from and you're like, I don't know how many bytes are in this stream, I'm just, you know, reading it, by the time you're writing the body, you should have already written the header that says the length.
191
00:15:45.520 --> 00:15:46.180
Yes.
192
00:15:46.180 --> 00:15:52.320
So you have to consume the whole stream in order to figure out how many bytes you're going to send.
193
00:15:52.320 --> 00:15:59.880
Which is, I mean, it's fine if you're serving static content off of a disk, where you're like streaming it from disk and you can go, how long is this file?
194
00:15:59.880 --> 00:15:59.880
So you can put that in the header.
195
00:15:59.880 --> 00:15:59.880
Right.
196
00:15:59.880 --> 00:16:13.180
And it's immutable. It's not going to change on me. Okay. It's two terabytes. Cool. Content length, two terabytes. Here it comes. But yeah, if it's being streamed by a process for some reason, it's like outputting, you know, you're catting a file or whatever, something ungodly.
197
00:16:13.180 --> 00:16:13.600
Don't do this in the web.
198
00:16:13.600 --> 00:16:17.820
Or another socket that you're reading from, right? Like you're trying to act as a proxy perhaps.
199
00:16:17.820 --> 00:16:36.020
Right, then you don't know ahead of time how long the thing is going to be to write it into the header, because you've already sent it. And I mean, anyone who's written web server software has almost certainly had some kind of exception thrown when they've tried to do something silly in the body handler, where it's like, no, you can't do that because the header has already been sent.
200
00:16:36.020 --> 00:16:38.040
Mm-hmm.
201
00:16:38.040 --> 00:16:48.360
Like, you know, this is, forget, I've definitely hit it with, oh, crikey, I can't think of the name. But in some cases where you're trying to do something, it's like, no, look, I've already sent the header.
202
00:16:48.360 --> 00:16:56.820
I can't now mutate this. I can't like turn it into a streaming request. I can't turn it into a different size request because we've committed. We're done now. You just have to send text now.
203
00:16:56.820 --> 00:16:57.060
Mm-hmm. Mm-hmm.
204
00:16:57.060 --> 00:16:57.950
Sorry about that.
205
00:16:57.950 --> 00:16:58.300
Mm-hmm.
206
00:16:58.300 --> 00:17:03.140
Yeah, there is multi-part though, which is a pretty crazy hack.
207
00:17:03.140 --> 00:17:31.820
Right, right, multi-part uploads. If you've ever used like an object store that supports HTTP like Amazon S3 and other ones like this, you wind up sometimes doing multi-part uploads, because otherwise, if you have a lot of data to upload and something goes wrong on the 99th percentile byte of your five terabyte file, you've got to do it all over again, which is very inconvenient.
208
00:17:31.820 --> 00:17:41.060
I think there might also be, for S3, now that I say it out loud, there might also be another portion of the S3 protocol that allows you to do that.
209
00:17:41.060 --> 00:17:54.550
I was going to say, I think those are separable, because certainly S3 lets you configure, like, where do the slices of a multi-part upload go and how long do they live before I delete them away.
210
00:17:54.550 --> 00:17:54.670
Yeah.
211
00:17:54.670 --> 00:17:56.160
Yeah, yeah, yeah. How long they live. Right. Right.
212
00:17:56.160 --> 00:18:06.640
But then there's multi-part, which is more like, you say, it's almost like a streaming protocol where you say, hey, I'm going to keep sending you data. Here is the delimiter, and you come up with some string that you hope to God doesn't appear in your actual data.
213
00:18:06.640 --> 00:18:07.760
Right.
214
00:18:07.760 --> 00:18:12.300
And then you say, this is the thing that tells us that we've done this chunk, going to give you some more headers, I think.
215
00:18:12.300 --> 00:18:12.300
Right.
216
00:18:12.300 --> 00:18:17.580
I think that's how that works. Now, I'm very much in dodgy territory here.
217
00:18:17.580 --> 00:18:17.660
Yep.
218
00:18:17.660 --> 00:18:27.240
I've looked at it with WebSockets before. There's some kind of thing that you have to do to get it to, during the handshake, at least WebSockets as they were in the 2010s. So 15 years ago.
219
00:18:27.240 --> 00:18:34.930
Yeah, yeah. Okay, so exploring other areas where you might do this, and we've explained this simple model of the world, we're sending text, we're receiving text, everything's fine.
220
00:18:34.930 --> 00:18:35.250
Oh, God. Right.
221
00:18:35.250 --> 00:18:35.360
Right.
222
00:18:35.360 --> 00:18:37.700
And you sit down with Telnet and you're like, I'm gonna do what Ben and Matt said and gonna request the thing.
223
00:18:37.700 --> 00:18:47.980
Oh, we didn't really, we didn't see, so, yeah, the curl, for example, or if you Telnet directly, you will see the headers of the response, but curl was hiding them from me because I just did curl, the end.
224
00:18:47.980 --> 00:18:48.360
Yes.
225
00:18:48.360 --> 00:19:00.430
So typically you'll see something along the line of, first of all, something that says 200 OK, the response thing that says, like, your get verb and whatever, you get a response document that starts with the...
226
00:19:00.430 --> 00:19:01.460
Yep, yeah, there you go, yep.
227
00:19:01.460 --> 00:19:10.230
What was the code? And you know, we all have seen the codes. The two XX ones are like, everything's fine, and they have slightly different flavors, but usually 200 OK is the one that you'll see.
228
00:19:10.230 --> 00:19:10.300
Right.
229
00:19:10.300 --> 00:19:21.910
I think there's a, one of the two XX ones is like, OK but no data, is like, just, yes, this is fine, but there's nothing to come, which you see only very occasionally.
230
00:19:21.910 --> 00:19:23.460
Yeah. Yes.
231
00:19:23.460 --> 00:19:28.960
Three XX ones are like exceptional case type stuff. Is that right? What is the definition of three XXs?
232
00:19:28.960 --> 00:19:29.300
I think the...
233
00:19:29.300 --> 00:19:31.440
Cause they're like moved and stuff like that, isn't there?
234
00:19:31.440 --> 00:19:34.220
Yeah, like there's temporarily moved, permanently moved.
235
00:19:34.220 --> 00:19:34.780
Yeah, you know, like...
236
00:19:34.780 --> 00:19:39.360
I mean, it's most of the, I think all the redirects are in 300. There might be other things in 300.
237
00:19:39.360 --> 00:19:45.620
That's the kind of edge casey thing where there might be. And then certainly 400 XXs are errors of sorts, and like, the world famous 404.
238
00:19:45.620 --> 00:20:06.280
Yeah, the world famous 404, which you probably know even if you're not a programmer. But there's the one that Twitter proposed and I think actually got introduced to the standard, 420 Enhance Your Calm, for rate limiting responses, which is a reference to the movie Demolition Man.
239
00:20:06.280 --> 00:20:06.330
Yeah.
240
00:20:06.330 --> 00:20:06.760
Oh. Oh, no, I'd not seen that.
241
00:20:06.760 --> 00:20:16.530
Oh, that's cool. I know there's "I'm a teapot," which is one of the things, which is an error code that a coffee, a tea machine may return if it is asked to serve coffee or something like that.
242
00:20:16.530 --> 00:20:16.820
Yep.
243
00:20:16.820 --> 00:20:17.880
Yep, exactly.
244
00:20:17.880 --> 00:20:30.260
And then obviously if something breaks in the server itself and the server wants to let you know that something hideous went wrong internally, it'll give you a five-XX. What are the 1XX things?
245
00:20:30.260 --> 00:20:33.100
Are those like continuations, I think, for stuff like multi-part?
246
00:20:33.100 --> 00:20:34.040
I think so.
247
00:20:34.040 --> 00:20:36.720
Oh, no. See, I'm trying to do this without Googling because I have the loudest mechanical keyboard.
248
00:20:36.720 --> 00:20:36.910
I know.
249
00:20:36.910 --> 00:20:44.360
So everything you're hearing, dear listener, is coming off the top of our heads, and that, possibly, other parts of our anatomy, depending on how accurate they are.
250
00:20:44.360 --> 00:20:45.060
That's right.
251
00:20:45.060 --> 00:20:55.440
[laughing] So anyway, and then after that, you get to see a sequence, sorry, of headers that look like the ones that you sent up.
252
00:20:55.440 --> 00:20:55.440
Right. That's exactly right.
253
00:20:55.440 --> 00:21:02.550
So it's, you know, something colon something else. And then there'll be two new lines, and then off you go with the data, the response.
254
00:21:02.550 --> 00:21:02.700
Mm-hmm.
255
00:21:02.700 --> 00:21:04.900
So that's, it's pretty cool.
256
00:21:04.900 --> 00:21:05.210
Mm-hmm.
257
00:21:05.210 --> 00:21:07.420
It's, yeah.
258
00:21:07.420 --> 00:21:09.940
It is. It is. And so this is perfect, actually.
259
00:21:09.940 --> 00:21:10.000
Yeah.
260
00:21:10.000 --> 00:21:22.040
This is perfect. So coming back to what I was saying before about, like, okay, I'm going to do what Ben and Matt said, I'm going to sit down and I'm going to like type these things out in Telnet, and I'm going to get back some response. And when I do that, all I see is a bunch of gobbledygook.
261
00:21:22.040 --> 00:21:24.190
I don't see any text. I don't see any HTML.
262
00:21:24.190 --> 00:21:24.340
Yes.
263
00:21:24.340 --> 00:21:40.100
What is going on? One of the things that might be going on is that the server has compressed the response. And it will tell you that it has done that, and it will tell you which compression algorithm it has used to do that in the headers.
264
00:21:40.100 --> 00:21:44.860
So you will see a compression header, and then you will see...
265
00:21:44.860 --> 00:21:47.310
Yes. So the header, the text-based headers are still there, just to be clear, right?
266
00:21:47.310 --> 00:21:47.420
Yes. Yes.
267
00:21:47.420 --> 00:21:56.800
And then there'll be a new line, and then you'll see garbage, which, although a well-configured server ought not to do that unless you said I would accept a compressed thing.
268
00:21:56.800 --> 00:22:10.730
But nevertheless, this is exactly the kind of thing, to your point at the beginning of this, is like, when something goes wrong, you know, I'm getting an endpoint that just assumes every client can support a compression algorithm, which I believe you're about to go and talk about.
269
00:22:10.730 --> 00:22:10.740
That's true.
270
00:22:10.740 --> 00:22:10.750
Right.
271
00:22:10.750 --> 00:22:10.760
Right, right.
272
00:22:10.760 --> 00:22:12.020
So I don't want to keep, I don't know.
273
00:22:12.020 --> 00:22:24.380
Well, yeah, no, and that's another thing that curl is, again, you know, peeling back the abstraction here, that's another thing that curl is doing for you, right? It's reading that header, it's applying the correct compression. I think you might need to tell it.
274
00:22:24.380 --> 00:22:32.400
I can't remember if, like, if you just, but can you disable it? Can you tell it not to do that?
275
00:22:32.400 --> 00:22:51.880
I don't remember either. I know I've had problems before because, and this is going to lead to one of my favorite observations, or rather a mutual friend's observation, you know, whether it's transparently decompressing because it says it's deflated, and so curl is then going to say, well, I'll undeflate it then, because that's what it says that I should do.
276
00:22:51.880 --> 00:23:08.210
Or whether, you know, that just means the fact that the compression, whether the compression is there or not, is now opaque to you. You don't know if it's happening or not, which is the kind of dichotomy of transparency: it means that now it's impossible to see whether it's actually happening or not, you know?
277
00:23:08.210 --> 00:23:08.340
Right.
278
00:23:08.340 --> 00:23:08.600
Yes.
279
00:23:08.600 --> 00:23:18.930
So our friend coined the term, which I introduced to somebody at work yesterday, transpaque, which is like that kind of combination of, like, well, it's like you're meant to not know it's there,
280
00:23:18.930 --> 00:23:19.550
Yeah, yeah, yeah.
281
00:23:19.550 --> 00:23:19.960
Right.
282
00:23:19.960 --> 00:23:25.460
but if it is there, now you can't tell if it's there or not, and maybe it's broken, and now you're just being gaslit by your system.
283
00:23:25.460 --> 00:23:25.960
Right.
284
00:23:25.960 --> 00:23:37.620
You know, I think we had this with at least one of the things at work where a vendor was sending gzipped data, but whatever...
285
00:23:37.620 --> 00:23:52.840
accept header we were getting was then un-gzipping it for us, even though we'd asked, you know, like, get bob.gz or get, you know, this.gz. And then the library was like, oh, cool, I'll un-gzip that for you. And then it was storing it as, and we were writing it to disk as, you know, bob.gz.
286
00:23:52.840 --> 00:24:02.000
And you're like, why is everything so huge? Oh, wait, everything's called .gz because that's what we asked for, but something transparently, again, transpaquely...
287
00:24:02.000 --> 00:24:03.680
Transpaquely.
288
00:24:03.680 --> 00:24:22.120
un-gzipped it for us, being helpful, thinking it was being helpful, and it wasn't, right? Anyway, so, but yes, it could be deflate compressed, I think. Or what are the other ones that are out there? Zstandard, I think, is that standardized? I don't know, but deflate's the one that, you know, everyone uses, which is to say gzip.
289
00:24:22.120 --> 00:24:34.450
Pretty much, to a first approximation, deflate is the algorithm that powers gzip, and then there's a little tiny bit of a header associated with an actual gzip file, which I think is slightly different, and a CRC.
290
00:24:34.450 --> 00:24:35.340
Okay.
291
00:24:35.340 --> 00:24:39.020
So that's definitely something that can go wrong.
292
00:24:39.020 --> 00:24:48.320
Yeah. Yeah. Do we want to try to get into TLS and compression and HTTPS? Is that a bridge too far for this conversation?
293
00:24:48.320 --> 00:24:56.390
Well... that might be a bridge too far. Certainly, I don't know enough about it. So what we've been talking about so far, let's just graze the topic and see if we feel comfortable with this.
294
00:24:56.390 --> 00:24:56.520
Yeah, right.
295
00:24:56.520 --> 00:25:17.380
And then our listener can roll their eyes at how wrong we are. But what we've been talking about is like the HTTP protocol. And in HTTPS, there's another layer, another sort of wrapping of the protocol where the, a certain amount of, well, a certain amount, some,
296
00:25:17.380 --> 00:25:36.390
encryption is applied, and authentication and certificate exchange, all that kind of stuff. But once you've peeled that back, you're back to HTTP again. So like, effectively, from the point of view of the web server itself and from the web client, you can almost, almost ignore the fact that the tunnel that connects them is...
297
00:25:36.390 --> 00:25:36.680
Mm-hmm.
298
00:25:36.680 --> 00:25:55.260
an encrypted tunnel, as opposed to just a regular Telnet. So obviously you can't Telnet to port 443 and type anything and expect it to happen, which is unfortunate. What you can expect, what will happen is, if you're very lucky, you might see a lot of garbage bytes being sent back to you, and then the connection is closed. That's probably what you're gonna see if you do anything at all, if you Telnet to port 443.
299
00:25:55.260 --> 00:26:23.460
Something that surprised me the other day is, I naively assumed that you connect to port 443, some kind of negotiation happens with a certificate and key exchange and the whole Diffie-Hellman thing to, like, you know, get a unique key on both sides, and then, and then, and only then, you start saying, hi, I'd like to get this, you know,
300
00:26:23.460 --> 00:26:24.440
Right. I was going to say this exact same thing.
301
00:26:24.440 --> 00:26:42.920
But, yeah, yeah. But it turns out, for all that, for the same reason that we added the host colon to the sort of, like, the message body rather than just relying on the Telnet, you know, the IP address that you're connecting to being unique for that website.
302
00:26:42.920 --> 00:26:53.920
We lose that if we encrypt it, which means that, like, you know, the web server you're talking to, if it's, you know, google.com is co-located with yahoo.com, unlikely.
303
00:26:53.920 --> 00:26:58.280
But whose certificate should the server give you when you connect to it?
304
00:26:58.280 --> 00:26:58.360
Right.
305
00:26:58.360 --> 00:27:03.500
You're like, ah, good question. I don't know. Which means that you, the client, need to tell it first.
306
00:27:03.500 --> 00:27:04.200
Yes.
307
00:27:04.200 --> 00:27:12.640
But how do you encrypt that so that no one can tell which site you're actually browsing? And the surprising answer is, you don't.
308
00:27:12.640 --> 00:27:13.640
Yep.
309
00:27:13.640 --> 00:27:15.340
Which was like, mind blown.
310
00:27:15.340 --> 00:27:16.110
Yep. Yep.
311
00:27:16.110 --> 00:27:18.840
Oh.
312
00:27:18.840 --> 00:27:21.280
Not as secure as you thought. All that information is being exposed to the interwebs.
313
00:27:21.280 --> 00:27:27.680
So... does this mean that all these adverts that I keep fast-forwarding through about VPNs...
314
00:27:27.680 --> 00:27:36.900
And, dear listener, do not worry, this is not about to segue into an advert for one of those. This is, it's not a way of announcing that finally Surfshark or whatever is...
315
00:27:36.900 --> 00:27:37.280
We've taken sponsorship. Yeah.
316
00:27:37.280 --> 00:27:38.050
No, no sponsorship.
317
00:27:38.050 --> 00:27:39.260
Uh huh.
318
00:27:39.260 --> 00:27:39.760
This is just my fun.
319
00:27:39.760 --> 00:27:40.540
Right.
320
00:27:40.540 --> 00:27:52.100
We're not turning into anything else. But no, that is the one argument I can see for that, because I just assume, you know, like, fine, HTTPS everything, and it's, you know, no one need know what I'm doing.
321
00:27:52.100 --> 00:28:01.800
But it's, you know, you are actually, if you cracked out Wireshark and visited a website (which maybe we should do live sometime, we need to work out a way of screen recording this or something).
322
00:28:01.800 --> 00:28:03.240
Yeah. Yeah, yeah, yeah.
323
00:28:03.240 --> 00:28:15.190
Because, yeah, that's, I think the first time you show somebody Wireshark and then you're browsing or doing anything trivial on the network is the first time the scales fall.
324
00:28:15.190 --> 00:28:15.340
Mm-hmm.
325
00:28:15.340 --> 00:28:19.900
Certainly was for me. Like, oh my gosh, look how much stuff's going on on my network or on my computer.
326
00:28:19.900 --> 00:28:20.120
Mm-hmm.
327
00:28:20.120 --> 00:28:21.220
Oh, golly.
328
00:28:21.220 --> 00:28:21.840
Mm-hmm.
329
00:28:21.840 --> 00:28:25.120
But anyway, yeah, so, popping back, you were going to say the same thing about this. Surprising, was it SNI or something, was the name of it?
330
00:28:25.120 --> 00:28:48.070
Yeah, you should be very careful about assuming that the data that is in your request, the URL, including the query parameters, the headers that you are either adding yourself or being added on your behalf by the tools that you're using, like curl, are actually secure, because they're almost certainly not, in many, many cases.
331
00:28:48.070 --> 00:28:48.320
Hmm.
332
00:28:48.320 --> 00:28:50.080
So when you make that request,
333
00:28:50.080 --> 00:28:56.780
I was thinking it was only the host that was the thing that was leaked, just purely so the correct certificate could be served back to you. But I may be wrong.
334
00:28:56.780 --> 00:29:14.950
So I think that, so my understanding of this is that you're doing this sort of, like, escalation request, you're like, I'd like to escalate to this other protocol. And I would not be very confident in telling anybody that I knew for certain that, in all cases, it could only be these things, right?
335
00:29:14.950 --> 00:29:16.600
No, that's fair.
336
00:29:16.600 --> 00:29:24.620
It's, yeah, it's one of those things where it's like, if it's important to you that that information is encrypted, then you should check, probably using Wireshark or something else like that.
337
00:29:24.620 --> 00:29:49.180
I've just fired it up. I couldn't help myself. Oh, I need to sudo make me a sandwich, of course, because, why, I shall. Yeah, okay, given that I have to escalate, I'm not going to do that now, otherwise, you know, you are going to hear my long password rattling out. But no, that's definitely an exercise for the reader. And I'm sure, hopefully people will comment and tell us where we're going wrong here. But again, this is from the top of our heads. So, yeah, no. So.
338
00:29:49.180 --> 00:29:55.020
Okay, I think that's probably the most we should do on the encryption, because it's clear that we've reached the limit of our knowledge.
339
00:29:55.020 --> 00:29:55.100
Yeah, yeah.
340
00:29:55.100 --> 00:29:57.280
We've hit the layer of abstraction that we can recall sitting here.
341
00:29:57.280 --> 00:30:02.470
I mean, there is definitely some things to do with certificates and things, and, you know, the little lock that you see.
342
00:30:02.470 --> 00:30:02.820
Yeah.
343
00:30:02.820 --> 00:30:03.640
And a little bit about that.
344
00:30:03.640 --> 00:30:07.620
Not going to pretend to know nearly enough about that to explain it to the world.
345
00:30:07.620 --> 00:30:11.000
Well, I mean, there is like a chain of trust, isn't there? We can probably talk a little about that, where...
346
00:30:11.000 --> 00:30:13.680
Mm-hmm. Yeah, that's true. Certificate authorities.
347
00:30:13.680 --> 00:30:34.900
Yeah, so your browser and curl or whatever, and your operating system, may provide a list of certificate authorities, which are like trusted third parties that have said, we will sign other people's certificates to say it's definitely them. Trust me, bro. And then, as long as you trust the person who signed the certificate, you can trust.
348
00:30:34.900 --> 00:30:42.600
Google.com is in fact Google.com because it's signed by VeriSign or whoever the heck actually signs Google. Google probably are a CA, aren't they? Probably, and that's fine.
349
00:30:42.600 --> 00:30:43.900
Yeah, probably at this point.
350
00:30:43.900 --> 00:30:59.960
And that's kind of how you know that you are actually looking to the correct site, the authentication part of HTTPS. And "know" has got air quotes here, because obviously it really does depend how much you trust the people who signed these certificates to say it really is who they say that it is.
351
00:30:59.960 --> 00:31:00.020
Mm-hmm.
352
00:31:00.020 --> 00:31:14.340
And if you look at the number of CAs that there are inside your browser, if you go and look at the list, and there's like a bewildering number of them, including some from, to me, things that would raise an eyebrow, and I'm like, oh, apparently we trust this company.
353
00:31:14.340 --> 00:31:14.560
Mm-hmm.
354
00:31:14.560 --> 00:31:18.200
I don't know who they are, and it doesn't look very trustworthy to me.
355
00:31:18.200 --> 00:31:19.560
Right, right, right, right.
356
00:31:19.560 --> 00:31:37.240
Similarly, in a corporate environment, you might discover, reasonably, your own company's root CA is in that list, because then you can sign, first of all, you can sign all of your internal applications with your own certificate that doesn't have to go out to the outside world, doesn't have to be signed by VeriSign.
357
00:31:37.240 --> 00:31:48.780
It means that you can go to, you know, benscoolapp.internal.com, and you can get a certificate for that, and your browser won't go, wait a second, this is just self-signed.
358
00:31:48.780 --> 00:31:53.590
That's no good. You know, it can be signed by, you know, benscoolcompany.com, which is then in there.
359
00:31:53.590 --> 00:31:53.960
Mm-hmm.
360
00:31:53.960 --> 00:32:10.060
But it also allows potential for, like, security scanning software to decrypt, by man-in-the-middling you in your own, you know, on your own network, to be able to check that things aren't going outside, which is really important in, certainly in a large corporate environment.
361
00:32:10.060 --> 00:32:11.080
I don't know that it happens in any, like, the companies we're in, but.
362
00:32:11.080 --> 00:32:21.420
No, no, I mean, that's definitely a thing. I mean, again, like, you know, the theme of this being, like, you know, breaking people's expectations around abstractions and things like that.
363
00:32:21.420 --> 00:32:21.780
Right.
364
00:32:21.780 --> 00:32:36.360
Like, it's probably been told to you in indirect ways by people at your company if they're doing this, but maybe not in as direct ways as, we're reading your personal email if you check your personal email at work, right?
365
00:32:36.360 --> 00:32:36.520
Right.
366
00:32:36.520 --> 00:32:39.680
Like, the little security lock in the browser that you're running, right?
367
00:32:39.680 --> 00:32:47.610
Is not as strong as you think it is, because if you're using your company, I mean, obviously, if you're on your personal device on the guest Wi-Fi of your company or whatever, fine.
368
00:32:47.610 --> 00:32:47.650
Yeah.
369
00:32:47.650 --> 00:32:47.740
Sure. Right.
370
00:32:47.740 --> 00:32:57.940
And that's a very good reason to do that, right? But yes, if you are, most companies have at least a clause that says something like, there's no expectation of privacy, which is reasonable, right? You're at work, you're doing work things.
371
00:32:57.940 --> 00:32:59.260
Yeah.
372
00:32:59.260 --> 00:33:00.730
I mean, reasonable people could have disagreements about this.
373
00:33:00.730 --> 00:33:00.800
You're on a work computer doing work things.
374
00:33:00.800 --> 00:33:24.960
I get it. I get it. I get it. But like, that is the way in which they can potentially look at the encrypted data, is that they can say, well, any website you go and get, if you go to secure.com, I will say, yes, sure, I'm secure.com, and I can prove I am because I'm signed by this certificate, which, by the way, your company-given security system says I can sign anything I damn well like, and that's fine.
375
00:33:24.960 --> 00:33:25.140
Yeah.
376
00:33:25.140 --> 00:33:25.500
Yeah, right.
377
00:33:25.500 --> 00:33:39.700
And that's, yeah. So yeah, just knowing that that exists is all you need to do. And it's an exercise to the listener to go and see whether that's something that's happening to them or not. And I mean, it's a fairly standard thing.
378
00:33:39.700 --> 00:33:49.160
Anyway, popping the stack all the way back, right? So we said, what about curl, right? You're right, with curling this endpoint. What else? Is there anything we've missed? Let's just think about this.
379
00:33:49.160 --> 00:33:50.780
Well, I got, so I got a good one.
380
00:33:50.780 --> 00:33:51.300
OK.
381
00:33:51.300 --> 00:34:16.020
So let's say that, what you see, when you do this request, and maybe you're trying to drop down a level here and you're gonna use, like, the Telnet approach, is that you connect to the site, you type in your headers, you get back your response headers, and then you get some of the body, and then it stops.
382
00:34:16.020 --> 00:34:18.580
Ah, yeah,
383
00:34:18.580 --> 00:34:27.770
What happened? What is happening? Like, how do you even know that it's not done? What is the mechanism for that, if I recall correctly?
384
00:34:27.770 --> 00:34:27.960
Yeah.
385
00:34:27.960 --> 00:34:40.010
It's like, two new lines is the fancy technology that they use to tell you that the body of the document is complete. It's that we're gonna do a new line and then we're gonna do another new line, and that's how you know that it's done.
386
00:34:40.010 --> 00:34:40.800
I think that's the hardcore thing.
387
00:34:40.800 --> 00:34:44.180
But let's say that you don't see those two new lines and it just stops.
388
00:34:44.180 --> 00:34:44.860
Right.
389
00:34:44.860 --> 00:34:48.040
What is curl doing in that situation?
390
00:34:48.040 --> 00:34:54.740
That's an excellent question. I don't actually know the answer to that.
391
00:34:54.740 --> 00:35:11.240
Yeah, I don't know, because, like, my instinct, if this was an interview question and you, Ben Rady, were asking me what's going on here, I would be like, oh, this sounds like the remote end has decided to keep the connection open.
392
00:35:11.240 --> 00:35:29.440
But it has, in fact, finished the document. And I don't know if it puts something at the end of that. I don't think it does. Maybe that's one of those 100, 100 continue is like one of those things, as it puts it in. I don't know. But, totally separately, and just pause there, I did, in fact, just Telnet to google.com and do get space slash enter.
393
00:35:29.440 --> 00:35:30.780
I think you could get something like that, yeah.
394
00:35:30.780 --> 00:35:36.500
And it doesn't even tell me that I'm at the wrong address, because it doesn't know that I came to the wrong address. It doesn't try and redirect. It just gives me the whole...
395
00:35:36.500 --> 00:35:42.020
you know, minimized google.com thing pukes out. So it still works with even, you know, HTTP 0.9.
396
00:35:42.020 --> 00:35:42.300
Interesting.
397
00:35:42.300 --> 00:35:47.900
Okay. What happened then, Ben? Tell me the answer. What was going on with my hung connection, or what might be happening?
398
00:35:47.900 --> 00:35:58.910
So this could be, this could be a number of things, but one scenario in which you would see this is that there was, like, an error, like a programming error on the server side, where it like read a document,
399
00:35:58.910 --> 00:36:00.220
Oh.
400
00:36:00.220 --> 00:36:08.340
and encountered an error and for whatever reason didn't close the connection. Right, maybe it's waiting for, itself is waiting for data to show up that just isn't showing up.
401
00:36:08.340 --> 00:36:08.680
Right.
402
00:36:08.680 --> 00:36:12.310
It's not going to come, or it's deadlocked in itself trying to do something.
403
00:36:12.310 --> 00:36:19.030
Right. It's, like, in the midst, stuck on some database query or streaming some data from another source or whatever it might be, and it's just stopped.
404
00:36:19.030 --> 00:36:19.280
Yeah.
405
00:36:19.280 --> 00:36:19.530
Yeah.
406
00:36:19.530 --> 00:36:39.460
curl has a number of different timeouts that you can configure. And part of the reason for those timeouts is situations like this, because you don't want some process to just be stuck forever because it made some request that is now stuck forever.
407
00:36:39.460 --> 00:36:41.300
It has a connection timeout, which is more of a TCP level thing. It is a TCP level thing. It's like, hey, I want to connect to this IP address and send it some text so I can get my documents.
408
00:36:41.300 --> 00:36:59.500
If it never establishes the TCP connection, or it might take some time to establish a TCP connection, there's a timeout that you can control for that. I believe there is also a response timeout, to say, like, if I start getting data and then I stop getting data, just whacked my microphone.
409
00:36:59.500 --> 00:37:01.660
That's fine, I'm sure that will be edited away.
410
00:37:01.660 --> 00:37:04.500
It won't be edited away!
411
00:37:04.500 --> 00:37:07.780
Yeah.
412
00:37:07.780 --> 00:37:22.040
And you stop getting data, and then you continue to not get any data, how long should I wait before I just give up? And it's important to know, like, with all of these protocols, we use it often enough, you almost think of them as, like...
413
00:37:22.040 --> 00:37:28.830
You almost think of it as being, like, this synchronous thing, right? Like, almost like an atomic thing, where it's like, I'm going to do the request thing and we get the response, and it all works.
414
00:37:28.830 --> 00:37:29.280
Right, right, right, right.
415
00:37:29.280 --> 00:37:49.860
And it is entirely possible that you get some, but not all, of the response. It is entirely possible that you get halfway through sending the request and then the connection just dies. It's entirely possible that you start sending the request and, depending on exactly what is happening on your network, it may not be clear how much of it was actually sent.
416
00:37:49.860 --> 00:37:54.760
Like, did you, I'm doing this, like, put request or delete request, right?
417
00:37:54.760 --> 00:37:54.980
Oh.
418
00:37:54.980 --> 00:37:58.680
Like, did you actually delete it on the other side?
419
00:37:58.680 --> 00:37:59.300
Right.
420
00:37:59.300 --> 00:38:00.660
I'm not sure.
421
00:38:00.660 --> 00:38:08.840
That is, yeah, the sort of, like, what's the name of it? The Byzantine generals problem or whatever, where you've got, like, the two, you're like, I sent a thing to you that says, did you delete it?
422
00:38:08.840 --> 00:38:08.900
Yeah.
423
00:38:08.900 --> 00:38:17.340
And you go, I got it, but I didn't hear your response. You're like, well, did you then? I don't know. I don't know, because I didn't, and like, there's any number of acks on both sides that you, and you can never be a hundred percent sure. And yeah.
424
00:38:17.340 --> 00:38:18.590
Exactly, exactly.
425
00:38:18.590 --> 00:38:18.860
Yeah.
426
00:38:18.860 --> 00:38:24.160
So that is definitely another place where this sort of abstraction can break down, where it's like, well, I did the delete request and it failed.
427
00:38:24.160 --> 00:38:24.340
I've...
428
00:38:24.340 --> 00:38:26.300
So clearly it's not deleted.
429
00:38:26.300 --> 00:38:27.790
There's at least five different timeouts.
430
00:38:27.790 --> 00:38:27.900
Well,
431
00:38:27.900 --> 00:38:33.420
I've just been looking through the man page of curl, five different termouts, termouts, timeouts for different conditions.
432
00:38:33.420 --> 00:38:34.360
Termouts.
433
00:38:34.360 --> 00:38:45.420
You know, like, how often, how long do you wait before connection? As you said, overall timeout for the entire request, the first response, and then, you know, subsequent chunks of data, that kind of thing.
434
00:38:45.420 --> 00:38:46.080
Mm-hmm.
435
00:38:46.080 --> 00:38:58.540
And then some other things to do with IPv4 and IPv6, if you're trying to do both at the same time and get the first one to respond, some really clever things like that. But yeah, a lot of those things are sort of, yeah, as you say, we think of these things as atomic.
436
00:38:58.540 --> 00:39:16.580
We think about, you know, like, everyone talks about RESTful APIs and you're like, okay, it's like a magic RPC that I just do, and it happens or it doesn't happen and I'm done. And you're like, you could see an error on your side that says the connection never made, you know, the request never went through, but it did in fact make it through.
437
00:39:16.580 --> 00:39:23.380
It was just the response didn't get to you and the TCP was torn down, but it did in fact take effect on the remote end, and now you're stuck.
438
00:39:23.380 --> 00:39:32.800
You didn't get back, you know, the response. So it's a tricky world out there, but yeah.
439
00:39:32.800 --> 00:39:34.400
Absolutely. Yeah.
440
00:39:34.400 --> 00:39:36.440
No, there's many timeouts. Oh my gosh, yeah.
441
00:39:36.440 --> 00:39:37.720
You said there's five timeouts?
442
00:39:37.720 --> 00:39:42.240
I think so. I've got connect timeout, expect 100 timeout.
443
00:39:42.240 --> 00:39:47.940
"Maximum time in seconds you'll wait for curl for a 100 continue." Oh, good. I wasn't completely making that up.
444
00:39:47.940 --> 00:39:49.000
That's good.
445
00:39:49.000 --> 00:39:52.780
See, you also, connect timeout. Happy eyeballs timeout. That's hilarious to me.
446
00:39:52.780 --> 00:39:54.040
What? Happy eyeballs timeout?
447
00:39:54.040 --> 00:40:19.140
Yeah, I'm going to read this to you because it's joyous. "Happy Eyeballs", with a capital H and a capital E, "is an algorithm that attempts to connect to both IPv4 and IPv6 addresses for dual-stack hosts, giving IPv6 a head start of the specified number of milliseconds." So it's like a way of, you know, preferring IPv6, but not necessarily...
448
00:40:19.140 --> 00:40:21.960
Oh man, that's good.
449
00:40:21.960 --> 00:40:38.910
So yeah, we've got, see also, a max timeout, retry max time. So there's an overall max time, and then there's a retry maximum time, which is like the number of times it will, including, yeah, retry time is reset before the first... It's complicated, but I mean...
450
00:40:38.910 --> 00:40:41.230
And then we're going to keep talking.
451
00:40:41.230 --> 00:40:44.180
Yeah, and we're back.
452
00:40:44.180 --> 00:40:54.260
What happened there was a terrible break in the abstraction layer, because my USB hub crashed and put us into a very bad state. And we've just about recovered it.
453
00:40:54.260 --> 00:40:57.400
And, can't remember what on earth we were talking
454
00:40:57.400 --> 00:41:00.760
No idea. No idea at all. It's like it was a different dimension.
455
00:41:00.760 --> 00:41:11.550
about. And if only, I mean, if only we had some kind of permanent record of what we'd said. But sadly, neither you nor, you said you could download the MP3s okay, and it was working. So at least we've got that on our side.
456
00:41:11.550 --> 00:41:14.160
I did do that, yes.
457
00:41:14.160 --> 00:41:24.760
Okay, right. So we've got the previous part. So, dear listener, we apologize. Whatever we were talking about before has slipped our minds. But we didn't want to leave you in the lurch, or, more importantly, we didn't want to leave editing Matt with a job of, like, how do you finish an episode where...
458
00:41:24.760 --> 00:41:24.760
Right.
459
00:41:24.760 --> 00:41:37.660
Clearly something went wrong. But I think there's another abstraction layer we can talk about another time. It's like, how on earth do our devices talk to each other in a sort of day-to-day environment?
460
00:41:37.660 --> 00:41:41.280
Yeah, USB actually would be kind of a cool one.
461
00:41:41.280 --> 00:41:46.150
USB, USB-C and DVI and digital video and all those kinds of magic.
462
00:41:46.150 --> 00:41:46.300
Maybe a bit ambitious, but I think I like that one.
463
00:41:46.300 --> 00:41:49.000
I mean, it's a miracle that it works as well as it does, frankly.
464
00:41:49.000 --> 00:41:50.760
Yeah. Yeah. Yeah.
465
00:41:50.760 --> 00:41:58.020
But yeah, we were looking at curl, and I think we were just about looking through all the timeouts. That's what I'd recall us talking about.
466
00:41:58.020 --> 00:41:59.560
Yeah. Oh, the happy eyeballs.
467
00:41:59.560 --> 00:42:09.120
Happy eyeballs for IPv6, IPv4, which is crazy. I saw someone put an IPv8 proposal out there, which I think is somewhat tongue-in-cheek, but also I read through it and I was like, this isn't as stupid as it looks.
468
00:42:09.120 --> 00:42:09.120
What?
469
00:42:09.120 --> 00:42:11.940
Maybe this is real. You know, I can't tell.
470
00:42:11.940 --> 00:42:13.440
And they skipped seven because we skipped five. That's how, yeah.
471
00:42:13.440 --> 00:42:31.040
I guess. I think the, it uses, like, a 64-bit IP address rather than the full crazy IPv6, I don't know, 256-bit, 128, whatever bit size that is, where it's like, hey, all this information about the host, you're like, I don't want to leak that outside my network.
472
00:42:31.040 --> 00:42:31.910
Thank you very much.
473
00:42:31.910 --> 00:42:32.600
Yeah, yeah, yeah.
474
00:42:32.600 --> 00:42:43.840
Right, so it was, and it's sort of backwards compatible with IPv4 because it's essentially, all existing IPv4s are 0.0.0.0.0.0, the rest. You're like, oh, the rest, you're like, oh, that makes sense.
475
00:42:43.840 --> 00:42:43.900
Interesting.
476
00:42:43.900 --> 00:42:56.120
Maybe that's got legs. Anyway, that's really off topic, and I have, that's about as much as I know about it. My friend, I don't know if there's anything that we can recover from where we were going.
477
00:42:56.120 --> 00:42:58.500
So I think we should probably just call it at this point here.
478
00:42:58.500 --> 00:42:58.840
No, yes.
479
00:42:58.840 --> 00:43:06.900
And having introduced the world to transpacity, I feel like, you know, we've at least achieved something.
480
00:43:06.900 --> 00:43:08.230
Transpacity?
481
00:43:08.230 --> 00:43:08.410
Yeah.
482
00:43:08.410 --> 00:43:08.780
Transpacity. Okay.
483
00:43:08.780 --> 00:43:10.040
I guess it would be, wouldn't it?
484
00:43:10.040 --> 00:43:10.240
Yes.
485
00:43:10.240 --> 00:43:10.440
Yeah.
486
00:43:10.440 --> 00:43:11.040
Yes, it is.
487
00:43:11.040 --> 00:43:12.040
Transpacity. Opacity, transpacity.
488
00:43:12.040 --> 00:43:14.020
That's exactly, transpacity.
489
00:43:14.020 --> 00:43:20.010
We'll have to ask our friend and see if maybe he'll join us on the podcast sometime to talk about it. But it's definitely a thing.
490
00:43:20.010 --> 00:43:20.780
Yeah, there you go.
491
00:43:20.780 --> 00:43:23.790
All right, mate. Well, it's been a journey today.
492
00:43:23.790 --> 00:43:25.200
Yep. Fun times.
493
00:43:25.200 --> 00:43:28.640
I will chat to you another time.
494
00:43:28.640 --> 00:43:30.640
Sounds good.
1
00:00:18.180 --> 00:00:19.420
Hey, Ben.
2
00:00:19.420 --> 00:00:21.100
Hey, Matt.
3
00:00:21.100 --> 00:00:28.040
I can't believe we're laughing already. Ben, very, very, I don't know what the right word would be, but like, you've been auspiciously auspicious.
4
00:00:28.040 --> 00:00:28.040
Auspiciously?
5
00:00:28.040 --> 00:00:39.990
That's not even, though that's not the right word, but you took your microphone and kind of bent it out of the way so we could have a nice talk, and your headset microphone isn't in the way.
6
00:00:39.990 --> 00:00:40.120
Yeah, right.
7
00:00:40.120 --> 00:00:45.940
Oh gosh, it's too early in the week to be recording a podcast, and yet here we are.
8
00:00:45.940 --> 00:00:47.160
You mean Friday? It's...
9
00:00:47.160 --> 00:00:51.060
Yeah, I guess. I don't know, now everyone knows when we record these.
10
00:00:51.060 --> 00:00:52.660
There's not a lot of time left, my man.
11
00:00:52.660 --> 00:01:00.860
That's true. Somehow we're in May as well, which, this podcast will go out in May as well, because it has to.
12
00:01:00.860 --> 00:01:02.320
Yeah. Yeah, right, right.
13
00:01:02.320 --> 00:01:03.580
Otherwise, we're in trouble.
14
00:01:03.580 --> 00:01:04.720
We've not got any in the bank.
15
00:01:04.720 --> 00:01:05.240
Okay.
16
00:01:05.240 --> 00:01:08.560
Yeah, so what are we talking about today, Ben?
17
00:01:08.560 --> 00:01:32.250
I thought we could do at least one podcast, and maybe we'll do this again, on kind of like how computers actually work. And I think a way in which to cover the extremely broad topic of how computers actually work is to look at some of the maybe non-intuitive behaviors that they have.
18
00:01:32.250 --> 00:01:33.010
I love it.
19
00:01:33.010 --> 00:01:33.260
Oh.
20
00:01:33.260 --> 00:01:38.000
Things that, you know, people kind of take for granted as sort of like an abstraction that you rely on.
21
00:01:38.000 --> 00:01:48.890
And, you know, peel back the covers a little bit and explore how it works. And maybe even, if we get to it, some like non-obvious ways in which it breaks.
22
00:01:48.890 --> 00:02:00.490
Right. And like when it doesn't work, what is actually happening? Because I think that sort of understanding the failure modes of these things is a great way to understand what's actually going on.
23
00:02:00.490 --> 00:02:02.190
So that's my general idea.
24
00:02:02.190 --> 00:02:16.520
For certain, yeah. OK, so we've got like this sort of general idea of the theme of how computers really work, which obviously is very close to my heart as well. So this sounds brilliant.
25
00:02:16.520 --> 00:02:28.160
And then we were discussing some things before we started, and you know, shooting around ideas. And I sort of suggested the, there used to be like an old Google interview question, which was, what happens, you know, you type google.com into a browser, you hit enter, what happens?
26
00:02:28.160 --> 00:02:34.960
And you know, I always liked that because there's almost any number of ways that you can go, but how about we simplify that bit?
27
00:02:34.960 --> 00:02:43.700
What happens if I have curl rather than a whole web browser there, and I curl some simple URL, right?
28
00:02:43.700 --> 00:02:44.580
Right.
29
00:02:44.580 --> 00:02:46.760
Not, well, not JavaScript or anything. Well, I guess it doesn't matter, but yeah.
30
00:02:46.760 --> 00:02:47.600
I mean, could be JavaScript. A document.
31
00:02:47.600 --> 00:02:47.800
I guess so.
32
00:02:47.800 --> 00:02:49.280
You're fetching a document with curl.
33
00:02:49.280 --> 00:02:52.160
It doesn't even matter at that point, because it'll just print it out in front of you.
34
00:02:52.160 --> 00:02:52.710
That's right.
35
00:02:52.710 --> 00:02:54.100
Yeah.
36
00:02:54.100 --> 00:03:01.550
So, so yeah. When should we try something like that? What happens if we curled google.com? In fact, I did this only earlier today as I was playing around with sandboxes.
37
00:03:01.550 --> 00:03:02.010
Yeah.
38
00:03:02.010 --> 00:03:02.940
Oh, perfect.
39
00:03:02.940 --> 00:03:07.710
And I curled exactly HTTPS google.com. And I know exactly what it returns.
40
00:03:07.710 --> 00:03:11.200
Can I, can I offer a guess before you tell me the answer?
41
00:03:11.200 --> 00:03:12.100
You can offer away.
42
00:03:12.100 --> 00:03:19.220
Yes. Does it give you a redirect? Something in the HTTP 300 range would be my guess.
43
00:03:19.220 --> 00:03:26.540
It does. It tells me the document has moved, and it tells me the document has moved to www.google.com, which was like, oh, really?
44
00:03:26.540 --> 00:03:26.860
Yes, okay.
45
00:03:26.860 --> 00:03:29.800
Yeah, I guess because I was very lazy.
46
00:03:29.800 --> 00:03:32.940
Yeah, so how does it tell you that?
47
00:03:32.940 --> 00:03:43.740
Well, in this instance, the curl told me just a document. It printed out a document in front of me that said 300 or whatever it was, moved, document moved.
48
00:03:43.740 --> 00:03:50.380
And then it had a tiny little bit of HTML that came to me, because I didn't have, you know, dash D dash or whatever the thing is to print out all the headers and stuff like that. But no.
49
00:03:50.380 --> 00:04:06.180
Okay. Okay. So you didn't see, so this was going to be my question. My understanding of this is, whenever you request a document over HTTP or HTTPS, you're getting two things back: you're getting the headers and you're getting the document, the document body.
50
00:04:06.180 --> 00:04:06.400
Aha.
51
00:04:06.400 --> 00:04:06.620
Yeah.
52
00:04:06.620 --> 00:04:16.180
Right. And so when it was redirecting you, all you were seeing there was the document body, which was some HTML. Is that correct?
53
00:04:16.180 --> 00:04:25.580
Correct. A very small piece of HTML, which you would normally never see. You know, a browser would render that, really because the real meat of it was in the headers, right?
54
00:04:25.580 --> 00:04:26.420
Mm-hmm. Mm-hmm. It would just redirect you. Yeah.
55
00:04:26.420 --> 00:04:35.100
Yeah. Okay. So then when you do that, and I'm just kind of going through, I'm trying to like answer the interview question here a little bit.
56
00:04:35.100 --> 00:04:36.760
That's right, yeah.
57
00:04:36.760 --> 00:04:38.980
So when you, or maybe I'm interviewing you. I don't even know how this is supposed to work.
58
00:04:38.980 --> 00:04:44.680
I don't know which way around this is anymore, because I thought it was going to go the other way and now we seem to have flipped it. Yeah, so carry on.
59
00:04:44.680 --> 00:04:46.720
I don't know who's, who's, I mean, you know, it's a collaboration.
60
00:04:46.720 --> 00:04:48.020
What even job is this?
61
00:04:48.020 --> 00:04:49.060
We're collaborating here. No one knows.
62
00:04:49.060 --> 00:04:50.780
What are the benefits anymore?
63
00:04:50.780 --> 00:04:51.880
We don't have jobs anymore. They're all taken by robots.
64
00:04:51.880 --> 00:04:53.400
Oh, yeah, that's true.
65
00:04:53.400 --> 00:05:09.460
So when you make this request, you're going to get back headers and you're going to get back the body. Like, how is the request delivered and how exactly is the body returned? Like, what protocol is it using to do that?
66
00:05:09.460 --> 00:05:16.620
Right, right. Well, I mean, I know that the particular command I was using is absolutely HTTP slash 1.1.
67
00:05:16.620 --> 00:05:17.300
Mm-hmm.
68
00:05:17.300 --> 00:05:39.760
And I mean, this is one of those cool things. I know that maybe this is showing our age a little bit, or certainly my age. I think I've got a few years on you. But at university in the late 90s, in the mid 90s, darn, you know, learning that you could Telnet to a port on a machine and just type stuff in.
69
00:05:39.760 --> 00:05:56.000
And get them to do things. So like, you know, the most happy day was to discover how to send emails from other people. I don't recommend that folks try this anymore. I don't think it works anymore, but you used to be able to Telnet to the mail port and give it just raw commands.
70
00:05:56.000 --> 00:05:56.280
Yeah.
71
00:05:56.280 --> 00:05:58.850
And it would be like, very trustingly, say, sure.
72
00:05:58.850 --> 00:06:02.780
Right,
73
00:06:02.780 --> 00:06:22.970
that you're god@universe.com, fine, send your emails please, right? So you kind of got this idea that everything was a text-based thing. And I remember this early web thing, and this was HTTP one, so you would just literally Telnet to port 80 of some machine and type get space slash and hit enter, and then you'd get the document back. No headers, no nothing. It was just, that was the whole thing.
74
00:06:22.970 --> 00:06:23.580
Right. Right.
75
00:06:23.580 --> 00:06:35.280
But nowadays, HTTP 1.1, we have to tell it more information. So you are going to connect to a TCP port on the machine, usually port 80. Sorry, 443 nowadays.
76
00:06:35.280 --> 00:06:45.250
Everything's HTTPS. So for the purposes of talking about the protocol, at least this level of the protocol, HTTP, let's ignore SSL for now.
77
00:06:45.250 --> 00:06:45.370
Yes.
78
00:06:45.370 --> 00:06:45.940
For HTTPS it's 443.
79
00:06:45.940 --> 00:06:47.060
Let's see if we can get to it in time.
80
00:06:47.060 --> 00:06:47.620
Right.
81
00:06:47.620 --> 00:06:47.990
But I don't know.
82
00:06:47.990 --> 00:06:48.200
Right. Like Tom said in his most recent video.
83
00:06:48.200 --> 00:07:06.110
Correct. And I'm so sorry, this is my family coming home in the background, and we thought we'd beat them, but no. So my dog is announcing their arrival. Much as you announce your arrival at the HTTP server by asking it...
84
00:07:06.110 --> 00:07:06.610
Yeah.
85
00:07:06.610 --> 00:07:08.760
Whoa, segue there.
86
00:07:08.760 --> 00:07:09.500
Meta.
87
00:07:09.500 --> 00:07:25.360
So get, it's still a verb, like get or put or post. And most of these things would be like get. Then the URL, rather, the document locator on that machine itself, and I forget the exact terminology for it, but that's the bit after the host name,
88
00:07:25.360 --> 00:07:25.960
Yeah.
89
00:07:25.960 --> 00:07:34.400
but before the hash, if there's one in your URL. So like in Google, it will probably just be slash, forward slash, if it was your Google, because that's the only thing you've got.
90
00:07:34.400 --> 00:07:34.760
Mm-hmm.
91
00:07:34.760 --> 00:07:43.510
If you're going to google.com, then you put a space and then you tell it that it's HTTP slash 1.1, which tells it, hey, I'm talking to you in the new way, the new new way.
92
00:07:43.510 --> 00:07:43.760
Mm-hmm. Mm-hmm.
93
00:07:43.760 --> 00:08:02.660
I mean, there's QUIC and there's HTTP 2 and there's all other things, but like, for simplicity, let's stick with this. Then you're expected to send a bunch of headers. Each header is a bunch of ASCII bytes followed by a colon, followed by a space, I think. Maybe the space is optional. And then some other stuff and the new line.
94
00:08:02.660 --> 00:08:03.780
Yeah.
95
00:08:03.780 --> 00:08:13.800
So it's pretty human readable, as these things go. And the one thing that you will be most required, most required, I don't know if it can be more or less required, but...
96
00:08:13.800 --> 00:08:14.860
The utmost requirement.
97
00:08:14.860 --> 00:08:35.180
The highest requirement is to put host colon and say, actually, what host (again, my dog in the background, you'll have to apologize), but what host I'm actually requesting from. Because this was like the big unlocker, I think, in like the early aughts, whenever this thing came out, which allowed you to have more than one website per IP address.
98
00:08:35.180 --> 00:08:36.620
Right, right.
99
00:08:36.620 --> 00:08:42.420
Because if you just do get slash, you're like, well, which website did you want? I don't know, right? It's whatever.
100
00:08:42.420 --> 00:08:43.140
Mm-hmm. Mm-hmm.
101
00:08:43.140 --> 00:08:50.790
But that was the way to solve it. You know, maybe it would have made more sense to do get HTTP colon blah and just give it the whole URL and say, you work it out. But anyway.
102
00:08:50.790 --> 00:08:51.440
Maybe.
103
00:08:51.440 --> 00:09:11.150
But host colon google.com is what we would have said. And then, I mean, from my own personal experience of Telnetting to google.com to see how it works, I know that all you need to do to get it to work at all is to do get slash HTTP 1.1, host colon google.com return, and then a blank line that says, I'm done with the headers now.
104
00:09:11.150 --> 00:09:11.600
Mm-hmm.
105
00:09:11.600 --> 00:09:17.420
And then you get the document that says, no, you meant, surely you meant www.google.com. You're like, ah,
106
00:09:17.420 --> 00:09:19.120
Right, right.
107
00:09:19.120 --> 00:09:31.050
but nevertheless, connectivity is there. And then it will close the connection, I believe. So that's what I think would happen if I'm doing it manually myself with Netcat or Telnet or something of that nature.
108
00:09:31.050 --> 00:09:34.200
What am I missing? What's next?
109
00:09:34.200 --> 00:09:50.800
No, that's good. And I mean, I love doing that exercise when kind of explaining HTTP to people, because it takes the magic out of it. Now, you introduce a whole lot more magic when you add in like TLS and a bunch of WebSockets and other things like that.
110
00:09:50.800 --> 00:10:00.400
There's a fantastic quote by one of our friends. It was to do with optimization, as it happens, but, you mentioned magic and I wanted to sort of bend it to this, which is that he said, you know, like, HTTP is like magic.
111
00:10:00.400 --> 00:10:05.620
And you're like, well, it isn't. It's just like a bunch of ASCII being sent over it.
112
00:10:05.620 --> 00:10:06.600
It's just text.
113
00:10:06.600 --> 00:10:07.310
Yeah. And then...
114
00:10:07.310 --> 00:10:07.660
It's words.
115
00:10:07.660 --> 00:10:16.740
And that is like magic, because if you actually show people how it's done, it's kind of disappointing afterwards. You're like, oh, it's just, oh, I see. Yeah, there's just two of them. Like, oh, yeah.
116
00:10:16.740 --> 00:10:17.830
The rabbit was already in the hat.
117
00:10:17.830 --> 00:10:17.980
Yeah.
118
00:10:17.980 --> 00:10:19.000
It was there the whole time.
119
00:10:19.000 --> 00:10:19.640
That's right. It was there the whole time.
120
00:10:19.640 --> 00:10:22.290
It's not special.
121
00:10:22.290 --> 00:10:22.420
Oh.
122
00:10:22.420 --> 00:10:23.600
It's not special. It's not a special hat.
123
00:10:23.600 --> 00:10:23.600
It's not special.
124
00:10:23.600 --> 00:10:26.420
It's not a special rabbit. It's just, the rabbit was in there the whole time.
125
00:10:26.420 --> 00:10:32.080
It was there the whole time. Exactly. But I love that feeling of, like, magic is that. That's what magic means, really.
126
00:10:32.080 --> 00:10:33.020
Yeah, yeah, yeah.
127
00:10:33.020 --> 00:10:34.940
It's disappointing when you know how it's done, actually.
128
00:10:34.940 --> 00:10:35.860
Right, right.
129
00:10:35.860 --> 00:10:44.580
But anyway, not disappointing, because now you know how it's done. You can go, oh cool, this is something I can build on and I understand. I can debug. And so on.
130
00:10:44.580 --> 00:10:45.660
Mm-hmm.
131
00:10:45.660 --> 00:10:46.220
But this obviously is not curl.
132
00:10:46.220 --> 00:10:46.420
Mm-hmm.
133
00:10:46.420 --> 00:10:48.720
This is just me doing it manually. And you do it manually.
134
00:10:48.720 --> 00:10:56.700
Right. So when curl is doing this, in the base case, when it just works as expected, it is, yes,
135
00:10:56.700 --> 00:11:00.420
Bass. Not, not that kind of bass.
136
00:11:00.420 --> 00:11:05.650
It is opening a TCP connection to whatever IP address was resolved from the host name that you gave it.
137
00:11:05.650 --> 00:11:05.840
No treble.
138
00:11:05.840 --> 00:11:06.520
Yes.
139
00:11:06.520 --> 00:11:16.820
It is sending those, that initial request with the HTTP verb, as it's sometimes called, the get, the post, the put, whatever it is.
140
00:11:16.820 --> 00:11:27.900
It's sending all the headers that it might send. And curl sends a lot of headers on your behalf, right? You don't, you can tell it to send specific headers, but you can also not. I just want this document, please.
141
00:11:27.900 --> 00:11:34.680
And it will add in whatever headers it thinks are appropriate, including a user agent.
142
00:11:34.680 --> 00:11:37.660
For example, like the user agent, yeah, hey, I'm curl this version and stuff like that.
143
00:11:37.660 --> 00:11:38.480
Yep, yep.
144
00:11:38.480 --> 00:11:41.410
And what type of documents they'll accept, I think, is a list of things.
145
00:11:41.410 --> 00:11:42.200
Mm-hmm.
146
00:11:42.200 --> 00:11:50.720
And then potentially something to do with the connection, in terms of closing the connection or keeping the connection open for further requests, which, yeah.
147
00:11:50.720 --> 00:12:04.510
Mm-hmm. Okay. One, side note here, one of the interesting ways that you can detect which client is actually connecting to you... because the user agent is supposed to say, but obviously that's like kind of voluntary.
148
00:12:04.510 --> 00:12:04.760
You can change that, yeah.
149
00:12:04.760 --> 00:12:15.790
You can put whatever you want in there. But one of the interesting things that you can do is you can look at the order of the headers, because the order of the headers is not specified in the protocol.
150
00:12:15.790 --> 00:12:18.540
So it can be whatever you want it to be.
151
00:12:18.540 --> 00:12:21.660
Oh, but it's a sort of, it's a fingerprint for the client code that connected.
152
00:12:21.660 --> 00:12:30.380
Yes, certain clients tend to put the headers in certain orders, and depending on what client you're using, that can be an indication of what it actually is.
153
00:12:30.380 --> 00:12:32.100
Oh my, that's sneaky.
154
00:12:32.100 --> 00:12:33.320
Yes.
155
00:12:33.320 --> 00:12:34.740
Ben, you are a sneaky man.
156
00:12:34.740 --> 00:12:34.960
Right.
157
00:12:34.960 --> 00:12:44.710
What is, is the sort of like fingerprinting based on the fact that, yeah, well, you know, curl always sends them in this order, and Chrome's fetcher does this and Firefox does this?
158
00:12:44.710 --> 00:12:44.860
Right.
159
00:12:44.860 --> 00:12:50.060
So it doesn't matter how you configure it. It's, you know, to say like, lie, user agent liar, is like, yeah, but we know you're Chrome really.
160
00:12:50.060 --> 00:12:59.940
Yeah, yeah. Similarly with like, you know, as you said, like spaces in between the headers and the values, again, optional, and you can do it either way.
161
00:12:59.940 --> 00:12:59.940
Right.
162
00:12:59.940 --> 00:13:13.960
And that creates, also the casing on the values is another thing that can create sort of a fingerprint here. So curl is choosing to do it the way that curl does it. And even if you change the user agent to be like, oh, no, actually I'm Firefox,
163
00:13:13.960 --> 00:13:26.400
it's possible that somebody else might be able to figure that out. But the header order is not actually really, and I might be lying here, but I think it is specifically not important.
164
00:13:26.400 --> 00:13:27.100
Like, the order is completely unspecified.
165
00:13:27.100 --> 00:13:38.770
I think it is unspecified which sequence they're in. Yeah, I think so. But yeah, reader, dear reader, listener, sorry, not reader, gosh. Listen, but again, it's, it's been a week.
166
00:13:38.770 --> 00:13:40.180
Yeah.
167
00:13:40.180 --> 00:13:45.360
You know, check what we're saying here, but this is just two old farts comparing notes on what they remember about how this works.
168
00:13:45.360 --> 00:14:01.200
Right, right, right. And, well, and as you were saying before, oftentimes web servers are very tolerant of any kind of, you know, sort of garbage they may receive. And I think you said that you literally were just doing get slash on Google earlier today with no protocol.
169
00:14:01.200 --> 00:14:04.930
No, this one had a, yeah, no, this one did have a get slash.
170
00:14:04.930 --> 00:14:05.080
Okay.
171
00:14:05.080 --> 00:14:07.340
No, this was still using curl. I didn't actually Telnet today.
172
00:14:07.340 --> 00:14:07.520
Okay.
173
00:14:07.520 --> 00:14:18.160
But like, I think nowadays, if you do get space slash, it will tell you to go away, because that's HTTP 0.9 or whatever, the one that didn't even have the version in it, pre-versioning.
174
00:14:18.160 --> 00:14:18.160
Yeah.
175
00:14:18.160 --> 00:14:19.990
Yeah, the unversioned version. Yeah, right.
176
00:14:19.990 --> 00:14:23.960
Yeah, something like that.
177
00:14:23.960 --> 00:14:42.660
Exactly. So yeah, so those headers, you're sending those headers whether you know curl is doing it for you or not. You can obviously be explicit about it. But if you don't do anything, it'll do it for you. And then, depending on the type of verb you choose, the request itself may also have a body.
178
00:14:42.660 --> 00:14:45.760
Oh, yes, that's a good point.
179
00:14:45.760 --> 00:14:59.400
And this is particularly important for put and post and probably patch, but you can also do it with get.
180
00:14:59.400 --> 00:14:59.620
And patch, I think, is another one I'm thinking of.
181
00:14:59.620 --> 00:14:59.620
Oh.
182
00:14:59.620 --> 00:15:01.600
You can, I'm pretty sure that you can include a request body with get.
183
00:15:01.600 --> 00:15:07.310
I think one of the headers, one of the header fields, is content length. Is that right? Is that the thing that gates this?
184
00:15:07.310 --> 00:15:07.560
Ooh.
185
00:15:07.560 --> 00:15:15.770
And then you tell it how much you're going to be sending it or receiving. You know, certainly the response has a content length that says, this is how long it's going to be, or it can do.
186
00:15:15.770 --> 00:15:16.080
Yeah, yes.
187
00:15:16.080 --> 00:15:20.980
But if you're going to do anything that requires you uploading, you need to tell it where the end of your thing is, your data.
188
00:15:20.980 --> 00:15:32.070
I think you're right. I think you're right. Which is why, by the way, it is particularly difficult, if not impossible, to stream the body of a request.
189
00:15:32.070 --> 00:15:32.380
Uh-huh.
190
00:15:32.380 --> 00:15:45.520
Like, if you have some input stream that you're reading from and you're like, I don't know how many bytes are in this stream, I'm just, you know, reading it, by the time you're writing the body, you should have already written the header that says the length.
191
00:15:45.520 --> 00:15:46.180
Yes.
192
00:15:46.180 --> 00:15:52.320
So you have to consume the whole stream in order to figure out how many bytes you're going to send.
193
00:15:52.320 --> 00:15:59.880
Which is, I mean, it's fine if you're serving static content off of a disk, where you're like streaming it from disk and you can go, how long is this file?
194
00:15:59.880 --> 00:15:59.880
So you can put that in the header.
195
00:15:59.880 --> 00:15:59.880
Right.
196
00:15:59.880 --> 00:16:13.180
And it's immutable. It's not going to change on me. Okay. It's two terabytes. Cool. Content length, two terabytes. Here it comes. But yeah, if it's being streamed by a process for some reason, it's like outputting, you know, you're catting a file or whatever, something ungodly.
197
00:16:13.180 --> 00:16:13.600
Don't do this in the web.
198
00:16:13.600 --> 00:16:17.820
Or another socket that you're reading from, right? Like you're trying to act as a proxy perhaps.
199
00:16:17.820 --> 00:16:36.020
Right, then you don't know ahead of time how long the thing is going to be to write it into the header, because you've already sent it. And I mean, anyone who's written web server software has almost certainly had some kind of exception thrown when they've tried to do something silly in the body handler, where it's like, no, you can't do that because the header has already been sent.
200
00:16:36.020 --> 00:16:38.040
Mm-hmm.
201
00:16:38.040 --> 00:16:48.360
Like, you know, this is, forget, I've definitely hit it with, oh, crikey, I can't think of the name. But in some cases where you're trying to do something, it's like, no, look, I've already sent the header.
202
00:16:48.360 --> 00:16:56.820
I can't now mutate this. I can't like turn it into a streaming request. I can't turn it into a different size request because we've committed. We're done now. You just have to send text now.
203
00:16:56.820 --> 00:16:57.060
Mm-hmm. Mm-hmm.
204
00:16:57.060 --> 00:16:57.950
Sorry about that.
205
00:16:57.950 --> 00:16:58.300
Mm-hmm.
206
00:16:58.300 --> 00:17:03.140
Yeah, there is multi-part though, which is a pretty crazy hack.
207
00:17:03.140 --> 00:17:31.820
Right, right, multi-part uploads. If you've ever used like an object store that supports HTTP like Amazon S3 and other ones like this, you wind up sometimes doing multi-part uploads, because otherwise, if you have a lot of data to upload and something goes wrong on the 99th percentile byte of your five terabyte file, you've got to do it all over again, which is very inconvenient.
208
00:17:31.820 --> 00:17:41.060
I think there might also be, for S3, now that I say it out loud, there might also be another portion of the S3 protocol that allows you to do that.
209
00:17:41.060 --> 00:17:54.550
I was going to say, I think those are separable, because certainly S3 lets you configure, like, where do the slices of a multi-part upload go and how long do they live before I delete them away.
210
00:17:54.550 --> 00:17:54.670
Yeah.
211
00:17:54.670 --> 00:17:56.160
Yeah, yeah, yeah. How long they live. Right. Right.
212
00:17:56.160 --> 00:18:06.640
But then there's multi-part, which is more like, you say, it's almost like a streaming protocol where you say, hey, I'm going to keep sending you data. Here is the delimiter, and you come up with some string that you hope to God doesn't appear in your actual data.
213
00:18:06.640 --> 00:18:07.760
Right.
214
00:18:07.760 --> 00:18:12.300
And then you say, this is the thing that tells us that we've done this chunk, going to give you some more headers, I think.
215
00:18:12.300 --> 00:18:12.300
Right.
216
00:18:12.300 --> 00:18:17.580
I think that's how that works. Now, I'm very much in dodgy territory here.
217
00:18:17.580 --> 00:18:17.660
Yep.
218
00:18:17.660 --> 00:18:27.240
I've looked at it with WebSockets before. There's some kind of thing that you have to do to get it to, during the handshake, at least WebSockets as they were in the 2010s. So 15 years ago.
219
00:18:27.240 --> 00:18:34.930
Yeah, yeah. Okay, so exploring other areas where you might do this, and we've explained this simple model of the world, we're sending text, we're receiving text, everything's fine.
220
00:18:34.930 --> 00:18:35.250
Oh, God. Right.
221
00:18:35.250 --> 00:18:35.360
Right.
222
00:18:35.360 --> 00:18:37.700
And you sit down with Telnet and you're like, I'm gonna do what Ben and Matt said and gonna request the thing.
223
00:18:37.700 --> 00:18:47.980
Oh, we didn't really, we didn't see, so, yeah, the curl, for example, or if you Telnet directly, you will see the headers of the response, but curl was hiding them from me because I just did curl, the end.
224
00:18:47.980 --> 00:18:48.360
Yes.
225
00:18:48.360 --> 00:19:00.430
So typically you'll see something along the line of, first of all, something that says 200 OK, the response thing that says, like, your get verb and whatever, you get a response document that starts with the...
226
00:19:00.430 --> 00:19:01.460
Yep, yeah, there you go, yep.
227
00:19:01.460 --> 00:19:10.230
What was the code? And you know, we all have seen the codes. The two XX ones are like, everything's fine, and they have slightly different flavors, but usually 200 OK is the one that you'll see.
228
00:19:10.230 --> 00:19:10.300
Right.
229
00:19:10.300 --> 00:19:21.910
I think there's a, one of the two XX ones is like, OK but no data, is like, just, yes, this is fine, but there's nothing to come, which you see only very occasionally.
230
00:19:21.910 --> 00:19:23.460
Yeah. Yes.
231
00:19:23.460 --> 00:19:28.960
Three XX ones are like exceptional case type stuff. Is that right? What is the definition of three XXs?
232
00:19:28.960 --> 00:19:29.300
I think the...
233
00:19:29.300 --> 00:19:31.440
Cause they're like moved and stuff like that, isn't there?
234
00:19:31.440 --> 00:19:34.220
Yeah, like there's temporarily moved, permanently moved.
235
00:19:34.220 --> 00:19:34.780
Yeah, you know, like...
236
00:19:34.780 --> 00:19:39.360
I mean, it's most of the, I think all the redirects are in 300. There might be other things in 300.
237
00:19:39.360 --> 00:19:45.620
That's the kind of edge casey thing where there might be. And then certainly 400 XXs are errors of sorts, and like, the world famous 404.
238
00:19:45.620 --> 00:20:06.280
Yeah, the world famous 404, which you probably know even if you're not a programmer. But there's the one that Twitter proposed and I think actually got introduced to the standard, 420 Enhance Your Calm, for rate limiting responses, which is a reference to the movie Demolition Man.
239
00:20:06.280 --> 00:20:06.330
Yeah.
240
00:20:06.330 --> 00:20:06.760
Oh. Oh, no, I'd not seen that.
241
00:20:06.760 --> 00:20:16.530
Oh, that's cool. I know there's "I'm a teapot," which is one of the things, which is an error code that a coffee, a tea machine may return if it is asked to serve coffee or something like that.
242
00:20:16.530 --> 00:20:16.820
Yep.
243
00:20:16.820 --> 00:20:17.880
Yep, exactly.
244
00:20:17.880 --> 00:20:30.260
And then obviously if something breaks in the server itself and the server wants to let you know that something hideous went wrong internally, it'll give you a five-XX. What are the 1XX things?
245
00:20:30.260 --> 00:20:33.100
Are those like continuations, I think, for stuff like multi-part?
246
00:20:33.100 --> 00:20:34.040
I think so.
247
00:20:34.040 --> 00:20:36.720
Oh, no. See, I'm trying to do this without Googling because I have the loudest mechanical keyboard.
248
00:20:36.720 --> 00:20:36.910
I know.
249
00:20:36.910 --> 00:20:44.360
So everything you're hearing, dear listener, is coming off the top of our heads, and that, possibly, other parts of our anatomy, depending on how accurate they are.
250
00:20:44.360 --> 00:20:45.060
That's right.
251
00:20:45.060 --> 00:20:55.440
[laughing] So anyway, and then after that, you get to see a sequence, sorry, of headers that look like the ones that you sent up.
252
00:20:55.440 --> 00:20:55.440
Right. That's exactly right.
253
00:20:55.440 --> 00:21:02.550
So it's, you know, something colon something else. And then there'll be two new lines, and then off you go with the data, the response.
254
00:21:02.550 --> 00:21:02.700
Mm-hmm.
255
00:21:02.700 --> 00:21:04.900
So that's, it's pretty cool.
256
00:21:04.900 --> 00:21:05.210
Mm-hmm.
257
00:21:05.210 --> 00:21:07.420
It's, yeah.
258
00:21:07.420 --> 00:21:09.940
It is. It is. And so this is perfect, actually.
259
00:21:09.940 --> 00:21:10.000
Yeah.
260
00:21:10.000 --> 00:21:22.040
This is perfect. So coming back to what I was saying before about, like, okay, I'm going to do what Ben and Matt said, I'm going to sit down and I'm going to like type these things out in Telnet, and I'm going to get back some response. And when I do that, all I see is a bunch of gobbledygook.
261
00:21:22.040 --> 00:21:24.190
I don't see any text. I don't see any HTML.
262
00:21:24.190 --> 00:21:24.340
Yes.
263
00:21:24.340 --> 00:21:40.100
What is going on? One of the things that might be going on is that the server has compressed the response. And it will tell you that it has done that, and it will tell you which compression algorithm it has used to do that in the headers.
264
00:21:40.100 --> 00:21:44.860
So you will see a compression header, and then you will see...
265
00:21:44.860 --> 00:21:47.310
Yes. So the header, the text-based headers are still there, just to be clear, right?
266
00:21:47.310 --> 00:21:47.420
Yes. Yes.
267
00:21:47.420 --> 00:21:56.800
And then there'll be a new line, and then you'll see garbage, which, although a well-configured server ought not to do that unless you said I would accept a compressed thing.
268
00:21:56.800 --> 00:22:10.730
But nevertheless, this is exactly the kind of thing, to your point at the beginning of this, is like, when something goes wrong, you know, I'm getting an endpoint that just assumes every client can support a compression algorithm, which I believe you're about to go and talk about.
269
00:22:10.730 --> 00:22:10.740
That's true.
270
00:22:10.740 --> 00:22:10.750
Right.
271
00:22:10.750 --> 00:22:10.760
Right, right.
272
00:22:10.760 --> 00:22:12.020
So I don't want to keep, I don't know.
273
00:22:12.020 --> 00:22:24.380
Well, yeah, no, and that's another thing that curl is, again, you know, peeling back the abstraction here, that's another thing that curl is doing for you, right? It's reading that header, it's applying the correct compression. I think you might need to tell it.
274
00:22:24.380 --> 00:22:32.400
I can't remember if, like, if you just, but can you disable it? Can you tell it not to do that?
275
00:22:32.400 --> 00:22:51.880
I don't remember either. I know I've had problems before because, and this is going to lead to one of my favorite observations, or rather a mutual friend's observation, you know, whether it's transparently decompressing because it says it's deflated, and so curl is then going to say, well, I'll undeflate it then, because that's what it says that I should do.
276
00:22:51.880 --> 00:23:08.210
Or whether, you know, that just means the fact that the compression, whether the compression is there or not, is now opaque to you. You don't know if it's happening or not, which is the kind of dichotomy of transparency: it means that now it's impossible to see whether it's actually happening or not, you know?
277
00:23:08.210 --> 00:23:08.340
Right.
278
00:23:08.340 --> 00:23:08.600
Yes.
279
00:23:08.600 --> 00:23:18.930
So our friend coined the term, which I introduced to somebody at work yesterday, transpaque, which is like that kind of combination of, like, well, it's like you're meant to not know it's there,
280
00:23:18.930 --> 00:23:19.550
Yeah, yeah, yeah.
281
00:23:19.550 --> 00:23:19.960
Right.
282
00:23:19.960 --> 00:23:25.460
but if it is there, now you can't tell if it's there or not, and maybe it's broken, and now you're just being gaslit by your system.
283
00:23:25.460 --> 00:23:25.960
Right.
284
00:23:25.960 --> 00:23:37.620
You know, I think we had this with at least one of the things at work where a vendor was sending gzipped data, but whatever...
285
00:23:37.620 --> 00:23:52.840
accept header we were getting was then un-gzipping it for us, even though we'd asked, you know, like, get bob.gz or get, you know, this.gz. And then the library was like, oh, cool, I'll un-gzip that for you. And then it was storing it as, and we were writing it to disk as, you know, bob.gz.
286
00:23:52.840 --> 00:24:02.000
And you're like, why is everything so huge? Oh, wait, everything's called .gz because that's what we asked for, but something transparently, again, transpaquely...
287
00:24:02.000 --> 00:24:03.680
Transpaquely.
288
00:24:03.680 --> 00:24:22.120
un-gzipped it for us, being helpful, thinking it was being helpful, and it wasn't, right? Anyway, so, but yes, it could be deflate compressed, I think. Or what are the other ones that are out there? Zstandard, I think, is that standardized? I don't know, but deflate's the one that, you know, everyone uses, which is to say gzip.
289
00:24:22.120 --> 00:24:34.450
Pretty much, to a first approximation, deflate is the algorithm that powers gzip, and then there's a little tiny bit of a header associated with an actual gzip file, which I think is slightly different, and a CRC.
290
00:24:34.450 --> 00:24:35.340
Okay.
291
00:24:35.340 --> 00:24:39.020
So that's definitely something that can go wrong.
292
00:24:39.020 --> 00:24:48.320
Yeah. Yeah. Do we want to try to get into TLS and compression and HTTPS? Is that a bridge too far for this conversation?
293
00:24:48.320 --> 00:24:56.390
Well... that might be a bridge too far. Certainly, I don't know enough about it. So what we've been talking about so far, let's just graze the topic and see if we feel comfortable with this.
294
00:24:56.390 --> 00:24:56.520
Yeah, right.
295
00:24:56.520 --> 00:25:17.380
And then our listener can roll their eyes at how wrong we are. But what we've been talking about is like the HTTP protocol. And in HTTPS, there's another layer, another sort of wrapping of the protocol where the, a certain amount of, well, a certain amount, some,
296
00:25:17.380 --> 00:25:36.390
encryption is applied, and authentication and certificate exchange, all that kind of stuff. But once you've peeled that back, you're back to HTTP again. So like, effectively, from the point of view of the web server itself and from the web client, you can almost, almost ignore the fact that the tunnel that connects them is...
297
00:25:36.390 --> 00:25:36.680
Mm-hmm.
298
00:25:36.680 --> 00:25:55.260
an encrypted tunnel, as opposed to just a regular Telnet. So obviously you can't Telnet to port 443 and type anything and expect it to happen, which is unfortunate. What you can expect, what will happen is, if you're very lucky, you might see a lot of garbage bytes being sent back to you, and then the connection is closed. That's probably what you're gonna see if you do anything at all, if you Telnet to port 443.
299
00:25:55.260 --> 00:26:23.460
Something that surprised me the other day is, I naively assumed that you connect to port 443, some kind of negotiation happens with a certificate and key exchange and the whole Diffie-Hellman thing to, like, you know, get a unique key on both sides, and then, and then, and only then, you start saying, hi, I'd like to get this, you know,
300
00:26:23.460 --> 00:26:24.440
Right. I was going to say this exact same thing.
301
00:26:24.440 --> 00:26:42.920
But, yeah, yeah. But it turns out, for all that, for the same reason that we added the host colon to the sort of, like, the message body rather than just relying on the Telnet, you know, the IP address that you're connecting to being unique for that website.
302
00:26:42.920 --> 00:26:53.920
We lose that if we encrypt it, which means that, like, you know, the web server you're talking to, if it's, you know, google.com is co-located with yahoo.com, unlikely.
303
00:26:53.920 --> 00:26:58.280
But whose certificate should the server give you when you connect to it?
304
00:26:58.280 --> 00:26:58.360
Right.
305
00:26:58.360 --> 00:27:03.500
You're like, ah, good question. I don't know. Which means that you, the client, need to tell it first.
306
00:27:03.500 --> 00:27:04.200
Yes.
307
00:27:04.200 --> 00:27:12.640
But how do you encrypt that so that no one can tell which site you're actually browsing? And the surprising answer is, you don't.
308
00:27:12.640 --> 00:27:13.640
Yep.
309
00:27:13.640 --> 00:27:15.340
Which was like, mind blown.
310
00:27:15.340 --> 00:27:16.110
Yep. Yep.
311
00:27:16.110 --> 00:27:18.840
Oh.
312
00:27:18.840 --> 00:27:21.280
Not as secure as you thought. All that information is being exposed to the interwebs.
313
00:27:21.280 --> 00:27:27.680
So... does this mean that all these adverts that I keep fast-forwarding through about VPNs...
314
00:27:27.680 --> 00:27:36.900
And, dear listener, do not worry, this is not about to segue into an advert for one of those. This is, it's not a way of announcing that finally Surfshark or whatever is...
315
00:27:36.900 --> 00:27:37.280
We've taken sponsorship. Yeah.
316
00:27:37.280 --> 00:27:38.050
No, no sponsorship.
317
00:27:38.050 --> 00:27:39.260
Uh huh.
318
00:27:39.260 --> 00:27:39.760
This is just my fun.
319
00:27:39.760 --> 00:27:40.540
Right.
320
00:27:40.540 --> 00:27:52.100
We're not turning into anything else. But no, that is the one argument I can see for that, because I just assume, you know, like, fine, HTTPS everything, and it's, you know, no one need know what I'm doing.
321
00:27:52.100 --> 00:28:01.800
But it's, you know, you are actually, if you cracked out Wireshark and visited a website (which maybe we should do live sometime, we need to work out a way of screen recording this or something).
322
00:28:01.800 --> 00:28:03.240
Yeah. Yeah, yeah, yeah.
323
00:28:03.240 --> 00:28:15.190
Because, yeah, that's, I think the first time you show somebody Wireshark and then you're browsing or doing anything trivial on the network is the first time the scales fall.
324
00:28:15.190 --> 00:28:15.340
Mm-hmm.
325
00:28:15.340 --> 00:28:19.900
Certainly was for me. Like, oh my gosh, look how much stuff's going on on my network or on my computer.
326
00:28:19.900 --> 00:28:20.120
Mm-hmm.
327
00:28:20.120 --> 00:28:21.220
Oh, golly.
328
00:28:21.220 --> 00:28:21.840
Mm-hmm.
329
00:28:21.840 --> 00:28:25.120
But anyway, yeah, so, popping back, you were going to say the same thing about this. Surprising, was it SNI or something, was the name of it?
330
00:28:25.120 --> 00:28:48.070
Yeah, you should be very careful about assuming that the data that is in your request, the URL, including the query parameters, the headers that you are either adding yourself or being added on your behalf by the tools that you're using, like curl, are actually secure, because they're almost certainly not, in many, many cases.
331
00:28:48.070 --> 00:28:48.320
Hmm.
332
00:28:48.320 --> 00:28:50.080
So when you make that request,
333
00:28:50.080 --> 00:28:56.780
I was thinking it was only the host that was the thing that was leaked, just purely so the correct certificate could be served back to you. But I may be wrong.
334
00:28:56.780 --> 00:29:14.950
So I think that, so my understanding of this is that you're doing this sort of, like, escalation request, you're like, I'd like to escalate to this other protocol. And I would not be very confident in telling anybody that I knew for certain that, in all cases, it could only be these things, right?
335
00:29:14.950 --> 00:29:16.600
No, that's fair.
336
00:29:16.600 --> 00:29:24.620
It's, yeah, it's one of those things where it's like, if it's important to you that that information is encrypted, then you should check, probably using Wireshark or something else like that.
337
00:29:24.620 --> 00:29:49.180
I've just fired it up. I couldn't help myself. Oh, I need to sudo make me a sandwich, of course, because, why, I shall. Yeah, okay, given that I have to escalate, I'm not going to do that now, otherwise, you know, you are going to hear my long password rattling out. But no, that's definitely an exercise for the reader. And I'm sure, hopefully people will comment and tell us where we're going wrong here. But again, this is from the top of our heads. So, yeah, no. So.
338
00:29:49.180 --> 00:29:55.020
Okay, I think that's probably the most we should do on the encryption, because it's clear that we've reached the limit of our knowledge.
339
00:29:55.020 --> 00:29:55.100
Yeah, yeah.
340
00:29:55.100 --> 00:29:57.280
We've hit the layer of abstraction that we can recall sitting here.
341
00:29:57.280 --> 00:30:02.470
I mean, there is definitely some things to do with certificates and things, and, you know, the little lock that you see.
342
00:30:02.470 --> 00:30:02.820
Yeah.
343
00:30:02.820 --> 00:30:03.640
And a little bit about that.
344
00:30:03.640 --> 00:30:07.620
Not going to pretend to know nearly enough about that to explain it to the world.
345
00:30:07.620 --> 00:30:11.000
Well, I mean, there is like a chain of trust, isn't there? We can probably talk a little about that, where...
346
00:30:11.000 --> 00:30:13.680
Mm-hmm. Yeah, that's true. Certificate authorities.
347
00:30:13.680 --> 00:30:34.900
Yeah, so your browser and curl or whatever, and your operating system, may provide a list of certificate authorities, which are like trusted third parties that have said, we will sign other people's certificates to say it's definitely them. Trust me, bro. And then, as long as you trust the person who signed the certificate, you can trust.
348
00:30:34.900 --> 00:30:42.600
Google.com is in fact Google.com because it's signed by VeriSign or whoever the heck actually signs Google. Google probably are a CA, aren't they? Probably, and that's fine.
349
00:30:42.600 --> 00:30:43.900
Yeah, probably at this point.
350
00:30:43.900 --> 00:30:59.960
And that's kind of how you know that you are actually looking to the correct site, the authentication part of HTTPS. And "know" has got air quotes here, because obviously it really does depend how much you trust the people who signed these certificates to say it really is who they say that it is.
351
00:30:59.960 --> 00:31:00.020
Mm-hmm.
352
00:31:00.020 --> 00:31:14.340
And if you look at the number of CAs that there are inside your browser, if you go and look at the list, and there's like a bewildering number of them, including some from, to me, things that would raise an eyebrow, and I'm like, oh, apparently we trust this company.
353
00:31:14.340 --> 00:31:14.560
Mm-hmm.
354
00:31:14.560 --> 00:31:18.200
I don't know who they are, and it doesn't look very trustworthy to me.
355
00:31:18.200 --> 00:31:19.560
Right, right, right, right.
356
00:31:19.560 --> 00:31:37.240
Similarly, in a corporate environment, you might discover, reasonably, your own company's root CA is in that list, because then you can sign, first of all, you can sign all of your internal applications with your own certificate that doesn't have to go out to the outside world, doesn't have to be signed by VeriSign.
357
00:31:37.240 --> 00:31:48.780
It means that you can go to, you know, benscoolapp.internal.com, and you can get a certificate for that, and your browser won't go, wait a second, this is just self-signed.
358
00:31:48.780 --> 00:31:53.590
That's no good. You know, it can be signed by, you know, benscoolcompany.com, which is then in there.
359
00:31:53.590 --> 00:31:53.960
Mm-hmm.
360
00:31:53.960 --> 00:32:10.060
But it also allows potential for, like, security scanning software to decrypt, by man-in-the-middling you in your own, you know, on your own network, to be able to check that things aren't going outside, which is really important in, certainly in a large corporate environment.
361
00:32:10.060 --> 00:32:11.080
I don't know that it happens in any, like, the companies we're in, but.
362
00:32:11.080 --> 00:32:21.420
No, no, I mean, that's definitely a thing. I mean, again, like, you know, the theme of this being, like, you know, breaking people's expectations around abstractions and things like that.
363
00:32:21.420 --> 00:32:21.780
Right.
364
00:32:21.780 --> 00:32:36.360
Like, it's probably been told to you in indirect ways by people at your company if they're doing this, but maybe not in as direct ways as, we're reading your personal email if you check your personal email at work, right?
365
00:32:36.360 --> 00:32:36.520
Right.
366
00:32:36.520 --> 00:32:39.680
Like, the little security lock in the browser that you're running, right?
367
00:32:39.680 --> 00:32:47.610
Is not as strong as you think it is, because if you're using your company, I mean, obviously, if you're on your personal device on the guest Wi-Fi of your company or whatever, fine.
368
00:32:47.610 --> 00:32:47.650
Yeah.
369
00:32:47.650 --> 00:32:47.740
Sure. Right.
370
00:32:47.740 --> 00:32:57.940
And that's a very good reason to do that, right? But yes, if you are, most companies have at least a clause that says something like, there's no expectation of privacy, which is reasonable, right? You're at work, you're doing work things.
371
00:32:57.940 --> 00:32:59.260
Yeah.
372
00:32:59.260 --> 00:33:00.730
I mean, reasonable people could have disagreements about this.
373
00:33:00.730 --> 00:33:00.800
You're on a work computer doing work things.
374
00:33:00.800 --> 00:33:24.960
I get it. I get it. I get it. But like, that is the way in which they can potentially look at the encrypted data, is that they can say, well, any website you go and get, if you go to secure.com, I will say, yes, sure, I'm secure.com, and I can prove I am because I'm signed by this certificate, which, by the way, your company-given security system says I can sign anything I damn well like, and that's fine.
375
00:33:24.960 --> 00:33:25.140
Yeah.
376
00:33:25.140 --> 00:33:25.500
Yeah, right.
377
00:33:25.500 --> 00:33:39.700
And that's, yeah. So yeah, just knowing that that exists is all you need to do. And it's an exercise to the listener to go and see whether that's something that's happening to them or not. And I mean, it's a fairly standard thing.
378
00:33:39.700 --> 00:33:49.160
Anyway, popping the stack all the way back, right? So we said, what about curl, right? You're right, with curling this endpoint. What else? Is there anything we've missed? Let's just think about this.
379
00:33:49.160 --> 00:33:50.780
Well, I got, so I got a good one.
380
00:33:50.780 --> 00:33:51.300
OK.
381
00:33:51.300 --> 00:34:16.020
So let's say that, what you see, when you do this request, and maybe you're trying to drop down a level here and you're gonna use, like, the Telnet approach, is that you connect to the site, you type in your headers, you get back your response headers, and then you get some of the body, and then it stops.
382
00:34:16.020 --> 00:34:18.580
Ah, yeah,
383
00:34:18.580 --> 00:34:27.770
What happened? What is happening? Like, how do you even know that it's not done? What is the mechanism for that, if I recall correctly?
384
00:34:27.770 --> 00:34:27.960
Yeah.
385
00:34:27.960 --> 00:34:40.010
It's like, two new lines is the fancy technology that they use to tell you that the body of the document is complete. It's that we're gonna do a new line and then we're gonna do another new line, and that's how you know that it's done.
386
00:34:40.010 --> 00:34:40.800
I think that's the hardcore thing.
387
00:34:40.800 --> 00:34:44.180
But let's say that you don't see those two new lines and it just stops.
388
00:34:44.180 --> 00:34:44.860
Right.
389
00:34:44.860 --> 00:34:48.040
What is curl doing in that situation?
390
00:34:48.040 --> 00:34:54.740
That's an excellent question. I don't actually know the answer to that.
391
00:34:54.740 --> 00:35:11.240
Yeah, I don't know, because, like, my instinct, if this was an interview question and you, Ben Rady, were asking me what's going on here, I would be like, oh, this sounds like the remote end has decided to keep the connection open.
392
00:35:11.240 --> 00:35:29.440
But it has, in fact, finished the document. And I don't know if it puts something at the end of that. I don't think it does. Maybe that's one of those 100, 100 continue is like one of those things, as it puts it in. I don't know. But, totally separately, and just pause there, I did, in fact, just Telnet to google.com and do get space slash enter.
393
00:35:29.440 --> 00:35:30.780
I think you could get something like that, yeah.
394
00:35:30.780 --> 00:35:36.500
And it doesn't even tell me that I'm at the wrong address, because it doesn't know that I came to the wrong address. It doesn't try and redirect. It just gives me the whole...
395
00:35:36.500 --> 00:35:42.020
you know, minimized google.com thing pukes out. So it still works with even, you know, HTTP 0.9.
396
00:35:42.020 --> 00:35:42.300
Interesting.
397
00:35:42.300 --> 00:35:47.900
Okay. What happened then, Ben? Tell me the answer. What was going on with my hung connection, or what might be happening?
398
00:35:47.900 --> 00:35:58.910
So this could be, this could be a number of things, but one scenario in which you would see this is that there was, like, an error, like a programming error on the server side, where it like read a document,
399
00:35:58.910 --> 00:36:00.220
Oh.
400
00:36:00.220 --> 00:36:08.340
and encountered an error and for whatever reason didn't close the connection. Right, maybe it's waiting for, itself is waiting for data to show up that just isn't showing up.
401
00:36:08.340 --> 00:36:08.680
Right.
402
00:36:08.680 --> 00:36:12.310
It's not going to come, or it's deadlocked in itself trying to do something.
403
00:36:12.310 --> 00:36:19.030
Right. It's, like, in the midst, stuck on some database query or streaming some data from another source or whatever it might be, and it's just stopped.
404
00:36:19.030 --> 00:36:19.280
Yeah.
405
00:36:19.280 --> 00:36:19.530
Yeah.
406
00:36:19.530 --> 00:36:39.460
curl has a number of different timeouts that you can configure. And part of the reason for those timeouts is situations like this, because you don't want some process to just be stuck forever because it made some request that is now stuck forever.
407
00:36:39.460 --> 00:36:41.300
It has a connection timeout, which is more of a TCP level thing. It is a TCP level thing. It's like, hey, I want to connect to this IP address and send it some text so I can get my documents.
408
00:36:41.300 --> 00:36:59.500
If it never establishes the TCP connection, or it might take some time to establish a TCP connection, there's a timeout that you can control for that. I believe there is also a response timeout, to say, like, if I start getting data and then I stop getting data, just whacked my microphone.
409
00:36:59.500 --> 00:37:01.660
That's fine, I'm sure that will be edited away.
410
00:37:01.660 --> 00:37:04.500
It won't be edited away!
411
00:37:04.500 --> 00:37:07.780
Yeah.
412
00:37:07.780 --> 00:37:22.040
And you stop getting data, and then you continue to not get any data, how long should I wait before I just give up? And it's important to know, like, with all of these protocols, we use it often enough, you almost think of them as, like...
413
00:37:22.040 --> 00:37:28.830
You almost think of it as being, like, this synchronous thing, right? Like, almost like an atomic thing, where it's like, I'm going to do the request thing and we get the response, and it all works.
414
00:37:28.830 --> 00:37:29.280
Right, right, right, right.
415
00:37:29.280 --> 00:37:49.860
And it is entirely possible that you get some, but not all, of the response. It is entirely possible that you get halfway through sending the request and then the connection just dies. It's entirely possible that you start sending the request and, depending on exactly what is happening on your network, it may not be clear how much of it was actually sent.
416
00:37:49.860 --> 00:37:54.760
Like, did you, I'm doing this, like, put request or delete request, right?
417
00:37:54.760 --> 00:37:54.980
Oh.
418
00:37:54.980 --> 00:37:58.680
Like, did you actually delete it on the other side?
419
00:37:58.680 --> 00:37:59.300
Right.
420
00:37:59.300 --> 00:38:00.660
I'm not sure.
421
00:38:00.660 --> 00:38:08.840
That is, yeah, the sort of, like, what's the name of it? The Byzantine generals problem or whatever, where you've got, like, the two, you're like, I sent a thing to you that says, did you delete it?
422
00:38:08.840 --> 00:38:08.900
Yeah.
423
00:38:08.900 --> 00:38:17.340
And you go, I got it, but I didn't hear your response. You're like, well, did you then? I don't know. I don't know, because I didn't, and like, there's any number of acks on both sides that you, and you can never be a hundred percent sure. And yeah.
424
00:38:17.340 --> 00:38:18.590
Exactly, exactly.
425
00:38:18.590 --> 00:38:18.860
Yeah.
426
00:38:18.860 --> 00:38:24.160
So that is definitely another place where this sort of abstraction can break down, where it's like, well, I did the delete request and it failed.
427
00:38:24.160 --> 00:38:24.340
I've...
428
00:38:24.340 --> 00:38:26.300
So clearly it's not deleted.
429
00:38:26.300 --> 00:38:27.790
There's at least five different timeouts.
430
00:38:27.790 --> 00:38:27.900
Well,
431
00:38:27.900 --> 00:38:33.420
I've just been looking through the man page of curl, five different termouts, termouts, timeouts for different conditions.
432
00:38:33.420 --> 00:38:34.360
Termouts.
433
00:38:34.360 --> 00:38:45.420
You know, like, how often, how long do you wait before connection? As you said, overall timeout for the entire request, the first response, and then, you know, subsequent chunks of data, that kind of thing.
434
00:38:45.420 --> 00:38:46.080
Mm-hmm.
435
00:38:46.080 --> 00:38:58.540
And then some other things to do with IPv4 and IPv6, if you're trying to do both at the same time and get the first one to respond, some really clever things like that. But yeah, a lot of those things are sort of, yeah, as you say, we think of these things as atomic.
436
00:38:58.540 --> 00:39:16.580
We think about, you know, like, everyone talks about RESTful APIs and you're like, okay, it's like a magic RPC that I just do, and it happens or it doesn't happen and I'm done. And you're like, you could see an error on your side that says the connection never made, you know, the request never went through, but it did in fact make it through.
437
00:39:16.580 --> 00:39:23.380
It was just the response didn't get to you and the TCP was torn down, but it did in fact take effect on the remote end, and now you're stuck.
438
00:39:23.380 --> 00:39:32.800
You didn't get back, you know, the response. So it's a tricky world out there, but yeah.
439
00:39:32.800 --> 00:39:34.400
Absolutely. Yeah.
440
00:39:34.400 --> 00:39:36.440
No, there's many timeouts. Oh my gosh, yeah.
441
00:39:36.440 --> 00:39:37.720
You said there's five timeouts?
442
00:39:37.720 --> 00:39:42.240
I think so. I've got connect timeout, expect 100 timeout.
443
00:39:42.240 --> 00:39:47.940
"Maximum time in seconds you'll wait for curl for a 100 continue." Oh, good. I wasn't completely making that up.
444
00:39:47.940 --> 00:39:49.000
That's good.
445
00:39:49.000 --> 00:39:52.780
See, you also, connect timeout. Happy eyeballs timeout. That's hilarious to me.
446
00:39:52.780 --> 00:39:54.040
What? Happy eyeballs timeout?
447
00:39:54.040 --> 00:40:19.140
Yeah, I'm going to read this to you because it's joyous. "Happy Eyeballs", with a capital H and a capital E, "is an algorithm that attempts to connect to both IPv4 and IPv6 addresses for dual-stack hosts, giving IPv6 a head start of the specified number of milliseconds." So it's like a way of, you know, preferring IPv6, but not necessarily...
448
00:40:19.140 --> 00:40:21.960
Oh man, that's good.
449
00:40:21.960 --> 00:40:38.910
So yeah, we've got, see also, a max timeout, retry max time. So there's an overall max time, and then there's a retry maximum time, which is like the number of times it will, including, yeah, retry time is reset before the first... It's complicated, but I mean...
450
00:40:38.910 --> 00:40:41.230
And then we're going to keep talking.
451
00:40:41.230 --> 00:40:44.180
Yeah, and we're back.
452
00:40:44.180 --> 00:40:54.260
What happened there was a terrible break in the abstraction layer, because my USB hub crashed and put us into a very bad state. And we've just about recovered it.
453
00:40:54.260 --> 00:40:57.400
And, can't remember what on earth we were talking
454
00:40:57.400 --> 00:41:00.760
No idea. No idea at all. It's like it was a different dimension.
455
00:41:00.760 --> 00:41:11.550
about. And if only, I mean, if only we had some kind of permanent record of what we'd said. But sadly, neither you nor, you said you could download the MP3s okay, and it was working. So at least we've got that on our side.
456
00:41:11.550 --> 00:41:14.160
I did do that, yes.
457
00:41:14.160 --> 00:41:24.760
Okay, right. So we've got the previous part. So, dear listener, we apologize. Whatever we were talking about before has slipped our minds. But we didn't want to leave you in the lurch, or, more importantly, we didn't want to leave editing Matt with a job of, like, how do you finish an episode where...
458
00:41:24.760 --> 00:41:24.760
Right.
459
00:41:24.760 --> 00:41:37.660
Clearly something went wrong. But I think there's another abstraction layer we can talk about another time. It's like, how on earth do our devices talk to each other in a sort of day-to-day environment?
460
00:41:37.660 --> 00:41:41.280
Yeah, USB actually would be kind of a cool one.
461
00:41:41.280 --> 00:41:46.150
USB, USB-C and DVI and digital video and all those kinds of magic.
462
00:41:46.150 --> 00:41:46.300
Maybe a bit ambitious, but I think I like that one.
463
00:41:46.300 --> 00:41:49.000
I mean, it's a miracle that it works as well as it does, frankly.
464
00:41:49.000 --> 00:41:50.760
Yeah. Yeah. Yeah.
465
00:41:50.760 --> 00:41:58.020
But yeah, we were looking at curl, and I think we were just about looking through all the timeouts. That's what I'd recall us talking about.
466
00:41:58.020 --> 00:41:59.560
Yeah. Oh, the happy eyeballs.
467
00:41:59.560 --> 00:42:09.120
Happy eyeballs for IPv6, IPv4, which is crazy. I saw someone put an IPv8 proposal out there, which I think is somewhat tongue-in-cheek, but also I read through it and I was like, this isn't as stupid as it looks.
468
00:42:09.120 --> 00:42:09.120
What?
469
00:42:09.120 --> 00:42:11.940
Maybe this is real. You know, I can't tell.
470
00:42:11.940 --> 00:42:13.440
And they skipped seven because we skipped five. That's how, yeah.
471
00:42:13.440 --> 00:42:31.040
I guess. I think the, it uses, like, a 64-bit IP address rather than the full crazy IPv6, I don't know, 256-bit, 128, whatever bit size that is, where it's like, hey, all this information about the host, you're like, I don't want to leak that outside my network.
472
00:42:31.040 --> 00:42:31.910
Thank you very much.
473
00:42:31.910 --> 00:42:32.600
Yeah, yeah, yeah.
474
00:42:32.600 --> 00:42:43.840
Right, so it was, and it's sort of backwards compatible with IPv4 because it's essentially, all existing IPv4s are 0.0.0.0.0.0, the rest. You're like, oh, the rest, you're like, oh, that makes sense.
475
00:42:43.840 --> 00:42:43.900
Interesting.
476
00:42:43.900 --> 00:42:56.120
Maybe that's got legs. Anyway, that's really off topic, and I have, that's about as much as I know about it. My friend, I don't know if there's anything that we can recover from where we were going.
477
00:42:56.120 --> 00:42:58.500
So I think we should probably just call it at this point here.
478
00:42:58.500 --> 00:42:58.840
No, yes.
479
00:42:58.840 --> 00:43:06.900
And having introduced the world to transpacity, I feel like, you know, we've at least achieved something.
480
00:43:06.900 --> 00:43:08.230
Transpacity?
481
00:43:08.230 --> 00:43:08.410
Yeah.
482
00:43:08.410 --> 00:43:08.780
Transpacity. Okay.
483
00:43:08.780 --> 00:43:10.040
I guess it would be, wouldn't it?
484
00:43:10.040 --> 00:43:10.240
Yes.
485
00:43:10.240 --> 00:43:10.440
Yeah.
486
00:43:10.440 --> 00:43:11.040
Yes, it is.
487
00:43:11.040 --> 00:43:12.040
Transpacity. Opacity, transpacity.
488
00:43:12.040 --> 00:43:14.020
That's exactly, transpacity.
489
00:43:14.020 --> 00:43:20.010
We'll have to ask our friend and see if maybe he'll join us on the podcast sometime to talk about it. But it's definitely a thing.
490
00:43:20.010 --> 00:43:20.780
Yeah, there you go.
491
00:43:20.780 --> 00:43:23.790
All right, mate. Well, it's been a journey today.
492
00:43:23.790 --> 00:43:25.200
Yep. Fun times.
493
00:43:25.200 --> 00:43:28.640
I will chat to you another time.
494
00:43:28.640 --> 00:43:30.640
Sounds good.
