Full Transcript for DNS Cold Start

5:51 fine name, fine website, all that kind of stuff. And I rebooted my resolver and I ran TCP dump. I'm doing packet capture. So what do I see? Well, the first thing it does is it goes. I've been given a set of hints for the IP addresses of the root of the DNS, the root name servers. Don't trust them, so I'll pick one at random, one of these addresses I've been given, and ask it for all the name servers of the root. That's query number one. Hi, root root server. Tell me about the root name service. It's called the priming query. I'm not sure how old the stuff is that came with the software release. Let me refresh it with live information. Query one. George Michaelson 6:29 I could kind of imagine having picked one at random if it was a bad hair day and that one at random was offline or unreachable. There's an obvious outer loop. All right, pick a different one and try again, and there are 13 of them, and they both have four and six. Geoff Huston 6:44 four and six, and there are 26 queries before you give up in disgust. But by then, you'd have to have snipped the wire on your own computer not to have got an answer. George Michaelson 6:52 Yeah, one of these is going to have an anycast instance close to you, and it's going to say, "Hi, here's the complete list of all the other root servers addresses and names. Go for your life. Geoff Huston 7:03 So I pick one. Doesn't matter anyone. And I go dot.dot.potaroo.net. I wonder who are the name servers of.net. Don't forget, I've got no cache, no brain, no nothing. I just had a root hints file. I've now got a true version of the root servers, but now I need to know the net servers. So I ask a root server at random. George Michaelson 7:22 Just before we move on, Geoff, was that literally only two queries for you, or did it actually turn out even that bootstrapping was more than two queries? Geoff Huston 7:31 No, no, one, one query, George Michaelson 7:33 one query. Geoff Huston 7:34 Ask a root name server that came pre-packaged for the name servers of the zone dot. George Michaelson 7:40 One question. Geoff Huston 7:41 Query type NS. Query name dot. One question, one answer. [George: right] The next question. Pick any of those IP addresses of the root name servers. Just one. Pick them. Pick one and say, tell me the name servers for .NET. Fine. One answer. Here is a list of the names of the name servers, and because because the name servers are in bailiwick, the name servers for .NET are in .NET A.GTLD servers .NET, B. You know, then it actually gratuitously says, "Look, I know strictly speaking, I should only tell you the names of the name servers, but look, I'm going to be really friendly, and you're not going to find out this information any other way because you don't know the name servers until I tell you. I'm

going to tell you the IP addresses of those name servers. George Michaelson 8:30 And again, this is extra data given for free, but in one query you get a bundle of info. Geoff Huston 8:37 I get this so called glue, and quite frankly, if I didn't get the glue, I'd be stumped because I don't know who the name servers for .NET. If you give me said, "Well, here's the name of a name server in .NET that happens to be in .NET, I'm sitting there going, "That's circular dude. I can't get out of this hole. You got to give me some help. And so it does. The DNS gives you help, so that's my second query, .NET. So I now have some name servers for .NET and their IP addresses. Thank you, glue. And I go third query. I'm after a name that's actually potaroo.net. So I ask any of the name servers for potaroo.net. [George: yeah] And we're now entering into my territory. I've got two name servers, and they're both again in bailiwick. [George: yeah] Don't you love that term, bailiwick? George Michaelson 9:19 I love that word, bailiwick. It makes me think of a beef eater at the Tower of London holding a halberd and saying, "You are in my bailiwick. Geoff Huston 9:28 It's a wonderful word that reflects the history of England. bailiwick. Bailey is a French word, and it actually comes from ribs and law and justice. It talks about the extent of power. Bailey, French word Norman invasion. Thank you, Normans. The Normans brought over a whole bunch of words about prisons, police, and justice. They must have been a fun-loving crowd when they invaded England. George Michaelson 9:49 Well, and beef. Geoff Huston 9:50 Yeah, we're going to lock all you up, you bastards, you Saxons. We hate you. And Wic, Wic is a wonderful word. It's a Viking word meaning village. George Michaelson 9:58 The Internet is a village. Geoff Huston 10:00 It's a classic term, and it actually got inherited into the DNS from 18th century Americans, where the Americans wanted to show how erudite they were, so they started picking up old medieval English words and using them again. And someone in the DNS, and I'm going to point the finger of accusation at Paul Vixie and go, Paul. I blame you. Probably wrong. You revived this and sort of resurrected the term Baliwick for the DNS. The rest of us haven't heard this word ever. It's a wonderful word. George Michaelson 10:29 Well, if it wasn't Paul Vixie, it might have been Paul Mockapetris. So I think you're safe accusing somebody called Paul. Geoff Huston 10:36 Oh, accuse Paul. Yeah, Paul. It's all your fault. So anyway, I'm onto query three. I'm still resolving potaroo.net slowly. George Michaelson 10:45 There's a moment here. A priory to use a bit of French. You don't actually know if potaroo is a label directly in .net or if you're going to be told I don't know. You have to

ask somebody else, Geoff Huston 11:00 George. I'm a very up-to-date, fashionable queryer. I follow all the latest RFCs. If I was a bearded old timer, crusty from the 1990s, I would actually ask the root servers, not for the name servers of .NET. Nothing so silly. I would ask one of these root name servers, "Tell me the IP address, the A record for www.potaroo. net, and the root server would come back going in a gently chiding fashion. No, I don't know that, Geoff. But you seem to want to know my neighbor. The name servers for .net. Go ask them. [George: Right] Oh, referral answer. Hi, .net. Tell me about dub dub dub. potaroo.net. Geoff, calm down. I'm a .net server. I can't answer that, but I can tell you. And here's a referral for potaroo.net. Go ask there. But we stopped doing that because it leaks information like a sieve. George Michaelson 11:55 Oh well, a whole host of people have just learned that you, Geoff, are looking for this label. Unassociated parties have learned a lot about you and your query. Geoff Huston 12:05 My dot very dot secret dot name dot I don't want other people.to know I'm dot asking about dot yes that name George Michaelson 12:12 yeah a lot of people know Geoff Huston 12:14 the current fashion and most resolvers are now there and it's you know up around well over half of users query minimization only ask what's relevant at the point where you're asking. So when I say you ask the root for the name server is the root, and you ask a net name server for you know the next one. Now you're deliberately only minimally exposing information. That's good. So now I've asked for potaroo.net. George Michaelson 12:37 So to keep in the background the idea, there's a conversation about caching here. Having learnt everything about the route and having learnt the things you need to know in future to ask about .NET, you have immediately popped them into your local cache. Geoff Huston 12:54 My resolver will never ask that question again for the time to live, the cache time of those records. So I'm never going to ask it again for quite some time, hours, [George: yeah] sometimes even days, depending on what those GTLs have been set. George Michaelson 13:08 Great, keep that thought alive. Keep on down the line, Geoff. You haven't got to the end point yet. Geoff Huston 13:13 Well, I've got glue records. I've done my third query: is the the name service for Potaroo that says, "Go ask these people. I go, aha! Here's the name dub dub dub dot potaroo. Tell me the answer, oh name server for Potaroo, and it goes. Here's an IP address. You know, have fun, fill your boots, do what you need. Four queries, three labels, four queries, standard stuff. Yep. George Michaelson 13:33 But Geoff teams.microsoft.com. Geoff Huston 13:36 Ah, let's go to Teams. George Michaelson

13:38 Three labels and dub dub dub.potaroo.net. Yeah, Geoff Huston 13:42 why isn't it four queries? Why is it 247? George Michaelson 13:45 Yeah, this is not good. Geoff Huston 13:47 Oh, teams.microsoft.com is not actually teams.microsoft.com. It's an alias. There is a weird redirection pointer in the DNS called a Cname, canonical name, and the DNS says, "Actually, you didn't want that name". What do you mean? I didn't. You didn't. This is not what you're looking for. With a wave of the hand. George Michaelson 14:07 This is not the name you're looking for. Geoff Huston 14:09 This is not the name you're looking for. You're looking for teams.office. com. Oh, well, okay. Teams.office. com. Then I have some very sad news for you. This too is not the name you are really looking for. What do you mean? Oh, you happen to be in this part of the world looking for this kind of name. Oh, so what kind of name am I looking for? TMC-g2 dot tm four.office.com. Sounds silly. George Michaelson 14:35 I'm guessing that's also a C name, isn't it? Yeah, there's Geoff Huston 14:38 Another C name coming. Teams-office-com dot s-triple 05 dot dual s-msh.net. Oh, surely, to God, that's the right name. Surely, no, there's another Cname. George Michaelson 14:52 Can we just come back a moment because you started looking for a thing in.com, which meant from cold start state you had to get the ns. for Dot com, and you were then looking for Microsoft, which absolutely exists, and you will have been told go and ask Microsoft. You hit Microsoft, you're now at three queries, and you enter this living hell of aliases, and one of those aliases says no, no, you need to go ask . net, which means you have to go back to the root and say oh root where's .net? Geoff Huston 15:23 It's worse than you think. It's way worse than you think. You see, teams.office.com. That's an interesting name, and that's the second name in the Cname chain. Don't forget, every time you find a Cname, you throw everything you've learned into the cache and start again. Now you've got a cache, but each time the whole query resolution process starts with the new name. And of course, with Office. com, you're going to ask the .com servers for the name servers of Office. com. Tell me all the name servers for Office. com. Ooh, ns one hyphen o5 dot azure dns.com. Oh, that's in bailiwick. That's great. Ah, there's more ns two hyphen o5 dot azure hyphen dns.net. Damn, I've got to go and ask.net. George Michaelson 16:13 That's not in bailiwick. Geoff Huston 16:14 It's not ns three hyphen o5 dot azure hyphen dns.org. [George: what?] Why do you punish me about this? I'm not finished. There's a fourth name server for office. com ns four hyphen o five.as your hyphen dns dot info. Jaw drops. George Michaelson 16:31 That has to be a deliberate decision. Four discretely different top level domains are the NSes behind this service. Geoff Huston 16:40 I think I see inside their brains when they were working this out. I want resilience. What if the .com servers aren't working? And and normally the answer would be at this point abandoned all hope. The Internet is dead. Go out and play. No, we will have a backup in .net. What if they're both down? We will have a backup in .org. What if all three are down? Well, we will have a backup in .info. Someone's going to answer out of all of this. Well, possibly. George Michaelson 17:05 So, their risk management belief was that four discretely different top level domains got out of the if but maybe, and I'm looking at that going four discretely different top level domains costs a cold starter four bootstrapping queries on the route, and you then intruded on top of that alias transiting. Geoff Huston 17:29 Let's again look at these name servers for Office. com. By strict definition, when I ask for the name servers of office.com. I will get a list of these names, and I will get their IP addresses as additional glue records, George Michaelson 17:44 just like you did when you were bootstrapping at the root for .com. Geoff Huston 17:48 Should I use them? Ah, well, for the one called ns105.azuredns.com, I've kind of got no choice. It's in bailiwick. It's the domain I want to go to. I kind of have to accept the glue, okay. But what if I get glue for .NET? No, I can't use that. Why not? It's free, or it might be a lie. [George: right] I have to resolve it from scratch. George Michaelson 18:14 This is a moment which has been exploited by hackers in the past. They're functionally doing a thing that I think is called poisoning the DNS because Geoff Huston 18:22 poisoning the cache that I deliberately give you the answer that you want, but I add these additional IP addresses of name servers that are mine, and I try and get you, Poor gullible you to believe them and accept them on face value. And the real thing is, you should only ever take the in bailiwick. Still love that word. The in bailiwick addresses because you've got no choice. You're over a barrel. But all the others don't use them. You could be being misled. What does that mean? George Michaelson 18:52 You've got to find the address. Geoff Huston 18:54 I've got to ask them. I can't accept helpful hints. So I'm now generating a lot of queries, but let's go one step further because there's this whole thing about who is authoritative for the name service of a delegated zone. Ooh, interestingly enough, if you go through the 300 or so RFCs that define the DNS, you will find that the parent, the one you've just asked, is not the authoritative answer point. So when you ask a server in .NET, tell me the name servers

for potaroo.net and it says, "Well, NS1 and NS2. The answer is, "Well, that's good, but is it right?" And the answer is, "Well, you're asking the wrong person, dude". Just for a second, suspend disbelief and ask one of those name servers, i.e. the child, for the name servers of this name. Hi, ns1.potaroo. Net. Tell me the name servers of potaroo.net. and you will find that that answer is authoritative. George Michaelson 19:52 It might not include the thing you just asked in that answer. It could give you anything as an answer. Geoff Huston 19:58 Whenever you get replicated in. In the DNS, you have to live with the possibility that might not be replicated. It might be different. George Michaelson 20:04 Yeah, we touched a little bit on this when we discussed the DELEG draft on this, and the idea of how to establish authoritatively what's going on. Geoff Huston 20:13 And the issue is, if you're appropriately paranoid, you must always believe the child, not the parent. But that also means you've got to query the child more queries. George Michaelson 20:21 Another query. [Geoff: Another]. Is there no end to the number of queries? I am beginning to get a sense why the number got so large, Geoff. Geoff Huston 20:31 We're building up to 247. We're building this up really quickly. Part of it is: Do you want to be fast, or do you want to be thorough? Do you want to follow every last buy way, or you just want to get to the answer, and that's a really interesting question. Should I resolve the name of every name server of Office. com, all four of them, but I'm only going to use one of them? Should I be assiduous and thorough? Should I be the right version of obsessive compulsive that matches the impulses of people who code, or should I just go? Damn, milliseconds are on the line here. I'll have one with the first answer. George Michaelson 21:05 Kind of want to divert just a moment. I like simple, and there's a part of me that's sitting here going, the Internet police, the ones who don't exist, probably should be hitting at Microsoft saying Cname chains are a really bad idea, and the number of NSes you put behind a domain-it's not your belief that it needs to be four independent top-level domains. You've gone too far, and I'm kind of thinking their rebuttal would be: a) you're not the Internet police, and b) you're not the boss of me, and c) we actually are using this to do some things because when you do that Cname lookup, we're in the position of using a CDN and we can give you a different answer to the guy over there. Geoff Huston 21:05 Ah, you see, that's the thing. You see, I actually run a name server for potaroo.net. and that's me being a crusty old greybeard because that's not the thing you do. The thing you do is go to anyone who says, "I'll host your web server for

you", and you say, "Well host it". Now, in the dim dark days, they'd say, "Well, fine, let me run your entire DNS zone, potaroo.net". And I go, "No, no, no, no, no, just the name server, dude", George Michaelson 21:05 mate. That's a lot of asset I've handed over to you. Geoff Huston 21:05 I'm not going to do that. And so the answer was: How do I lift? How do I forklift one name out of one zone and drop it into the control of somebody else? [George: yeah] And the answer is Cnames, because what it's really saying is that name dub dub dub.potaroo.net. That's not a real name. That's just a bit of lipstick on the pig. The pig is over there in a different sty, and it's got a different name, and it might be an Akamai name or a Cloudflare name or whatever else, or an Azure name. But the target of the Cname is where that forklift is happening. Okay. Yeah. So let's go one step further because I can. All Akamai names tend to be 2 Cnames deep. Why two? One to lift it out of its home. Hi, that wasn't the name you thought. It's actually under control of Academy as a host name. Two. Oh, where are you, George? You're in Australia. I have a server in Brisbane. I'm going to Cname the generic name to a server really close to you. [George: yeah] So that when you go to this resource, the speed's going to rock. George Michaelson 21:05 Right. Geoff Huston 21:05 So I do the two to one to isolate, two to steer. Now Microsoft are doing five because George Michaelson 21:05 because the Internet police haven't knocked on their door, Geoff Huston 21:05 and you kind of go, "couldn't you do better". And I think the answer was, "Well, we could have, but we didn't [George: so] yeah, George Michaelson 21:16 you have given me a nice feeling about cold start involving a lot more query than I possibly could have imagined, and I feel like there are two stories in this that we probably could have some fun talking about. And the first one is this word "cache, and so I get a thing from you, and I decide to hang on to it for the length of time you tell me you think you're going to keep it before you might change it. Geoff Huston 24:09 Ah, it's not a rule; it's only a guideline. Only George Michaelson 24:12 guideline, and I'm in this beautiful position that I've built up a model of all the things I've ever been told and the age they're in. And the thing is, that model means I might not be asking all these questions. I had to under cold start, but the next time I go and look at guardian.net, I don't have to do this dance over.net. Yeah, exactly. The second time you go to teams.microsoft.com or one of your app starts, it comes back in an instant because it's gone through that rigmarole and loaded the cache with basically the answer. It's gone through all the intermediate stages of discovering that answer.

And interestingly, when you ask a second time, it doesn't try and rediscover. It says to the recursive resolver, "Hey, have you got teams. Microsoft. Com's IP address?" Yeah, here it is. Do." Done. I'd look up. If we think about the investment that people like Cloudflare and Google have in public DNS, at the cost of them knowing everything I do, they're probably not in a cold start world very often. I mean, thermonuclear devices have not gone off in Western America, so the service that operates out of Mountain View that feeds their DNS had a cold start when they turned it on, but they've been running continuously now for a decade, and it's not cold start. They have a lot of data. Geoff Huston 25:33 What you've got to worry about is an engineering compromise between two things. Nothing is permanent. I might want to change stuff so that if I say to you, "Here's an answer, George, cache it for a year", then if I want to change it, I've got to be awful patient. And I'll tell you a tiny story here. When Slack were signing their Slack.com domain name with DNSSEC, they managed not they people others managed to shoot their foot off a bit and get it wrong. And what hit the cache was that there is no names in Slack. com. George Michaelson 26:10 Oh wow! Geoff Huston 26:11 And that was cached for the TTL that they had suggested the time to live, the cache cache lifetime, which in Slack's case happened to be 24 hours. So here's this enormously popular service, forcibly offline because DNS caching for 24 hours. George Michaelson 26:27 But you did just say it's only a suggestion; it's not an instruction. Geoff Huston 26:31 Yes, and some folk make it longer, and some folk stupidly make it shorter. But it is only a suggestion, and certainly when I say to you, "Here's an answer. Remember it for three seconds." Most recursive resolves would say, "Geoff, Geoff, get serious. If I'm gonna going to process this answer, three's too short. Minimum of pick a number: 15, 20 60, seconds. George Michaelson 26:55 Yeah, but if you have a giant foot gun and you and the person publishing decide, yeah, it's a day. Then your harm of mispublishing has 24 hours of duration in the distributed world. Geoff Huston 27:07 Right, but if I'm looking at teams.microsoft. com and I'm looking at Cold Start, if I put a cache lifetime of how long, one second, then once it expires, I'm back into the hell of rediscovery. George Michaelson 27:20 Right, because cache ejection doesn't make you go and re query it. You wait till somebody needs it, at which point you don't have the cache state. Geoff Huston 27:28 Well, people keep on tinkering, and some implementations will query for you just in case to be sure, to be sure. But most of the time, no. The next person who comes along will find basically an empty cache and have to do it all over again. Now it won't be a full top-down discovery, but certainly bits of this will have disappeared. You've got to discover a lot more than you thought. So there is this issue about what are you optimizing for: performance or resilience? [George: yeah] and that's the real trade-off when you're designing a DNS system. I can put in 13 name servers. Look at the root, each with a v4 and a V6 address. That's 26 points. Is anyone really going to hang around while your recursive resolver unsuccessfully makes 26 queries, only to learn you pulled the Ethernet wire out of the back of your laptop? No. So you know you don't need to go full overboard. You know all guns blazing on resilience. You don't need to over equip, and most folks think four is a good number for name servers. Oddly enough, odd numbers are evil. Six is a good number. Five is an evil number. Don't know why, but they never go the full 13 or something outrageous like that. That just really doesn't work. George Michaelson 28:40 Is there a strong sense whether it should all be in bailiwick or there should be some out of bailiwick, or is there no guidance in this? Geoff Huston 28:47 There's a whole bunch of guidance saying do not accept gratuitous offers of glue authority records for names that are outside the bailiwick of the name server. Do not accept them because people will lie and will poison. So, if you're after performance in bailiwick, if you're after resilience, keep a few to one side. The don't use all out of bailiwick because that means every time you got a problem. Probably unwise to use all in bailiwick for the same kind of issue around poison. But in bailiwick, certainly better than out of bailiwick on the whole. George Michaelson 29:20 Yeah, and we have no better mechanism than 2 Cnames to provide hand off to a third party and localization to a specific location. Geoff Huston 29:30 Well, 2 3 4, depends on what you're trying to achieve administratively, but it is how do you lift names out of one space and get them to another? There was one other technique that I thought was truly, truly, truly awesomely weird, which was weird games in the DNS. We called it Napter, N-A-P-T-R, and it actually said something entirely different. A C name says, "Don't look at me, look at this name, go and resolve that. A NAPTR is. Brain explodes. A regular expression, George Michaelson 30:03 A Turing complete program. Geoff Huston 30:05 You shouldn't have been drinking while while I said that. It is honestly apply this transform for the name you're looking at to get a new name and go and look for that instead. So if I say that star.com becomes star.foo.baz.com, then no matter what I ask for in.com, the regular expression transform will match it to a new name generically by using regular

expressions. George Michaelson 30:30 Wow, that's powerful. That's scary. That's complicated. Geoff Huston 30:31 Well, it never took off. It never took off. It was very scary, and it was so scary that they invented a new one called Snapter. What was Snapter? That was the simple Napter because the power of regular expressions mean. You know that game, The Towers of Hanoi. George Michaelson 30:50 Oh yeah, Geoff Huston 30:51 you could play that in the DNS with Napter Records. [George: yeah] Geoff Huston 30:53 because the awesome power of regular expression. So, so there's a number of techniques of how do you move something. You see, the problem is that URLs have this sort of duality about what it is and where it is. [George: Yeah]. And the conventional URL, as distinct from a URI, a URL is actually a locator, and it's not an identity. But we all use it as identities. So when I go and change the organization of my web page and put things in different places and so on? How do I keep all the old URLs, or do I not change the organization of my web page because dot dot dot? George Michaelson 31:33 Right, I'm a strong believer in the persistence of URLs because the behavior, and if we invoke the caching word, there is a massive amount of offline cached state held in everybody's browser tabs and browser history, and if you randomly decide you don't want to support the name, but I've gone into history search and find the critical document under the old name, I'm in a world of hurt. Geoff Huston 31:56 You are, but the problem is that when people you know move to new servers, go to a different commercial provider, etc. etc. They actually want to reorganize their data. They actually want to reorganize things, but they've got to do so with the same URLs, which you actually can't use as a true locator. So you need translation systems in the DNS. George Michaelson 32:18 Yeah, if we walked into URIs as the default instance of how we do this? We'd have the necessary level of indirection that Cnames is providing, and we might be able to stave URIs as persistent things. Geoff Huston 32:32 Oh, everyone knows that academics live in ivory towers. And George Michaelson 32:35 thank you for calling my apartment an ivory tower. Does that mean it's worth more? Geoff Huston 32:39 And no one listens to them now. It made sense, but it gelled with the people at the time. Locators sort of, "Well, right, done there. On to the next problem. [George: Yeah], you know, I've spent two minutes thinking about this. On to the next. So yeah, you're right. George Michaelson 32:52 Last time we spoke, there was a component of this that was the kick off was the tragedy of the commons, and I think there's a flavor of risk of tragedy in the commons in the bootstrap situation you've just described. Here we have something that is truly critical for the global community that use Microsoft product. And if you walk into a world where there was a local power outage, you have a 250 query burden in the global DNS framework for you to learn how to get to Microsoft. That doesn't sound to me like a good situation. Geoff Huston 33:25 Oh, optimizing for disasters is never fun, and I don't think that's worthwhile. But what is more realistic is that bits of the Internet may not be there at the exact time you need it. So when we're talking about resilience, you actually do want to spread your assets, but for performance you want to think hard. So in domain name server names, out of domain, prefer in domain. Spread them across many top levels versus don't don't spread them. Whatever happened with Office. com and its use of com Info. Net and org, that's a bad idea. It doesn't help resilience. A Cname chain that's five long is three too long. It really is. I can understand one. That's brilliant. I can understand the administrative convenience of two: one to lift and a variable one to translate. [George: Yeah]. I can't really understand three. Don't use them. They're expensive. And as for the number of name servers of a domain, two, if they're any cast, in other words, they're implemented everywhere, but it's two distinct names with each of them have a v4 and a V6 address. That's four addresses. It's good. Three is kind of George Michaelson 34:31 odd numbers are evil. Geoff Huston 34:33 Odd numbers are evil, and what disaster are you thinking about? And even when you get to four, it's kind of if you needed four, each with two addresses, something's happening so badly that it's not worth optimizing for. Don't bother. Two is a good answer. So resilience and diversity, you've got to think about how and what level you want to do it. [George: yeah] Geoff Huston 34:51 trying to do it all in the DNS is pushing the DNS well beyond it. So let's have a look at a couple more examples, and this one I kind of like. Google. It has four name servers, which I would argue is two too many, and they're all anycast. They're everywhere. Each of those four have one IPv4 address and one IPv6 address. George Michaelson 35:10 So eight different address instances can supply Google, Geoff Huston 35:14 and they're all in Google.com. George Michaelson 35:16 Is that resilient? Geoff Huston 35:17 So if I ask for the name servers of Google.com, I get back an answer that goes: Here are four names, and here are eight IP addresses. Go for it. Nothing more to ask. George Michaelson 35:28 But they all reside within Google's own assets of BGP routing, global connectivity, and naming. Geoff Huston 35:35 No, no, hang on, hang on. They might all reside within Google's assets, but they're in different routed prefixes,

but in both four and six [George: right] They don't fate share. We're learning, George. We are getting better at this, [George: right] And so that's not a bad idea. Let's go over to Cloudflare. net. George Michaelson 35:52 There's kind of an ancillary behavior that means Google's operational practices now has to have a black book requirement: you never ever play with the routing plane in all of these IP spaces at the same time. If you have to do something, Geoff Huston 36:07 that'd be shooting your own foot off. That would be a very bad idea. Yeah, George Michaelson 36:09 they're obviously moving risk into different aspects of reliability engineering for provision of what it is to be Google. Geoff Huston 36:16 Well, and it's a minimal piece of engineering, but I think it gives resilience and performance. Let's go to Cloudflare. Cloudflare. net. Five name servers. That's an evil number. George Michaelson 36:27 Odd number. Geoff Huston 36:27 But okay, five. The odd number. I'm suspicious too. They're inventively numbered NS1, NS2, all up to NS5. Are they in bailiwick? Yes, they're all in Cloudflare. net. So far, so good. Each name has three IPV4 and three IPV6 addresses. Yeah, of course you can do that. It's legal because it's not the names that matter. It is the addresses. But now those five, [George: yeah] right, have become 15 times two, which is 30. George Michaelson 36:57 That's a lot of Geoff Huston 36:58 IP addresses. George Michaelson 36:59 I've tried A. I've tried B, I've tried C, I tried D. Geoff Huston 37:02 You're going to give up. Resolvers aren't that persistent, and you kind of go. I'm not sure what you were thinking, Cloudflare. You've kind of gone overboard, and there's a whole bunch of stuff that no one is ever going to do. It's good that you're in bailiwick. Full points, putting three V4 and three V6 against every name server. [George: yeah] I think you had an enthusiastic afternoon. I think that was actually a silly thought. You should have gone to sleep instead. George Michaelson 37:26 I would have said do two and have three of each if you like, but do two and have one of each probably would have been as good. Geoff Huston 37:33 Direct correlation between names and addresses is for the DNS simple. Stacking IP addresses up against a single name is I don't know. I think the best adjective I can use is I think it's distasteful. It's not illegal. It's not wrong. It just it offends me. And you kind of go why instead of oh you're hanging too much off a single name. Don't do that. [George: yeah] keep it simple, one to one. George Michaelson 37:57 So there are different strategies out there. Geoff Huston 37:59 Oh, there are very different strategies because what you're trying to do, and this gets back to the whole thing about what Microsoft

was trying to achieve and why, and even the whole thing about caching, is that you're trying to optimize both performance and resilience. Now, caching will give you most of the performance you need, except when the cache expires, and if you're using short time to live because I want to change things on the fly. So if you have very long time to live in the cache, everything is difficult to change. So if you want stuff that's malleable and efficient, you've got to think about what information you're loading into the DNS and where and how, and try and optimize the DNS performance. Maybe there's an AI tool that will help you. Maybe there isn't. I suspect you will find hallucinations more easily than you're going to find truth. But there's an awful good stuff to read about this topic, and some good presentations to watch about engineering both resilience and performance in the DNS. George Michaelson 38:56 So Ondřej's presentation covered off on a lot of this. Geoff Huston 38:59 No, it just was a mere starting point as to there are hidden gotchas in the DNS. His story was actually, you know, bind and 247 queries. [George: Yeah], we started trimming out the fat. We started saying, "I'm not going to ask the child. I'm just going to rely on the parent. I'm not going to resolve all the name seven names. I'm only going to use one. I'm going to take the short path to the answer and only go down the byways and you know diversions if the shortest path didn't work. I'm going to code for optimal performance, [George: yeah], and only exercise resilience if the shortest path didn't work. So his story was about how engines negotiate, but the DNS is two parts. It's the engines that go through the minefield that the folk who will actually configure their names set for those engines. And the answer is, don't get too clever when you're setting up this sort of field of coordination and configuration for your names. Keep it simple. George Michaelson 39:56 But you've written up the story that you found as a trigger point in this, the aspect of behavior here that is what we've just been looking at. Geoff Huston 40:04 I've written up a story about what I think it leads to. This is why there are a couple of really good, you know, suggestions about configuring your names. Use in bailiwick names. Avoid spreading out to multiple top levels. Keep your Cnames short, and quite frankly, don't have a whole heap of name server names or addresses. You're just wasting everyone's time, including your own. Four guidelines. If you do that, you won't get into too much trouble. There are so many ways you can handle yourself. Try not to use this set. George Michaelson 40:33 Going to emerge as a formalism in IETF, or you think this one will just stay in soft operational practice? Geoff Huston 40:39 Oh, a lot of the DNS is folklore, and I think that's the way it should be. I think the mania for trying to document everything and having it change in two weeks has led to 2000. How much we up to now? RFC 10,000. Coming soon. No one's read them all. We got to the point where they've actually ceased to be a useful reference. And in some ways, the folklore, the common knowledge in the DNS, is more topical, more up to date, and more accurate. Oh, George Michaelson 41:03 fightin talk. Geoff Huston 41:05 Whoa. Well, you know, maybe that, or I'm just sick and tired of writing RFCs, so I've stopped one or the other. George Michaelson 41:10 That's been fascinating, Geoff. Thank you. Geoff Huston 41:13 It's been fun, and I hope, dear listener, you're kept along for the ride. Thanks indeed. George Michaelson 41:18 If you've got a story or research to share here on Ping. Why not get in contact by email to ping@apnic.net or via the APNIC social media channels? Also, remember the measurement at APNIC. net mailing list on Orbit is there to discuss and share relevant collaborative opportunities, grants and funding opportunities, jobs and graduate placings, or to seek feedback from the community on your own measurement projects, be sure to check out the APNIC website for all your resource and community needs. Until next time.