Full Transcript for Does AI Replace Corpus Linguistics? | Randy Barnett, Jack Kieffaber & James Heilpern

joining us. 00:02:36.639 --> 00:02:41.360 And with that note, I'm going to introduce our vice president of debates, Louis Fong. 00:02:48.319 --> 00:02:51.520 Good evening, everyone, and thank you for coming to tonight's name. 00:02:51.840 --> 00:03:02.800 Our moderator is Professor Professor Randy Barnett, who is a Patrick Oton professor of constitutional law here at Georgetown University Law Center, and the faculty director at the Center of the Constitution. 00:03:03.039 --> 00:03:07.759 He tried many felony cases as prosecutor at the Cook County State Southern Soffice of Chicago. 00:03:08.000 --> 00:03:17.520 He's received a good high fellowship in constitutional studies and the Radio Thomas, and has authored 12 books and more than 100 articles in the news, as well as numerous authors. 00:03:18.159 --> 00:03:21.439 We also have Jackieber joining us, who is the founder of RA. 00:03:22.080 --> 00:03:36.240 He served as an adjunct professor at the Catholic University of America at the Columbus School of Law, where he teaches a course on AI and served as a law learner to judge anymore on the district board at the District of Columbia, and to judge Newson on the Court of Appeals and Library. 00:03:37.120 --> 00:03:45.919 We also have James Elmer joining us, who specializes in complex litigation and corpus linguistics in Germany, representing clients at all stages of litigation. 00:03:46.159 --> 00:03:55.120 He is a leading figure in law and corpus linguistics, having authored some of the first race in the country to employ corpus linguistics and offers training sessions as well. 00:03:55.360 --> 00:03:57.759 He clerked for the Honorable Richard Cleveland on the U.S. 00:03:57.840 --> 00:04:09.280 District Court in the Eastern District of Michigan, and he didn't have it with that one to say thank you for the introduction. 00:04:09.439 --> 00:04:14.479 Thank you for introducing the other debaters too, so now I don't have to do it, so that's that's great. 00:04:20.959 --> 00:04:24.319 I've in the past been a big supporter of corpus linguistics. 00:04:24.399 --> 00:04:53.759 I I have been characterized by some of its developers as one of the earliest practitioners of corpus linguistics, because I had an article published in the University of Chicago Law School Law Review in 2001, in which I surveyed the use of all the operative terms of the Commerce Clause, every use of the operative terms of the Commerce Clause in the Constitutional Convention, in the Federalist Papers, and in the ratification debates. 00:04:53.839 --> 00:05:22.000 And then I had a sequel article in which I had located a database of all the issues, all the issues of the Pennsylvania Gazette published between around 1710 to 1805, in which I surveyed every use of the word commerce and the rest of the operative terms in the Commerce Clause, of which there were about 1,500 mentions of the word commerce in that database, as a means of trying to identify what was the original public meaning of these words. 00:05:22.560 --> 00:05:31.839 And came to the same con came to the conclusion that Justice Thomas was largely correct in his concurring opinion in the Lopez cases, which is what I was trying to decide. 00:05:32.000 --> 00:05:37.759 I was trying to see if I could defend Justice Thomas's analysis in Lopez, and I provided that defense. 00:05:38.079 --> 00:05:47.279 Since then, corpus linguistics, of course, has greatly expanded the corpus beyond that which I discovered, which also had very, very rudimentary search uh capabilities. 00:05:47.360 --> 00:05:49.920 Uh, so it was really kind of annoying to try to use. 00:05:50.000 --> 00:05:54.800 Uh, but that was the first effort to try to objectify the search for objective meaning. 00:05:54.959 --> 00:06:11.040 Uh since then, um, I have been a supporter of corpus linguistics, and James has uh lectured uh uh uh one summer after another for the originalism summer seminar in order to bring students up to speed on how corpus linguistics works, since it's such an essential tool. 00:06:11.279 --> 00:06:30.560 But I have developed some doubts about it, um, primarily about whether it's fair to conclude that because a particular usage was very, very common, perhaps by far the majority use of a particular word, that that means it was what was meant, uh, the public meaning of the Constitution in context. 00:06:30.639 --> 00:06:43.920 Uh, when in fact um for for example, the right to bear arms might have had a primarily, might have been primarily a military connotation, but that doesn't mean it had an exclusively military connotation in the Second Amendment itself. 00:06:44.079 --> 00:06:56.240 And there is, I think, a natural tendency, as some people in the Second Amendment have discovered, to try to equate the dominant usage with the origin with the conventional understanding of the words in the Constitution uh in context. 00:06:56.319 --> 00:06:58.639 So that has actually given me some pause about it. 00:06:58.879 --> 00:07:04.800 Add to that the development of AI, as Sarah mentioned, I'm actually a pretty avid user of AI. 00:07:04.879 --> 00:07:07.279 I'm a I I think it's a very important tool. 00:07:07.360 --> 00:07:11.439 Um, and I think it has enormous potential. 00:07:11.600 --> 00:07:20.160 Um and it could, AI could do to corpus linguistics what AI is currently doing to search, which is basically making search obsolete. 00:07:20.319 --> 00:07:22.639 Um, search like Google, search like Bing. 00:07:22.879 --> 00:07:34.319 Um, because in fact, corpus linguistics could be considered a sophisticated, specialized form of search uh rather than an analysis that the kind that uh or a synthesis of the kind that AI is capable of. 00:07:34.720 --> 00:07:38.800 Having said that, we're all we all know that AI is capable of hallucinating. 00:07:38.959 --> 00:07:45.680 Um it loves to please you by delivering a result that sounds good to you, um, even if it turns out not to be based on fact. 00:07:45.839

--> 00:07:50.319 And there are precautions you can take against that eventuality, but you always have to be wary of it. 00:07:50.399 --> 00:08:04.000 And right now, the use of AI in the practice of law is extraordinarily risky, and people are having their law licenses questioned because they've submitted uh fraudulent uh citations to the courts on the basis of an AI engine, uh large language model. 00:08:04.399 --> 00:08:11.680 So, with that as context, I welcome the opportunity to moderate this debate just because I wanted to hear the debate. 00:08:11.839 --> 00:08:20.720 Uh I I'm I'm not even sure this is going to be a debate as opposed to an informed discussion uh by two experienced users of the technologies that they're going to be discussing. 00:08:20.800 --> 00:08:29.839 And I, for one, have a very open mind about how it can be used, whether AI will supplant corpus linguistics, what it would mean to say that it's supplanting corpus linguistics. 00:08:30.000 --> 00:08:34.080 And so I'm just looking forward to hearing what each of the which each of the speakers say. 00:08:34.559 --> 00:08:41.440 I've been told they're each given seven minutes to do their opening statements, and then we'll have an exchange for how many rounds do is the is the procedure? 00:08:42.240 --> 00:08:47.279 One remote and then Okay. 00:08:47.440 --> 00:08:49.279 So hopefully I'll have some questions. 00:08:49.519 --> 00:09:00.080 And so I guess the affirmative, which is the position that AI is going to uh supplant uh corpus linguistics, um, is going to be Jack. 00:09:00.320 --> 00:09:00.879 So take it away. 00:09:01.200 --> 00:09:01.600 Thanks, Professor. 00:09:01.759 --> 00:09:03.679 It's an honor to be here again with you guys. 00:09:03.840 --> 00:09:05.840 I'm not sure how much of a debate this is going to be. 00:09:06.080 --> 00:09:08.960 At minimum, we are sort of rival CLE providers. 00:09:09.120 --> 00:09:19.519 I've got this business with Brian Garner, who I do believe is still the leader in what they call baby judge schools, but JEI, fast uh approaching judicial uh education institute is one of this. 00:09:20.159 --> 00:09:24.159 But yeah, I mean, we've got this robot today, smarter than anybody to ever live, they're saying. 00:09:24.320 --> 00:09:30.879 It builds your bioweapons, it's firing nukes, and so you get the question that's on everybody's mind uh what's going to happen to corpus linguistics? 00:09:31.120 --> 00:09:32.399 That's the top-level stuff. 00:09:32.480 --> 00:09:34.799 But I mean, it is a more serious question than that. 00:09:34.879 --> 00:09:38.000 And I what I really want to do today is insofar as Mr. 00:09:38.159 --> 00:09:45.759 Hubert wants to incorporate more AI and what he's doing, which sounds like he might want to, I just want to let him know sort of what he's sitting on top of kind of the powder keg. 00:09:45.919 --> 00:09:51.519 You know, when you're selling a methodology in this day and age with AI, I call you a methodologue. 00:09:51.600 --> 00:10:02.879 You're kind of playing for world domination here, because if enough judges latch on to what you're doing the same way they latched on to Scully and Garner, you know, cited over 5,000 times of circuit courts, a hundred SCOTA sites. 00:10:03.039 --> 00:10:06.639 It's different in kind than selling them a book if you sell them a bot. 00:10:06.799 --> 00:10:09.840 You kind of are a legislature at that point. 00:10:10.000 --> 00:10:12.559 Nobody elected you, nobody cast a vote for you. 00:10:12.639 --> 00:10:17.039 You just convinced a bunch of judges to use your product, and suddenly who controls those weights and models? 00:10:17.200 --> 00:10:22.480 And as a backdrop of this conversation, I should let you know, um, Claude and GPT are already in the lead on that. 00:10:22.559 --> 00:10:36.960 You're gonna get judges lazier and lazier, because I know the way all you guys study, less and less reading, more and more querying bots, and if that's kind of the trajectory, the provision's on, these things are gonna have a lot of influence on how laws come out and what they mean. 00:10:37.200 --> 00:10:39.039 But basically, there's two things I want to talk about. 00:10:39.120 --> 00:10:40.960 It's corpus and method. 00:10:41.120 --> 00:10:43.759 Those are kind of the two main things to interpretation. 00:10:43.840 --> 00:10:45.919 I think we're sort of in dialogue on. 00:10:46.080 --> 00:10:49.279 And by corpus, there used to be sort of higher-minded notions of this. 00:10:49.360 --> 00:10:52.559 I think anybody who's done originalism or textualism knows what I mean by corpus. 00:10:52.720 --> 00:10:54.159 It's a big stack of PDFs. 00:10:54.320 --> 00:10:57.120 You work at a law firm, they say, we want you to do a textual analysis. 00:10:57.279 --> 00:11:06.320 You go download as many PDFs as you possibly can, and you're crossing your fingers, give me a John Adams, give me a Thomas Jefferson, you're control Fing in the old days for firearm. 00:11:06.639 --> 00:11:10.799 Sometimes there's a fire hyphen arm, but you didn't control F for the hyphen, so you missed that one. 00:11:10.879 --> 00:11:14.879 And the better lawyer, the better law student, he types the hyphen in because he's really careful. 00:11:15.120 --> 00:11:16.159 That was the project. 00:11:16.320 --> 00:11:25.840 And in sort of the pre-Mortzen years, the pre-Steve Moritzon, the guy who invented corpus linguistics, you were kind of in the Wild West where there was no fixed corpus and no fixed method. 00:11:26.000 --> 00:11:31.840 I'm more of a John Adams guy in the sense that Adams said at the founding there should be an enumerated dictionary. 00:11:32.000 --> 00:11:32.559 He was big on. 00:11:32.639 --> 00:11:35.279 This is kind of a Benthamite, Jeremy Bentham type point. 00:11:35.600 --> 00:11:41.679 If you're going to have a more formalist legal system, what the words mean actually needs to be legislated. 00:11:41.759 --> 00:11:48.559 This isn't something we

should sort of leave up to, you know, the beauty and intricacies of human interaction, human language. 00:11:48.639 --> 00:11:50.320 And I do very much agree with that. 00:11:50.399 --> 00:11:58.960 I think there should be legal language, and that's why the most controversial thing that might set us off on a debate, I don't really care which method you use. 00:11:59.039 --> 00:12:00.480 I don't even really care what the words mean. 00:12:00.559 --> 00:12:02.639 What I care is that we decide what they mean. 00:12:02.799 --> 00:12:08.320 We do so consistently, such that you can give Congress sort of a building block language to write statutes in. 00:12:08.480 --> 00:12:14.559 My answer to no vehicles in the park is that that's a poorly written statute, and you need a better word than vehicles. 00:12:14.639 --> 00:12:20.399 And I think corpus linguists have us somewhere down the road to making that language. 00:12:20.559 --> 00:12:22.799 I'll give you a hint, the language is binary code. 00:12:23.039 --> 00:12:24.720 By the end of this, you're going to see kind of what I mean. 00:12:24.879 --> 00:12:27.120 But pre-Mortzin, there was no fixed corpus, right? 00:12:27.200 --> 00:12:28.639 Guys are pulling dictionaries off the shelf. 00:12:28.799 --> 00:12:32.480 Sometimes they'd even pull out dictionaries from the era in which the statute was drafted. 00:12:32.639 --> 00:12:38.320 And of course, the dictionary writer, Webster, who have you, he's already done his own corpus collection and methodology. 00:12:38.480 --> 00:12:43.519 Garner owns all these classic old school 600, 700 dictionaries, 700s dictionaries. 00:12:43.679 --> 00:12:45.039 That's what lexiconers do. 00:12:45.200 --> 00:12:49.919 So you're deferring to their corpus and method, and then every judge had his own method for interpreting that document. 00:12:50.080 --> 00:12:52.159 It was kind of the wild west. 00:12:52.320 --> 00:12:54.159 And this is the Steve Moritzon revelation. 00:12:54.240 --> 00:12:59.440 He comes along with corpus linguistics and says, well, let's at least decide what corpus we're working from. 00:12:59.600 --> 00:13:02.000 Let's fix some documents. 00:13:02.480 --> 00:13:05.360 And right now, you know, there's an entity, I believe it's BYU. 00:13:05.519 --> 00:13:07.120 BYU fixes the corpus. 00:13:07.519 --> 00:13:09.279 Here are the PDFs you're using. 00:13:09.519 --> 00:13:11.759 I do think that's a pretty good start on this. 00:13:11.840 --> 00:13:16.240 If you want to go towards the Adams position of, well, how would you even start to enumerate a dictionary? 00:13:16.399 --> 00:13:22.799 Well, let's start by saying these are the official PDFs of the United States, or perhaps more realistically, of my contract. 00:13:22.960 --> 00:13:30.320 I also built talking contracts as my day job, a contract you can ask its own AI built-in AI judge for ex ante opinions. 00:13:30.480 --> 00:13:32.720 You'll see how this is kind of conducive to that. 00:13:32.960 --> 00:13:36.720 So that's what we have right now, and I think Professor Barnett teased up very well. 00:13:36.799 --> 00:13:44.639 It's all about how do you query this thing, how do you get to individual instances, but I want to make note of what the corpus linguist actually does when he has the instances at his fingertips. 00:13:44.799 --> 00:13:48.159 He gets 200 of them, let's say, and he lines them all up. 00:13:48.240 --> 00:13:51.519 And what I've heard so far is kind of in keeping with what I've heard Mr. 00:13:51.679 --> 00:13:56.240 Halpern and uh Judge Thomas Lee, who's also a big adherent of this stuff, talk about. 00:13:56.799 --> 00:13:58.639 You need to be able to get those 200 faster. 00:13:58.799 --> 00:14:00.399 But what do you do when you have the 200? 00:14:00.559 --> 00:14:08.399 Well, you take each individual one, you kind of bucket like two conflicting interpretations of the word that you're doing in that particular case, and you go like this. 00:14:08.559 --> 00:14:10.320 You go zero, one, zero, one. 00:14:10.559 --> 00:14:11.919 You literally, they call it coding. 00:14:12.000 --> 00:14:14.559 You code them one box or the other. 00:14:14.799 --> 00:14:25.279 And what with the robot being smarter than everybody and building the bioweapons and the nukes and all that, you kind of got to ask yourself, and so far as is at least to what Justice Lee has put in writing, this is kind of his operative question. 00:14:25.440 --> 00:14:26.559 Could a robot do that? 00:14:26.720 --> 00:14:29.120 Could it do that pretty consistently? 00:14:29.600 --> 00:14:30.960 I kind of think yeah. 00:14:31.120 --> 00:14:34.879 You've got it down to this very sort of already mechanical level. 00:14:35.039 --> 00:14:38.080 Do you need the human to do that last step of bing, bing, bing, bing? 00:14:38.240 --> 00:14:39.519 He might, in theory, mess it up. 00:14:39.600 --> 00:14:41.919 And that's always been my one problem with corpus linguistics. 00:14:42.000 --> 00:14:45.919 At the end of the day, you build this corpus, and they say, I've read a lot of Justice Lee's papers. 00:14:46.080 --> 00:14:47.919 He cites to these big books on linguistics. 00:14:48.000 --> 00:14:49.200 Well, how do you actually do the coding? 00:14:49.279 --> 00:14:50.720 Well, read those, come back. 00:14:50.960 --> 00:14:51.440 Uh-oh. 00:14:51.679 --> 00:14:58.080 I don't know if judges are going to be any more uniform in doing that than they have been in doing originalism and textualism generally. 00:14:58.240 --> 00:15:03.120 But you've got this Lee quote from the article he wrote against uh Judge Newsom, my old boss in Snell. 00:15:03.200 --> 00:15:07.200 And I was there the Snell term, so I have some uh I've got something wrapped up in this. 00:15:07.360 --> 00:15:14.480 He said, some researchers, this is a quote, some researchers should be tempted to ask, could be tempted to ask, AI to analyze the quantitative results of coding. 00:15:14.720 --> 00:15:15.120 Do this.

00:15:15.279 --> 00:15:19.759 And he says, that would be a step too far, too close to robojudging. 00:15:20.000 --> 00:15:36.559 And to that I say, and I'm not even sure if uh James would disagree with this, but if we distilled text into a coding, where you'd go on bing, bing, bang, for individual instances, I don't know, if your job was to make judging more human, I don't know if you would have gone down the entire corpus linguistics road to begin with. 00:15:36.720 --> 00:15:43.039 I think you might have pulled up a little short of that, you know, maybe in the more Vermulean camp of things, if you wanted to add to the humanity. 00:15:43.200 --> 00:15:49.120 If we're down this mechanical road, I might be the rare uh opponent of corpus linguistics who says, well, it's not mechanical enough. 00:15:49.200 --> 00:15:50.720 Usually here it's too mechanical. 00:15:50.879 --> 00:15:54.080 I think there's got to be a third step that fixes the method. 00:15:54.240 --> 00:15:57.120 Right now, the corpus is fixed, the methods with slapping on the breeze. 00:15:57.279 --> 00:15:58.639 This is why I called Brian Garner. 00:15:58.799 --> 00:16:07.840 I was doing some talk at SMU, and I managed to get dinner with him, and after that we went to his house where he has three big libraries, which I like, because you're looking at those libraries and thinking, well, that's a corpus. 00:16:08.080 --> 00:16:09.919 I just when I look at a library, I'm sick in the head. 00:16:10.000 --> 00:16:11.919 I see PDFs everywhere, a lot of PDFs. 00:16:12.080 --> 00:16:14.639 I thought you could print all those out, you could take those, put them in a corpus. 00:16:14.879 --> 00:16:19.279 And then you got to think, well, whose method has kind of been deciding on these things? 00:16:19.519 --> 00:16:26.720 You know, when Kagan says we're all textualists now, um, in a way she's kind of saying we're all doing Scalia and Garner now. 00:16:26.799 --> 00:16:28.720 That was the big move that kind of crystallized this. 00:16:28.799 --> 00:16:39.279 I mean, these guys invented the cannons of construction, they distilled them from the past, but a lot of the names were applied literally by Justice Scalia, in a lot of cases by Brian Garner while writing this book. 00:16:39.519 --> 00:16:42.240 So you've got a tested method that a lot of people are using. 00:16:42.399 --> 00:16:45.759 The question is, could you distill that into a bot? 00:16:45.919 --> 00:16:47.200 I thought, how would I do that? 00:16:47.440 --> 00:16:55.759 The best I come up with is, well, I want to make sort of a starting baseline rapper that's read all of his stuff, and then I want Brian Garner to train the thing. 00:16:55.919 --> 00:16:57.279 I want him to grade outputs from it. 00:16:57.440 --> 00:17:00.000 If you know anything about Brian Garner, he does this for a living. 00:17:00.159 --> 00:17:08.400 He'll go to your law firm, he'll go to your baby judge school, and he will grade outputs from these judges, textual

reviews from his students all the way up to circuit judges, and he's got a lot of notes. 00:17:08.559 --> 00:17:12.480 We read all those notes, and we said, all right, do it again talking directly to this bot. 00:17:12.720 --> 00:17:16.559 So, in a way, we've been sort of enumerating the Garner consciousness. 00:17:16.640 --> 00:17:23.440 And indeed, he's got this follow-up sort of a spiritual successor to reading law coming out, and uh it's not going to be a book. 00:17:23.519 --> 00:17:25.359 It's coming out as a bot. 00:17:25.440 --> 00:17:30.079 We're gonna have five copies, we'll put them on the table, and then we'll whisk them away the second anybody wants to know what's in there. 00:17:30.240 --> 00:17:32.160 You want to know what's in the bot in the book writer. 00:17:32.400 --> 00:17:34.640 You've got to look at the bot. 00:17:34.880 --> 00:17:41.759 And the thought process, and I'm very interested because I know that James sits on a pretty uh coherent corpus in his own right. 00:17:41.920 --> 00:17:47.039 If you've got that corpus, you've got the predictable method, what do you get if you garnerize, you freeze the method? 00:17:47.119 --> 00:17:52.480 Well, you kind of get that talking statute, kind of binary code law, because what does the back end look like, the thing? 00:17:52.640 --> 00:18:00.160 The back end is a black box zeros and ones, and now you do have that more sort of flogged into binary, that more predictable language. 00:18:00.319 --> 00:18:06.000 You could get advisory opinions on the front end, asking a garnerizer, and if enough people are using it, it's right. 00:18:06.160 --> 00:18:09.039 So that's why I mean a methodologue is playing for world domination. 00:18:09.119 --> 00:18:11.200 So from one methodologue to another. 00:18:11.599 --> 00:18:12.000 Thank you. 00:18:12.079 --> 00:18:13.440 That was right on time. 00:18:14.400 --> 00:18:15.119 All right, okay. 00:18:16.000 --> 00:18:20.559 Thank you, Randy, thank you, Jack, and thank you, Sarah, for inviting me. 00:18:20.720 --> 00:18:28.960 Um, so I think we want to take a step back right now and uh and and just sort of cue this up by talking about what are the purposes of textualism. 00:18:29.039 --> 00:18:34.640 I I assume that if you're in this room, you know what textualism is, so we'll skip that step and talk about why do we care. 00:18:34.960 --> 00:18:40.799 And I think, you know, the whole corpora, whole large language models have been written about this topic. 00:18:41.039 --> 00:18:47.759 But I think we can kind of crystallize it down to textualism respects the separation of powers. 00:18:48.319 --> 00:18:55.680 It fundamentally concedes that our legislatures should be the ones who write the law. 00:18:56.319 --> 00:19:01.839 If we're in this room as different members of society, we're gonna disagree about a whole bunch of stuff. 00:19:02.160 --> 00:19:16.799 And then through our elected representatives, we're gonna argue about that stuff, and before we pass laws that say thou shalt do this or thou shalt not do that, we're gonna crystallize an agreement that at least a majority of our representatives agree to. 00:19:17.039 --> 00:19:20.480 And we're gonna, you know, memorialize that in writing. 00:19:20.880 --> 00:19:35.039 And then our constitutional system says that those words that our representatives agreed to remain the law until that legislative process uh is used to change it. 00:19:36.480 --> 00:19:39.359 So uh I'm not breaking any new grounds there. 00:19:39.519 --> 00:19:58.480 Uh the point is that textualism has been criticized over the years because the tools that that folks have been using have not necessarily delivered on the promise of objectivity and uh and adherence to that law that has been produced by our legislators. 00:20:00.000 --> 00:20:15.839 And corpus and and so what I want to talk about, uh I'm the first one to acknowledge that a lot of the tools that uh textualists have used over the time have been uh lacking and haven't necessarily delivered on that promise of objectivity. 00:20:16.000 --> 00:20:20.960 Uh but I think that corpus linguistics can help us get there in ways that AI can't. 00:20:21.119 --> 00:20:23.680 And so let me just sketch a couple of these. 00:20:23.920 --> 00:20:29.599 Uh number one, and Jack cued this up, corpus linguistics is transparent. 00:20:29.839 --> 00:20:32.559 Okay, and it's transparent in at least a couple of different ways. 00:20:32.720 --> 00:20:46.400 Number one, all of the databases that are used by judges and by lawyers in corpus linguistics are freely accessible, and it is totally clear what's in the database. 00:20:46.559 --> 00:20:56.640 They've been curated by linguists, they've been curated by historians to try to present an objective representation of a particular community. 00:20:57.200 --> 00:20:59.519 Uh AI, not so much. 00:20:59.839 --> 00:21:04.640 AI may have a lot of stuff, but we don't necessarily know what's in it. 00:21:04.880 --> 00:21:06.160 And that's a problem. 00:21:06.319 --> 00:21:09.839 Now, Randy talked about uh the bear arms question. 00:21:09.920 --> 00:21:17.599 Now, there's a database out there uh that is called the Corpus of American Soap Operas. 00:21:18.240 --> 00:21:34.880 And you could do the greatest corpus analysis in the history of the world, analyzing all of the all of the scripts of all of the soap operas that have ever been written, and it wouldn't tell you diddly squat about what the Second Amendment means. 00:21:35.200 --> 00:21:48.640 We don't know when we use large language models, we don't know if the very confident answer that it is spitting out is as a result of American soap operas or modern usage. 00:21:48.799 --> 00:21:49.759 We just don't know. 00:21:50.240 --> 00:21:52.400 Number two, it's replicatable. 00:21:52.559 --> 00:21:52.880 Okay? 00:21:53.039 --> 00:21:57.680 Now I wrote a brief in a very boring case called Rimini Street v. 00:21:57.759 --> 00:21:58.319 Oracle.

00:21:58.400 --> 00:21:59.599 It was a$13.5 million. 00:22:00.240 --> 00:22:04.240 Dispute that boiled down to the meaning of the word full in the Copyright Act. 00:22:04.319 --> 00:22:08.480 And I'm just going to leave that as the summary because you don't want to know more because it's very boring. 00:22:09.839 --> 00:22:16.720 I produced a brief that I submitted to the Supreme Court that relied on four or five different corpora. 00:22:17.839 --> 00:22:29.759 I provided hundreds of examples, coded them, showed all of the different ways in which full was used in all of its many different contexts in different time periods and among different communities. 00:22:29.839 --> 00:22:37.039 And I submitted it to the court and I said, regardless of what your jurisprudence is, here's some data that will be helpful. 00:22:37.680 --> 00:22:44.880 Now, the we ended up, uh our our our analysis ended up supporting Ramini Street. 00:22:45.519 --> 00:22:47.440 Oracle was represented by Paul Clement. 00:22:47.680 --> 00:22:50.559 He clearly read that brief, was clearly nervous about that brief. 00:22:50.640 --> 00:22:55.279 He went out and found his own amicus of linguists to write a counterbrief. 00:22:55.839 --> 00:23:12.880 And among the other things that he said, that this amicus said was, well, we've looked at all of the hundreds of examples that James has put together using corpus linguistics, and we disagree with the way that he coded one of them. 00:23:14.319 --> 00:23:23.279 I could have kissed that linguist on the mouth because he just told the Supreme Court that my brief was 99% accurate. 00:23:24.880 --> 00:23:26.720 You don't get that with AI. 00:23:27.039 --> 00:23:28.559 It's not replicatable. 00:23:28.720 --> 00:23:32.400 In fact, about a year ago, I was in a room like this. 00:23:32.559 --> 00:23:36.559 There were 12 judges, mostly district court judges and state Supreme Court justices. 00:23:36.640 --> 00:23:38.400 We did the Snell opinion. 00:23:38.640 --> 00:23:49.759 We had them pull up their the various uh large language models, we had them run the question of whether or not uh an in-ground trampoline was landscaping. 00:23:50.079 --> 00:23:52.960 It was almost exactly 50-50. 00:23:53.119 --> 00:23:54.799 I think it was seven judges. 00:23:54.960 --> 00:23:58.720 The LLM spit back and said, yes, it is landscaping. 00:23:58.880 --> 00:24:00.079 Seven said it wasn't. 00:24:00.240 --> 00:24:01.759 That's not replicatable. 00:24:02.319 --> 00:24:08.160 Um final point, and then I'll queue up some of my other points in the back and forth. 00:24:08.480 --> 00:24:10.880 Corpus linguistics combats bias. 00:24:11.200 --> 00:24:11.519 Okay? 00:24:12.000 --> 00:24:22.720 The very first judicial training that I did, we had as an example the Muscarello opinion, which asked what the meaning of carry a firearm meant. 00:24:22.880 --> 00:24:28.960 And there was a judge on the Idaho Supreme Court who sat down and he said, I've read the Muscarello opinion. 00:24:29.119 --> 00:24:33.200 I am 100% sure that you can carry a firearm in your car. 00:24:33.440 --> 00:24:35.440 I had a case almost exactly like this. 00:24:35.519 --> 00:24:36.880 I've thought about it a lot. 00:24:37.119 --> 00:24:38.799 Justice Breyer was right. 00:24:39.119 --> 00:24:48.079 We spent 25 minutes with the corpus analyzing this, and then he looked at me and he said, I was wrong. 00:24:50.640 --> 00:25:10.000 Um AI, because of what's called the sycophancy problem, has the risk of actually reaffirming and doubling down on our biases, and then obscuring it so that the judge doesn't even realize that that's what's happening. 00:25:10.240 --> 00:25:12.400 So that's fair. 00:25:12.640 --> 00:25:19.839 I mean, one thing to take away from that, the test for AI, and indeed the test for judging, appears to be consistency. 00:25:20.000 --> 00:25:21.039 I just want to put that out of there. 00:25:21.119 --> 00:25:22.640 That's the textualist test for this stuff. 00:25:22.799 --> 00:25:34.079 If all 12 GPTs get the same outcome, that is, they take the same fact pattern, phrased however many times, and they get the same why outputs, that's considered good textualism. 00:25:34.319 --> 00:25:40.559 Anybody who's kind of looking to put some humanity in the whole process, I do just want to note before I go further, I don't think you're a textualist. 00:25:40.799 --> 00:25:43.200 Like that is the endeavor. 00:25:43.279 --> 00:25:55.039 And getting into these very tough questions about can you make a fully, you know, end-to-end syllogistic language whereby all 12 bots get the same thing, that's the future of textualist debate. 00:25:55.279 --> 00:26:00.160 So you've got to ask yourself, you're in the textualist or in the federal society, you've got to ask yourself, do I like that? 00:26:00.319 --> 00:26:01.119 What does that look like? 00:26:01.359 --> 00:26:06.559 If you're one of these humanistic judgy enterprises, I think you might have to uh, I don't know if this is the room for you. 00:26:06.880 --> 00:26:16.799 With that said, I very much uh agree with the separation of powers point, and I'm kind of taking this a step back before, so I'm concerned with what language do they write in. 00:26:17.200 --> 00:26:23.359 You look at all these uh thinkers and judges who have their own bespoke responses to no vehicles in the park, and it gets kind of cartoonish. 00:26:23.519 --> 00:26:24.240 Very smart guys. 00:26:24.400 --> 00:26:29.519 One guy has got nine things, he's got a bike and a toy, and the other guy's got twelve, they all got 13. 00:26:29.680 --> 00:26:32.079 It kind of looks like a losing endeavor. 00:26:32.240 --> 00:26:37.599 The problem, seem to me, is not sort of how we're parsing the vehicle, it's the English language itself. 00:26:37.839 --> 00:26:40.720 There should be a more granular way of denoting this. 00:26:40.880 --> 00:26:44.559 The Congress should know to speak in on the front end, something more binary. 00:26:44.720 -->

00:26:46.079 Think of the flagellation thesis. 00:26:46.240 --> 00:26:52.240 If we apply literalistic textualism from the bench, in theory, we flog the legislature into writing things that are more specific. 00:26:52.319 --> 00:26:57.519 They don't save vehicles, they give you a list of various things such that you could do, you know, a used them generous with them at least. 00:26:57.680 --> 00:27:00.400 I don't know why that couldn't apply to languages themselves. 00:27:00.480 --> 00:27:05.200 And that I think is the idea behind placing a robot on the methodology part. 00:27:05.359 --> 00:27:08.640 Though I do agree with your point, it's got to be consistent. 00:27:08.799 --> 00:27:18.960 But when we've got the thing distilled down to this very bare essence where we're literally coding one way or the other, that seems like a lot simpler process to have a machine try its hand at, and it's very testable. 00:27:19.119 --> 00:27:23.599 You have various different phrasings run through the thing, and you see, do I go 12 for 12? 00:27:23.759 --> 00:27:25.119 Do I go 100 for 100? 00:27:25.279 --> 00:27:31.119 The question I think text world has got to put on the table going forward is can we give Congress a Lego block language? 00:27:31.279 --> 00:27:34.960 And by this I mean, you know, stone's great for carving art. 00:27:35.039 --> 00:27:36.880 You're not gonna get two statues of David. 00:27:37.039 --> 00:27:44.400 Maybe the superintelligence idea is that yes, you could have the bot say, go make me something more beautiful than David, it would carve the stone, and you'd say, wow, that's even more beautiful than David. 00:27:44.480 --> 00:27:45.279 That's super intelligence. 00:27:45.519 --> 00:27:48.960 Short of that, it kind of takes a human to carve some stone and get David. 00:27:49.279 --> 00:27:50.640 You get them once in a while. 00:27:50.799 --> 00:27:52.559 You don't get them very much, though. 00:27:52.720 --> 00:27:53.920 When you do, it's really nice. 00:27:54.079 --> 00:27:55.519 And then there's Lego blocks. 00:27:55.920 --> 00:27:59.279 You can make a statue of David out of Lego blocks, and you're gonna know it's a statue of David. 00:27:59.359 --> 00:28:01.680 It's not gonna look as pretty, but a five-year-old can build it. 00:28:02.000 --> 00:28:04.160 Ten-year-old, twenty-year-old, fifty-year-old. 00:28:04.319 --> 00:28:10.559 It's easy to re to replicate, to your point, replicability, because you've got these individual bespoke blocks. 00:28:10.640 --> 00:28:15.359 So I feel like English is too much like Congress trying to make art out of stone. 00:28:15.680 --> 00:28:20.160 I kind of looking for more of a Lego block language that's much uglier. 00:28:20.240 --> 00:28:28.319 I mean, you try to read the code of Google, try to read JavaScript, it's hideous, but it runs these vast apparatuses very predictably. 00:28:28.480 --> 00:28:33.920 And to me, I think based on the conversation we've had, the highest textual slowstraw you're looking at is predictability. 00:28:34.079 --> 00:28:45.440 So if we could get a method and really distill it down to that basic 0-1 essence, which I think corpus liquidistics takes you right to the doorstep of, you just gotta take that one other step over, and we're making Lego block law. 00:28:45.519 --> 00:28:46.400 I like that. 00:28:47.440 --> 00:28:53.200 So I I I I think we have more disagreement uh than you think. 00:28:53.839 --> 00:29:02.480 So, first of all, I do not think that consistency is the uh be and end all of textualism. 00:29:02.640 --> 00:29:04.319 It is an important component. 00:29:04.559 --> 00:29:07.759 I also think we want to be right. 00:29:08.400 --> 00:29:19.200 We want to capture what the words actually meant at the time the statute or the constitutional provision was enacted. 00:29:20.480 --> 00:29:28.319 This Lego block idea does not match the reality of how language is used. 00:29:28.960 --> 00:29:32.079 There is a phenomenon called linguistic drift. 00:29:32.400 --> 00:29:39.440 Language changes, sometimes dramatically, for no apparent reason. 00:29:39.920 --> 00:29:56.000 And we can have, it is possible in the world of textualism, because, as Justice Gorsett says, we care about the language of the statute at the time it was enacted. 00:29:56.240 --> 00:30:05.359 Uh, it is possible to have two statutes that use the same word and have those words mean different things. 00:30:05.599 --> 00:30:12.799 In fact, you invoked Adams and his desire for some sort of interpretive dictionary. 00:30:13.599 --> 00:30:31.839 Uh in the Jefferson and Adams papers, they actually have a whole series of letters where they sort of wax eloquent about how American English is just different than British English and how it's just changed over time, and language continues to change over time. 00:30:32.400 --> 00:30:36.799 Um I'm not sure that we can get there. 00:30:37.039 --> 00:30:39.440 You know, just to provide a couple of examples. 00:30:39.680 --> 00:30:51.039 Let's imagine that uh we are interested in interpreting the meaning of the rules of Major League Baseball. 00:30:51.279 --> 00:30:51.599 Okay? 00:30:52.880 --> 00:30:56.559 And we're interested in figuring out what the word hit means. 00:30:56.960 --> 00:31:01.920 Now, there's lots of definitions of the word hit. 00:31:02.559 --> 00:31:05.519 If I punch Randy in the face, I hit him. 00:31:05.680 --> 00:31:13.759 You know, if I go down to the corner and I find a drug dealer and I say, I want a hit, I mean something very different. 00:31:14.960 --> 00:31:19.039 Context is what makes the difference. 00:31:19.279 --> 00:31:25.920 Not, there's no bespoke Lego block definition of the word hit. 00:31:26.079 --> 00:31:38.880 What corpus linguistics does is we can look, well, A, we could select a corpus that is exclusively baseball-related productions. 00:31:40.880 --> 00:31:57.839 Or if we were look, if there wasn't one of those available, we could look more broadly at sort of an

ordinary American English corpus, and we could notice, oh, every single source that is discussing baseball is using the word hit differently and distinctly. 00:31:58.000 --> 00:32:05.519 And we can get we can figure that out because it's transparent and because it's replicatable, which is not something we can get with a robot. 00:32:05.920 --> 00:32:09.839 Uh I have a question for each of you, just to chime in here for a minute. 00:32:10.079 --> 00:32:16.240 Um we've been discussing we're sort of assuming, I think, in our minds that we're talking about a large language model. 00:32:16.480 --> 00:32:19.279 But what if we're talking about what if we're talking about a small language model? 00:32:19.680 --> 00:32:22.640 What if we just use the corpus that BYU has? 00:32:23.200 --> 00:32:24.079 What is the name of the corpus? 00:32:25.039 --> 00:32:25.839 Oh, there's there's various. 00:32:26.880 --> 00:32:28.160 So you pick pick the right one. 00:32:28.319 --> 00:32:42.720 Pick the one that you would use uh uh otherwise, that you would be using as the basis of your analysis, and then have a chatbot and code what you've discovered in your in your database, in your small language model. 00:32:42.960 --> 00:32:48.720 Um would that address the problem of linguistic drift, for example? 00:32:48.880 --> 00:32:50.319 Because you'd be using the same database. 00:32:50.640 --> 00:32:53.839 And my question for you, Jack, is garbage in, garbage, garbage out. 00:32:54.640 --> 00:32:58.640 All I've heard from you is basically, I think, large language model. 00:32:58.960 --> 00:33:10.880 And we're gonna start who what is the database that you think AI is going to be applied to so as to address the concerns that uh that we've that James has said has articulated. 00:33:11.200 --> 00:33:12.640 Those two questions are two sides of a coin. 00:33:12.720 --> 00:33:15.039 I'm very much in agreement that there needs to be that fixed corpus. 00:33:15.279 --> 00:33:17.119 And here's the kind of the money point here. 00:33:17.440 --> 00:33:19.200 I mean, the law is made up. 00:33:19.440 --> 00:33:24.559 We can, you know, if here's doctor.ai and the bot's only this good, you gotta kind of wait for it to get good. 00:33:24.640 --> 00:33:26.400 The heart attack, you can't change what a heart attack is. 00:33:26.640 --> 00:33:28.480 Laws, you can change the corpus. 00:33:28.720 --> 00:33:31.759 You could say, well, you know, the the broader corpus is too big. 00:33:31.839 --> 00:33:34.400 And there's a big question here that we're kind of elighting here. 00:33:34.559 --> 00:33:50.319 If we were gonna start a textualist country from scratch, nothing written yet, I think the textual should definitely want to do it my way, limit the corpus at the outset, such that if the corpus changes, words change over time, that is actually a legal change. 00:33:50.480 --> 00:33:52.559 It's reflected, there's something that tracks it. 00:33:52.640 --> 00:34:01.920 You update that PDF base and you kind of keep a tight lid on it for the specific purpose of building artificial building block style language. 00:34:02.160 --> 00:34:05.759 The question then is, well, we don't live in that country, we live in this one. 00:34:05.920 --> 00:34:12.559 But when you refer to the meanings of hit, I'll do this briefly, you know, you're saying there's a lot of different contexts of hit, so they can't be Lego block. 00:34:12.800 --> 00:34:13.119 I don't know. 00:34:13.199 --> 00:34:15.119 I see in the dictionary, I see about nine blocks. 00:34:15.199 --> 00:34:18.480 There's hit one, hit two, hit three, there's drug hit, there's baseball hit, there's various types. 00:34:18.559 --> 00:34:24.880 And I think as you build in various corpora over different eras, there might be like hit five million, hit five million and one. 00:34:25.199 --> 00:34:27.840 The question is, are there infinite contexts? 00:34:28.079 --> 00:34:28.880 I don't think so. 00:34:29.039 --> 00:34:36.800 I think if a human being can use these corpus linguistics tools to get down to these individual things, at the end of any given corpus linguistics analysis is a score. 00:34:36.880 --> 00:34:38.880 You get an output, and that gets stored away. 00:34:38.960 --> 00:34:40.079 There's your new Lego block. 00:34:40.239 --> 00:34:41.760 That's hit five million and two. 00:34:42.239 --> 00:34:47.519 At some point, if languages have any finite meaning, you gotta run out of hits. 00:34:47.679 --> 00:34:53.599 So I'm not saying there's ten pieces in the box or a thousand, it could be a lot of pieces, but these are really big computers. 00:34:53.760 --> 00:34:54.639 That's my take. 00:34:54.880 --> 00:34:59.519 Yeah, so so let me address Randy's question first, and then I'm gonna circle back and respond to Jack. 00:34:59.679 --> 00:35:01.039 So 100%. 00:35:01.760 --> 00:35:03.519 Totally in agreement, okay? 00:35:03.679 --> 00:35:19.840 I I think we need to move away from this idea of LLMs, and I I like this term, a small language model, class, or what or what sometimes we we refer to as a rag uh in sort of computer science, this idea that we're selecting the corpus. 00:35:20.400 --> 00:35:23.599 And I do think that this is the future of corpus linguistics. 00:35:23.840 --> 00:35:42.000 I think the future of corpus linguistics is that we incorporate the AI technology in order to interface with a specifically curated uh corpora in order to do corpus linguistics more efficiently and faster. 00:35:42.800 --> 00:35:45.920 Um I think it still needs to be transparent. 00:35:46.239 --> 00:35:55.599 I think i i if we're just getting the AI spitting out and saying, hey, I looked at the corpus, 72% agree with you. 00:35:56.000 --> 00:35:56.800 Trust me. 00:35:57.679 --> 00:36:00.320 I think that that that's dangerous land. 00:36:00.400 --> 00:36:33.679 Uh there was uh if you think back in the Bostik case, okay, there was an amicus brief that was submitted by a couple of corpus lingu corpus linguists who

have admitted to me that they were paid a lot of money to write a brief that said that at the time in 19 when whenever the Title VII was um, or is it Title IX, whenever that statute was drafted, that the word for sex was totally ambiguous. 00:36:34.000 --> 00:36:48.079 And they did not tell the court what corpus they used, they did not provide any examples, they just said, we looked at it, and this is what we found, here are the raw statistics, trust us. 00:36:49.039 --> 00:36:55.920 And I am fairly confident that all nine of the justices took that brief and threw it in the trash. 00:36:56.239 --> 00:36:58.480 Because you just can't trust that. 00:36:58.559 --> 00:37:01.119 It was so blatantly biased. 00:37:02.239 --> 00:37:05.760 So, a small language model, it needs to still be transparent. 00:37:05.920 --> 00:37:11.039 We need to be able to say, okay, here's the universe, and here's how we coded it. 00:37:11.280 --> 00:37:19.440 And maybe there's even an interface where it codes all of the easy answers, and then it says, like, look, here are some hard ones where it could go either way. 00:37:19.760 --> 00:37:30.400 Um I do think that that's I I I'm actually in the process of I've been working with a scholar at MIT and a scholar at Washington and Lee to try to work on this. 00:37:31.039 --> 00:37:40.159 You know, AI models are really great if you give it one concordance line and it says code this hit from the corpus. 00:37:40.400 --> 00:37:41.840 It does that like a champ. 00:37:42.320 --> 00:37:46.639 If you give it a hundred, it starts making stuff up. 00:37:46.800 --> 00:37:48.320 It really doesn't want to do that. 00:37:48.559 --> 00:37:50.159 Now there's ways of doing that. 00:37:50.239 --> 00:37:55.920 We're working on programming and creating an agent in order to do that, but I think that is the that is the future. 00:37:56.159 --> 00:38:02.880 Um to respond to Jack, again, I just don't think that that's how language works. 00:38:03.199 --> 00:38:03.519 Okay? 00:38:04.159 --> 00:38:09.039 You know, it and it actually sounds a little 1984-esque. 00:38:09.280 --> 00:38:25.039 You know, I I I love in 1984 there's this moment where there's a sort of a side character who's at a pub and he's talking about how he just wants a pint of beer, but they've eliminated the concept of a pint. 00:38:25.119 --> 00:38:30.719 And so he can have a half liter and that's too small, or he can have a whole liter and that's too much. 00:38:30.960 --> 00:38:33.440 And he just wishes that he could have a pint. 00:38:34.320 --> 00:38:36.400 Leader is completely uncivilized. 00:38:39.360 --> 00:38:42.079 The metric system is just abominable. 00:38:44.079 --> 00:38:46.000 And so I think that that that there's a risk. 00:38:46.079 --> 00:38:50.719 I don't think that's necessarily what you're saying, but but like recognizing that language changes. 00:38:50.800 --> 00:38:53.360 So let me give you uh just a couple examples. 00:38:53.840 --> 00:38:54.079 Okay? 00:38:54.559

--> 00:38:56.559 Uh this is potentially apocryphal. 00:38:56.639 --> 00:38:59.199 Linguists and historians argue about this story. 00:38:59.360 --> 00:39:06.639 But after the Great Fire of London, uh there was a contest to design the new uh St. 00:39:06.800 --> 00:39:08.159 Paul's Cathedral. 00:39:08.480 --> 00:39:18.159 And Sir Christopher Wren, who was an astronomy professor at Oxford, was told that his design had been selected because it was simultaneously the most awful and the most artificial. 00:39:19.360 --> 00:39:24.719 Not great praise today, but at the time it was the highest compliment. 00:39:25.039 --> 00:39:27.039 Language has just moved on. 00:39:28.239 --> 00:39:30.079 Number closer to home. 00:39:30.320 --> 00:39:34.880 Think about 2005, 2004. 00:39:35.519 --> 00:39:37.280 You hear the word viral. 00:39:38.960 --> 00:39:40.639 What does it mean? 00:39:41.119 --> 00:39:42.880 What does it mean today? 00:39:43.199 --> 00:39:44.880 Very, very different. 00:39:45.840 --> 00:39:50.639 Um third example from the Constitution, okay? 00:39:50.800 --> 00:39:51.599 In the Trump v. 00:39:51.840 --> 00:40:11.360 Anderson litigation about the meaning of Section 3 of the 14th Amendment, some really good friends of mine who are con you know originalist scholars uh made an argument well before Trump was president uh that Section 3 of the 14th Amendment does not apply to the president because the president is not an officer of the United States. 00:40:11.599 --> 00:40:42.880 And it was really great intra-textual analysis, but over and over and over again in this series of articles, these scholars, who again are friends of mine, said that they were based on the assumption that the Constitution hardwires a distinction between appoint and elect, and that the president is very clearly elected, not appointed, and so therefore the appointments clause and does not apply, and therefore he's not an officer. 00:40:43.199 --> 00:40:59.760 The problem is that when you start to scratch the surface and you look at how people actually used the word appoint and the word elect at the time of the founding, you find over and over and over and over and over again that those words were used interchangeably. 00:41:01.840 --> 00:41:08.000 Language, that is not the case today, but language moved on, linguistic drift. 00:41:08.320 --> 00:41:24.880 So I again I I I again I believe that there's a way of harnessing this technology, but I'm not, I I I just am not convinced that we can narrow or even predict the way that language is gonna change in the future. 00:41:25.280 --> 00:41:28.400 Well, I mean, is textualism dangerously close to newspeak? 00:41:28.559 --> 00:41:29.119 Yeah. 00:41:29.360 --> 00:41:31.199 Does it compel newspeak? 00:41:31.360 --> 00:41:32.079 I tend to think so. 00:41:32.159 --> 00:41:34.480 I've never claimed to be sort of an out-and-out textualist. 00:41:34.559 --> 00:41:35.760 Textualism scares me a little bit. 00:41:35.840 --> 00:41:40.559 It's kind

of a nihilistic flip of the coin of like, you know, we're not doing God's law out here. 00:41:40.639 --> 00:41:42.559 We've got to kind of make our own thing up. 00:41:42.639 --> 00:41:46.239 We've got to make our own text has kind of become this sort of civic. 00:41:46.400 --> 00:41:55.519 Religion is a lot of newspeak, but the problem is, I think as the corpus linguistics tools become more available, you're gonna accidentally get a lot of this if you don't guard against it. 00:41:55.599 --> 00:42:01.760 Because when you say those guys came up with a bias court and said, trust us, the question is, well, how would they have shown the court their work? 00:42:01.840 --> 00:42:05.519 Well, they would have given a big diagram of all the various instances they coded. 00:42:05.599 --> 00:42:11.599 They would have shown them each zero and one so the court could have gone instance by instance the way they did in yours, found one out of the hundred they didn't like. 00:42:12.159 --> 00:42:13.920 I think a robot can do that. 00:42:14.079 --> 00:42:16.079 I think a robot could produce that exact sheet. 00:42:16.239 --> 00:42:17.920 I think it can then store that exact sheet. 00:42:18.000 --> 00:42:26.559 I think it could do it thousands of more times and start grouping those sheets together, learning the way it does it each time, and start slowly closing in on these things. 00:42:26.800 --> 00:42:27.599 I love the example. 00:42:27.679 --> 00:42:31.199 I've listened to your talk, you're talking about artificial, etc., and it's meaning over time. 00:42:31.760 --> 00:42:36.639 I still want to say we're not talking about thousands of nuanced micro changes here. 00:42:36.719 --> 00:42:38.719 We're talking like two or three viral. 00:42:38.800 --> 00:42:40.719 We got old viral, new viral, it's two. 00:42:41.199 --> 00:42:45.599 That doesn't seem just dramatically beyond a robot's comprehension. 00:42:45.679 --> 00:42:56.320 If we do update the corpus's, you know, the corpora with some regimentation when there are new PDFs that are coming in, which is there's a quite open question, is that what the LMs are doing now by combing the entire internet? 00:42:56.719 --> 00:43:05.679 Are they keeping an up-to-date PDF log that is allowing the bot to be just as dynamic a tracker of human language as humans are speakers of human language? 00:43:05.840 --> 00:43:16.239 If that's the case, I think your nightmare scenarios, people say I like corpus linguistics a lot, and I just had Claude make me a 200-line code, and I'm gonna submit that to the court. 00:43:16.400 --> 00:43:17.760 Now, what's the court really gonna say? 00:43:17.840 --> 00:43:24.639 Maybe they disagree on the margins here or there, but if you keep doing it enough, I don't know why that can't replace the analysis. 00:43:24.800 --> 00:43:26.559 And then you do have that predictability. 00:43:26.719 --> 00:43:33.119 You've taught the machine to speak at a level of granularity, you can say, hey, I just did this, I got this parking ticket, X facts happen to me. 00:43:33.280 --> 00:43:34.000 What result? 00:43:34.719 --> 00:43:42.079 And if you've got the sort of newspeak, you can kind of get a pretty credible advisory opinion from whatever bot you know the court will be using. 00:43:42.320 --> 00:43:45.920 That's kind of a dystopian judge AI world, that's what I write about. 00:43:46.000 --> 00:43:51.599 But I think corpus linguistics walks you all the way to how you might, in our lifetime, pretty briskly get there. 00:43:52.400 --> 00:43:59.760 So I I I I still don't think that corpus linguistics gets you anywhere near the dystopic judge. 00:44:00.320 --> 00:44:01.280 AI world. 00:44:01.519 --> 00:44:06.159 What it does is it gives judges more and better evidence. 00:44:06.320 --> 00:44:08.800 And then the judge got a judge. 00:44:09.199 --> 00:44:12.960 There will be cases that are hard. 00:44:13.679 --> 00:44:23.840 There will be cases where, in fact, what the corpus shows and what uh what the data shows is that actually this statute is ambiguous. 00:44:24.079 --> 00:44:31.519 And then, you know, per Brian Gardner, you know, there are ways of dealing with ambiguity. 00:44:32.480 --> 00:44:45.039 You know, but the but what corpus linguistics does is it gets, it forces judges to show their work, and it guards against motivated reasoning and navel gazing. 00:44:45.360 --> 00:44:58.719 And what I'm concerned about robot judges is that we are outsourcing to not just navel gazing because AI doesn't have a belly button. 00:45:00.000 --> 00:45:10.719 It's just it's just trusting someone who doesn't have a soul to make determinations about our fundamental law. 00:45:11.119 --> 00:45:31.440 And and one one more point that I want to just make in response is there are um so much of the Textualist project is about tying the words on the page to the context and time in which they were enacted. 00:45:31.760 --> 00:45:35.840 And again, I think we solved this with a small language model or a rag. 00:45:36.880 --> 00:45:52.480 But large language models, if you look at the Snell opinion, and and Judge Newsom acknowledges that this is a defect, he's not asking what did these words mean at the time that the contract was written. 00:45:52.639 --> 00:45:57.360 And and and some of that was because the contract that was at issue was written in 2017. 00:45:57.599 --> 00:46:00.719 So we it had this close nexus to the time. 00:46:01.039 --> 00:46:03.039 But but AI can't. 00:46:03.280 --> 00:46:14.719 I mean, I'm developing a course for for the originalism seminar on how to use AI to do originalism research better, and there's lots of ways that it can be helpful. 00:46:14.800 --> 00:46:17.039 I use it all the time in my legal practice. 00:46:17.360 --> 00:46:25.519 But if you just ask AI what did this mean, and you ask the ultimate legal question, what did this mean? 00:46:26.480 --> 00:46:28.239 It it makes stuff up. 00:46:28.639 --> 00:46:29.199 Oh, totally. 00:46:29.360

--> 00:46:36.400 But if you make it go through all of your steps, the the problem with this is the 50-50 splits are really kind of scary if you're textual. 00:46:36.719 --> 00:46:41.679 I've got 500 instances of this, I have 250, 200 one way, 250 the other. 00:46:42.079 --> 00:46:45.280 I mean, how much more can we make this a science? 00:46:45.519 --> 00:46:52.320 And if it's still an art at that last step, one question is, are there objective answers in textualism? 00:46:52.480 --> 00:46:56.320 If there aren't, that seems like a commentary on the entire project. 00:46:56.400 --> 00:46:58.960 It seems as indeterminate as morals or anything else. 00:46:59.119 --> 00:47:03.199 I feel like corpus linguistics gives you hope that no, there are specific answers. 00:47:03.280 --> 00:47:10.559 And then, like you say, there is a toolbox, and you're afraid, it's the Garner toolbox when you hit that sort of uh question mark, that construction zone. 00:47:10.719 --> 00:47:14.639 It's weird to me that you could still hit a construction zone after this many data points. 00:47:14.800 --> 00:47:23.119 And even when there's a question of if there aren't right answers, shouldn't we contrive a system where there are, just because we say there are? 00:47:23.280 --> 00:47:27.360 Can't we shrink the corpus or make some sort of have something make a judgment call? 00:47:27.519 --> 00:47:37.039 And I know it's a sort of, you're afraid of this black box, and that's what I'm trying to make sure, because a lot of that salesmanship on who's designed the black box, you need a name, like a garner, etc., whose reputation bears that weight. 00:47:37.199 --> 00:47:39.840 You can tell that the guy who trained it really knows what he's doing. 00:47:40.000 --> 00:47:42.960 But on some of them, maybe you just say nothing need something to make a call. 00:47:43.119 --> 00:47:44.159 And what do you know in that case? 00:47:44.239 --> 00:47:45.519 You at least know what the call is. 00:47:45.599 --> 00:47:51.280 You don't have to litigate for decades to find out the meaning of a word that a new court ten years later might find something different. 00:47:51.360 --> 00:47:56.239 You know it empirically, and now it's kind of the FedSocopia, there's nothing for the judge to do anymore. 00:47:56.400 --> 00:47:59.119 You don't go protest at the Supreme Court as Bork abhorred. 00:47:59.280 --> 00:48:04.880 You say, legislature, use those words that the Congress, use the ones we objected to the meaning of, and change them. 00:48:05.119 --> 00:48:06.000 Change the outcome. 00:48:06.079 --> 00:48:08.079 The legislative outcome is the only one possible. 00:48:08.159 --> 00:48:10.320 To me, that's platonic separation of powers. 00:48:10.480 --> 00:48:11.440 I think that's where this goes. 00:48:11.519 --> 00:48:12.800 I don't think it's bad. 00:48:13.360 --> 00:48:15.760 Jack, do you think there's a construction zone? 00:48:16.400 --> 00:48:20.559 It scares me that this much data there could still be a construction zone. 00:48:20.719 --> 00:48:26.800 If there is, I think we might have to start looking at the project more broadly and say, can we just get rid of this damn thing even arbitrarily? 00:48:26.880 --> 00:48:28.239 And you hit it with the shrinking corpus. 00:48:28.480 --> 00:48:29.039 There's ways to do it. 00:48:29.280 --> 00:48:32.079 But in principle, there is such a thing as a construction zone. 00:48:32.400 --> 00:48:34.400 I think there is given current tools. 00:48:34.559 --> 00:48:41.760 I am very interested to see, and I think the analysis of Snell, or no, it was Lee did what the other case, the companion case that you did. 00:48:41.920 --> 00:48:42.639 It's very weird. 00:48:42.719 --> 00:48:45.440 There was a lot of for one, the data point's kind of underwhelming. 00:48:45.519 --> 00:48:48.960 There was like 50 total, and they changed corpuses over like 60. 00:48:49.119 --> 00:48:54.800 But if you get just thousands of instances, uh it's shouldn't it scare me that there's construction with that much data? 00:48:55.199 --> 00:48:58.000 Has Brian changed his opinion on the existence of a construction zone? 00:48:58.159 --> 00:49:03.679 Because in the Scalia Garner book, he rejects the distinction between interpretation and construction pretty emphatically. 00:49:03.920 --> 00:49:04.960 Has he has he evolved? 00:49:05.280 --> 00:49:09.199 I never want to speak for him on this stuff, and I won't ask him that question directly. 00:49:09.360 --> 00:49:16.079 I think that, you know, as to where this project's going, yes, it is far there has to be some answer for construction. 00:49:16.159 --> 00:49:18.880 Yes, I think it has to be interpretation all the way down. 00:49:19.039 --> 00:49:23.360 If there's constructing that you're doing, I think that's you've got to admit that's inherently anti-textualist. 00:49:23.599 --> 00:49:25.920 You're building something that was not previously there. 00:49:26.079 --> 00:49:32.079 And if the answer is that for all this data, that's inevitable, there has to be some humanity there, well, that's a whole different project. 00:49:32.159 --> 00:49:36.400 Well, then if we're letting humanity in that little kernel, where does the flood floodgates stop? 00:49:36.559 --> 00:49:42.159 The alternative is you say, well, can we build this syllogistic world whereby we decrease the amount of construction? 00:49:42.400 --> 00:49:47.760 I think the textualists, and even Brian, on some level would want to be able to say, we'd like to get rid of construction. 00:49:48.000 --> 00:49:49.039 We don't like that it's there. 00:49:49.119 --> 00:49:51.599 It's a necessary evil we should be trying to eradicate. 00:49:51.920 --> 00:50:04.719 Well, if construction is a result of irreducible ambiguity, which is your hypothetical, you just can't tell one third or the other, you know, whether arms means one thing or what arms means, or whether a direct tax is this or a direct tax is that, then construction is

inevitable. 00:50:04.800 --> 00:50:06.800 And then the question is, how do you do construction? 00:50:07.039 --> 00:50:18.159 And according to like one proposal, you do it you do it according to the spirit or purpose or object of the provision as opposed to its meaning, because its meaning is run out. 00:50:18.320 --> 00:50:26.559 The other reason for construction is if language is vague and you're trying to apply vague language, and you can't make by interpretation, you can't make vague language unvague. 00:50:26.639 --> 00:50:32.079 There's a core meaning and then there's a penumbral meaning, and you have to decide whether some borderline case is in or out. 00:50:32.239 --> 00:50:33.840 That also has to be done by construction. 00:50:34.000 --> 00:50:36.159 So I don't see how you're going to eliminate it entirely. 00:50:36.480 --> 00:50:38.800 Well, I mean, this is sort of textualism untell. 00:50:38.880 --> 00:50:44.079 There's a point at which you do have to open this sort of it's the Pandora's box for kind of a reason. 00:50:44.159 --> 00:50:48.159 And the question is, I mean, I don't think Brown would say this point, but in the papers I've written, I have said it. 00:50:48.400 --> 00:50:52.159 I'd be alright if a machine took out a coin and flipped it and froze its results. 00:50:52.400 --> 00:51:00.800 As long as the coin came out the same way every single time, as long as we could have that front-end consistency, now you give the legislature some building blocks they can actually move around. 00:51:00.880 --> 00:51:02.880 You completely absolve the judge. 00:51:03.119 --> 00:51:06.800 I don't think a textualist should be satisfied with construction. 00:51:06.880 --> 00:51:19.679 And it just, again, if we really maximize the data and we could go through a corpus the size of what GBD has at its disposal with sort of corpus linguistics style tools to really narrow and make it transparent what's going on, transparent and replicable. 00:51:19.840 --> 00:51:25.519 If you've still got construction after that, I think there would be far a small, a smaller amount of construction. 00:51:25.760 --> 00:51:30.239 If there's any residual amount left, heck, take out the coin and flip it in that case. 00:51:30.400 --> 00:51:32.320 At that point, nothing's going to get an answer. 00:51:32.400 --> 00:51:33.119 We just need something. 00:51:33.199 --> 00:51:34.800 I think a bot can provide something. 00:51:35.119 --> 00:51:40.559 Well, we have 10 minutes left, and so we I'd like to open it up to questions from the students who are here. 00:51:40.719 --> 00:51:47.199 Um you've got two experts in front of you on a topic that I'm sure you've given a lot you've given a lot of thought or to, or you've you certainly wondered about. 00:51:47.360 --> 00:51:51.119 So what questions do you have for these experts while you have them here in front of you? 00:51:51.599 --> 00:51:56.559 And it it may feel ridiculous, but we're committed to podcasting our events this year. 00:51:56.800 --> 00:52:01.760 So when you raise your hand, I'm gonna bring you over this tiny microphone to say your question into. 00:52:11.360 --> 00:52:12.480 Hi, thank you for coming. 00:52:12.639 --> 00:52:13.519 My name's Donald. 00:52:13.679 --> 00:52:33.199 I was wondering, um, how do you with large language models, how do you avoid the same pitfalls that uh just modern language has with shifting meaning, especially as you can't see on the back end what has gone into this model uh to spit out the result. 00:52:33.360 --> 00:52:47.119 Whereas at least when judges who are rationalizing their interpretation of how they're defining something, they're even if they're not using a large language model, they're still explaining, okay, well, this is what I think the meaning is. 00:52:47.360 --> 00:52:53.760 And so, like what happens when a judge just pulls from the what the language large language model spics out. 00:52:54.159 --> 00:52:59.280 I mean, I take a lot of the criticisms that you want to know more what's in some giant, fathomless model like Claude. 00:52:59.440 --> 00:53:00.880 In theory, there is a list of that. 00:53:01.039 --> 00:53:03.199 That's even somewhat a problem with corpus linguistics, though. 00:53:03.280 --> 00:53:06.000 There's 500 documents, you could put that as an appendix, you know. 00:53:06.159 --> 00:53:08.239 I would hope everybody's reading all that stuff. 00:53:08.320 --> 00:53:11.039 They probably aren't if you maximize that to the millions. 00:53:11.199 --> 00:53:16.639 But the problem is I don't know why you wouldn't want corpus linguistics with millions of PDFs. 00:53:16.719 --> 00:53:19.679 It seems like more data points equals better. 00:53:20.239 --> 00:53:24.719 And at that point, um, if you wanted to query it in theory, you could. 00:53:24.880 --> 00:53:27.360 The PDFs are all sitting there. 00:53:28.239 --> 00:53:32.320 I do think you're gonna, it's the difference between a classifier model and an LLM. 00:53:32.400 --> 00:53:34.480 You know, we've got a paper about this because the guy tried this out. 00:53:34.559 --> 00:53:40.960 He had AI rephrase various questions, same structure but different words, and LLM's butchered it. 00:53:41.039 --> 00:53:43.840 But this arbiters thing I work at makes the talking contract. 00:53:44.000 --> 00:53:45.360 They want it to be a classifier model. 00:53:45.440 --> 00:53:47.599 So, what an LM does is classifying the very next token. 00:53:47.840 --> 00:53:53.599 Still a classifier, but with a very open-end set of what should the next set of pixels be in this strain of pixels that look like words. 00:53:53.760 --> 00:53:57.519 A classifier model, in the contract sense, it groups fact patterns. 00:53:57.679 --> 00:54:00.320 These facts are win facts, these are lose facts, they go together. 00:54:00.400 --> 00:54:03.519 It's kind of like the sort of analogical reasoning you see in common law. 00:54:03.760 --> 00:54:06.559 I think James offers a really nice way to

do this linguistically too. 00:54:06.719 --> 00:54:11.760 You're offering discrete instances, usages, word, words around it, word and context. 00:54:11.920 --> 00:54:16.079 You can bucket those, you can classify those, and then you want to be running the specific tests. 00:54:16.159 --> 00:54:19.840 Not does the thing sound real smart at the end, but does it get consistent outcomes? 00:54:20.000 --> 00:54:24.719 Can you phrase something a billion different ways and it still knows, okay, that's that particular context. 00:54:24.800 --> 00:54:29.440 That is hit number 5 million and three, or applying hit number 5 million and three. 00:54:30.400 --> 00:54:31.840 I think you can do that. 00:54:31.920 --> 00:54:34.000 If you can't, again, this is the construction of a point. 00:54:34.159 --> 00:54:36.400 It should really scare you if you're a tax realist. 00:54:36.559 --> 00:54:42.559 And at that level of granularity, you still think there is a human interpretive endeavor to undertake. 00:54:44.320 --> 00:54:58.079 Um I want to just sort of throw in two thoughts about whether or not we should be trusting AI for this sort of interpretive endeavor. 00:54:58.480 --> 00:55:05.920 Uh number one, similar to the dictionary, somebody just made the AI. 00:55:06.320 --> 00:55:06.639 Okay? 00:55:07.039 --> 00:55:13.679 So let me give you a real example of a conversation that I had with ChatGPT uh a couple of weeks ago. 00:55:13.920 --> 00:55:19.280 Um so my my wife and I, we're we're expecting our sixth child. 00:55:19.519 --> 00:55:27.599 Uh my wife, who is a sometimes college professor, is is right now mostly a full-time mom by choice. 00:55:27.760 --> 00:55:29.039 And so I just asked. 00:55:29.280 --> 00:55:34.719 I asked out of curiosity to ChatGPT, uh, what do you think? 00:55:34.880 --> 00:55:43.679 Is it uh is a good idea for a woman in you know a monogamous, great marriage to be a full-time mom? 00:55:44.400 --> 00:55:52.000 And ChatGPT gave me all of these reasons about why this was a terrible idea. 00:55:52.719 --> 00:56:09.920 And so I said, okay, my wife, I said she was a college professor, she's a sociologist by training, and so I said, I want you to tell me what is the sociological paradigm that you are using to inform your answer. 00:56:10.159 --> 00:56:12.639 And they said, Oh, well, that's a great question. 00:56:12.880 --> 00:56:18.239 Uh, it's mostly conflict theory uh with a heavy feminist lens. 00:56:18.480 --> 00:56:20.880 And that's that's my priority. 00:56:22.400 --> 00:56:31.840 Now, nobody sat down and thought, okay, at some point in time, someone's gonna ask if they're gonna be a stay-at-home mom. 00:56:33.119 --> 00:56:38.320 But the way that these large language models and AI models are done is exactly what Jack said. 00:56:38.480 --> 00:56:39.679 Someone trains them. 00:56:40.719 --> 00:56:56.480 And my understanding is when you collect in in phase one, where you just collect everything that's on the internet and all of the garbage

that's on the internet, all of these large language models are the worst element of human beings. 00:56:57.119 --> 00:57:05.760 And then there's a huge operation to ask questions and and train it and say, no, don't ask that, don't say that. 00:57:05.840 --> 00:57:06.800 You can't say that. 00:57:07.039 --> 00:57:16.880 And it gradually absorbs the worldview that happens to be the worldview of Palo Alto. 00:57:17.679 --> 00:57:20.800 And the question is, do we do we do we trust that? 00:57:21.280 --> 00:57:23.679 Um you trust Palo Alto, Jack? 00:57:24.400 --> 00:57:26.480 Never known as a big Palo Alto, trust myself. 00:57:26.639 --> 00:57:30.639 I mean, if you want to write law review articles about this space, this is very fertile ground. 00:57:30.800 --> 00:57:34.000 Training a model is a legislative task. 00:57:34.159 --> 00:57:37.360 This is why they're hiring all these legal philosophers over at Claude. 00:57:37.440 --> 00:57:39.360 They got their own constitutionalist at Claude. 00:57:39.440 --> 00:57:42.880 It's already that, given how widespread the stuff is. 00:57:43.039 --> 00:57:44.000 There's an open question. 00:57:44.239 --> 00:57:48.559 How do you change weights and vectors in a legislative way to change outcomes? 00:57:48.639 --> 00:57:49.360 They clearly do this. 00:57:49.440 --> 00:57:53.360 You can't ask for bomb-making instructions where you could or somebody legislated that. 00:57:53.440 --> 00:57:56.559 And I was, I mean, corpus linguistics is made too. 00:57:56.719 --> 00:57:57.599 People made that. 00:57:57.760 --> 00:58:01.840 It's more scrutable in that, for one, they just don't have as many documents. 00:58:02.079 --> 00:58:02.960 You can see them all. 00:58:03.119 --> 00:58:10.880 They got a lot less fewer documents than Claude does, so it's it seems like a more manual process, and you do see this very the coding thing is more bespoke. 00:58:11.039 --> 00:58:11.840 You can see it play out. 00:58:12.000 --> 00:58:15.360 But when you let the human go into it, now you've got the flesh and bone machine coming in. 00:58:15.440 --> 00:58:17.760 This is what you always have to compare it against the human user. 00:58:17.920 --> 00:58:22.000 Once you bring in an entire judiciary, it's okay, you guys get it over the final step. 00:58:22.159 --> 00:58:25.199 All those same black box concerns apply. 00:58:25.360 --> 00:58:33.440 If you got a black box anyway, better than the black box that is a singularity that I can predict directly and that I can patch it all at once. 00:58:33.599 --> 00:58:39.280 To patch the human flesh and bone part of the corpus linguistics endeavor, well, you've got to take guys off courts, you gotta put them on. 00:58:39.360 --> 00:58:43.039 It's the Leonard Leo project suddenly, and that has had, you know, mixed results. 00:58:43.199 --> 00:58:47.199 Some guys don't do what you thought they were gonna do when you put that cog in the machine. 00:58:47.519 --> 00:58:52.000 Part of me kind of likes the one

thing that I can directly legislate through. 00:58:52.400 --> 00:58:58.159 I have to say, when I did my pr proto-corpus linguistics search, coding was the challenge. 00:58:58.320 --> 00:59:05.440 So I had two uh student uh uh student research assistants each code every instance separately. 00:59:05.519 --> 00:59:08.000 I gave them the categories and I said, you code them. 00:59:08.159 --> 00:59:16.320 And I basically wanted to know any usage they found that was contrary to what Justice Thomas and I thought might be the dominant usage. 00:59:16.480 --> 00:59:19.920 Um and but and even if it's borderline, they should report it. 00:59:20.079 --> 00:59:22.960 And it they each coded uh differently. 00:59:23.280 --> 00:59:39.119 Um one person was very generous and the other person was very stingy in terms of what were the outliers, and then but then but then the idea here is where they uh disagreed or where they produced these outliers, that those are the ones that I would take a look at and I would judge. 00:59:39.280 --> 00:59:45.519 So we had two independent coders, um, and then I would come in and break the ties by examining those sorts of things. 00:59:45.679 --> 00:59:46.719 Is that kind of thing done? 00:59:47.039 --> 00:59:50.800 Yeah, that's that's sort of uh I I guess you could say the gold standard. 00:59:51.039 --> 01:00:08.800 You have independent coding, and then usually what happens is you take a small set, uh, you know, maybe maybe it's 50 each, and they're they're coding the exact same, and then they unmass them and they talk about their differences. 01:00:09.760 --> 01:00:15.519 Uh now I did this, I've got a couple of papers where I've applied corpus linguistics to trademark law. 01:00:15.840 --> 01:00:16.960 And it was really interesting. 01:00:17.039 --> 01:00:24.480 We had four coders, two lawyers, two linguists, and we were trying to track whether genericide had happened. 01:00:24.960 --> 01:00:37.679 And what we found is that the linguists and the lawyers, the linguists always agreed with each other, and the lawyers always agreed with each other, and they were diametrically opposed to each other because we were using different standards. 01:00:38.079 --> 01:00:47.199 Uh the lawyers were looking for evidence of things having gone generic. 01:00:47.840 --> 01:00:52.880 Whereas the linguists were looking for evidence that it was still a brand. 01:00:53.039 --> 01:01:03.760 And we realized after we did that initial coding, whoa, we gotta we gotta define our terms and and really sort of put together a metric of exactly what we're looking for. 01:01:04.079 --> 01:01:13.360 And then after we did those initial, you know, 50, we then coded several hundred, and we had lots more agreement after that. 01:01:13.760 --> 01:01:14.880 Well, we're at the end of our time. 01:01:14.960 --> 01:01:19.199 I want to give each of our speakers like two minutes to sum up and then we'll thank them. 01:01:19.519 --> 01:01:23.599 I'll say that I kind of feel

like the guy who invented paper money when I give these talks. 01:01:23.679 --> 01:01:24.239 It's a hard sell. 01:01:24.480 --> 01:01:29.119 You're gonna say, we're gonna pretend that these 10 papers are equal to one ox. 01:01:29.280 --> 01:01:30.880 And the guy says, We are? 01:01:31.119 --> 01:01:31.599 That's not one ox. 01:01:32.000 --> 01:01:32.400 We're gonna pretend. 01:01:32.480 --> 01:01:33.280 He goes, Who's gonna pretend? 01:01:33.440 --> 01:01:37.360 We go, everybody! We're all gonna pretend that these are equal to one ox. 01:01:37.519 --> 01:01:39.199 That's a hard thing to say the first time. 01:01:39.280 --> 01:01:43.920 There's so much beautiful nuance to how one trades an ox for a chicken to all these various things. 01:01:44.000 --> 01:01:47.119 Trade and barter is a deeply human enterprise. 01:01:47.360 --> 01:01:55.440 But if you just listen to the crazy guy saying, take out the little 5%, you know, uh weirdness and just standardize it, you can have modern economics. 01:01:55.519 --> 01:02:02.960 You can trade these things like widgets, you can, you know, turn factual substrate into just a mercantile crop that you go harvest and say, are these win or lose? 01:02:03.119 --> 01:02:10.559 You solve access to justice crisis, you do all these things, you give people perfect knowledge up front, you don't have to pay for lawyers anymore, you live in the future. 01:02:10.800 --> 01:02:14.320 But there's still this push of, but no, there's a humanity in it. 01:02:14.480 --> 01:02:17.679 And is barter a better reflection of what things are worth? 01:02:17.840 --> 01:02:20.079 It probably is at scale. 01:02:20.239 --> 01:02:31.280 But if you just fudge a little bit and treat it like Lego block law, you open up a lot of doors I just don't think people are aware can open in a legal system. 01:02:31.679 --> 01:02:32.639 Final word, James. 01:02:32.800 --> 01:02:36.079 Yeah, so uh first I'm gonna just sort of emphasize what I'm not saying. 01:02:36.159 --> 01:02:37.280 I'm not a Luddite. 01:02:37.760 --> 01:02:39.440 I'm not anti-AI. 01:02:39.679 --> 01:02:41.679 I use AI every single day. 01:02:41.920 --> 01:02:48.239 I also want to emphasize that I agree with Randy that there will be AI applications to corpus linguistics. 01:02:48.559 --> 01:02:56.719 I do think that there are real, real scary implications to a judge AI world. 01:02:56.960 --> 01:03:05.519 Um every single AI model is hackable through a process called social engineering. 01:03:06.239 --> 01:03:07.599 Every single one. 01:03:07.840 --> 01:03:37.760 And if we have a world where we take out the human element and just turn it over to a machine, not only is Palo Alto going to be informing the values and the way that you you determine that, but it also will open up a cottage industry where rather than lawyers, we will have uh coders who are designing briefs to hack the AI arbitrator or the AI judge. 01:03:38.000 --> 01:03:39.039 And that's scary. 01:03:39.360 --> 01:03:40.880 Brush up on your common good. 01:03:47.840 --> 01:03:53.199 I'm frankly stunned that our most entertaining debate ever was about corpus linguistics. 01:03:56.159 --> 01:03:59.039 Um and then I also wanted to just say a quick plug. 01:03:59.119 --> 01:04:18.880 Um, if you like this kind of stuff, if you like these types of conversations about constitutional interpretation, I highly, highly recommend you apply for Bradley Scholars through the uh Center for the Constitution and apply for the originalism seminar, which this past year James Halpern was a speaker at. 01:04:19.039 --> 01:04:22.159 I don't know, maybe Jack will have to do a a session on AI. 01:04:27.519 --> 01:04:28.480 Thank you, everyone.