บทถอดความฉบับเต็มของ Raphaël Millière on large language models

an explosion of interest since around mid to late 2022, which coincides with the release of chat G B T maybe a little bit earlier than that. Uh, and I, I, I guess that's , um, you could argue , uh, that the interest has exploded because the capabilities have exploded, and it really seems like there has been a large increase in capability and capacity in the last few years. So what has happened in the last few years that , uh, we didn't see earlier? Um, I guess we, we can , maybe we could start off with , uh, I suppose a discussion of what LLMs are doing mm-hmm . , um, and how they actually work. Uh, but it , I'd be interested to understand, I guess, what's changed in the last few years in terms of architecture. I, I , just to prompt you, I know transformer architecture , uh, some have argued, has been a big, major change in from around 2017. So could you maybe give us a brief discussion of , uh, how large language models work , uh, if there's such a thing, and what's changed in the last few years to increase these capabilities? 00:06:53.959 --> 00:19:19.740 Yes. So , uh, in some sense the question, how do language models work and what are they really doing is the million dollar question that , uh, you know, we can , we could spend hours discussing. But I'll give you the , uh, the, the kind of first pass version of, of an answer. Uh , so just a , a brief historical overview , um, since the very beginning of, of , uh, you know, empirical work with, with computers in the very first computers were invented , uh, say in the 1940s, people have been interested in using , uh, this new technology to , um, try to process language in some ways and do, do things, automate some, some tasks , um, with language. For example , uh, one of the early research projects with , with the first computers pretend to machine translation, trying to build systems that could translate from one language another. Uh, and there was a big part of research in AI in the postwar post second World War period , uh, with obviously tremendous potential applications in various industries. And so , um, early on there was this kind of divide between two different approaches to this field of natural language processing with computers. One of them was the most symbolic approach that was heavily inspired by theoretical linguistics , uh, and the works of people like Nom Chomsky . And the idea that , uh, we could design systems that would , uh, um, draw inspiration from knowledge in theoretical linguistics about grammar, for example , to , uh, um, hand build some rules for these , these systems to process language programmatically. So, for example , uh, people at the time would design these syntactic parsers that would just , uh, uh, embed some , uh, grammatical rules , uh, provided by linguists , uh, to try to break down, to process , uh, uh, sentences , uh, break them down and , uh, try to interpret , uh, their meaning , uh, using these, these , you know , uh, handcrafted ad hoc rules , uh,

inspired from linguistics . The other paradigm that was developed in parallel , uh, you could call it the stochastic or statistical approach , um, was trying to do things in a different way instead of using this kind of top-down hand-built set of rules , uh, trying to , uh, work from the bottom up , uh, with a large amount of data, with a corpus of text, and try to learn from the statistical properties of this corpus , uh, without this , this hand handbuilt rules. And so, initially, that work was done more in the, in the field of speech recognition. But then , uh, I'll spare you the, the , the , the details of, of all this history. It's a very interesting , uh, uh, history. But , uh, uh, eventually there was this , uh, this idea that emerged as a very powerful , uh, intuition , uh, that people call the distributional hypothesis. Uh , is, is the idea that the meaning of a word , uh, is , uh, determined by the context in which the word , uh, appears. So it's , that was initially developed in by structural linguists in the fifties. Uh, one of the big slogans , uh, from, you know, summarizing this kind of approaches , uh, you shall know a word by the company keeps . And this article kind of percolated into the statistical approach to an LP with the idea that we can just , um, train , uh, or, or, you know, learn, have a model that learns from a large corpus, from the statistical distribution of words, not corpus to , uh, kind of induce or learn some , uh, information about the meaning of words and the syntactical roles of, of words. Uh, and that's essentially the intuition that, that , uh, led with a lot of intermediate steps to current language models. So the way in which we get to current language models is that , uh, people started to try to operationalize that kind of intuition around the , uh, a set of models that would try to , uh, use that single properties of words , uh, co current statistics. So, for example, the fact that the word apple and pear both occur very frequently next to the word juice , uh, that tells you that , uh, that must be something in common in the meaning of apple and pear . In fact, they're above fruits and they're above fruits that , that , uh, uh, can be juiced and can be , uh, used for , for , uh, to flavor juices. Uh, it's just one example of how the patterns of distribution of words and, and the , uh, the , the company that words keep , uh, can , uh, uh, give you a lot of information about what the words mean. And so , uh, people realize that you could use that approach looking at a statical approaches of words to try to represent the meaning of words as vectors in a high dimensional vector space. Um, so we don't need to get too deep into the , the weights here, but , uh, that's a very powerful idea, because it means that then you can , uh, look , uh, you know, represent , um, uh, words as these vectors where each dimension of the multidimensional vector space in which , uh, these vectors , uh, are , uh, placed , um,

captures some aspect of the meaning , uh, and potentially syntactic roles of words. And then you can look at the distance between two word vectors, the vectors that stand for two different words, to give you some insight into the, the semantic distance between two words. So two, two vectors that are close together in the space , uh, uh, like , uh, the vector for apple. And the vector for pair would be for two words that are very close together in terms of a meaning. And vectors that are far away would be vik victors , uh, vik victors for words that are very different. Um, so people started working on these world emitting models. Uh, a big one was what to VEC in 2013 , uh, that really showed the power of this kind of approach , uh, uh, after decades of it being more of a, you know , uh, kind of proof of concept time models, it started to show real promise , uh, around that time. And then finally, we get to , uh, the modern language models , uh, basically with this , uh, turning point in 2017 that I mentioned already with this paper, attention is all you need by Ian , colleagues that extended the kind of approach to the modeling of sequences, not just words. So obviously, language is more than just a collection of words when you take a sentence. Uh, the order in which the words appear is important sentence is not just bags of words , uh, to use a term that's used in an rp. Uh, and so if you want to model the whole sequence, you have to also model the relationships within the words in that sequence, including the grammatical structure of the sentence. And , uh, this paper from 30 17 introduced this transformer architecture that turns out to be a really powerful neural network architecture. So it's an architecture for an artificial neural network that can learn from data. And it has this , uh, this mechanism called attention that basically en enables the model to , uh, attend to different kinds of relationships between words or to model different kinds of relationship between words, including syntactic or grammatical , uh, uh, relationships between words such as subject verbal agreement , for example , uh, on many others. So really what it does is that it enables these models to , uh, as it were, learn the rules of grammar , uh, and learns all sorts of other can of relationships between words and sequences in sentences such that these models with, you know, this one version of this family of transformer models is enabled to generate text, to produce text. And so that gave rise to this , um, family of G p T models. G P T just trained stands for generative pre-trained transformer. So you have this transformer architecture there in the name, it is generative because it can generate texts, so you feed it some text, and then you can continue that text , uh, generate new text, or just answer questions. And it is pre-trained because you , uh, pre-train it on a massive amount of data, a whole, a subset of the whole internet, a really large corpus with all of

Wikipedia, hundreds of thousands of books and millions of webpages. And once it's been pre-trained in this way , uh, it can perform a bunch of different tasks just by having the user ask in the input, in the prompt to the model, ask the model to do something, a certain, certain tasks. It could be summarize this paragraph or answer that question, or , uh, write a story about something, write a new story about something. And these models tend out to be really good at doing that, even though they hadn't really been pre-trained specifically for these tasks. So the pre-training objective, what we call the learning objective in national learning for these models , uh, again, by being trained on this whole very large corpus , uh, script from the internet, is very simple, is just next work prediction. So the models will sample sequences of text from this corpus and try to predict which word follows from that sequence. And it does that over and over again, millions of times. And at the beginning, it's, it's no better than chance. It's really bad at it. Uh, it just predicts random words, and as it goes, it gets better and better, because every time it makes this prediction of the word that follows a sequence that's randomly sampled from this very large corpus, there is a method, a procedure to calculate how wrong it was, so how far off it was with its prediction, and then use that measured error to correct, to update the weights of the model . So it's like adjusting the knobs inside the model to make it a little bit better. So every time it predicts a word, it doesn't get it very right. You can adjust the knobs slightly, you know , uh, cor course correct , get the model to be a little bit better, and then do this millions of times. And eventually you get your pre-train model that is really, really good at NEXTWork prediction. Um , and the thing that no one really saw coming, I think , uh, 10 years ago, people would've thought would be crazy, is that just by doing this, just by playing this NEXTWork prediction game, you can end up with a model if it's big enough and trend enough data, like GT three, GT four , that can do a really remarkable , uh, range of things , uh, that maybe you wouldn't have thought , uh, abstractly without having seen this result that you could do just by playing this next prediction game. So sometimes people will say things like, these models are just fancy to complete just doing export prediction, as if that's kind of a limitation on the range of things , um, they can do , uh, once they've been pre-trained. But it's important to keep in mind that in the process of doing export prediction so well on so many different linguistic contexts. So again, if you think the model is sampling sequences from the training data, it will sample some sequences from books, some sequences from WPI articles, some sequences from , uh, read threats of discussion of threats . It will have to be really good at export prediction in virtually any kind of context on any kind of topic. And in order to become so good , uh, in all of these different contexts. In fact, these models are superhuman and , and export prediction compared to humans to do much better at predicting the next word, presumably, that forces the model in order to become so good to learn a lot in the process, right? Including learning how to produce grammatical sentences that are not just gibberish, but also learning a lot of world knowledge, learning, a lot of knowledge about how , uh, things are in the world , uh, such that the syntheses , uh, are not, you know, nonsensical, even if they're grammatically well formed , that they actually , uh, you know, if you want to predict the next word correctly, you have to know a lot about what is the most plausible next word, based on how the world actually works. Uh, and all sorts of other kinds of knowledge that people are still kind of trying to probe and find out exactly how much these models know, how much they can do, what kind of capacities we can ascribe to them. But the, the, the, the bottom line is how these models work. Short answer is by being pre-trained with this next work prediction, learning objective, just playing this next work prediction game on the large amount of data. A long answer is in that process, they seem to learn a lot in the process of doing export prediction very well, and researchers are still trying to find out exactly what kinds of , uh, capacities or computations they learn in that process, and people disagree about that. 00:19:20.069 --> 00:19:44.579 Great. Thank you. I think that was a , a really good primer to go a little bit deeper. I think you've already touched on this a little bit, but I know that you've said previously there is a tendency for polarization in talking about large language models , uh, between two very strong sets of claims , uh, which we've called deflationary and inflationary claims. So I'd just like to give you an opportunity to , uh, explain what those claims are and , uh, why you think those are problems. 00:19:45.180 --> 00:25:02.779 Yes . So that's something that I think is very salient in the current discourse on language models. And you see that if you, you know, go in the opinion section of the New York Times and u CSAs , uh, op-eds on, on chat , G P T essentially, you know, a lot of them will be either saying something like, the headline will be something like, you know, Chad Pity is , uh, it's is not smart at all. It's as dumb as a toaster. It's , uh, it's not what you think it is. It's again, just next for prediction or FTO complete . It's a cheap trick. And , uh, it is , uh, uh, it is not an, it is not a , uh, uh, uh, it is not making any progress towards generating artificial intelligence. Or you will be, say , you will see people saying that , uh, Chad g p t and similar models are harbinger of human-like intelligence, and that they're on the verge of, you know , uh, breaking through , uh, to, you know, artificial

general intelligence and maybe even superhuman intelligence. And that, you know, just a few years we might be facing existential risk from super intelligent ai , uh, that , uh, threatens to eradicate the human species. So you get these really radical views that are extremely polarized on buff extremes of the spectrum. So either these models are not intelligent at all , uh, they're really dumb . They , during this really cheap trick of next word prediction, and some people say the stochastic para , such as haphazardly stitched together bits and pieces from the training data without producing anything novel or anything , uh, that is deserving of, of, of the, of the description of any psychological capacity or cognitive capacity, or they exhibit sparks of artificial general intelligence. And they're almost, almost there, almost almost , uh, at the level, at human level on various tasks. And I find that a little frustrating as a researcher because I think , um, there is a huge rich , uh, middle ground between these two extremes that hasn't been as present in the public discourse, at least. I mean, I , I , of course, a lot of serious researchers are working on charting that middle ground and taking various positions , uh, in between. Uh, but if you look again at op-eds, public op-eds, people on Twitter, things like that , uh, including the outputs of some researchers, it tends to be quite polarized. And I think that's, that's a shame because I personally think the , uh, the truth is somewhere in between. Uh , these models are, as I said, just because they're doing expert prediction, doesn't tell you really that much about what they learned to do in the , in the process of doing expert prediction. So, well , uh, it seems that it do genuinely acquire some non-trivial capacities in that process, but they're also severally limited in other ways. And certainly I think they're extremely far from something like , uh, general human-like intelligence in various respects. But the devil is in the detail, and I, I think it's , um, I understand why people tend to gravitate towards these polarized positions because there's something comforting about being able to, to argue for a very determinate stance about what these models are and what they can do. Uh, either, you know, taking on this very deflationary tech and saying , well, they're just fancy to complete. They can't do , uh, much just cheap tricks. So , you know, you have this sense that you, you, you kind of really know what , uh, what, what , what heights be beyond the , the hype that is ped by , uh, you know, venture capitalists and some companies. And you can say, well , uh, don't be fool . This is what these models really do. Uh, or on the other hand of the spectrum, people , uh, you know, thinking that , uh, um, they , they've uncovered the true , uh, capacities of these models that , uh, would be close to human, human intelligence in various respects . And also, there , there's something contorting about feeling that you, you are lucid about what these models can do, and you have, you have the answer. And the truth is, I think people are still kind of stumbling , uh, in the dark as it were in this line of research, trying to find out what these models can and cannot do, how they work and reverse engineer the computations that they , they , they induce during training. And that's a long and painstaking process, as is often the case in science. And I think , uh, you know, we should be more kind of practical and persist about this and to perform experiments and gradually learn more and more about how these models work. But I think no one really has a fully worked out answer just yet. And that's not to say that, you know, anything goes, and that perhaps we'll find out. These models are basically, it's , it's not just to , to kind of endorse a kind of mystical line of, of , uh, discourse in these models, that it's to say that, oh , these are alien intelligences and we don't really understand how they work . They might be really advanced, and we just don't know . That's not what I'm saying, that clearly , uh, you can probe their behavior and see that they have other limitations. So it's not to say that, it's just to say that to really truly fully understand how they achieve the behaviors that they can achieve, which is quite remarkable, and , uh, how similar , uh, the , uh, mechanisms that they've implemented to achieve that behavior might be to the mechanisms that , uh, human brains or animal brains , uh, uh, implement in order to achieve some of their intelligent behaviors. That's a , a bit of an open question, so we should be pragmatic about this. 00:25:03.490 --> 00:25:49.579 Yeah, unfortunately , uh, it's not too surprising to me that we see such polarization in something like a New York Times op-ed. Um, I guess that applies to many topics. Uh, I guess reasonable , uh, opinion pieces don't sell too well. And we don't know how many reasonable opinion pieces were submitted to New York Times. They just chose the, the most polarized. But , uh, I guess I'd like to hear a little bit more about what the dangers are of under claiming and overclaiming. So Samuel Melman's paper, I'm, which I'm sure you're familiar with , uh, dangers of under claiming the talks about this. Uh, but I , I , I guess you could argue that , um, Murray Shanahan's paper talking about large language models maybe is more on the dangers of Overclaiming side. Uh, so what are the, the risks, I guess, with under claiming or overclaiming? 00:25:50.368 --> 00:33:16.099 Yeah, so given this , this prioritization of this course and the fact that , um, on the one hand you have these very deflationary text , on the other hand, it's very, oh , inflationary text . If you think that both of these , uh, extreme views are , uh, mistaken in some ways that , uh, uh, you know , uh, one of them is, is not giving enough credit to what this language most can do. The other one is, is , uh, uh,

inflating what they can do and giving way to, to , uh, hype. I think there are dangers to, to both excesses so dangerous to under claiming what these models can do and dangerous to over claiming what they can do. Uh, we can start with, with the risks of Overclaiming, because I think they're , they're the most salient. Um , certainly the , uh, there have been , there has been a lot of breathless hype about these language morals lately, especially since the release of Chad g PT and the love that hype is fueled by , uh, economic incentives. There's a lot of money to be made. There are now big companies like Open AI and others , uh, that have a lot of skin in the game. And , uh, of course, it's, it's partly in their interest to hype up , uh, these systems and to kind of dangle in front of people the promise of something like , uh, agi, artificial general intelligence, which is explicitly the goal of , uh, open ai, for example, to build a gi We can come back to that. I have some, some qualms about this very notion of agi. I think it's not always a very coherent notion , uh, or at least in the way it's used by some of these companies. But giving that aside , uh, I think, I think there is this dynamic where , uh, some of these companies, the venture capitalists that invest in them have this incentive to hype up these systems. And there's a risk here because , uh, essentially you , uh, end up having people misrepresenting what these models can really do, and convincing people who don't know any better because they don't work in that field, and they don't , uh, fully understand as far as anyone can really fully understand how these models work. They don't even know the very basics. And because it feels, it can feel, of course, borderline magical to interact with these systems. They're very impressive. Uh, and because we humans have a , a remarkable tendency for anthropomorphism when interacting with systems that can converse fluidly , fluently in language, it's very easy for people to just buy wholesale into this idea that these models are harbinger of superior or human, or human-like intelligence. There's this old , you know , uh, phenomenon in, in research in AI that people call the Eli I effect. Uh, dating back to the , uh, uh, observation of Joseph Eisenberg at m I t uh, in the sixties, he built this chatbot called Eliza that was a very rudimentary chatbot compared to what chatt can do , uh, using a bunch of kind of clever tricks , uh, using , uh, handmade rules that you could talk to it, write on a keyboard in which respond to you in a somewhat MRUs elusive way. Uh , but people back there , back then in , uh, in the sixties on the MIT campus were really quick to start , uh, treating the system and talking to it as if they were talking to another human. So people are really quick to project certain anthropomorphic traits onto the systems and all the more of language models . So I think that's the clear risk of Overclaiming is , uh, to essentially , uh, mislead people with all sorts of potential nefarious, don't seem consequences behind people as scrubbing too much weight to what these models produce , uh, giving too much credence to some of the things they say. And sometimes they , uh, they can generate , uh, uh, answers that are factually incorrect. They can generate dangerous answers, they can , uh, do all sorts of problematic things. Uh, and also you have companies that go even further and , uh, I think are really, have a really unethical business model, which is to try to market AI companions. Uh, so I think of companies like replica that , uh, will explicitly market these , uh, you know , use fine tune versions of language models like, like GP three to sell access to these , these so-called quote unquote virtual companions are just, you know, dressed up language models that , um, uh, they sell as the market as , uh, AI companions that really care that you can confine in, that you can treat as a friend, a mentor, or a consular or even a , a romantic partner or sexual partner, explicitly pushing that really anthropomorphic angle. So I think that's, I mean , it's already, you see people already, if you go online, people falling in love with these systems or at least , uh, reporting that they do. And I think that's , uh, that's , uh, disaster waiting to happen. So that's the danger of , of a claiming under claiming on the other hand , um, well , a lot of people who I think might be erring on the side of under claiming, are actually motivated by ethical concerns precisely because they're, they want to resist over claiming they want to resist the hype. So I think it comes from, it comes from a good place , is it , where it comes from a , from a concern for the risks of overclaiming, but I think , uh, they are overcorrecting in the other direction in a lot of cases. And I think that comes with its own risks, including ethical concerns, which , uh, is, is a shame because, you know , if you're concerned about AI ethics, you should really be concerned about calibrating your characterization of , uh, what current systems can and cannot do in the right way so that you can inform the public as well as possible about potential risks. Uh, and if you err on the sign of under claiming and you say, these are, this is just false fancy to complete, it's really no smarter than , you know , to complete feature on your phone and you write a text or it's just a stochastic power that's just powering sentences from the training data, well, you, you will , um, kind of create a blind spot in people's mind if they, if they buy into that picture wholesale where they are not seeing , uh, the potential disruptive applications of the systems and the things, the things that can, that can already or real soon be able to do. And again, that's not to say that we're on the verge of anything like super intelligent AI or that , uh, concerns about existential threats to humanity's existence are , uh, uh, well placed

necessarily. I, I'm , uh, rather skeptical about that. Uh, but , um, it's more that there are everyday concerns, you know, more , more mundane risks that are nonetheless very concerning that , uh, are on the horizon about , uh, whether it's about using language models for misinformation or , or manipulation of people , uh, impersonation in ways that , uh, are , you know, that mirror stochastic power or a language model that , uh, would be just to kind of fancy the , the complete feature like on your phone, wouldn't be able to do so . Uh, uh, I think on the horizon are sophisticated forms of misuse or risks of language models that are really tied to the kinds of cognitive capacities you can ascribe to them, or I would say proto intelligent behavior. You can't ascribe to them , uh, where the, the, you know, the , these problematic behaviors precisely come from the fact that they are quite capable in some respects. They're not just these dumb , uh, you know, dumb stochastic parts that some people say they are. So in a , of that's, I think, the risks. And I think we, we ought to map out the middle ground , uh, not just if we want to gain a better theoretical understanding of these models, which is a good and interesting goal in your son , right ? But also , uh, if we want to take the full measure of, of , uh, these , uh, uh, minefield of ethical issues. 00:33:16.549 --> 00:33:40.819 Great. Thanks for that. Raphael. There's a long list of things that ais might have, like compositionally intelligence and so on. Uh, so I'm curious to ask about , uh, one by one , go through these , uh, ask about what they are, do AI have these, are they important for ais? But before we go through them one at a time, do you have a general map of how you see these different characteristics or some other way to summarize them that might be useful? 00:33:41.380 --> 00:38:12.750 Yeah , so generally, I like to think about this as , um, we have these systems, right, that , uh, artificial systems, and we want to know what kind of capacities we can ascribe to them. And specifically we want to know in particular , uh, when it comes to discussions about intelligence or , um, comparisons to humans , uh, the human mind, we want to know what kind of psychological or cognitive capacity we can ascribe to them. Uh, so we humans have a whole suite of cognitive capacities or psychological capacities, and we have different concepts to talk about them and carve them out, some of which are very broad concepts, very vague concepts like intelligence or , uh, consciousness , uh, uh, uh, or just a very concept of a mind. It's a very kind of fuzzy concepts with blurry boundaries. Uh, and we ascribe, we , we apply these concepts also to non-human animals , uh, to various , uh, species of animals that , uh, we deem more or less intelligent. Uh, we talked , uh, about animal minds or animal cognition. Uh, we talk about remote consciousness as well. And then we have some concepts that are a bit more specific. For example, we , um, when it comes to human cognition, talk about certain specific faculties or capacities like memory, long-term memory, short term memory, you can make furthers sub distinctions, capacity for thinking, capacity for reasoning , uh, which might be slightly different capacities for , uh, language understanding, understanding language, but also understanding the world around us, visual understanding , uh, perceiving the world. We have all of these different terms that we apply to, to , uh, carve out different aspects of human cognition and perception, and generally the human mind. And again, we apply these terms more or less liberally, liberally or conservatively to different non-human animal species, depending on where you stand. And we have a long, traditional, long history of trying to do that in a more principled, empirical way , uh, in the field of comparative psychology, comparing animal cognition and human cognition. So now we have these new systems that exhibit be exhibit behavior that up until now was the kind of behavior that , uh, only humans , uh, could exhibit , uh, first and foremost speaking, producing language. So up , uh, until recently, fluidly, fluently producing language in the way that , uh, systems like church , you can , uh, do was a clear sign of a whole suite of cognitive capacities were scrub to humans. Only humans could do that. And the capacity to, to , to , to generate this kind of fluid , uh, fluent , similar intelligent language was something that , uh, was always associated with , uh, everything else that was with human minds. And so now we have this problem that we have the systems that can do that, but for the first time seem to come into question the association between the capacity for language production and , and all of these other capacities. Um, so the way I like to think about the question you just asked is, let's take this on a , on a case by case basis , um, have kind of divide and concur approach where we take these different capacities, these different psychological , uh, properties that we can ascribe to humans and maybe, maybe we can ascribe to non-human animals and on a case by case basis, decide, okay, based on the evidence and based on the specific definition of this capacity, that has to be precise, specific enough that we can discuss it without , uh, falling into merely verbal dispute that we don't talk past each other. Can we reasonably ascribe that particular capacity to language models? Uh, and of course, often the, oftentimes the answer is not gonna be that clear cut right now because it's still , uh, early days, and , uh, this line of research is still emerging. But that's the kind of approach I like to, to, to take. So , uh, uh, and it's helpful then for , to go from the, from the very general psychological properties like intelligence that are very vague and not necessarily very helpful or , or understanding. Another one, understanding is, you know, it can mean

very many different things, and people mean mean very many different things by this term . And so they tend to talk past each other. So go from these to more specific sub distinctions , uh, of more specific kinds of capacities where we have a , uh, perhaps a better idea and a better empirical grip on to kind of arbitrate between , uh, uh, different positions where people disagree on whether we can ascribe the capacity to language models. 00:38:13.739 --> 00:38:30.800 So let's, let's start with intelligence , uh, which you've already mentioned. Maybe we could start with the definition of what intelligence means in the context of ai. And I know you've said that it's, it's , uh, maybe hard to pin down exactly what that means, but , um, your favorite definition is, is fine just to, just to get this rolling. Yeah. 00:38:30.860 --> 00:44:24.500 So that's, that's a really hard question. I mean, for various reasons, also, because, so intelligence, as we said, is a , is a vague concept. It is also a somewhat problematic , uh, concept that is laden with all this , uh, historical baggage, including this , uh, uh, really unfortunate baggage from the, the , the atoms to measure intelligence and psychometrics that is really historically intertwined with, with racist ideas about , uh, intelligence. So, for example, the whole history of the , uh, of, of the, of the iq , uh, uh, measurement , uh, uh, is, is intimately tied with , uh, with extremely pragmatic , uh, ideas about, about what constitutes intelligence and, and that were motivated initially by , uh, by eugenicists , uh, and , uh, uh, racialist , uh, um, uh, ideology. So it's, it's problematic for these two reasons. So we , when we want to operationalize notion of intelligence, we, we want to, to be able to measure it in a more objective way. And often that leads to things like the IQ measurement that , uh, kind of centralize specific manifestations of intelligence in a way that , uh, really reduce the range of things we consider to be , uh, in intelligence in, in a somewhat problematic way. Uh, and the same is true when we look at, say, human animal comparisons. And , uh, we kind of essentialize certain human manifestations of, of, of intelligence to deny intelligence to certain animal species where , uh, intelligence might be manifested in different ways through different behaviors in , in , in species that don't have things like language, for example. So these are complicated questions. Uh, I do like this paper by Francois Charle , uh, called on the Measure of Intelligence. That was a really interesting paper trying to breach this kind of historical theoretical dis discussions about intelligence , uh, with discussions about how could we measure intelligence in AI systems. And , uh, he comes up with this , um, pretty broad characterization of intelligence. It's been a really long time since I actually read about the paper, but, but , um, if I recall correctly, the 30 definition goes something like this. So intelligence is characterized by, by two things. So , um, the scope of applicability of kind of , uh, general problem solving skills. So systems will manifest our intelligence by being able to solve problems. And the range of problems that they're able to solve , uh, is one dimension of intelligence. This the scope of applicability of their intelligent problems, of skills. A system, for example, like , uh, deep blue that , uh, was seasoned by IBM to play chess and beat Kerry Kasparov , uh, uh, the world chess champion at the time, chess playing system. That's a narrow system that has a very narrow scope of applicability , uh, right? So people call it narrow AI because , uh, it, it's really designed to do just one thing and do it pretty well, but just do one thing, which is plain chess, and so can only solve a very narrow range of problems. So that's one dimension. And I could ensure that there is this other dimension, which is , uh, kind of the generalization power of the system . So how quickly and efficiently , uh, can it generalize to unseen situations , uh, on unseen tasks, anden problems where I'll just, you know, generalize from one domain to another. So if it's learned to , uh, solve a problem in a specific domain, and it's suddenly put in a completely new situation or a new domain that hasn't encountered before, how efficiently can it generalize to solving tasks in that domain? So say for example, you, you train a robot , uh, you build a robot that , uh, is designed to, to make coffee, and you , uh, train it, you have it learn how to make coffee in a specific kitchen where things are placed in certain ways. There's a specific layout of the kitchen, and there's a specific coffee maker, coffee grinder, the bin are in the place and so on. It might become very good at that task, right? And make coffee super quick, excellent coffee every time, very consistent, much better than a human. But then suppose you take that robot and place it in a totally new kitchen where things are different places, layout is different, the beans are different. The, the , the , the , the purchase is different. It might be that your robot is going to completely fail and not be able to perform even the basic first steps to make coffee just because it's not able to generalize to this new situation. So we, humans are very good at generalizing, and children from a very young age are extremely good at generalizing to novel situations very quickly. So we're very efficient in this way, and that's one of the key characteristics of human intelligence. And to a large extent, you know, non-human animals also are very good at that , uh, not as good as humans because we have an extreme form of generalization where we are able to generalize to completely new domains that would range all the way to putting a man on the moon by building a rockets, things that no one has ever done before. So animals don't generalize that far, but it can generalize to novel situations , uh,

generally better than , uh, AI systems. So I like this kind of approach to intelligence because it's quite broad. It's not taking , uh, a really specific stance about , uh, I mean, the paper goes on about to , to discuss how we could measure intelligence, but that's an , that's different question, but you know, it's not, it's not reducing intelligence to a very narrow , uh, kind of , um, range of problem solving behavior that the kind of thing that would be measured by the IQ test . And it's broad enough to be applicable to various kinds of systems, including humans, non-human animals, and artificial intelligence systems. 00:44:25.559 --> 00:44:57.539 Listeners might be aware that , uh, recently GPT four was able to beat 90% of lawyers , uh, on a bar exam, for example. Um, and there are many more examples of , uh, various l LMS performing quite well on , uh, written exams. What does test scores like a 90th percentile on the bar exam mean for AI compared to what they mean for human, A human? So I guess , uh, and more generally , um, when an AI performs as well as a human on one of these narrow tasks, what does that mean for intelligence? 00:44:58.440 --> 00:53:25.380 That's a really great question , uh, that a few thing I I would like to say here. So the first , uh, is just to , to, to tell your audience. There was this seeming lip between GT three and GT four , like there was between GT two and GT three , ban AI released GT four , um, just a few weeks ago. And , um, uh, they showed that , uh, indeed this new model , uh, performs really well on various exams that were designed for humans cleaning the bar exam and various other exams where GPT three was not doing that well. So GPT three , um, perhaps was in their , um, um, 30th percentile or something like that. And , uh, GPT four on a lot of these exams is in the 90th percentile, so performing as well or better than 90% of , uh, humans. So that's at least seemingly on the face of it. Very impressive and it is impressive, and I don't want to take away from how impressive it is, but , uh, there are two, I think two main concerns about these results. So one first concern is about this , this problem called , uh, data contamination, which is that current systems like for are trend on so much data cuz uh, they are extremely data hungry and they need a lot of text to perform well. Uh, and the trend has been , uh, from , uh, you know, the first consumer models in 20 17, 20 18 up to GP four that we train bigger models. So we have , uh, bigger neural networks and we train them on more data. These two things have to scale together. So the more, the bigger your model, the more data you need to train it on. And we realized we need a lot of data to get good models, and by a lot , it's really hard to wrap your head around it. It's , uh, more than a trillion words , uh, in the case of systems like GPT four . So again, it's a significant subset of the whole internet. And then there is even speculation that , uh,

we're running out of data, but just scraping the open web for text. And that open AI had to use , uh, their in-house transcription algorithm called whisper to transcribe things like perhaps podcasts , uh, or YouTube , uh, videos to get even more text, more words to train the models on. So it's certain , a humongous amount of data and that data will basically encompass anything that humans talk about. And so it becomes harder and harder to rule out that the model has been trained on some of the things that you want to test it on. And that would include , uh, questions for an exam. Of course, there are many, many websites where people will discuss examples of questions from various exams. And , um, that's a problem if you want to claim that your system has , uh, achieved excellent performance on an exam because it could be that a system has memorized the right answers to certain questions. And this is also very good at memorization. They are generally very , uh, lazy in the sense that if they can solve a problem by just brute forcing it and memorizing the answer, they will do it. So , um, um, you find that there is a lot of memorization going on and that's, that's a feature and not a bug in a lot of cases because it means that they know about a lot of world knowledge because they just memorized that and they know about, say, the great work , great works of literature because they memorized that. And so , uh, if you ask GP four to recite the pram , uh, you know, the Tiger by John Kitts , it will do it because it's memorized it, but you don't want, when you're testing the model's capacity to have it just regurgitate , uh, and sources memorized and , uh, data contamination and leakage of the training data, that's very testing data into the training data is a very hard problem to mitigate , uh, because the corpus is so large that it's really hard to actually check that what you're testing the system on hasn't occurred anywhere in the training data, including in some variations . So maybe, you know, someone, maybe there's a question in the bar exam that's not quite the same as question that've been asked in previous years. It's phrased in a different way, somewhat, slightly different, but it's still similar enough that having encountered the slight variation of the question from a previous year is gonna help you a lot to answer that question. So opening I took some precautions , uh, in a technical report to GBT to GP four , they talk about it, that they tried to control for that, but , uh, it's debatable how well they were able to control for that. And , uh, it's a really interesting question actually at which, you know, at where do you , where do you draw the line about which when it becomes cheating for model to have memorized some variations on a question because obviously, you know, humans also cram when they , uh, uh, you know, pass exams and they also learn from previous exam questions and things like that. Uh, but still it's a concern. But the second, and perhaps even more substantive concern is about how to interpret good performance on an exam that was designed for humans and not for LLMs, not for language models. And there we're hitting on this really big important issue in , uh, the philosophy , uh, of , uh, AI and discourse and AI generally, which is this question of the, the gap between performance and competence. So one thing you can , uh, investigate when you look at the behavior of the model by testing it on exams, on benchmarks and so on, and looking at the quality of the outputs is the performance of the model. How well is it doing, how, how well is its core at the exam, how well is its core on the benchmark? That's all about the performance of the model. And so you can compar models, language models in humans in terms of that performance, right? But by doing that, you don't, you don't just care about the performance, the exams core , what you really care about is the , is the underlying competence that is supposed to be measured by the performance. So with humans, you know that the bar exam is supposed to measure human competence at certain , uh, you know , uh, legal knowledge, for example. Uh, and you want to know that , uh, a lawyer is going to have that knowledge and, and that , uh, you , you , you , you, you would hope that a lawyer that passes the bar , uh, that is good evidence for an underlying competence , uh, at being a lawyer or an incompetence in the law. Um, but that's an assumption. And you need to , uh, uh, kind of empirically validate that assumption by validating the construct validity of your exam or benchmark that was gonna tell you, well, that's a well-formed, well-designed exam or benchmark that really gets at the underlying competence. And by the way, there , there could be concerns about that even with humans, because could be that , uh, you know , some exams, I mean, I , I , I'm , I'm a professor teacher, I, I , um, you know, I'm , I'm well aware of that with , with students. Sometimes you design an , uh, a question for an exam and you realize it can be brute force in some ways, or there's a shortcut that students could learn to get good performance on that question that is not actually indicative of students having really learned the , uh, the thing you wanted them to learn having acquired the underlying competence. So that's not just a consent for models, but it's a , especially in concern when you wanna translate, you know, take an example what isn't for humans and just apply to language models without further scrutiny or reflection of what this means. And so, models getting good performance an exam doesn't necessarily mean that they have the same competence that would enable good performance in that exam for humans, because it might be that they solve that exam in a different way than humans do it. Uh , again, with some memorization, with some shortcut learning, with some brute forcing of ice heuristic and tricks that , uh, humans might not act ,

uh, use themselves because , uh, these two things, humans in language ones work very differently. So that's, that's the two of the main concerns. I mean, they're connected to some extent, and it's not to take away from how impressive it is that G D four can pass all these exams, but it's just to con contextualize that , and I think much more research will need to be done to fully probe , uh, how to understand how to interpret these results. And part of the problem here is that the trading data for G four is not public. In fact, there is hardly any information publicly available about what the model is, even what , uh, what is the architecture, what is the training data, how we were trained , uh, and that makes it really hard to, to conduct rigorous research about this kinda question. Now 00:53:25.380 --> 00:53:36.579 Might be a good time, I think, to talk briefly about artificial gender intelligence , uh, which you've already mentioned a little bit. But , uh, first of all , um, how, how exactly do we define artificial gender intelligence? 00:53:38.349 --> 01:00:34.340 So I, I think this is one of these notions that , uh, it's not always very helpful because I think people, a lot of people assume that , uh, it refers to a specific determinate thing , um, uh, when in fact I think people , um, have mean different things by the term. And, and , uh, and generally I think the term might be a little unhelpful if, if , uh, if it is coherent at all. So , uh, artificial general intelligence, the , the very broad way to characterize it is by contrast with artificial narrow intelligence. So again, think of the scope of applicability of the system. If a system is designed to only play chess and can only play chess and do nothing else, it's a very narrow system. And by contrast, humans and non-human animals, in fact , uh, to a large extent, are able to solve a very broad range of problems. So the intelligence that humans have that many non-human animals have to a certain extent , uh, seems very general in comparison to say a chess playing system. Um, so by contrast with narrow systems, narrow ai, we talk about agi, artificial general intelligence to refer to this , uh, goal of creating an artificial intelligence systems that would have a more much broader range of applicability, scope of applicability in terms of its problem solving behavior than humans , uh, sorry, than current systems do . Um, there are different concerns though, with this notion. So , uh, one of them is that it's a very context sensitive notion because the very notion of generality is context sensitive. Uh, it's a vague predicate. And what I mean by that is that , uh, take for example, the concept of , uh, the , the , the , the adjective toll . Um, so you can talk about a toll building, but if I go to a small village in Italy and I, I refer to a toll building in that village , uh, or I come, you know, where I live here in New York City and talk about the toll building, I'm gonna refer to very different things. What's a toll building in a small village is very different from what's a toll building in New York. Um, and , uh, and the reason is just context sensitive, right? Uh, it depends on the height of the surrounding buildings and it might not really make sense to talk about at all building in absolute terms , uh, without having that kind of context again, because it depends what your reference point is. And similarly for the context concept of, of agi , um, what seems like a , a general system in some contexts in comparison with other systems might not be so general in comparison with others. So , uh, certainly the problem solving skills or the intelligence of a rat seems extremely general compared to that of something like a chess playing , uh, uh, algorithm. But compared to humans, it doesn't seem general at all. And so that's one of the concerns people have is that , um, it's not really clear that there is a line in the sand, in the sand that would be the point at which we reach agi, where all of a sudden we go from a narrow system to an a , to a general system. Uh, it's more of this spectrum, this gradient of generality , uh, where systems can have a , a more narrow or a broader scope of applicability of their , uh, problem solving behavior and general intelligent behavior. So, for example, companies like Open AI talk about agi more like , uh, sometimes more like a goal where there's a clear line in the sense . Uh, so for example, in a recent statement, they talk about the first, when we reached the first AGI system, the first agi . And it's somewhat unclear for me what the first AGI would be, probably would be a matter of interpretation. You know, some people might already, some people say that GPT four, they think of it like a form of agi and it's a bit meaningless cuz it's like, well, yes, it's, it seems arguably more general in its , uh, in its scope of applicability than , uh, GT three is , uh, it's also much less general , uh, to, to , um, to really understate it. I mean, it's really, really much , much less general than humans , uh, in terms of risk capability . Um, so, so that's one concern. No, no bright light in the sand. Uh, there is a broader concern that people say, well, there's no such thing as general intelligence simpl error . There's just, it is just an incoherent notion. Uh, and what people have in mind when they say these things , uh, would be something a few different things. So one , there is this theorem in computer science and not freelance theor that says that , uh, any , uh, algorithm can learn effectively, must be somewhat specialized. You must have some inductive biases to, to, to learn , uh, certain solutions to certain problems. You can't have a maximally general , uh, universal intelligence in that sense. Uh, that that's , uh, you know, it's, it's linked to this mathematical theorem . And, and, and , uh, you know, there's a sense in which the , of a maximally general or universal intelligence

is, is somewhat in Korea from that perspective. And then there is a more , um, specific concern, which is that , uh, even if you live at as sad and if you take human intelligence, well , even human intelligence is not that general in some sense, it's been , it's been molded by evolution to feed a certain niche and to apply to certain specific set of problems that humans are concerned with. And , um, and so in that sense, it's not, it's not general , not that general, it's been, it's been kind of finely tuned to the range of problems that human enc , encount humans encounter. And , uh, all the algorithms, non-human animals have intelligence that has been tuned for there an ecological niche and is different. And , uh, perhaps there is a sense in which it might be a little misleading to say to, to set up the comparison between humans and sometimes human animals just by saying humans have a marginal intelligence. It might be, you know, if you talk to some biologists , it might be, well , perhaps it's more informative to say that humans have intelligence that's more tailored to their niche and they've evolved in a certain way, and that led them to be able to do all sorts of things that , uh, is unhelpful and , uh, you know, for other organisms in other niche , uh, to, to evolve towards. Um, so that's another concern that, you know, this se setting up this whole discussion in terms of this single dimension of generality of intelligence is overly reductionists. And , um, that , uh, it leads to, you know, it is almost, it's almost kind of reproducing this whole idea for medieval philosophy of the scale and asura this kind of scale of nature where you have, you know, humans at the top and like worms at the bottom or something, and maybe God above humans. And it's all on this single continuum, which , uh, where you have superior beings and inferior beings. And , uh, that's very p problematic for various reasons. 01:00:35.000 --> 01:01:09.099 It sounds like , uh, the , with these claims of , uh, GPT four , maybe having early signs of artificial gender intelligence, and I'm not sure if you've seen this paper Sparks of artificial gender intelligence or experiments , the GPT four . It sounds like you're quite bearish on this though. Could you , uh, uh, maybe explain a little bit more why you don't think this is sparks of agi? And I , I know you've already talked about why the definition of AGI is , uh, quite vague and up to open to interpretation. But what makes you, I guess, what , what do you think makes you more bearish on this than say the authors of this paper? Right. 01:01:09.340 --> 01:06:10.489 So I would say that , um, it's not so much that I'm bearish on the outlook of this paper as much as I, I think it's not the most helpful way to set up the discussion about the capacities of language models having this kind of dichotomy, again, very polarized dichotomy between these either stochastic powers or sparks of hgi . Um, and instead, again, I would advocate for this divide and conquer approach where we take things on a, on a case by case basis, we take different capacities and we say, where do language must fall? Uh, with respect to the capacity? Do they have anything like the human-like version of that capacity? Do they have something different? Can we get more specific by looking at the computations that these models learn to implement during training and try to reverse engineer what's going on inside. So the sparks of AGI paper, you know, one of my qualms with that paper as I think many other people's qualms about it , is that it's , it's, it's , um, it, it diverges quite a bit from the normal standards of , uh, say, rigorous scientific research about language models. Uh, it's just , um, a paper that's very long paper, in fact, that's , uh, almost 150 pages that was written by researchers on Microsoft that got access to GT four , six months, I think before the release, the unveiling of the model , uh, and got to test it in a bunch of different , uh, tasks. And so the, the , the paper is , uh, first of all, it's a pre-print, so it's not peer reviewed. Uh , and it's, it's this kind of collection of somewhat anecdotal tests where it just, you know, tested the model on some things I thought were interesting and reported , uh, the performance of the model. Uh, but there is little in the paper that is really systematic , uh, even in terms of behavioral testing. So , uh, testing on, on, on rigorous well designed benchmarks. So it's more giving examples of successes and failures, but mostly successes of the model. And, and so I think many people are concerned that it was pandering to the hype a little bit because, you know, it's hard to resist the impression reading the , the paper that's saying, look how cool, like , uh, gyp can do, GP four can do that, they can do that, they can do this. And as a researcher you read that and it's a bit like, okay , uh, yes, it's , uh, it's, it's intriguing. It's just saying , now is this , uh, is this , uh, consistent? Can the model , uh, if I change the prompt, can it do just as well? Uh , is it, is it systematic? What does it actually mean in terms of the underlying capacity of the model? Can we test that more systematically? And can we go beyond behavior by opening up the model, looking inside, trying to figure out how the model is doing via all of these behaviors that I think is the most interesting line of research, not just , not just, you know, not just looking at behavior, and especially not just cherry picking some anecdotal results and saying, look how cool. And the problem here is that, again, GP four is not a very open system. There is an API that no people can use to test the model, but , um, one of the big problems is that you can't, you can't go beyond the api, so you can , you don't access the weights of the model. So you can actually probe the model and use various techniques to go beyond the behavior, but also even just looking at behavior,

even if you , even if you just want to systematically evaluate the , the performance of the model on a benchmark, for example, or on a specific set of prompts, open AI keeps updating it on the back end . And so what you see a lot nowadays is that someone will point out a failure mode of the model on Twitter, say, and then a week later you try the very same prompt and the model will succeed and give you the right output. And people have speculated with good reason that probably OpenAI is patching the model, playing kind of failure mode guacamole on the back end , uh, and silently patching the model as they go. So even with , um, even with the , uh, the box of a GI paper, it's kind of tricky because , uh, you know, some people have tried some of the pros they use and it didn't work with the model that we have access to , to through the api. They say it wasn't the same model . So it's very opaque, basically. And , uh, and there is, you know, it , it kind of straddles the line for me and for many other people where it's not really, it's kind of hard to see that as scientific research at this point. It's more like , uh, almost like , uh, product advertisement , uh, for something that's no longer a research artifact, but more of a , of a product being marketed. So that's, that's , uh, my, my broader concern. But, but you know, in terms of the being bearish , I mean, no, I , I , I, I do think that there are highly nontrivial capacities that these systems, especially GT four have clearly that it's not just a sarcastic part . And so I just fall somewhere in between. But again, I think we need to rigorous research, we need to progress with this kind of reis mindset where we rigorously test things, we keep an open mind and we do do things on a case by case basis. 01:06:11.610 --> 01:06:44.369 I want to talk a little bit about consciousness , uh, and maybe sentience as well. Now. Uh, so I guess I could ask about the definition of consciousness. And are large language models conscious now? Uh , can they be , uh, and of course, those are two very different questions. Um , but it's also interesting to me that with large language models, we seem to focus more on , uh, consciousness than sentience. And I'm just curious about if you have any thoughts about why that is, but maybe we can just start with a definition of consciousness, and do you think large language models are conscious and can they be? 01:06:45.699 --> 01:17:58.670 So consciousness is, is another one of these big words that , um, are a little vague. Uh, often people don't necessarily have a very specific definition in mind. Uh , uh, rely more on intuitive grasp of the concept or, or , uh, uh, might , uh, disagree and, and, and to verbal disputes or talking past each other because they don't, they don't intend the same thing by the term. Generally in philosophy, when people talk about consciousness , uh, they refer to subjective experience. Now you might say, what is subjective experience? And here

, uh, is the problem, which is that , uh, famously it's hard to give a non circle definition of consciousness. So best thing you can , you can do is try to kind of appeal to what , uh, uh, your point to certain things that you, you hope that everyone has an intuitive familiarity with from just being a human being that , uh, experiences the world. So you can say, well , uh, consciousness is that thing that is present, you know, when you wake up in the morning, but is lost in dreamless, dreamless , uh, deep sleep or in the coma. It's that things that manifested , uh, whenever you have any kind of experience, whether it's tasting , uh, uh, an apple , uh, or looking at a sunset , uh, and it's a subjective feel of , uh, all of these different experiences, what it , what it is like, that's what the term that feels the first generally used , what it's like to undergo a certain experience. And , uh, you know, presumably , uh, there is nothing, it's like to be a rock, because rocks don't have subjective experiences. And so that's why we said rocks are not conscious. And there is something it's like to be a human , uh, at least when we are awake, when we are not , uh, in a deep dreamless sleep or in a coma. And presumably also, there is something it's like to be , uh, uh, uh, many non-human animal species , uh, although it might be hard to exactly know what it's like , uh, from the inside as it were. Uh , there's a famous paper from Thomas Nagel , or what is it like to be a bat that makes this point , right? It's kind of hard to know what it's like to be a bad , because , or sensory approaches is very different from that . So sometimes people draw this distinction between consciousness and sentience, but a lot of people in philosophy , uh, use these two terms interchangeably. And in fact, I myself, in my work , have used these two terms interchangeably . But when people draw a distinction, generally, they think that consciousness is perhaps a slightly higher bar and slightly more sophisticated capacity. So sentience would just be the capacity to experience valence , uh, states such as pleasure and pain. So , uh, the capacity to have certain experiences that have , uh, subjectively valuable or dis valuable, that , uh, feel good or bad, for example , uh, and perhaps consciousness if you draw that kind of distinction, might be something a little more sophisticated that might involve perhaps having a sense of self or having the capacity to , uh, be aware of your experiences as you undergo them, perhaps , uh, uh, reflect upon them, have some kind of metacognition, something like that. So there are values, views there, but generally when people want to distinguish consciousness and sentience, that's roughly how they think of the distinction. And so you might say, well, maybe some very basic simple non-human animal species, like said , species of of worms or , or mosks of aaba might have sentient in the sense that they can experience pleasure or pain if you harm

them, they might experience, but maybe they don't have consciousness in the full-blown sense in which we humans have conscious experiences that have a , it's kinda a rich tapestry of experiences that include sense of self. And , uh, again , a metaawareness of our experiences and all of these things. But I personally generally prefer to just talk about consciousness, broadly speaking as a single thing and without distinction of sentence and consciousness to avoid verbal disputes, avoid, avoid this problem that people mean different things. And just stick to this very, very broad definition. Consciousness is just having subjective experience, having something, it's like to be something. And so , uh, in that sense, if the worm is capable of feeling pain, that that counts in my book as being conscious. And then you can have a further discussion about whether there might be degrees of consciousness. It could be more or less conscious, but at least there's some, the lights are on in some sense. There's something it's like to be that, that animal . Okay. So now language models, are they conscious? Do they have, is there anything it's like to be GT four ? That's a bit of a fr discussion. It's really coming to the LA night recently because , uh, especially when this, this engineer from Google, Blake Lemoine , uh, started claiming that the , uh, in-house chatbots called Lambda was , uh, sentient and , uh, really kind of tried to blew the whistle about this ended up being fired from Google subsequently , uh, and there was, there were a lot of very sensationalistic press media article about this. It made the buzz. I think it's a little bit unfortunate how this whole discussion has been set up, because Blake Leman made a lot of baseless claims based on his interactions with the model. It's worth mentioning , uh, that he was not himself involved in the design of Lambda. He was not, you know, an expert in the design of the systems. He just was, had internal access at Google. And so starting playing with it, and he ended up , uh, forming the beliefs that Lambda was sentient as he confessed himself, or, or just acknowledged based on his , uh, deep religious beliefs . So he's a profoundly religious man, and his , uh, his conclusion that Lambda was sentient wasn't really based on the scientific method, but more , uh, premised on, on certain religious beliefs. And if you look at his interactions with them, that he asked a lot of leading questions. And the thing about language models is that they are very good at mimicking all sorts of , uh, linguistic behavior. There's, I have this, this piece that I wrote for not till this last summer, where I say, you know, instead of calling them stochastic paras , you could , you might call them stochastic chameleons, and maybe that's a better, that's accurate term, which is they can kind of , kind of blend in different sorts of linguistic concepts and, and mimic different kinds of agents including , uh, engaging in creative fiction. They're very good at that, and that's one of the best uses for language we have today. And if you prime them by asking them, you know, what do you feel? Or , you know, do you have any experiences and what it's like to be you, what is it like to be you? Well, of course, they're gonna engage in that mode because they've been trained on a bunch of literature. They've been trained on so many books and web webpages and texts that describes subjective experiences. And so they're perfectly good at , uh, engaging in that kind of mimicry and describing experiences that humans have and humans have described in the training data. Um, that obviously doesn't entail that they themselves experienced these experiences that they have , uh, subjective experiences and feelings. So you shouldn't take what they say at face value , uh, that goes for , um, checking the factual accuracy of statements they make about the world. Like when , uh, you ask them about factual statements and they give you an answer, you should always fact check that. And that goes for, especially when you ask them about their own experience , uh, because this is a leading question that will just lead them to confabulate various , various claims about when they feel. So that's really a very, very poor source of evidence about, about whether language models have conscious experiences or not. But the problem is that until now, verbal reports was one of the best sources of evidence about whether someone experiences something and what, what do they experience. That's what we do in a lot of experiments in psychology. We, we ask participants how they, what they felt with questionnaires and interviews and so on. So it works for humans. It doesn't work for language models . Uh , yeah, there is a double dissociation between , uh, linguistic behavior and consciousness, where on the one hand, we animals non-human animals, we see that they can't speak, they can't just tell us what they feel, but that doesn't necessarily mean that they don't feel anything. And in fact, the trend has been that we've realized both on the behavior and, and decades of studies in animal, on animal cognition, that a lot of animals probably have a rich type of sphere of experience, and that has all sort of ethical implications for , uh, factory farming and what we, how we treat these animals if they can feel pain. So we shouldn't discount them just because they can speak , uh, um, uh, they probably can experience a lot. And conversely, language models can speak in some sense, it can generate language, but that doesn't mean that they experience anything. So there's double dissociation that's really selling these days. Uh, so do they experience something? What all the source of evidence could we look at? And that's where things get really tricky, because we don't have a fully worked out scientific theory of consciousness at this point. And , uh, I think we still very far from that. So we don't, we really don't have a good

principled scientific way to arbitrate this question of other language models have , uh, conscious expenses . The best we can do is look at the current leading theories of consciousness and see whether language models , uh, have any of the requirements according to these various theories for consciousness. And if you look at that, well, it's extremely, extremely unlikely that current language models have conscious experiences because the architecture is extremely simple compared to , uh, you know, the architecture of the human brain or animal brains. And in particular, some of the features of animal brains or , or human brains that seem to be quite important with this and correlated to experiences, especially in the human case. We can look at the structures that tend to support consciousness in the lab. There is a long , uh, now history of neuroscientists trying to as to investigate the neural correlates of consciousness in the lab. You can see that , uh, things like recurrent processing, having the capacity to process information through recurrent feedback loops seems to be quite important, for example. And current language models are purely fit forward networks and don't have any recurrent loops. It's one example that don't seem to have any kind of capacity for metacognition in the way that , uh, humans can , and , and , uh, and the list goes on. So for all, for various reasons. And if your , if your , if your audience is , is interested in this, I recommend there is a good , uh, talk by David Chalmers , my colleague from NYU that's on YouTube , uh, on that question , uh, where he goes through that kind of consideration. But based on this evidence, I would say it's, it's extremely unlikely that current systems are conscious. Uh, could there , could they , sorry, could they one day be conscious, become conscious, presumably, yes. I mean, I don't see any reason to to doubt that unless you have a very strong view that says that consciousness capacity for consciousness is , is intrinsically tied to the biological makeup of , uh, animal and human brains. There's something specific about carbon based life form that you can't replicate on a silicon chip that you couldn't, you couldn't realize in a different kind of hardware. Uh, but if you don't hold that view, then , uh, yes , presumably someday we could get there, but I don't think we're there yet. 01:17:58.680 --> 01:18:43.630 Great. Thanks. And we'll throw up a link to , uh, David Chamas talk on in the show notes and also any other paper , uh, that we've referred to as well be there . Uh, moving on to theory of mind. So this is the capacity , uh, to understand other people by scribing mental states to them. Uh, children typically don't get this into around four to five years old, arguably. And a classic example of this is you might ask , uh, a child what they're doing during a phone call , uh, and they'll say they're playing with this , uh, not, they assume that you know what this is, even though

you can't see it. Uh, so they don't have , um, a theory of mind. Do , uh, do large language models have theory of mind? Is this important? Uh , how would we, how would we measure this? 01:18:44.699 --> 01:24:15.659 Yeah, so that's a very hotly debated question these days , um, because , uh, you know, GP four came out and , uh, even before that, GP 3.5 was already , uh, displaying this wide array of impressive behavior. And so some people were keen to check whether , uh, you could describe anything like a theory of mind to the systems. Um, so the capacity to , uh, uh, kind of model the psychological states of other agents and keep track of them and draw inferences about them. And this is quite important because, you know , uh, many people think that , uh, something like a fear of mine is, is , uh, is a vital part of the way in which we , uh, understand language in conversation because , um, the meaning of sentences that are uttered in a conversation beyond the face value, meaning of the sentence, there are a lot of pragmatic aspects to language that stem from inferences about what the other speaker means. So, for example , uh, if you ask me if I'm coming to the party on Friday, and I tell you, no, I have some student essays to grade, sorry, I don't say no, I say , uh, I have some student essays to grade. I haven't told you yes or no, whether I'm gonna come to the party. And in fact, my answer doesn't strictly entail that I'm not coming. I could have some student essays to grade, but still decide to come anyway. But there is an implicator there, what linguists call an implic ature, which is that I'm not, that I'm not coming. That's an example where you kind of have to infer from what I'm saying about what I'm intend to convey to communicate in the conversation. And there are more sophisticated, pragmatic , uh, phenomena , uh, that influenced understanding of meaning that seemed to be tied with , uh, in theory of mind inferences about the psychological states of the person you're speaking to , uh, what they want, what they believe , uh, in conversation. So if you're interested in language understanding, in language models, that seems to be also like a piece of the puzzle that's interesting to investigate. And generally, it's , uh, it's an important part of research on , uh, intelligent behavior in humans and non-human animals. So people have tried to , uh, take tests that have been used to assess , uh, capacities for theory of minds, say in children, and apply these tests to , uh, to language rules . So that would be , uh, things like little stories where , um, you , um, tell the person, the child language model about , um, assisting little scenario where , uh, two , there are two different agents and one has access to certain information that the other agent doesn't have access to . Uh, and , uh, and you , uh, ask a question that , uh, where it seems like giving a right answer would, would hinge on accurately tracking the states, the mental states

of the two agents, and knowing that one has certain beliefs and the other one doesn't. Um, so kind of tracking what people these agents know in the story. Um, so this paper that came out a few weeks back , um, I think from, from Kosinski from Stanford that was trying to , um, well , was claiming that the theory of mine has emerged in language models like GT 3.5 and g4 . And it was using this standardized, you know, standard , um, kind of task , uh, to probe that again in humans. Um, the problem is, that's a great example of this kind of performance competence distinction we discussed earlier. The question is, you know, the , the , so the , the , the finding was that the language models were doing really well. And so the , those researcher concluded, well, they must have induced like a , formed a fear of mind , uh, during training that it has emerged. It's an emerging capacity of these models. But actually , um, another researcher , uh, to Holman , uh, wrote this preprint , uh, in a direct answer to that particular work where he took the same stimuli, the same , um, um, uh, tasks , prompts that were used , uh, to probe the capacities of, of , uh, language models was with the theory of mine. And he , he just makes some tweaks to , uh, add some distractors or some, some , uh, kind of , um, um, uh, differences that , uh, for humans wouldn't make a big difference about how we, how we parse the prompt and, and the kind of answer we gave , but was able to totally throw off , throw off the, the language models where their performance totally broke down when you just introduced these tweaks in the , in the prompts. And that's pretty good evidence that , uh, it , it was a case of, of gap between performance and competence, where , uh, on the initial tasks the humans and language models were doing just as well in terms of performance. But the underlying competence was there for the humans, but not there for the language models . And just varying controlling for the experiment by introducing some variations in the prompts and some tweaks reveals that gap. So the bottom line is , I think currently the evidence shows suggests that no , uh, a three year of mine hasn't emerged in language models . Uh, they probably don't have, they , they're probably missing some, some components to the architecture to get there. Um, uh, so something missing probably from the systems to get there, but , uh, uh, I mean, we'll see. Time will tell, but, but it's a good lesson in, in trying to , uh, you know, how, how can we set up good rigorous experiments to test , uh, language models on, on certain tasks that humans have been tested on. You were 01:24:15.680 --> 01:24:22.020 On a panel recently talking about the challenge of compositional for ai. So what is compositionally in ai? 01:24:23.039 --> 01:30:40.930 So compositional, it is a , is a notion that comes from linguistics, and it's this idea that , uh, the meaning of a , of a whole, of a complex expression, like a sentence, is , uh, determined by , uh, the meaning of it if it's constituent parts, like the individual words in the sentence, together with a way in which they are syntactically combined. So the grammatical structure of the sentence, for example. So indeed, you know , uh, when you're listening to me speak, for example, I'm , uh, uttering sentences and the way in which you're understanding the sentences I'm , I'm measuring involves knowing the meaning of the individual words that I'm using, but also how to put them together into coherent, coherent holes based on the grammatical structure of the sentences. And , uh, the structure matters, of course, because , uh, if you say , uh, uh, uh, John loves Mary , that doesn't mean the same thing as Mary loves John, even though you're using the very same words. So , um, when it comes to ai , um, essentially , uh, there has been a long history of people thinking that the kinds of systems that current language models are based on in the artificial neural networks or collection systems are pretty bad at , uh, parsing complex expressions, compositionally or generally at , uh, combining simple , uh, representations into complex representation with this kind of compositional structure. So this is idea that compositional it is a challenge for language models and more generally for AI systems that are based on these artificial neural networks. Uh, it's not just a problem for discussions of language processing, it's more broadly a problem for trying to build artificial intelligence, generally, because people have , uh, extended discussion of compositional it not just to apply to language in the way we humans understand language and produce language by combining complex , uh, simple , uh, expressions, but also , um, apply that notion to thinking through thought as well, right? So it seems like thought as well has this compositional property where to think complex thoughts and to understand complex thoughts, we're putting together simpler representation , simpler concepts, and combining them in certain ways that have a certain compositional structure, a bit like the grammar of sentences. And again, this idea that , uh, we humans do that very well, we can do that in systematic ways, such that if I have the concepts, John marry and love the lo loving , uh, I can think John loves Mary , but I can also think Mary loves John and , um, artificial neural networks because of the way they work. And then the way in which the format in which their representations , uh, uh, are couched , which is , uh, with vector continuous vector representations , uh, it was traditionally thought they don't really have the resources to do that kind of thing to combine representations together with this kind of systematic structure, because to combine representations, they can blend vectors together, but that doesn't seem to have the right kind of structure. And , um, well , my own work, I have this project where I've been suggesting that , uh, this is a

little , this is , this is too simplistic, and that actually current language models and current deep learning architectures generally , um, seem to , uh, actually be able to , um, have, you know, form representations that have a rich compositional structure. And that goes a long way to, towards explaining why they can generate grammatical well-formed sentences. For example, in the case of language models, if you talk to a language model as everyone has who has talked to Taji , PTAN knows it's, it's, it's , uh, rarely generating a non grammatical sentence. Uh , so it's very good at putting words together into coherent, grammatically well-formed expressions, including totally interval sentences that it's never encountered before. Then it's just making up , uh, so it's not just brute force memorization. So the compositional performance seems to be there. There's also a lot of empirical evidence about compositional generalization of these models and various other pieces of evidence, but there's also work that goes beyond behavior that opens up these systems and tries to reverse engineer what's going on inside. And that's the kind of evidence I'm building on in this work that suggests to me that they're actually able to, to , to , to form these mechanisms to combine representations together, compositionally, but in a way that is different from the way in which classical symbolic system, these old call systems that we used to use before the advent of neur networks achieved that. So , uh, old school systems to form a complex representation, they would just concatenate simple representations into the complex representation . So , uh, uh, say if you want have system that , uh, processes or forms of complex representation, John loves marry , that will literally involve taking the representation for John, the representation for Mari and the representation for love, and concatenate them together , uh, with a certain structure and the complex representation. John loves marry , and by contrast, connection is systems like neural networks. They perform manipul , they , they perform transformations of vectors, so they take vectors and they modify them in certain ways. But what i, I , I don't wanna get too much to the details because that would involve a lot of neural algebra, and some are pretty , uh, technical empirical evidence, but I think the way in which they do these manipulations of vectors and these transformations in the transformer architecture actually affords them with the capacity to have the right kind of structure to , uh, uh, to generate sophisticated compositional behavior, but in a way that doesn't reduce to this classical symbolic old school kind of architectures. So it's not just merely implementing a classical symbolic architecture, it's doing something different. And I think that's a really interesting finding, because maybe it might open up new avenues of research to think about how we humans actually are able to achieve the compositional

behavior that we exhibit , uh, when we produce novel sentences or understand sentences. 01:30:41.319 --> 01:30:50.810 Yeah. And sort of related to that , uh, I guess is language understanding in large language models , uh, do large language models understand language in any meaningful sense? 01:30:51.319 --> 01:38:00.020 Yeah, that's also a very big question. Uh, and yet another one of these words understanding, which is , uh, fraught with , uh, verbal disputes and generally vagueness. So the way I like to put this , uh, I try not to be too long on this, but is try to put this notion of medicine to the side because it's confounded by associations people, for example, some people think of understanding as requiring consciousness, like understand language. You need to have an experience, a feeling of understanding. And I wanna put that aside because I think even if you think that language monos don't have any capacity for consciousness, you might still meaningfully ask about where they're able to , uh, in a, in a, in a more narrow sense, understand parse the meaning of sentences in some sense. So I'd like to talk about semantic competence instead of understanding to try to rule out these associations. Uh, and then you can further break this down. So you can talk about lexical semantic competence, which is about understanding word meanings or, you know , uh, having the right kind of capacity to having the right kind of knowledge about word, meaning the meaning of words or compositional Semitic competence or structural Semitic competence, which is about understanding , uh, complex expressions like sentences or paragraphs. And , um, you can also break, have , make another distinction between what you might call referential semantic competence, which is the capacity to , uh, connect words to what they refer to out there in the world. For example , uh, if I'm , uh, in the streets and I see a pigeon or a dog, I can point to it and recognize it and, and use the word dog or pigeon in connection to that thing I see. So I can connect that work to what I'm seeing out there in the world. Or if you , uh, if we're we're at dinner and you , uh, you ask me, you know, can you pass me the salt? I'll be able to pick up the, the , the shaker, the salt shaker and give it to you. So I connect that word salt to things out there in the world. And there is this other notion of inferential semantic competence that pertains to not connecting words or linguistic items to whatever they refer to in the world, but connecting words to each other. That's what we do. When we, for example, define words, we use more words to explicitly at the meaning of a word. So it's a word to word relation , uh, definitions is one example, or synonyms. If you , if i , if you ask me to , uh, give you a synonym for a word, I give you another word that's a word to word relation, paraphrasing, summarizing all of these things , uh, kind of involve having this network curve , uh, relationship between words. Um, so language models , uh, I would argue, and I, and I, and I have some work in which I , I , I do argue that , uh, have a really highly nonfic degree of, of inferential competence, at least when it comes to word meanings. So because they're trained to , uh, they learn from the statistical distribution of words in a very large corpus, they learn alsos of relationship between words that are exactly the kind of relationship that are useful to the use in definitions, paraphrase, summarization , uh, pseudonym and so on. And so that enables these systems to , uh, exhibit a highly neutral degree, in my opinion, of inferential semantic competence , uh, to have these very fine grade networks of relationship between words that map onto relationship between vectors that represent the words in the vector space that these systems use to generate, generate words. Uh, when it comes to referential components , it's a much more controversial question, because the systems don't have access to the external world, and therefore the , uh, degree to which you could say that they're able to connect the linguistic items they work with to whatever these items refer to in the external world, is very debatable and very controversial. I actually did argue in a recent preprint that I wrote with my colleague Dimitri Colo , that they might qualify , uh, as having some form of , uh, referential competence , uh, um, that hinges on a somewhat complex argument. Uh, but the idea is that , uh, some systems like shed g pt , they're not just trained from NEXTWork prediction. They also, after they're trained, pre-trained on NEXTWork prediction, they're fine tuned with human feedback. So they undergo this like further little bit of training where you ask the model to generate texts based on prompts. Like you ask them question, for example, questions about various things, and you ask humans to provide feedback about of different outputs, which one was the best, which one was the most informative, helpful, harmless, truthful. So that includes a normative component , which is truthfulness or truthness or honesty, how accurate the , uh, outputs of the model where , and you then fine tune the model on that. So it's not just export prediction, you're kind of infusing the model with a function that's not just a , a linguistic function like export prediction, but that's actually world involving cuz you provide feedback to the model on the accuracy of it output that's dependent on the actual state of the world. If you ask the model what's the capital of France ? And it tells you Berlin , uh, that's a wrong answer, and you give it feedback about the inaccuracy. So we argue based on this somewhat technical argument that this might be a way in which the model can actually have a learning process that has the right kind of descriptive normativity to , uh, um, have, you know, to, to achieve some form of, of referential competence. Uh, because you're providing feedback about the reference ,

uh, in there indirectly the reference of the world of the , of the words in the world , uh, and accuracy condition for that , that kind of reference. Uh, it's not the kind of direct competence we have because we interact directly with the world and we perceive the world and work around in the world, and we do all these things. Uh, it's much more mediated and indirect, but nonetheless , uh, we think it's there , uh, in a limited degree. Uh, so that's a long-winded answer. And so that's, that's, I've talked mostly about lexical competence, so what pertains to word meaning, but , uh, that would translate also to structural competence because the systems can very well actually put together words into sentences that are well formed , coherent, and so on. Uh, and so and so, yes, I would argue that also translate to some form of, of structural or compositional semantic competence. So the bottom line is do they have human-like language understanding? No, clearly. I mean, there are various failure modes. They don't understand language as humans do. They don't have, they're still limited with all the, the pragmatic dimension of language, certainly with the theory of mind thing we've discussed before. And they just don't learn language in the way humans do. Uh, children don't learn language by reading thousands of books , uh, to learn the meaning of , of the word dog. Uh , they , they , you know, people point out the dog and say, that's a dog. And you know, they have this kind of very pre perceptively grounded approach to language learning , uh, so that they're work very differently. They don't learn language different the same way, and they have , uh, a much more limited understanding of language. But nonetheless , uh, if you think that it's a matter of degree and not just an , uh, uh, an either or , uh, a dichotomy, I would argue they have some limited but non-trivial degree of semantic components 01:38:00.840 --> 01:38:16.699 In terms of reference now that we're attaching things like vision to large language models, is this all moot? Is , is that going to be enough to, to give that reference, for example, the GT four , uh, having vision attached in some way? Uh, how , how does that change this discussion, if at all? 01:38:16.930 --> 01:41:09.619 Yeah, I think it's a , it's a , it's an interesting and somewhat complicated question because some , sometimes people assume that just hooking up this language, windows two , uh, images , uh, in addition to text, is just gonna be enough to , uh, to, to, to bridge that gap in terms of referential competence. Um, and actually what we argue in that paper I mentioned that I wrote with my colleague , uh, Dimitri Colo , is that there is , uh, a double dis association between having the access two images and being able to achieve referential grounding or kind of ref referential competence. The reason for that is that a system like , um, like Dali two , it learns by being trained on cap pairs of captions and images. Uh,

so it learns by being trained on images and together with a caption, but it's not actually getting any feedback about really about what words refer to out there in the world. You can think of the images that the , uh, model is fed to as just more tokens that the , uh, being made of more tokens that the, that, that the , uh, model is trained on. So if you think that, that the words of the English language are , and the words that sentences are made of are tokens for the model to process, and images also are broken down into patches that correspond to tokens, like , uh, a vocabulary for images, like a , like a , kind of like visual words as it were for the model is just tokens all the way down. So the model is , is learning to associate certain visual tokens. We've word tokens, but it's still , uh, arguably stay at the level of this kind of inferential competence of relating tokens to other tokens without exiting this kind of merry-go-round of , uh, intra token relationships , uh, in order to ground representations external world. And by contrast, chat g pt, when it's fine tuned with this explicit human feedback, even though it doesn't have access to images , uh, we argue it might be achieving some limited form of ential components just by virtue of getting this direct human feedback. So somewhat contour intuitively, you might think that, well, adding images to the systems is what gives them reference, refresh , grounding and solve the grounding problem. And we say, actually, no, that , uh, that these are two different things. I mean, you can put them together and that's ideal. So what you would want to do ideally is having systems that , uh, would learn the meaning of words by interacting with the world where they would get some feedback from the world about what words mean. That wouldn't be just, you know, passively learning to , uh, match caption and images, for example , which is very different from how humans would learn the meaning of words. Uh, so that would get you closer to something like the, the human like , uh, referential competence for , uh, the use of various words. 01:41:10.460 --> 01:41:41.659 So we talk , we've talked about a number of different characteristics that large language models might have, and there are many more that we , um, didn't get a chance to talk about, but to try and tie this discussion together, which of these faculties are most useful to understand? Now, of course, that depends on what we're interested in. Um , but for, for example, if we wanted to know whether artificial gender intelligence was here or if it's close, what, which of these faculties might we look at which are most useful, and maybe for some other, for some other situations, which of these are most useful to understand? 01:41:42.119 --> 01:47:23.859 So there are a number of ways in which current models are still very limited. One of them is that they , uh, undergo this training phase. They get trained on a lot of data, and by the way, they require an enormous amount of data to, to, to learn, to perform well, much more than children , uh, actually need . So there are significantly less efficient , uh, at learning. And here there might be some connection between learning efficiency and intelligence. I think , again, of the , the form social point about power, generalization power being related to intelligence. So , so that's in and out of itself is a limitation. So they learn from the sum amount of data, and then once they're trained, they are frozen as people say. So they no longer learn. The weights of the model don't change, and you can just run inference on the system by prompting it and generating outputs, which does not, is not associated with any updating of the model. So there's no more learning, there is no more , uh, evolution of the system. It's totally frozen in that sense. And that's extremely different from the way in which , uh, biological systems leave evolve and learn and , uh, acquire , uh, intelligent capacities. Humans and non-human animals are capable of lifelong learning, as people say. So that just means that we, we constantly learning and adapting our knowledge and representations of the world based on our experiences and based on incoming , uh, simuli based on , uh, what we do and say and think and so on. And these language models don't. So that's one major limitation in my opinion. Having systems that can learn on an ongoing basis would be interesting. Uh, another major limitation that these systems don't really have anything like , uh, the kinds of, of memory capacities that humans have in the sense that they don't really have a long-term memory in the same way that we do. We can store things, experiences in long-term memory that's somewhat related to the point about lifelong learning in the sense that when you talk to the chat bot , say at the end of the conversation, he's gonna forget everything about what you've talked about. There's no long-term storage there , there is a kind of root force approximation of a long-term memory just because during the training phase, the model can just memorize whole sequences of text. But beyond that, there is no equivalent to how long-term memory works in human and unhuman animals. It's also debatable to what extent there is anything like the short, short-term memory in the way that it works in, in humans and non-human animals where, you know, these models can memorize the content of the prompt once in the context window that you use to prompt them and back and forth in the conversation for a few thousand words. But that's not quite the same as , uh, the way in which working memory works , uh, in humans , uh, and og human animals. So that's another, another , uh, uh, limitation in my opinion. Um, uh, of course you want also systems that , uh, don't just learn from text, but uh , learn from other modalities. And we, we are getting closer to that. We have nowadays a lot of systems that also learn from images. We are

slowly getting system that might start being able to learn from videos, but that's a much ha harder, much like , um, uh, much more challenging problem , uh, because then you want , you introduce dimens enough time and , uh, that's still a very tough nut to crack. So I think that will be a progress in the right direction when we can get models that can actually learn from videos consistently. But ultimately you want to go even beyond that because all of these things I mentioned would still be a form of passive learning, which just feed a bunch of data outta the system and just passively learns from that data by doing something like next time step prediction or next token prediction. Ultimately, if you want more human-like , uh, systems that exhibit more human-like intelligence, you would want them to actually learn like humans do by interacting with the world in a and so interactive learning, not just passive learning. You want something that's more active where the model is going to actually test out some hypothesis , uh, and learn back, learn from the feedback it's getting from the world about these hypothesis in the way children do should learn all the time. You know, if you observe young, you know, babies and toddlers, they are constantly experimenting. They're like little scientists, you know, you see babies grabbing their feet , uh, and kind of testing whether that's part of my body or not, and kind of learning gradually and know very quickly learning all these things. Language roles don't do that. Uh, they don't explore in this way. They don't have the capacity for interaction in this way. Uh, they learn purely passively. And that's a major difference with biological intelligence as well. So these are just a few examples are others. I think we need also, we probably need them to develop something like a metacognition, the ability to think about their own outputs and reflect upon their own thought processes, something like introspection as well . Being able to have some kind of internal representation of their own representations and, and, and reflect upon that. And maybe that could emerge from the bottom up spontaneously as kind of emerging capability or maybe you will need to build into the system. That is an open question. Uh , generally the debate on these questions in AI is that some people think you can get all of these things for free, or like, at least you don't need to make major modifications to current architectures to get a lot of these, you know, the remaining , uh, capacities that are missing from the systems to get them to emerge just by scaling up current architectures and training them on more data and training bigger models. I'm not so sure. I think we'll need to make , uh, some important modifications and I dunno what the next break , next breakthrough is going to be. Well, thank 01:47:23.859 --> 01:47:56.819 You for that. This has been really good , uh, really great overview of , um, some of the , the faculties as they relate to large language models. Uh , we could, there's many more topics that we could talk about and I would've liked to , uh, have gotten to, but unfortunately didn't have time. But this , I think this has been a really good overview, and hopefully our listeners find this useful. If anyone wants to find out more about you and your work , where could they go? And I, I know that you , um, quite active on Twitter and some of your Twitter threads are very informative and useful for, for my own research, for this interview. But where would you like to point people to go to if they'd like to see more about you and your work? 01:47:57.220 --> 01:48:43.619 Uh, yes . So , uh, I indeed , I , I have a Twitter account, so I , I'm at , uh, Rafael Miler , so r a p h a e l , my first name and my last name m i l l i e r e r Ami . Uh , on Twitter, I also have a website, which is just R ami do com . Uh, uh, I need to update. It's been a while celebrated , but usually I will put there just not just my academic work, but also my public engagement work. I'm quite passionate about public engagement. I do some public writing, and I actually have a piece , uh, that I co-authored with my colleague Charles Radko , that , uh, should come out soon in the authentic, that's precisely about this issue of the extreme polarization of discussion on AI and how we can cover out that mural ground . So yeah , if people are interested in that, you can probably find it , uh, soon. I'll share it on Twitter and yeah, that's , uh, that , that's about it. 01:48:43.960 --> 01:48:47.819 Sounds great. Well, thank you so much again for joining us. We really appreciate your time and all the best. 01:48:48.270 --> 01:48:50.180 Thank you for having me. This was really enjoyable. 01:48:50.229 --> 01:49:08.739 Thank you. It was a pleasure . Thanks for listening. I hope you enjoyed the episode. You can subscribe to the Sentient Institute podcast on iTunes, Stitcher, or any podcast app.