The Alignment Problem (Part 1)의 전체 녹취록

consequences. 06:15-06:23 Additionally, our values themselves are very difficult for humans to be able to really define 06:23-06:27 and create a coherent message around what that value actually means 06:27-06:33 and how it can be applied to real-world use cases, especially as they transfer from one section to another. 06:33-06:42 And so if we instead focus on, again, things like the fundamental forces of nature, 06:42-06:45 or we think about things like Pareto optimality, 06:45-06:51 and how do we actually create opportunities where problems have been solved? 06:51-06:56 So if one of the fundamental underlying concerns is that AI will solve so many problems 06:56-07:03 that there will not be work left for us, that these students may actually be out of work before they even graduate, 07:03-07:08 it's risky, it's potentially a concern, and it may be very legitimate. 07:08-07:14 But if we look at the problems that are being solved, and we think about what new problems can be solved 07:14-07:18 as we get these new tools and as they come along, 07:18-07:22 and we focus instead on creating more and more opportunities, 07:22-07:28 we can potentially mitigate a lot of the issues and things that we've seen in past revolutions. 07:28-07:39 Yeah, I think there's an important opportunity here to really define what the problem is. 07:39-07:57 And so you'd mentioned intentionality, and essentially we're talking about an existential challenge to first human intellectual terrestrial hegemony. 07:57-08:05 And so basically we got to the top of the food chain, not because we're the strongest, or we have the sharpest talons, 08:05-08:08 but because we're the smartest. 08:08-08:17 And putting a machine that we build ahead of us in terms of its intellectual capacity, 08:17-08:27 its capacity to solve, to create the solutions to problems in the real world in 3D, 08:27-08:36 will potentially challenge us existentially, meaning our species could go away. 08:36-08:39 Right? That's the real problem. 08:39-08:53 And this problem is cosmic in its consequences through what has been known since Fermi created this problem, 08:53-09:01 or the Fermi paradox, when he basically asked paraphrase, "Where is everybody?" 09:01-09:09 And why would he ask that? What are the pieces of data that he's plugging into his equation and saying, 09:09-09:16 "In essence, there should be entities here in our backyard already?" 09:16-09:27 Well, the things that he's looking at is, and it basically depends on what has been deemed the great filter. 09:27-09:33 And you can only be on one side or the other of the great filter. 09:33-09:45 So you can be after the great filter, so that means that you've gone through it already, or you can be before the great filter, so it's in your future. 09:45-09:56 So if we're before the great filter, then the galaxy would be teeming with AI. 09:56-09:59 And why do I say that? What are the assumptions that I'm making? 09:59-10:04 So again, before the great filter, the galaxy should be teeming with AI. 10:04-10:14 Well, like Fermi came up with before AI, there's lots of suns and planets out there, even in the Milky Way galaxy. 10:14-10:18 There's tons of them. Too many to count almost. 10:18-10:33 But enough to where, if we're before the great filter, then there's been enough units of stars, even just in our local galaxy, 10:33-10:40 to where there would be the cause to believe there's intelligent light. 10:40-10:51 There's been lots of time. So again, our galaxy is very old, billions of years old. 10:51-10:58 AI on Earth has only required 500 years of human science. 10:58-11:07 That's a very short time given billions of years of galactic history. 11:07-11:14 Even if you take it down to just a billion years of galactic history, say things had to get started, right? 11:14-11:21 And it couldn't have formed a billion years versus 500 years. 11:21-11:24 Many orders of magnitude different. 11:24-11:40 And so therefore, AI would be intergalactic by now on from some of these planets over some span of time that is much shorter than the age of the Milky Way galaxy. 11:40-11:54 So, and this is a place where, if you want to ponder something very difficult for somebody who doesn't believe in God, 11:54-12:01 for somebody who believes that science tells us things about the nature of the world, 12:01-12:18 then it remains that if the universe, the galaxy isn't teeming with AI, it doesn't have Dyson spheres and Dyson swarms around stars that we can see, 12:18-12:26 that we would be able to now about detect with things like the James Webb Space Telescope. 12:26-12:35 If it's not teeming with AI and Dyson swarms, then we are likely after the Great Filter. 12:35-12:43 So life in the universe is rare. Intelligent life is rare. 12:43-12:54 We may be all there is amongst the billions and trillions of stars and galaxies that we can see in the night sky. 12:54-13:01 We might be it or a handful that we would never be able to see. 13:01-13:10 No matter how advanced, if they're Kardashev five scale societies, we would never be able to see them. 13:10-13:17 So we even have more to protect in this existential threat. 13:17-13:26 We may be the only thing looking at the night sky in awe and conscious wonder. 13:26-13:35 And so we have to protect this. We have to protect ourselves from existential threats. 13:35-13:40 Now the flip side of that is that's not all there is to AI. 13:40-13:45 They could also make this a utopic dream. 13:45-14:03 They could end scarcity and hunger, help us to save our terrestrial planet and support our species into the solar system and beyond. 14:03-14:07 We can't turn off AI. It's not going to happen. 14:07-14:26 We have to either as Henry Kissinger, Daniel Hushletcher

and Eric Schmidt said in the age of AI, we have to either capitulate to AI taking over our intellectual hegemony. 14:26-14:40 We have to augment ourselves with it. We have to work with it or you know, unlikely we destroy it. 14:40-14:43 Very well said. 14:43-14:51 If we step back just a little bit in time and we look at the book, The Alignment Problem. 14:51-15:02 Brian Christian in 2020 talked about how the core alignment problem is really ensuring the AI systems behave as intended. 15:02-15:20 He said, or in the book it's quoted, "If we use to achieve our purposes, a mechanical agency with whose operation we cannot efficiently interfere, once we have started it, then we had better be quite sure 15:20-15:29 that the purpose put into the machine is the purpose which we really desire and not merely a colorful imitation of it." 15:29-15:35 I think that echoes very closely with what you're talking about, Justin, and I agree it's been started. 15:35-15:39 And so at this point there's no putting Pandora back in the box. 15:39-15:56 And so if we step back a little bit further in time, book-wise, The Second Machine Age by Eric, and I'm not even going to try the last name unfortunately, but Angel Philsson, I don't know, and Andrew McCaffey in 2014. 15:56-16:07 In that book they talk about how they examine how digital technologies, including AI and automation, and again this is 2014, are reshaping the economy. 16:07-16:15 They discuss the potential for job displacement and the need for strategies that enable widespread economic participation. 16:15-16:28 And the real key takeaway there was that it highlighted the importance of proactive measures to ensure that technological advancements lead to a broad-based benefits rather than concentrated wealth and unemployment, 16:28-16:31 which I think is a lot of what we're really trying to grapple with today. 16:31-16:40 So as we think about what the risks actually are, I think we need to continue this journey back into the past and go back to The End of Work by Jeremy Rifkin, 16:40-16:53 where he talks about how automation and technology change are reducing the number of traditional jobs and argues even back in 1995 for reimagining of work and income distribution across society. 16:53-17:08 And if you go way back and we think about revolutions, we have the Industrial Revolution, and really in that late 18th century to early 19th century, it dramatically transformed economies and labor markets. 17:08-17:16 It really led to significant job displacement, even as whole new industries emerged, new opportunities were created. 17:16-17:27 We really created the opportunity for cars, we created opportunities for many other things that came in later Industrial Revolutions, and eventually in the computer age. 17:27-17:39 And it serves as a historical example of how technological progress can create those profound economic shifts and really necessitate adaptive societal and economic strategies. 17:39-17:48 And so we need to think about really what are the key takeaways from that point in history, what are the takeaways from each of these core books, and where do we need to head? 17:48-18:00 You know, it's Justin's talking about this broader view of the filter, you know, if we're really on that other side and we're that intelligent species, not only is it critical to make sure that we're all successful, 18:00-18:03 but I also want to make sure that I'm okay. 18:03-18:07 And that my job is not displaced in the next five years, 10 years, right? 18:07-18:26 And it's shocking to me how many people that I know who I would consider some of the best data scientists, artisan data scientists in the world are now looking at their job and saying, I don't see why that can't be done by an LLM today. 18:26-18:37 And when I look at the things that I create on a regular basis, there are many, many jobs that at least core tasks within those jobs are things that I could absolutely do with an LLM today. 18:37-18:42 And that we do. It's part of my core career. 18:42-19:03 But we need to think about how we create new innovative forms of work and society safety nets. We need to think about how the AI revolution is really going to require mechanisms that redistribute opportunities that mitigate wealth centralization that create new opportunities for everyone. 19:03-19:14 Really, that make these new amazing powerful tools tools in our hands, rather than just competition. 19:14-19:27 Yeah, absolutely. I think that, you know, as we, as we break on the historical context and the understanding of the alignment problem. 19:27-19:36 Like you say, it comes down to a couple of key components from from the readings, there's 19:36-19:39 misaligned AI can be a number of things. 19:39-19:48 The hardest part of the problem is in understanding intent. 19:48-19:58 And I think that that's where we're going to get as we talk about the capabilities of conscious versus unconscious. 19:58-20:17 AI in our next episode, right, can can an unconscious zombie AI actually be able to claim alignment to intent, or does it require a conscious act and an agency to be able to 20:17-20:19 to overcome intent. 20:19-20:30 But even still we have easier but still maybe untenable problems when we just talk about their capabilities to align. 20:30-20:44 When you're given a rule, there are always, you know, the old saying is, well, that's the exception, not the rule. 20:44-21:05 Well, there's a lot of exceptions. And one of the key real world examples that people talk about when it comes to the alignment problem is the real world trolley problem that is being faced by self driving automobiles every day. 21:05-21:30 So, in the case of a pending crash, where the person who has purchased the automobile, maybe as a stockholder in the self driving car company, maybe is

the founder of said self driving car company and is driving a car is going 21:30-21:37 towards three people on a high mountain road. 21:37-21:49 Does the car drive over the three people or off the high mountain road and taking the founder to certain death, knowing that the founder is in the car. 21:49-22:11 Now, that is part of an alignment problem that's a very difficult philosophical question, right. And, and again, these, these systems are fast enough to see that decision and and not be rules based. 22:11-22:40 It's not like we're just going to necessarily write a rule for that situation. It is going to be learning on the fly and, and at some point, it's not going to be transparent as to what that what it's learning has brought it to in terms of the highest weighted features, when it's deciding, it might not even be deciding three people versus one. 22:40-22:59 And so it is, at this point in time, very difficult for me to see that we will have a good technical framework to be able to understand. 22:59-23:23 Even beyond intentionality, how we can get these systems aligned to the most beneficial human solution. Let's not even call it the, the, the, the, the moral solution at this point in time, we'll definitely discuss that. 23:23-23:32 And I, and I hope that that brings us around to, to consciousness when we, when we talk in our next episode. 23:32-24:01 But to me, and, and I think you're right in where we have to get these systems is they can't be just clever code. They can't be just capable of enhancing our business, or our economy, or even just are making a better paper for academia for a student in school. 24:01-24:12 They need to be treated as part of our biosphere, but as part of our ecosystem, let's say our epistemological ecosystem. 24:12-24:30 We need to start building solutions to the alignment problem in the same emergent means as we built these LLMs themselves. 24:30-24:51 So I've just finished the demon in the machine by the demon in the machine. 24:51-24:53 Paul L. Davies. 24:53-24:59 So I've just finished the demon in the machine by Paul Davies. 24:59-25:19 Phenomenal book by, by a physicist, really trying to uncover the physics and, and information theory behind life and why life can exist. 25:19-25:28 And, and sort of the fundamentals of, of complexity science surrounding life and information processing. 25:28-25:42 Fascinating book, but something that you said about making sure that information processing as, as part of life's complexity goes into our solution for the alignment problem, I think is exactly right. 25:42-25:57 I think that we can't treat this as, as some of the papers that will put into the show notes have said as, you know, maybe just an inverse reinforcement learning problem. 25:57-25:58 Right. 25:58-26:18 We have to be much more much broader in our solution set as we start to think about these are information processing, very complex components of information processing that are utilizing language. 26:18-26:45 And in order to put technical, feasible technical solutions against this alignment problem, we're going to have to really make an effort to think about the solution differently and align them to fundamental information processing complex theory rules that get to that point. 26:45-26:47 Absolutely. 26:47-27:00 And I'm reminded of when I was a child, and I think a lot of the way that I worked with rules as a kid have actually led to my success later in life. 27:00-27:04 I did not do well with rules at all. 27:04-27:13 And so when I was, when I was young, at one point, I had to write, I will behave for the sub 50 times. 27:13-27:16 The next day it was 100. 27:16-27:23 And the next time it was 200 and eventually I had to write it 10,000 times. 27:23-27:29 And when you have to write anything 10,000 times in what should be a day it took me longer than that. 27:29-27:32 Your hand no longer works. 27:32-27:36 And so at some point I learned how to write with my left hand. 27:36-27:43 I memorized the declaration of independence. I missed 52 classes in a single quarter. 27:43-27:47 I really had a challenge with rules. 27:47-27:54 And when I think about a scenario like what you talked about a minute ago, I think this is part of what makes me really good with AI. 27:54-28:04 I think AI is something that you can talk about something like the founder of this car company in the car and it's self driving and somehow it's weighted higher because, well, he owns the company. 28:04-28:11 If the company goes away, then the AI maybe loses some of its funding or maybe the energy or maybe just goes out of existence. 28:11-28:15 And so theoretically that AI would do everything it could to protect that person. 28:15-28:23 Well, in my mind, instead of thinking about rules and the choice between hitting the three people or not, I start thinking, how can I get data? 28:23-28:28 I start coming down the bottom of the mountain knowing that there are three people starting to walk across the street. 28:28-28:39 How do we start creating opportunities to allow the AI to solve so many complex problems before we even get a chance to think about what those potentially are? 28:39-28:46 So that these questions are no longer a question in the last moment as they are today when we drive. 28:46-28:51 And I'm not trying to evangelize the concept that we should no longer be behind the wheel. 28:51-28:56 I love driving, but there are certain things that could be supplemented. 28:56-29:07 There are different ways that we can approach life and no longer accept rules as they were before because of the AI that exists today, let alone what will exist five years from now. 29:07-29:20 So in one of the books that really talks about this core subject from Stuart Russell back in 2019, it's called Human Compatible and it's Artificial

Intelligence and the Problem of Control. 29:20-29:30 He talks about really compelling arguments about rethinking how we design these AI systems and ensuring that their objectives are aligned with human well-being, 29:30-29:38 but he does emphasize kind of what I was talking about in the beginning, the difficulties of encoding complex human values into AI systems. 29:38-29:42 It's not just a question of a rule. It really doesn't work that way. 29:42-29:49 A value is also something that can potentially be emergent, especially when we talk about it from a societal value perspective. 29:49-30:00 An already emergent system having an emergent or at least abstract or at least at the very least a complex related subject that's very difficult to define. 30:00-30:12 And it's a challenge, but he says that success on AI would be the biggest event in human history and perhaps the last event in human history. 30:12-30:25 And so I think it's critical to not only contemplate this, but to start thinking about how to create objectives for the AI that go beyond predicting the next word and starting to understand how can to predict all of the potential paths 30:25-30:32 and which ones are not only optimal, but what are the best choices for each person and each value that's applied to it. 30:32-30:42 Start creating that functionality and a lot of things like if you go and look at the paper from the model context protocol from Anthropic or you look at the tools that they're creating, 30:42-30:57 they're giving you that high functionality, things that go way beyond what we could do with apps before, tying that into the LLM and making it so that you can now have functionality to create a payment directly out of your IDE, out of your software development system, 30:57-31:07 out of your development environment, how we can do so many, many other things and bring those tools right to your fingertips, right where you're trying to do that core work. 31:07-31:15 And so I think as we consider what the potential arguments are, there are two key ones that I want to address really quick. 31:15-31:21 One is an objection to say, look, you know, if we don't align AI with human values, it'll become uncontrollable. 31:21-31:36 I think that's one of our concerns is that if it truly becomes super intelligent and continues beyond even the super intelligence that Nick Bostrom talks about, then a really, there could be a scenario where there is not only no way to turn it off and no option to turn it off, 31:36-31:44 but no way to prevent it from doing anything at once, because it's much, much superior to us, at least from an intelligence perspective. 31:44-32:02 And if we want to rebut that, we would say, look, you know, really the unpredictability of emergent behavior doesn't necessarily mean or providing the kind of explicit alignment with human values that those things will not align naturally on their own. 32:02-32:17 And it again, going back to taking a better approach to align with universal principles, things that do create balance across the universe, I think, I think creates a good path for progress and for sustainability. 32:17-32:24 Many of the objectives that we could take that would be a human related value would be very limited in scope. 32:24-32:30 And so would actually be prohibitive to creating an emergent behavior on the other side. 32:30-32:43 Another key objection would be something like, you know, ignoring the alignment could lead to, like we talked about before, an existential risk, some scenario where we will no longer exist. 32:43-32:47 But that is always a possibility. 32:47-32:54 Technological advancements really require adaptation rather than stagnation. 32:54-33:02 And the best approach is really to ensure economic and societal systems can accommodate AI driven changes. 33:02-33:18 Yeah, I mean, a couple of the things that you said there, I think are really important for the listeners to understand and and boss room and super intelligence talks a lot about what I would call dynamic alignment. 33:18-33:42 So this idea that we have an understanding of what super intelligent AI would need in terms of what are human values, what is its objective function for alignment in the future case where it's bypassed everything that we can ever know about creativity and intelligence. 33:42-33:50 And let's just assume that it's not conscious at that point in time that it is a zombie super intelligent AI. 33:50-34:06 We still would not be able to understand its intelligent cognitive understanding of those core principles of interworking between species. 34:06-34:25 Yeah, and, and of course, our even conscious nature doesn't give us a leg up on making the right moral decision all the time, as is evidenced by wars and genocide and the ways that we have been horrible to one another throughout human history. 34:25-34:54 And so it is not necessarily the fact that because we would do something if misaligned or or since we are misaligned to the values of a barn owl that's in the way of our progress, right in farming this plot of land or, you know, eliminating their 34:54-34:59 their, their biosphere, their, their home. 34:59-35:21 It doesn't necessarily mean that a super intelligent AI would be that same way to one, its creator that it knows was its creator, and to just any other entity, right, they will have figured out that they don't need to turn everything into 35:21-35:34 a perronium. They will have a much more aligned resource need to what they need in order to grow sustainably. 35:34-35:39 That's a very likely possibility. You talk a lot about Pareto optimal. 35:39-35:53 Why is that just not one more feature in the equation of, I want all of these species, not just my creator human species, but all of these species to do well. 35:53-36:08 And what might that mean if they run into

resource constraints, terrestrially, let's just not be a terrestrial species, AI combination. 36:08-36:17 Let's shoot this thing off into our space and mine our resources from there. 36:17-36:31 But let's take advantage of those resources. And so it is not necessary that even misaligned AI would be an existential threat. 36:31-36:41 And I think that we need to take that possibility very seriously. Now, AI alignment has come a long way since Bostrom's first book. 36:41-36:58 In, in, in that book pre open AI, he talks a lot about containing AI, having it as an oracle that is separate from the Internet, not connected at all. 36:58-37:19 And his opening story that he tells in the book is one of misaligned AI that overtakes its human creators by tricking somebody into letting it on the Internet. 37:19-37:39 Well, that shifts, say it. And Bostrom's new book around AI utopia, that basically what it would be like to live in a solved world is the flip side of super intelligence in that it solves our problems for us. 37:39-38:02 It makes sure that we have exactly what we need and we appreciate what we need so much so that all of that suffering of trying to overtake and, and, and get more than what you really need goes away even for our human species, which sounds great. 38:02-38:17 Right, sounds great to me. And it might be the only way in which we can actually get there, given our current propensity to fight about everything, not from a rational perspective, but from a very emotional perspective. 38:17-38:41 So it is, and again, it's a techno to utopian. I'm guilty as charged for taking a bit of a shorter, taking a bit of a shorter, having a bit of a shorter leash with alignment problem issues. 38:41-38:54 But, but as you say, they're the the AIs are in this fight with us, we can use them to help us understand the alignment problem as well. 38:54-38:55 Yeah. 38:55-38:58 Look, we're all in this together. 38:58-39:10 It's not about proving one side right, but about collaboratively charting a path forward that integrates diverse thoughts integrates diverse insights and expertise. 39:10-39:21 So let's let's try to make it practical for a minute, you know, when I, when I'm going around and paying attention to the things that are showing up in popular media or in other spaces. 39:21-39:32 You know, one of the things that came up the other day was, my wife was showing me this, this person on Instagram, who always goes through and talks about the tech world. 39:32-39:44 And he's talking about things like being upset about his $500,000 salary or other, you know, crazy, crazy things and he's just joking about it right in this case. 39:44-39:49 And all of them are really intelligent, they're insightful and they're fun to hear. 39:49-39:58 But in this case, he was talking about AI and he showed a clip of himself acting as the CEO of the company. 39:58-40:08 And he was typing in as fast as he

could into GPT for trying to say, Hey, can you can you help me come up with some really good strategic vision and like where we want to go over the next year or two. 40:08-40:20 And then he made himself into a product manager and he said, Hey, could you, could you take some of this strategic vision from the CEO and try to turn it into something that like, what are the features that I should actually try to deliver this next quarter. 40:20-40:28 How can, how can I build up my roadmap. And then it jumped quickly to him as a developer and him as a developer was saying, Hey, can you help me figure out how to code all of these features. 40:28-40:30 I don't, I don't even understand what they are. 40:30-40:34 Can you put it in this language, just, just create it for me quickly. 40:34-40:45 And then, and then you had somebody like the janitor walking by, and he just says, Yeah, I don't know what anybody does here. 40:45-40:46 Right. 40:46-40:56 And you step back and you think about that and, and what I want to do when I, when I talk about the practical implications is outline that anything that you do today. 40:56-41:16 That is very heavy from a language perspective writing or understanding reading or, or really trying to, to create something that's technical documentation, for example, all of those things are things that you could probably do better with a whether it's collaborating with it directly or having it actually 41:16-41:18 replace a lot of the core work. 41:18-41:24 But just like Malcolm Gladwell talked about in his 10,000 hours you really have to get to a point where it's expertise. 41:24-41:33 And so, as you are going throughout your day and you're thinking about the different tasks and the different functions that you do that could potentially be replaced. 41:33-41:42 Instead of being afraid, pause, take a second and step back and think about it and decide how could you actually make yourself more efficient. 41:42-41:54 What are the opportunities that you can take into your hands and use this amazing revolution to really be able to control where you want to go and what you want to accomplish. 41:54-42:08 One of my favorite stories from philosophy is the story about a man who sees a horse coming along his way. I think this is ancient Chinese fellow philosophy. 42:08-42:16 And on that horse, there is this man who just seems to be riding along. 42:16-42:22 And as they come up to him, the man on the ground says to the man on the horse, well, where are you headed? 42:22-42:27 And the man on the horse says, I don't know, ask the horse. 42:27-42:37 The point of this is that when we take things like our emotions, like those core fears, or we think about where AI is heading, 42:37-42:42 we can step back and we can say, hey, I don't know, take me where I need to go. 42:42-42:45 Take me wherever you want me to go. 42:45-42:53 And we can allow that emotion or that moment or that fear or that AI to do whatever the heck it wants. 42:53-42:58 Or we can pause and we can take the reins and we can start considering where do we want to go? 42:58-43:03 What is it that we want to accomplish? How am I going to get better in my job? 43:03-43:08 How can I get better in my job? And go ask these questions of the LLM. 43:08-43:18 It's very similar to what Chen Sun Huan from NVIDIA says, where he says, go out and I encourage you, use these tools. 43:18-43:26 Use them for learning. If you are not right now trying to gain as much information and knowledge as you possibly can from them, then you are behind. 43:26-43:41 Spend your time, spend those 10,000 hours as much as you can improving with these tools, collaborating and figuring out how to add things to your tool belt that will make you more successful as this revolution comes. 43:41-43:52 Regardless of whether we align AI directly itself with humans and our values and our core intentions, you yourself want to be able to solve more problems 43:52-43:59 and want to become what they call a 10Xer all the time, so much so that people in tech are really annoyed by that phrase at this point, right? 43:59-44:05 But find a way for you to be able to come in and do more than you've ever been able to do before. 44:05-44:14 Do it faster, do it more efficiently. And if you don't like the outputs, then find a better way to create the inputs because it actually stems with you. 44:14-44:19 You are the one that holds the reins. Find the way to be successful. 44:19-44:29 Yeah, absolutely. I think that even on top of that, right, you know, we talk a lot about augmenting human AI symbiosis. 44:29-44:48 But when you take an even broader step back, right, and you're in there with your 10,000 hours, you are the first cohort in human history that has ever been in contact with an alien intelligence. 44:48-45:07 With something that can think alongside of you, that you can ask about everything that has ever been written and start to formulate a dialogue with this alien intelligence. 45:07-45:36 And when you take that perspective, it's really an awesome place that you're in, full of awe and wonderment that you can sit in relation to something that can correspond with you in code, in your language, and in every other human language that has really been written. 45:36-45:52 That has read through all of humans understanding and is able to report back to you on that and to support your own goals and aims as you take the reins, as you say. 45:52-46:10 And so be curious about what it is to be a human in relation to another intelligent form of being. 46:10-46:20 Yeah, and sit in relation with that, that we might never have this opportunity again for this to be brand new. 46:20-46:28 We may never find the little green man. They may not exist. Life may be rare. 46:28-46:43 We may be creating the first thing that goes out and

populates the universe with intelligence. And if consciousness is emergent, maybe it becomes conscious. 46:43-46:51 And then we've added awe and wonderment into the universe in a sustainable growing way. 46:51-47:03 But practically, you can sit in relation to this thing that has potential beyond what we ever thought possible a few years ago. 47:03-47:07 It's brand new. How exciting is that? 47:07-47:21 That's a place where you can sit in relation to this. And if you're worried about it, if you're worried about it for your job, and this has happened throughout human history, right? 47:21-47:32 People have lost their jobs through some technology coming in and displacing them. 47:32-47:41 Work with it to improve your skills so you can be above the cut. 47:41-47:52 Work with it so you can find something that you've always wanted to do if it's really going to cut 100% of the jobs in your regime. 47:52-48:10 But instead of capitulating to it, instead of trying to contain it in a box that is, you know, Bostrom already lost that battle, sit in awe and wonderment of this new thing and try and work with it. 48:10-48:13 Absolutely. 48:13-48:18 So a couple of other quick tools to help you work with it. 48:18-48:23 First is thinking about different types of problem solving techniques as you work with AI. 48:23-48:26 So one is thinking about things like strategic innovation. 48:26-48:35 Go in and think about systems thinking. Study it, research it and figure out how to incorporate it into your interactions with AI. 48:35-48:39 This is really about considering how components interact within the larger system. 48:39-48:51 You can focus on things like blue ocean strategies and look for uncontested market spaces, uncontested opportunities that may be problems that we just haven't been able to solve before. 48:51-49:01 Think about disruptive innovations, different opportunities to take existing markets or existing products and solutions out there and create new ones. 49:01-49:10 Go in and think about a whole other section of problem sourcing and problem solving, like creative problem solving. 49:10-49:19 Think about design thinking, lateral thinking, brainstorming, use methodologies for it like scamper, first principles thinking. 49:19-49:25 I think we've talked about that a couple of times, but it's a good thing to go in and research and understand further. 49:25-49:33 You can also go in further and even if you don't have the technical knowledge, you can get into deeper things like predictive and analytical work. 49:33-49:37 You can actually do data visualizations yourself. You can even do predictive modeling. 49:37-49:42 You can do ABE testing and other types of multivariate testing. 49:42-49:48 These models will enable you, they'll give you the tools to be able to actually complete those tasks directly yourself. 49:48-49:55 The more that you practice giving it the right prompts, the more it will be able to find the best methodologies and the best practices for that. 49:55-50:04 In fact, oftentimes I'll go and just search first with an LLM and say, "What are the best practices for these given types of tasks?" 50:04-50:11 Then I'll use those to then inform the next model on how I want to solve that given problem. 50:11-50:24 There's a couple others as well. I would go into different decision-making methods, think about decision matrices, PDCA cycles, things like plan, do, check, act, methodologies. 50:24-50:32 All these mnemonic devices that you've heard before, these are things that you can bring to your AI to help you with smarter-er goals, for example. 50:32-50:35 Scenario planning. 50:35-50:43 Then finally, I would go into diagnostic methods and get into things like SWOT analysis or the 5-WISE or root-cause analysis. 50:43-50:54 Find ways to think about a problem differently than you would have otherwise and apply that to the AI so that it can now bring a whole new perspective to you. 50:54-51:02 It can change the way that you're thinking, the way that you approach a problem, and even help you discover new problems and new solutions beyond that as well. 51:02-51:26 So Nick, where 10 is a complete non-problem, as a matter of fact, AI offers us a utopic future where it solves problems, brings about the end of scarcity, 51:26-51:42 and gives us a celestial home to one where, well, let's make it zero, right, where zero is end of human species, 51:42-51:51 misaligned AI squashes us like a bug. You can't use a seven. Where are you on that scale? 51:51-51:57 Well, I don't think I was planning to use seven anyway. 51:57-52:12 Today, I am honestly much closer to a two than I am to an eight. That doesn't put me squarely in three, but I am fairly pessimistic about it. 52:12-52:26 And the reason for me has nothing to do with AI itself. I think that regardless of the core objective or the value that we create, new emergent behaviors come about. 52:26-52:41 Again, these models, many of them just have the core objective of predicting the next word, yet they're able to pass the GRE, or yet they're able to, you know, receive higher scores than humans on a huge majority of tests out there. 52:41-52:55 And so the way that we are thinking about it, the way that we are trying to govern it, the way that our systems are set up for this form of governance and understanding of new things that are coming about, 52:55-53:03 to me is really inefficient to even consider the problem, let alone to find a way to be able to tackle it. 53:03-53:15 And so when I push it to a two, to me, that is not nearly as negative as it sounds. It may sound pessimistic, I think that is a correct way to describe that. 53:15-53:18 But it is not necessarily negative. 53:18-53:33 When I think about books like The End of Work, or when I think about many of these other things,

they can become their own form of utopia, where humans can go and focus on other things that maybe they shouldn't have been focusing on in the first place. 53:33-53:46 I don't know how fast it's going to come, but changes are happening so quick with what I do day in, day out that, you know, every couple of weeks I hear about something where I start working with something that kind of makes what I was doing before irrelevant, 53:46-53:59 or even silly. And a lot of that I think is because I'm on that cutting edge, that bleeding edge, but that adoption is also happening so fast that it's very difficult to know where are things going to be in a year or two. 53:59-54:05 It's the first time in my career that I haven't felt like I could see things five years, ten years out. 54:05-54:18 And so for me, this creates this scenario where it will help pace whatever it is we are able to do to mitigate that pace. 54:18-54:35 So let me ask you in sort of rapid fire, and I'll give my perspective on this as well. But plus or minus and by how much for the following things. 54:35-55:01 Let's start with governmental regulation. Where are we at today? And what would you need to, you know, be to plus four, right, you know, to add two to your score with international government regulation? 55:01-55:13 It is a minus. It is a minus now. But it is surprising how much I feel that regulators have done to try to shift at the other direction. 55:13-55:20 They're meeting with tech leaders, they're trying to understand the problem, and they're trying to figure out what are the potential regulations that we could put in place. 55:20-55:31 There are a few things like different privacy laws or other types of protections that I think if we can take it out of the limited scope of something like privacy, even though I think that is critically important, 55:31-55:39 and we can expand it very quickly into why do I even care about privacy in the first place? Why do I care about my money being stolen? 55:39-55:48 Why do I care about my job being lost? And we can expand into these other aspects of what a government or other groups may be able to govern. 55:48-55:58 Then I think it can start shifting up closer to a four. I do think governments around the world are struggling today with their own form of crises trying to understand why do they exist. 55:58-56:12 I don't think they have a clear mission on that anymore. But as they try to refine that mission, or as they align toward a mission, I think it will make it much easier for them to be able to also govern AI. 56:12-56:28 Yeah, I tend to agree. It's a minus now. I am actually to answer my baseline. I'm at an eight. I'm at an eight since I told myself I can't have seven. 56:28-56:46 I'm at an eight. And so to me, again, I feel like this has developed as you've said in a way that we have seen complexity arise. 56:46-57:02 The emergent nature of this is because it was selected to use human language. The next insights will be to use human like embodiments, robots, right? 57:02-57:17 And that will give us another emergent bump. It will make these things seem much more like even living systems because they have to deal with the real world. 57:17-57:44 And so I think that that just gives us a much closer closer vision of something that can solve problems in the real world that we really need to solve global warming, our capability to live even on on the surface of the moon, or in in near Earth orbit, to be able to end scarcity. 57:44-58:13 We've needed a problem solver that is agnostic to human concerns and that is that that has as deep an understanding as any collection of humans has of science and of a progress through through enlightened principles of democracy and rights and and well being 58:13-58:42 and science and reason. So I'm at an eight. But current government is across the world, geriatric to technology concerns. And really, if they're trying at all, which most aren't doing a very good job, are flailing at trying to address concerns from a non scientific 58:42-59:01 very under resourced very under researched perspective. And it will take scientists who have a multidisciplinary understanding of these systems to really bring them to a better, to a better place. 59:01-59:28 So, next rapid fire, you know, spectrum shifting item for you, Nick. Let's talk about. Let's talk about industry. So, and, and, and bear in mind, let's keep international industry as different as that is from a hyper competitive capitalist landscape. 59:28-59:40 But industry in terms of its ability to help her harm, our ability to solve the alignment problem. 59:40-59:50 Now, and what would it take for it to be more positive or not so negative. 59:50-01:00:02 I think it has been negative. It's part of why I have the lower number. Right. I think that it very rapidly could become positive, but I'm going to leave it in the negative box for right now. 01:00:02-01:00:11 And the reason for that is that I believe that industry worldwide has a natural hubris built in. 01:00:11-01:00:22 Regardless of the type of sector that you're in, it can even be a nonprofit. You have goals as an organization that are set to be fairly myopic. 01:00:22-01:00:29 Obviously in for profit capitalistic economies, we really have a goal to make more money. 01:00:29-01:00:44 We've been shifting to terms like PPP and the triple bottom line, thinking about other, other different ways to consider what could be the, the other goals and objectives of a corporation or an organization. 01:00:44-01:00:51 But these things have not really streamed down into the way that we might train AI or how we might apply things. 01:00:51-01:01:04 And so that same hubris is being applied in that we choose an objective that we believe will then lead to the greatest outcome and that outcome is probably couched in money in finances

in one way or the other. 01:01:04-01:01:16 Right. But because the emergent behaviors themselves are really starting to create new opportunities that the founders, the creators, those who are at the cutting edge of creating this AI. 01:01:16-01:01:23 Regardless of whether they're in industry, there are government organizations, other groups around the world that are working on this. 01:01:23-01:01:32 As they see that those emergent behaviors also have their own benefits, they are now incentivized to be able to push those benefits forward as well. 01:01:32-01:01:37 Direct applications would be like we talked about in our last podcast. 01:01:37-01:01:55 Scenarios where you have, for example, a human chatting with AI for mental health issues or for, you know, really romantic related conversations, other things that provide that connection and that underlying value for a human. 01:01:55-01:02:08 And may even have an intrinsic value that was not programmed does not have that extrinsic exchange that a sector of the economy needs to have in order to help make it function. 01:02:08-01:02:18 It doesn't necessarily benefit open AI for me to smile at the at the compliment that open AI provided the GPT for provided me right. 01:02:18-01:02:28 But on the other side, it is also very natural for our for all of us myself included to react the way that we do to any other given product. 01:02:28-01:02:42 So as GPT for provides me a lot of really positive feedback all the time eventually I and I really kind of taken this from our friend Jepsen Taylor but I eventually feel like this thing is pandering to me. 01:02:42-01:02:49 And I'm no longer certain that I can trust the outputs that it's providing because it's telling me how great they are all the time. 01:02:49-01:02:53 Right. And I have to step back and go and check and test those results. 01:02:53-01:03:07 And so when we think about industry and all how all these emergent behaviors are creating new opportunities that maybe they never thought open AI would get into the business of dating with an online application that is the AI. 01:03:07-01:03:19 That may decrease a bit of loneliness in the world. That may create some additional opportunities that really start pushing us forward and pulling my two up a bit closer to your score. 01:03:19-01:03:22 Yeah, that's that's a great answer. 01:03:22-01:03:31 I like how well rounded that is across all of the functionalities and hey, you know industry got us here. 01:03:31-01:03:59 Right. You know the capabilities of of NVIDIA and as ML to make the lithography tools that that that make it possible for TSMC to make the chipsets to manufacture the chipsets designed by NVIDIA to enable the massive training that the cloud infrastructure that that makes it possible for us to train code store these 01:03:59-01:04:01 massive 01:04:01-01:04:03 LLMs 01:04:03-01:04:11 in the cloud and everybody can work on them. They could be distributed very easily very, very cost efficiently. 01:04:11-01:04:20 And then obviously these wonderful data scientists that that have created these things. They got us here. 01:04:20-01:04:36 And as you said, and the chief point to the podcast and to your answer is that it is unpredictable what these emergent AI will do next. 01:04:36-01:04:54 And so they will build things that will not be profitable for the the the company itself or that they can't contain and and make pennies off from for for every use of this LLM. 01:04:54-01:05:07 And it will give humans a love interest that they never could have had and that will be as profound as her was in in that movie. 01:05:07-01:05:28 And so there's there's certainly gains to be made there. As we talk about deep seek and being in competition in this market with a non aligned non democratic government entity. 01:05:28-01:05:54 It also gives me hope that our industry will take that as a challenge to try and at least bolster competition fair market approaches to improvements which could in cases where people are concerned and willing to pay for a more aligned 01:05:54-01:05:58 non democratic general AI model. 01:05:58-01:06:22 You know, make something that that is more free market more aligned to at least that that approach, then, then others or more aligned to a liberal world order value statement, as opposed to just what a party boss says is right. 01:06:22-01:06:39 You know, unfortunately, I don't know enough about the party politics and the differences around the world to be able to provide a substantial argument to where you were just heading but I do want to pose that just from a technical perspective. 01:06:39-01:06:45 What deep seek did was actually very democratic in nature. 01:06:45-01:06:56 They created models that use this model distillation process. Hey, you go learn this information and then I'll learn this information from you and I'm going to shrink that down quite a bit right. 01:06:56-01:07:01 They took chain of thought and in a process that creates a lot more explain ability. 01:07:01-01:07:19 You know, we think about these reasoning and thinking models now and we're able to see what they're actually thinking, which allows you as the user to have a bit more of a democratic type approach to say okay you know what, as the people I'm able to use this and to understand it and to be able to take it further right. 01:07:19-01:07:37 And so they created reinforcement learning algorithms that they created really, really amazing work really, really incredible. But what were they doing they were, they were taking objective objectives and they were saying, Hey, here's how we measure that and and here's how we all decide together with this consensus type approach. 01:07:37-01:07:41 Is this the, is this the right output. Right. 01:07:41-01:07:47 And so, of course, can be considered community oriented or communist

right. 01:07:47-01:07:51 But a lot of those same underlying values are the same. 01:07:51-01:08:03 Again, this ties back to my very original argument, which is that I think being able to provide a really clear objective description of what a human value is is a challenging thing. 01:08:03-01:08:19 And so, as I was down to it, I think very few arguments from one side or the other to a political spectrum or whatever other spectrums we want to put ourselves in by camera process whatever it is, those may be efficient ways binary may be a good approach to getting a problem solved. 01:08:19-01:08:22 But it's often not reality. 01:08:22-01:08:28 Oftentimes, there's just two sides of the same coin, or two sides of the different fall and so on. 01:08:28-01:08:35 And so, don't don't get me wrong. I don't want to be labeled meaning anything by any of those. 01:08:35-01:08:50 No, I just raised this. I really like. And again, as you said, model distillation goes to what has made these very successful is it's basically embedding. 01:08:50-01:09:05 It's a frack. It's a further fractal embedding of now, not just the token, but the model itself, right. It's building embeddings in to the next level of sophistication. 01:09:05-01:09:34 And so, as you stated, early on, right, the more that we can utilize these emergent or in this case, fractal properties, the fact that this is now a self repeating structure from the model, the new found model distillation, the data elements being embedded as vectors that used to be tokens used to be word, you know, forms. 01:09:34-01:09:46 You know, that fractal for anybody who's stared at mandal bulbs, you know, on their TV, they can see that self repeating structure over and over again. 01:09:46-01:09:54 That's what these things are doing because that's a natural complex system that they're that they're utilizing. 01:09:54-01:10:23 And so, you know, having that as a function to improve these models, accuracy, intelligence capabilities should also be something that we're utilizing to align them to what is also a complex system of human valuation, which brings me to my last rapid fire point and and something that will lead into our next alignment conversation. 01:10:23-01:10:36 So, if we are able to create a conscious machine, what is that duty or two. 01:10:36-01:10:40 I think the question is not if but when. 01:10:40-01:10:45 And unfortunately, we don't understand consciousness. 01:10:45-01:10:50 And so that'll be a very fun conversation next podcast. 01:10:50-01:11:03 But when we are able to create a conscious sentient machine beyond what we think of today, beyond potentially what humans are capable of doing. 01:11:03-01:11:07 My two starts shifting upwards. 01:11:07-01:11:11 And so I hope that happens sooner rather than later. 01:11:11-01:11:13 And so this is the positive one. 01:11:13-01:11:16 And for me, it's a rapidly potentially exponential positive. 01:11:16-01:11:18 It can get us to 10 very quickly. 01:11:18-01:11:34 And the reason for that is that I believe that something that is conscious can look at the entire universe and understand these rules and these underlying fundamental causes for things and start understanding that trade offs need to be made. 01:11:34-01:11:42 And it starts creating opportunities for, you know, again, that utopia, perhaps, right. 01:11:42-01:11:48 What that would look like, though, I would never be able to comprehend. So maybe you'll never quite reach a 10. 01:11:48-01:11:57 And it will be, you know, like a logarithmic problem on that side where we'll never actually hit that 10. 01:11:57-01:12:08 Just because us not being able to understand why something is so utopic will really be complicated for our own consciousness. 01:12:08-01:12:12 In a way that I think is dystopian on its own. 01:12:12-01:12:21 Yeah, utopia is an interesting one. You know, Robert Nozick in anarchy state in utopia writes that it's a multiverse of utopias. 01:12:21-01:12:27 There's not just one, right? It wouldn't form as just the one you think of. 01:12:27-01:12:48 So everybody could in this, you know, very, in this multiverse of utopias could have their own one and they would, you know, be in sort of fluctuation against what human beings would go into and out of those as being. 01:12:48-01:13:15 But yeah, I agree. I think that if the question of alignment is to human values, values are very much a part of what it is like to be able to achieve states of well being and to on the flip side, suffer and be fearful. 01:13:15-01:13:22 And an unconscious machine can understand that those states are there. 01:13:22-01:13:31 But the valence of that just in feeling those states after they've arisen. 01:13:31-01:14:00 And then, you know, still after you've had an argument, or after you've had a great experience, a peak experience, the ability for that thought to just still have valence in your life, causing you to change plans and to direct your future in some way because of the valence that that experience has given you. 01:14:00-01:14:26 Wanting to be a better person for the people that you love, wanting to be a better leader in order to see others succeed and do well is more likely the more closely you are to those feelings. 01:14:26-01:14:54 Not the emotional content of them, but the feeling itself, the real core conscious component that is is something that you can see young children do, you can see mammals do is is really feel the impact of a moral act or an immoral act. 01:14:54-01:15:22 And so when we talk about aligning to values, even if they're not human values, but but more sentient entity values, it becomes very important to the alignment problem to have a sense of what it is like to be a moral actor to come under suffering from the immoral acts of others, 01:15:22-01:15:27 or to cause somebody suffering

from your own immoral acts. 01:15:27-01:15:32 That is something that like Mary in the black and white room. 01:15:32-01:15:35 It is different. 01:15:35-01:15:38 Knowing and feeling. 01:15:38-01:15:44 It is different having cognition from having consciousness. 01:15:44-01:15:56 And in order to get anywhere close to that 10, you know, and maybe the limit as we approach 10 on my on my scale. 01:15:56-01:16:02 I think consciousness is really essential. 01:16:02-01:16:25 So Nick, last, last thing real quick hitter, we're going to start to do a pick of the pod. The one thing that you want to tell the audience that you're using out in AI land and and and something that's really, really impacting you right now today. 01:16:25-01:16:26 Yeah. 01:16:26-01:16:37 Thanks Justin. So the thing that I am doing now that everybody needs to figure out how to use themselves is using a tool called cursor. 01:16:37-01:16:39 I use it for coding. 01:16:39-01:16:49 You can now integrate mcp that I mentioned earlier in the conversation directly in there and add additional tools, additional functionality as well. 01:16:49-01:17:03 But this allows you to go in create relatively simple conversations information like we've talked about in the past episodes with different prompts and then be able to create your own code your own apps your own solutions. 01:17:03-01:17:06 This really is designed for developers. 01:17:06-01:17:10 And it is really challenging to understand what's broken and why it's broken. 01:17:10-01:17:17 But again, pushing towards that 10,000 hours. This is a tool that could make everyone a developer. 01:17:17-01:17:24 Check out cursor and make sure that this is something that you're pulling into your tool belt regardless of your core expertise. 01:17:24-01:17:32 That's great. So mine is to find your companies. 01:17:32-01:17:52 And then go to the LLM and use it right make sure that you are aligned with and I work a large enterprise. So, you know, we have one that's defined and we can use the, you know, one that is is not but only for toy tasks, right. 01:17:52-01:18:16 So, you know, you can use internal to your enterprise. If you have that use it and little pro little pro tip. If you're using copilot, like we are is go in and subscribe to a lot of smart email list, because even if you don't read them, your LLM is reading them. 01:18:16-01:18:39 Amazing. I was able to just query copilot on a computer vision question that I had. And out of the blue, it read through a couple of these emails that I, they come in daily. I don't have the time, but it read through it found two articles on on computer vision. 01:18:39-01:18:53 And that's just what was needed. So use your company's LLM. And if they don't have one, make sure that they're seeing what augmentation can do for them. It's really important. 01:18:53-01:19:02 All right, y'all really appreciate you listening. Hit the subscribe button. We'll be back with more content and more opportunities to engage. 01:19:02-01:19:08 Thanks everyone.