WEBVTT
00:00:00.160 -->
00:00:07.040What happens when AI stops simply using the tools humans give it and starts helping to build the next generation of AI?
00:00:07.280 -->
00:00:10.960That may sound like science fiction, but it's already beginning to happen.
00:00:11.199 -->
00:00:20.399AI systems are increasingly writing code, running the experiments, analysing the results, and contributing to the research used to create more capable AI.
00:00:20.640 -->
00:00:33.520And if that process keeps accelerating, we could reach a point where the biggest driver of AI progress is no longer the number of human researchers we have, but how effectively AI can improve the technology that improves AI.
00:00:33.759 -->
00:00:37.359This is called recursive self-improvement or RSI.
00:00:37.600 -->
00:00:41.359This raises a very different question from the one we normally ask about AI.
00:00:41.520 -->
00:00:44.240We usually ask, what can AI do for us?
00:00:44.479 -->
00:00:51.119But what happens when we have to start asking, what can AI do to make the next AI even more capable?
00:00:51.359 -->
00:00:55.520That's the subject of today's episode of Inspiring Tech Leaders with me, Dave Roberts.
00:00:55.679 -->
00:00:58.320And there is an important point to make from the outset.
00:00:58.560 -->
00:01:06.719We're not living in a world where AI systems are independently redesigning themselves, creating their successes, or operating without human involvement.
00:01:06.959 -->
00:01:25.519Well not yet, but the technology is moving towards increasingly automated AI research, and that raises some fascinating questions about how quickly AI could develop, how human oversight will remain necessary, and whether our ability to control increasingly capable systems can keep pace with their capabilities.
00:01:25.760 -->
00:01:29.040So how close are we really to AI improving AI?
00:01:29.200 -->
00:01:36.239And perhaps more importantly, what happens if AI starts improving AI faster than humans can understand what it's doing?
00:01:36.480 -->
00:01:39.120So the basic idea sounds relatively simple.
00:01:39.359 -->
00:01:47.840Instead of humans designing a better AI system, the AI itself increasingly participates in designing, testing, training, and improving the next generation.
00:01:48.079 -->
00:01:52.319That creates a potentially very different trajectory for artificial intelligence.
00:01:52.480 -->
00:02:06.799Because if an AI system can make improvements to itself and those improvements make it better at AI research, then the improved system may be able to make further improvements even more effectively, and the next generation can potentially do the same thing again.
00:02:07.120 -->
00:02:08.879That is the recursive part.
00:02:09.039 -->
00:02:23.680It creates a feedback loop in which AI is not simply becoming more capable because humans are giving it more computing power and better data, it is becoming more capable because increasingly capable AI is helping to create the next generation of AI.
00:02:23.840 -->
00:02:28.639And that's why recursive self-improvement has become such a significant subject.
00:02:28.879 -->
00:02:31.680But there is an important qualification right at the beginning.
00:02:31.840 -->
00:02:40.800We're not currently living in a world where AI systems are independently redesigning themselves and then launching increasingly powerful successes without human involvement.
00:02:41.039 -->
00:02:46.639Open AI says itself that fully autonomous recursive self-improvement is not happening today.
00:02:46.800 -->
00:02:51.680The interesting question is how much of that journey towards that destination has already begun.
00:02:52.000 -->
00:02:54.560So has RSI become the new AGI?
00:02:54.719 -->
00:02:56.960And there is a good reason for that comparison.
00:02:57.199 -->
00:03:02.960AGI or Artificial General Intelligence has never had one universally accepted definition.
00:03:03.199 -->
00:03:08.719Different organizations and researchers have different ideas about what constitutes AGI.
00:03:09.039 -->
00:03:11.120RSI has a very similar problem.
00:03:11.360 -->
00:03:18.319For some people, an AI system contributing to the development of another AI system could count as a form of self-improvement.
00:03:18.560 -->
00:03:21.280For others, that definition is far too broad.
00:03:21.520 -->
00:03:24.400The stronger definition is much more demanding.
00:03:24.639 -->
00:03:37.759It means an AI system can independently conduct AI research, identify ways of improving itself, implement those improvements, evaluate whether they have worked, and then use the improved system to repeat the process.
00:03:38.000 -->
00:03:43.199In the extreme version, humans would no longer be necessary to drive the research loop.
00:03:43.439 -->
00:03:59.599That distinction matters enormously, because there is a huge difference between an AI helping a researcher write some code and an AI deciding what research should be conducted, carrying out that research, evaluating the results, and then using those results to create its successor.
00:03:59.840 -->
00:04:07.680We are somewhere in between those two extremes, and the distance between them is becoming one of the most closely watched areas in AI research.
00:04:07.919 -->
00:04:12.080There are already examples of AI being used to accelerate AI development.
00:04:12.400 -->
00:04:19.920There has been experimentation with automated research systems that allow AI agents to conduct relatively straightforward machine learning experiments.
00:04:20.079 -->
00:04:27.120Reports say that these systems have so far been forced on relatively modest improvements rather than revelationary breakthroughs.
00:04:27.360 -->
00:04:30.319Other organizations are pursuing similar approaches.
00:04:30.639 -->
00:04:36.639The important point is not necessarily that these systems are capable of creating superintelligence, they're not.
00:04:36.879 -->
00:04:45.519The important point is that the development process itself is becoming increasingly automated, and that changes the economics of AI research.
00:04:45.839 -->
00:04:50.160Imagine that today a research team has 10 highly skilled AI researchers.
00:04:50.319 -->
00:04:55.920They develop an experiment, write code, run it, analyze the results, and decide what to try next.
00:04:56.160 -->
00:05:01.759Now, imagine that AI systems can conduct hundreds or thousands of those experiments in parallel.
00:05:01.920 -->
00:05:16.000They can work continuously, they don't need to sleep, they can analyse enormous quantities of results, they can identify patterns that humans might miss, and they can potentially take successful techniques and incorporate them into the next generation of models.
00:05:16.399 -->
00:05:26.399The AI industry has already been using AI to help build AI for some time, but recursive self-improvement represents the possibility of taking that process much further.
00:05:26.720 -->
00:05:32.319Anthropic has said that Claude was responsible for around 26% of its model research and development.
00:05:32.480 -->
00:05:36.160That does not mean that Claude is independently building Anthropic's next model.
00:05:36.319 -->
00:05:38.160Human researchers remain involved.
00:05:38.319 -->
00:05:48.240But it provides an interesting indication of how quickly AI is moving from being a tool used by researchers towards becoming more of an active participant in the research process.
00:05:48.480 -->
00:05:51.600And that is perhaps the most important concept to understand.
00:05:51.920 -->
00:05:55.519RSI is not necessarily a switch that gets turned on one day.
00:05:55.759 -->
00:05:57.839It's better understood as a continuum.
00:05:58.079 -->
00:06:00.399At one end, humans do virtually everything.
00:06:00.560 -->
00:06:09.519The AI assists with coding, AI runs experiments, analyses results, proposes new architectures, evaluates competing approaches, and implements improvements.
00:06:09.759 -->
00:06:14.879Humans supervise the process, and eventually the human role becomes smaller and smaller.
00:06:15.120 -->
00:06:17.680This is where the debate becomes particularly interesting.
00:06:17.920 -->
00:06:27.199Because the question is not simply whether AI can improve itself, it is whether AI can improve itself faster than humans can understand and evaluate those improvements.
00:06:27.519 -->
00:06:29.360That distinction is critical.
00:06:29.519 -->
00:06:38.959An AI system could theoretically become extremely effective at optimizing its own capabilities, while humans remain responsible for deciding what it should optimize.
00:06:39.199 -->
00:06:40.480That is manageable.
00:06:40.800 -->
00:06:52.000The much more difficult scenario is one in which systems become capable of deciding how to improve itself, while humans increasingly struggle to understand the mechanisms and consequences of those improvements.
00:06:52.160 -->
00:06:54.879This is where the concept of alignment becomes central.
00:06:55.120 -->
00:07:01.839Alignment is essentially about ensuring that AI systems behave in ways consistent with human intentions and values.
00:07:02.000 -->
00:07:07.600And as AI systems become more capable and autonomous, alignment becomes more difficult.
00:07:07.839 -->
00:07:21.519OpenAI's recent paper on building standards for the next phase of AI says that its mission is to ensure that AGI benefits humanity and identifies automated AI research as an important part of the next phase of development.
00:07:21.839 -->
00:07:24.160It also makes an important distinction.
00:07:24.399 -->
00:07:29.920OpenAI says that automated AI research can involve varying degrees of human supervision.
00:07:30.160 -->
00:07:41.439As AI systems take on more of the work involved in developing successive generations of AI, they can increasingly drive the process of recursive self-improvement, even while humans remain involved.
00:07:41.680 -->
00:07:46.560That means RSI does not necessarily mean removing humans from the loop immediately.
00:07:46.720 -->
00:07:53.680There can be stages, oversight, evaluations, limits, and there can be decisions about when humans must intervene.
00:07:54.000 -->
00:07:58.160Open AI also makes a fascinating argument about the potential benefits.
00:07:58.399 -->
00:08:03.759An automated AI researcher could potentially become an automated AI safety researcher.
00:08:04.000 -->
00:08:10.319In other words, the same technology that helps make AI more capable could also help us make AI safer.
00:08:10.639 -->
00:08:21.439A more capable AI system could potentially identify vulnerabilities, test alignment techniques, improve cyber defences, and help researchers understand increasingly sophisticated AI behavior.
00:08:21.920 -->
00:08:23.759That is the optimistic case.
00:08:24.000 -->
00:08:29.040The concern is that capability could advance faster than our ability to maintain control.
00:08:29.519 -->
00:08:38.080OpenAI explicitly says that fully autonomous RSI is not happening today and should not be pursued unless and until it can be done safely.
00:08:38.320 -->
00:08:45.279It also says that progress in alignment and safety cannot simply be assumed to keep pace with the progress in capabilities.
00:08:45.519 -->
00:08:47.279That is an important admission.
00:08:47.759 -->
00:08:54.639Because the traditional model of technology development assumes that if we can build something, we can gradually learn how to control it.
00:08:54.879 -->
00:08:58.960With increasingly autonomous AI, that assumption becomes less comfortable.
00:08:59.200 -->
00:09:06.480The system itself is becoming part of the research process, and potentially part of the process for making the system safer as well.
00:09:06.720 -->
00:09:09.039This creates an unusual situation.
00:09:09.279 -->
00:09:14.960We are effectively asking AI to help us solve the problem of safety developing more powerful AI.
00:09:15.200 -->
00:09:23.679That could be enormously valuable, but it also means that the safety mechanism itself increasingly depends upon the technology it is designed to control.
00:09:24.480 -->
00:09:28.799This is one reason why international standards are becoming part of the conversation.
00:09:29.039 -->
00:09:37.440OpenAI is calling for common international standards around frontier AI, including standards relating specifically to recursive self-improvement.
00:09:37.679 -->
00:09:50.480The proposal includes common approaches to measuring AI capabilities, evaluating how much autonomous research is being conducted, establishing appropriate levels of human oversight, and classifying and reporting incidents.
00:09:50.720 -->
00:09:53.039The idea is relatively straightforward.
00:09:53.200 -->
00:10:01.440If different AI companies and different countries measure capabilities in completely different ways, it becomes much harder to understand what is actually happening.
00:10:01.679 -->
00:10:06.240A common measurement framework could provide something closer to a shared language.
00:10:06.559 -->
00:10:08.080Think about aviation.
00:10:08.320 -->
00:10:14.639Aircraft manufacturers and airlines compete fiercely, but they operate within common safety frameworks.
00:10:14.879 -->
00:10:19.679There are standards for aircraft, communication, instant reporting, and safety procedures.
00:10:19.919 -->
00:10:23.440The existence of those standards does not mean innovation stops.
00:10:23.679 -->
00:10:28.320The argument from OpenAI is that Frontier AI could benefit from a similar approach.
00:10:28.559 -->
00:10:33.919That is particularly important because AI development is not confined to one company or one country.
00:10:34.159 -->
00:10:40.159If one organisation decides to slow down while another accelerates, the incentives become complicated.
00:10:40.399 -->
00:10:47.039If one country introduces strict controls while another doesn't, the same problem appears at an international level.
00:10:47.279 -->
00:10:57.600And if recursive self-improvement rarely does have the potential to accelerate AI development, then the consequences of one organisation moving faster could potentially affect everyone.
00:10:57.919 -->
00:11:06.799This is why RSI is becoming much more than a technical research question, is becoming a question about governance, economics, competition, and the future of work.
00:11:07.360 -->
00:11:09.759Let's consider the economic implications.
00:11:10.000 -->
00:11:18.000If AI becomes substantially better at conducting AI research, then the cost of developing increasingly sophisticated AI could fall.
00:11:18.240 -->
00:11:21.600That could accelerate innovation across almost every sector.
00:11:21.840 -->
00:11:40.480Scientific discovery could potentially become faster, drug discovery could become more efficient, engineering and material science could benefit, software development could change dramatically, and businesses could potentially have access to levels of computational intelligence that would previously have required very large teams of highly skilled people.
00:11:40.799 -->
00:11:43.039This is the positive economic argument.
00:11:43.200 -->
00:11:44.480But there is another side.
00:11:44.720 -->
00:11:54.559If AI becomes capable of performing more of the research required to improve AI, then the value of certain highly specialized human skills could change very quickly.
00:11:54.720 -->
00:11:58.000And we've already seen the beginnings of this in software engineering.
00:11:58.320 -->
00:12:01.360AI coding tools can write substantial amounts of code.
00:12:01.600 -->
00:12:04.480The next step is not simply better code generation.
00:12:04.720 -->
00:12:15.840It is AI systems capable of planning software projects, writing code, testing it, debugging it, evaluating alternatives, and potentially managing increasingly complex development processes.
00:12:16.080 -->
00:12:20.080Now imagine applying the same concept to AI research itself.
00:12:20.320 -->
00:12:25.519That is where the potential acceleration comes from, and it is also where the uncertainty comes from.
00:12:25.759 -->
00:12:32.000However, this may not come all at once, and instead there could be a series of significant milestones on this journey.
00:12:32.240 -->
00:12:36.639One proposed framework talks about adequacy, parity, and supremacy.
00:12:36.879 -->
00:12:46.799Adequacy means an AI system could conduct AI research without humans being directly involved, even if the quality of that research is still below human performance.
00:12:47.120 -->
00:12:52.399Parity would mean AI only research becomes roughly as effective as human-only research.
00:12:52.639 -->
00:12:59.519And supremacy would mean AI working without humans is actually more effective than the combination of humans and AI.
00:12:59.759 -->
00:13:06.639Those are very different milestones, and it is entirely possible that we reach one without immediately reaching the others.
00:13:06.960 -->
00:13:16.000It is important because public discussions about AI sometimes jumps from relatively modest examples of automation directly to the idea of superintelligence.
00:13:16.240 -->
00:13:19.039The reality is likely to be much messier.
00:13:19.279 -->
00:13:35.360There will be failures, bottlenecks, limitations in computing power, problems with data, unreliable experiments, challenges around verification, and there will be questions about whether AI systems can genuinely understand the goals and priorities involved in long-term research.
00:13:35.600 -->
00:13:43.919Some current AI systems may be very good at coding and technical tasks, but still struggle with things humans routinely do when conducting research.
00:13:44.159 -->
00:13:56.960They can struggle when deciding what matters, ambiguous objectives, verification, understanding organizational priorities, and they can struggle with knowing whether an apparently successful result is actually a meaningful result.
00:13:57.279 -->
00:14:05.840Those weaknesses matter because research is not simply about generating more experiments, it's about deciding which experiments are worth conducting.
00:14:06.159 -->
00:14:11.360That's one reason I think the phrase recursive self-improvement can sometimes be misleading.
00:14:11.519 -->
00:14:15.840It makes it sound as though AI simply presses a button and becomes better.
00:14:16.000 -->
00:14:24.000In reality, intelligence improvement is likely to involve a complicated chain of research, experimentation, evaluation and validation.
00:14:24.240 -->
00:14:27.440The question is how much of that chain can be automated?
00:14:27.759 -->
00:14:30.320And that is precisely where the technology is moving.
00:14:30.559 -->
00:14:39.039Anthropic's progress is particularly interesting because it provides a concrete example of AI becoming increasingly involved in the creation of future AI systems.
00:14:39.279 -->
00:14:51.279OpenAI has also described an automated research intern capable of carrying out well-defined research tasks under human direction, with an ambition to develop an automated AI researcher.
00:14:51.519 -->
00:14:56.000And other companies have discussed reducing the amount of human involvement in model development.
00:14:56.399 -->
00:14:58.799So we are seeing different approaches emerging.
00:14:59.039 -->
00:15:09.759Some organisations are emphasising human oversight, some are pursuing increasingly autonomous systems, while others are talking about keeping highly capable AI within carefully defined boundaries.
00:15:10.000 -->
00:15:14.080There is not yet a consensus about exactly where the right boundary lies.
00:15:14.240 -->
00:15:17.519And perhaps that is the most important message from all of this.
00:15:17.840 -->
00:15:24.480RSI should not be treated as either an inevitable doomsday event or an inevitable technology utopia.
00:15:24.639 -->
00:15:28.879It is a technological trajectory, and trajectories can be influenced.
00:15:29.120 -->
00:15:37.360The critical issue is whether we can build the measurement, governance, and safety mechanisms quickly enough to understand what is happening while it is happening.
00:15:37.600 -->
00:15:45.679Because once AI becomes significantly involved in AI research, the speed of development could become less dependent on the number of humans we have available.
00:15:45.840 -->
00:15:48.799They could fundamentally change the innovation cycle.
00:15:49.039 -->
00:15:51.840For technology leaders, this is particularly important.
00:15:52.080 -->
00:15:54.879We tend to think about AI in terms of productivity.
00:15:55.120 -->
00:15:57.759How can AI make our employees more productive?
00:15:58.000 -->
00:16:03.919How can it automate processes, improve customer experience, reduce costs, or make us make better decisions?
00:16:04.159 -->
00:16:08.720Those questions remain important, but RSI introduces another question.
00:16:08.960 -->
00:16:13.679What happens when AI starts improving the technology that is improving our businesses?
00:16:13.840 -->
00:16:16.080That creates a second-order effect.
00:16:16.320 -->
00:16:24.639Instead of simply deploying AI, organizations may increasingly operate in an environment where the underlying capability of AI itself is accelerating.
00:16:24.879 -->
00:16:31.519For CIOs, CTOs and chief digital officers, that means that the strategic planning horizon becomes much harder.
00:16:31.759 -->
00:16:38.399A three-year technology roadmap assumes that the technology being planned around will evolve at a reasonably predictable rate.
00:16:38.639 -->
00:16:42.080Recursive self-improvement potentially challenges that assumption.
00:16:42.320 -->
00:16:47.679The capabilities available in two years could be substantially different from the capabilities available today.
00:16:47.919 -->
00:16:55.759That does not mean that organizations should simply abandon long-term planning, it means they need to build greater adaptability into those plans.
00:16:56.000 -->
00:17:02.480The winners in this environment may not necessarily be the organizations that predict exactly what AI will look like in three years.
00:17:02.639 -->
00:17:06.880They may be the organizations capable of adapting quickly when it changes.
00:17:07.119 -->
00:17:09.839There is also a human leadership question here.
00:17:10.079 -->
00:17:18.880If AI becomes increasingly capable of conducting research, writing software, and making technical decisions, the role of the human technology leader could shift.
00:17:19.119 -->
00:17:29.359It could become less about personally understanding every technical detail and more about establishing the objectives, boundaries, governance, and culture in which the AI operates.
00:17:29.839 -->
00:17:31.599That is a profound shift.
00:17:31.839 -->
00:17:36.079We spent decades building systems that follow instructions created by humans.
00:17:36.319 -->
00:17:42.880We are now building systems that can increasingly interpret objectives and work towards them with much greater autonomy.
00:17:43.039 -->
00:17:46.559And recursive self-improvement takes that one step further.
00:17:46.880 -->
00:17:50.799The system could potentially help determine how the next system is built.
00:17:51.039 -->
00:17:53.279That is why alignment matters so much.
00:17:53.440 -->
00:18:05.119The goal cannot simply be to create an AI system that is extremely intelligent, it has to be an AI system whose increasing intelligence remains compatible with human control and human interests.
00:18:05.359 -->
00:18:12.160The AI industry is beginning to recognise that the next phase cannot be managed purely through bigger models and more computing power.
00:18:12.400 -->
00:18:20.880It requires institutions, standards, measurements and cooperation, and that may ultimately be just as important as the technology itself.
00:18:21.279 -->
00:18:23.119So where does this leave us now?
00:18:23.440 -->
00:18:30.480The short answer is that recursive self-improvement is becoming real in pieces, but fully autonomous RSI is not here yet.
00:18:30.799 -->
00:18:39.279AI is already helping researchers build better AI, writing significant amounts of software, and is conducting experiments and contributing to model research.
00:18:39.519 -->
00:18:52.480Companies are actively working towards increasingly automated AI research, but there is a significant gap between that and an AI system independently improving itself through an open-ended loop with no meaningful human involvement.
00:18:52.720 -->
00:18:56.480That gap could be difficult to cross, it could take years.
00:18:56.799 -->
00:18:59.519Or progress could accelerate unexpectedly.
00:18:59.680 -->
00:19:03.279The uncertainty is precisely why the subject deserves attention.
00:19:03.519 -->
00:19:18.160The most interesting question about RSI may therefore not be whether the machines will suddenly become superintelligent, it's whether the boundary between humans developing AI and AI developing AI will continue to move, because that boundary is already moving.
00:19:18.319 -->
00:19:29.920And once AI becomes increasingly capable of improving the systems that come after it, the pace in which technology changes could become something that businesses, governments and individuals may never have experienced before.
00:19:30.240 -->
00:19:34.240For technology leaders, that means that the conversation around AI needs to evolve.
00:19:34.400 -->
00:19:37.440It is no longer enough to ask what today's AI can do.
00:19:37.599 -->
00:19:43.119We need to think about what happens when tomorrow's AI is partly responsible for creating the AI that follows it.
00:19:43.359 -->
00:19:46.559That is the real significance of recursive self-improvement.
00:19:46.640 -->
00:19:48.880It is not simply another AI acronym.
00:19:49.039 -->
00:19:52.960It could represent a fundamental change in how technology progress happens.
00:19:53.200 -->
00:20:04.640And perhaps the most important challenge for the years ahead will be making sure that as machines become increasingly capable of improving themselves, humans remain capable of understanding, directing, and controlling the process.
00:20:05.039 -->
00:20:06.160Well that's all for today.
00:20:06.240 -->
00:20:08.480Thanks for tuning in to the Inspiring Tech Leaders podcast.
00:20:08.559 -->
00:20:14.160If you found this episode useful, make sure to subscribe, turn on notifications, and share the episode with your network.
00:20:14.319 -->
00:20:19.200You can find more insights, show notes, and resources at www.inspiringtechleaders.com.
00:20:19.359 -->
00:20:23.920Head over to the social media channels you can find Inspiring Tech Leaders on X, Instagram, Inspo, and TikTok.
00:20:24.000 -->
00:20:26.160And let me know your thoughts about RSI.
00:20:26.400 -->
00:20:32.000Thanks for listening, and until next time, stay curious, stay connected, and keep pushing the boundaries of what's possible in tech.
