World In-Context Learning for Agentic Robot Manipulation bölümünün tam dökümü

and world feedback, proposing a robust framework for agent-centric robot manipulation. 48 00:04:34,458 --> 00:04:43,228 Its capability to generalize across different environments and improve through reflection holds promise for future advancements in autonomous robotics. 49 00:04:43,332 --> 00:04:46,132 Evan: And that wraps up the introduction section of the paper. 50 00:04:46,132 --> 00:04:52,292 Stay tuned as we delve deeper into the methods and experiments that bring this innovative approach to life. 51 00:04:53,546 --> 00:04:54,806 Evan: Welcome back. 52 00:04:54,806 --> 00:05:04,266 Now, let's dive into the methods section of this paper and see how the researchers have designed the framework for World In-Context Learning, or WICL. 53 00:05:04,322 --> 00:05:12,282 Ashley: To implement WICL, the researchers developed RoboScript, a modular infrastructure tailored for agentic robot manipulation. 54 00:05:12,282 --> 00:05:19,522 This system is structured around four key stages: skill building, task learning, deployment, and reflection. 55 00:05:19,586 --> 00:05:22,026 Evan: Skill building sounds crucial. 56 00:05:22,026 --> 00:05:23,346 What does it involve? 57 00:05:23,402 --> 00:05:27,062 Ashley: Skill building is the first stage in RoboScript's pipeline. 58 00:05:27,082 --> 00:05:33,912 Here, skills are developed and categorized into shared entry points and environment-specific skills. 59 00:05:33,912 --> 00:05:43,902 Shared entry points include operations like observing, grasping, placing, corrective motions, reachability checks, and coordinated bimanual motion. 60 00:05:43,902 --> 00:05:46,762 These operations are environment-independent. 61 00:05:46,886 --> 00:05:50,296 Evan: And what about environment-specific skills? 62 00:05:50,384 --> 00:05:51,704 Ashley: Great question. 63 00:05:51,704 --> 00:06:01,244 Environment-specific skills involve unique conventions of the operational environment, like coordinate frames, gripper characteristics, and collision worlds. 64 00:06:01,244 --> 00:06:09,264 RoboScript first segments each demonstration into action primitives and identifies gaps that existing skills cannot cover. 65 00:06:09,264 --> 00:06:14,114 These gaps are then addressed by a coding agent that writes new skills to cover them. 66 00:06:14,192 --> 00:06:16,472 Evan: Next, we have task learning. 67 00:06:16,472 --> 00:06:18,532 How does RoboScript approach this? 68 00:06:18,578 --> 00:06:24,508 Ashley: Task learning in RoboScript is realized through a structured document known as the Task Memory. 69 00:06:24,508 --> 00:06:28,738 The process comprises two stages—imitation and rehearsal. 70 00:06:28,758 --> 00:06:35,708 In the imitation stage, a demonstration reader studies the recorded demonstration and drafts the initial Task Memory. 71 00:06:35,708 --> 00:06:42,538 The agent then practices the task, revising the memory based on its attempts until consistent success is achieved. 72 00:06:42,602 --> 00:06:45,522 Evan: What does rehearsal

add to this process? 73 00:06:45,578 --> 00:06:52,158 Ashley: Rehearsal extends the learning to new training instances, ensuring the agent can generalize its learned skills. 74 00:06:52,158 --> 00:07:02,688 This stage involves practicing on different instances, revising the Task Memory based on successful retries, and gradually stabilizing the learned specifications and parameters. 75 00:07:02,738 --> 00:07:03,778 Evan: That makes sense. 76 00:07:03,778 --> 00:07:05,438 Let's move on to deployment. 77 00:07:05,438 --> 00:07:11,008 How does RoboScript ensure the learned skills are effectively utilized in real-world scenarios? 78 00:07:11,066 --> 00:07:16,826 Ashley: At deployment, the agent acts with the frozen Task Memory on new, unseen instances. 79 00:07:16,826 --> 00:07:25,996 It anticipates how the task should unfold, verifies the final state against the demonstration, and recovers from any failures without resetting the environment. 80 00:07:25,996 --> 00:07:31,666 This approach ensures the task’s completion despite potential disruptions throughout the process. 81 00:07:31,730 --> 00:07:36,930 Evan: And finally, reflection—how does this contribute to the overall learning process? 82 00:07:36,986 --> 00:07:39,846 Ashley: Reflection is a key feature of WICL. 83 00:07:39,846 --> 00:07:46,396 After evaluating an episode, the reflection agent reviews the episode records and identifies failure patterns. 84 00:07:46,396 --> 00:07:52,226 It writes guidance documents for the next round, realizing Recursive Self-Improvement (RSI). 85 00:07:52,226 --> 00:07:58,256 This approach enables the model to iteratively improve without gradient updates or changes in weights. 86 00:07:58,322 --> 00:08:04,902 Evan: So the reflection agent essentially guides the next steps based on previous performance? 87 00:08:04,946 --> 00:08:06,036 Ashley: Exactly. 88 00:08:06,036 --> 00:08:17,366 And what's remarkable is that the agent uses only deployable observations—those that a real robot would have access to—which ensures the robustness and generalizability of the learned skills. 89 00:08:17,486 --> 00:08:23,506 Evan: All of these components seem meticulously designed to address various challenges in robotic manipulation. 90 00:08:23,506 --> 00:08:26,726 What about the physical deployment on different robot embodiments? 91 00:08:26,786 --> 00:08:31,716 Ashley: RoboScript’s adaptability across different robot embodiments is indeed impressive. 92 00:08:31,716 --> 00:08:39,926 The paper demonstrates successful deployment of Task Memories on a physical dual-arm robot, validating cross-embodiment memory transfer. 93 00:08:39,926 --> 00:08:46,536 This adaptation is achieved using a lightweight adapter that remaps embodiment-specific parameters for the new robot. 94 00:08:46,536 --> 00:08:50,686 There's no need for re-recording the demonstration or repeating the rehearsal. 95 00:08:50,738 --> 00:08:54,848 Evan: That's significant, especially in real-world applications. 96 00:08:54,848 --> 00:09:01,358 How did they ensure that the learned Task Memories function well in both simulated and real environments? 97 00:09:01,418 --> 00:09:05,398 Ashley: The researchers enforced strict data isolation during learning. 98 00:09:05,398 --> 00:09:13,488 This means every acting agent only observes inputs available on a real robot, such as camera images with depth and proprioception. 99 00:09:13,538 --> 00:09:17,998 Privileged simulator information is used purely for supervision during learning. 100 00:09:17,998 --> 00:09:24,538 This ensures the learned Task Memories are deployable on physical robots without modifying the learned context. 101 00:09:24,602 --> 00:09:31,002 Evan: It sounds like WICL effectively enables a robot to learn and generalize tasks in a semantic context. 102 00:09:31,002 --> 00:09:34,362 Can you explain how they validated the approach experimentally? 103 00:09:34,418 --> 00:09:35,568 Ashley: Definitely. 104 00:09:35,588 --> 00:09:44,598 The paper presents extensive experiments involving RoboTwin 2.0, which defines 50 dual-arm tasks in clean and randomized settings. 105 00:09:44,648 --> 00:10:00,888 Tasks were evaluated on episodes requiring dozens of model calls, and RoboScript achieved 83.4% mean success across clean and randomized settings using GPT-6.1-Sol, significantly outperforming the strongest trained policies. 106 00:10:00,998 --> 00:10:02,958 Evan: Such results are profound. 107 00:10:02,958 --> 00:10:06,798 They conducted these experiments using different models as well, correct? 108 00:10:06,842 --> 00:10:18,612 Ashley: Yes, they tested with GPT-6.1-Sol, GPT-5.6-Luna, and DeepSeek-V4.1-Flash, maintaining high success rates across all models. 109 00:10:18,612 --> 00:10:28,812 This consistency across models underscores the scalability and robustness of the WICL framework, making it a versatile solution for diverse deployment scenarios. 110 00:10:28,874 --> 00:10:31,204 Evan: That's a thorough validation for sure. 111 00:10:31,204 --> 00:10:33,664 And now, that concludes the methods section. 112 00:10:33,664 --> 00:10:37,524 Next, we’ll explore the experimental results and discuss related work. 113 00:10:37,524 --> 00:10:38,574 Stay tuned! 114 00:10:39,819 --> 00:10:41,229 Evan: Welcome back, listeners. 115 00:10:41,229 --> 00:10:44,289 We've covered the introduction and methods sections of the paper. 116 00:10:44,289 --> 00:10:51,749 Now, let's dive into the experiments and results that validate the World In-Context Learning (WICL) approach. 117 00:10:51,795 --> 00:10:58,825 Ashley: The experiments were extensive, focusing on validating WICL using the RoboTwin 2.0 benchmark. 118 00:10:58,825 --> 00:11:08,935 RoboTwin 2.0 defines 50 dual-arm tasks divided into clean and randomized settings, each presenting unique challenges for robotic manipulation. 119 00:11:08,979 --> 00:11:12,799 Evan: What kind of results can we expect from these experiments? 120 00:11:12,843 --> 00:11:22,793 Ashley: Firstly,

the researchers evaluated RoboScript using a single clean demonstration per task followed by rehearsal in simulation, without any gradient updates. 121 00:11:22,793 --> 00:11:35,273 The results were impressive—RoboScript achieved a mean success rate of 83.4% across clean and randomized settings, which exceeds the strongest trained action policy by 21.7 points. 122 00:11:35,331 --> 00:11:38,081 Evan: That’s quite a significant improvement. 123 00:11:38,081 --> 00:11:41,561 How did they compare this with traditional trained policies? 124 00:11:41,619 --> 00:11:52,109 Ashley: Indeed, they compared it against Vision-Language-Action models (VLAs) and World Action Models (WAMs) trained with 50 clean demonstrations per task. 125 00:11:52,109 --> 00:12:02,559 For instance, the strongest trained policy, 4D-WAM, reached 61.7% mean success across clean and randomized settings. 126 00:12:02,559 --> 00:12:05,769 RoboScript’s performance shows a remarkable leap. 127 00:12:05,835 --> 00:12:07,265 Evan: That is impressive! 128 00:12:07,265 --> 00:12:09,685 How did they ensure these tests were robust? 129 00:12:09,747 --> 00:12:17,097 Ashley: The robustness came from extensive evaluations—10 episodes per task in each setting, totaling 1,000 episodes. 130 00:12:17,097 --> 00:12:24,767 They ensured the tests covered various aspects, from unseen object instances to performance retention under randomization. 131 00:12:24,889 --> 00:12:30,399 Evan: Can you highlight some specific results regarding retention under randomization? 132 00:12:30,489 --> 00:12:31,579 Ashley: Of course. 133 00:12:31,599 --> 00:12:43,449 RoboScript retained over 95% of its clean success rate under randomized settings, demonstrating that the semantic knowledge in the Task Memory generalizes well across different environments. 134 00:12:43,469 --> 00:12:50,419 In contrast, traditional trained policies retained at most 65.1% of their clean success rate. 135 00:12:50,475 --> 00:12:54,495 Evan: That certainly highlights the value of semantic knowledge in context. 136 00:12:54,495 --> 00:12:56,505 What else did the experiments reveal? 137 00:12:56,571 --> 00:13:06,831 Ashley: The researchers also tested different foundation models—GPT-6.1-Sol, GPT-5.6-Luna, and DeepSeek-V4.1-Flash. 138 00:13:06,831 --> 00:13:14,671 All models performed exceptionally well, with success rates ranging from 77.9% to 83.4%. 139 00:13:14,671 --> 00:13:21,301 This underscores the versatility and scalability of the WICL framework across various foundation models. 140 00:13:21,443 --> 00:13:28,793 Evan: How did they break down the contributions of different components like demonstrations and world feedback? 141 00:13:28,851 --> 00:13:37,771 Ashley: They conducted detailed ablation studies, isolating the contributions of demonstrations, rehearsal, task memory, and various deployment protocols. 142 00:13:37,771 --> 00:13:43,911 For example, removing the world feedback dropped the success rate significantly, indicating its crucial role. 143 00:13:43,911 --> 00:13:52,191 Without the world, success rates were around 69%, but with rehearsal in simulation, it increased to 78.5%. 144 00:13:52,251 --> 00:13:55,531 Evan: What about the real-world application of these Task Memories? 145 00:13:55,587 --> 00:14:04,337 Ashley: They tested RoboScript’s Task Memories on a physical dual-arm PiPER robot, demonstrating successful cross-embodiment memory transfer. 146 00:14:04,337 --> 00:14:16,287 The adapted Task Memories achieved high success rates on tasks such as Click Bell, Grab Roller, and Stack Bowls Two, validating the method’s practicality and effectiveness in real-world scenarios. 147 00:14:16,347 --> 00:14:21,967 Evan: Did the paper discuss cost implications of using these foundation models for learning and deployment? 148 00:14:22,011 --> 00:14:23,351 Ashley: Yes, they did. 149 00:14:23,351 --> 00:14:31,141 They reported the costs associated with different stages—demonstration attempts, rehearsal instances, and evaluation episodes. 150 00:14:31,141 --> 00:14:46,241 For instance, the total cost of learning and evaluating tasks using GPT-6.1-Sol was around 1,911 USD, whereas with DeepSeek-V4.1-Flash, it was 739 USD. 151 00:14:46,241 --> 00:14:53,761 Despite the costs, once the Task Memory is frozen, it can be reused across episodes without additional learning expenses. 152 00:14:53,811 --> 00:15:01,271 Evan: It seems like they’ve covered all bases to ensure the approach is well-validated both in terms of performance and practicality. 153 00:15:01,323 --> 00:15:15,123 Ashley: The extensive experiments and detailed analyses provide strong evidence for the effectiveness of WICL and RoboScript in both simulated and real environments, showcasing significant advancements in robotic manipulation. 154 00:15:15,171 --> 00:15:17,571 Evan: That concludes the experiments section. 155 00:15:17,571 --> 00:15:23,181 Next, we'll discuss related work and see how this approach compares to other methods in the field. 156 00:15:23,181 --> 00:15:24,271 Stay with us! 157 00:15:25,557 --> 00:15:27,097 Evan: Welcome back, listeners. 158 00:15:27,097 --> 00:15:40,397 Now that we’ve covered the methods and experiments, let's delve into the related work section of the paper and explore how World In-Context Learning (WICL) compares to other approaches in robotic manipulation. 159 00:15:40,445 --> 00:15:51,055 Ashley: To give you a clearer picture of how WICL stands in the existing landscape, the paper reviews several key areas within robotic manipulation and related AI approaches. 160 00:15:51,161 --> 00:15:54,471 Evan: Let's start with action policies for manipulation. 161 00:15:54,471 --> 00:15:57,541 How has the dominant approach changed over time? 162 00:15:57,605 --> 00:16:04,945 Ashley: Traditionally, robot manipulation has relied on trained action policies that store skill directly in network weights. 163 00:16:04,945 --> 00:16:12,025 Vision-language-action (VLA) models and world action

models (WAMs) have been extensively used. 164 00:16:12,025 --> 00:16:18,315 VLAs generally learn visuo-motor skills from demonstrations and large cross-embodiment datasets. 165 00:16:18,315 --> 00:16:24,575 WAMs typically involve generating future observations or latent plans and deriving actions from them. 166 00:16:24,699 --> 00:16:29,429 Evan: Has reinforcement learning also played a significant role in this space? 167 00:16:29,507 --> 00:16:30,667 Ashley: Definitely. 168 00:16:30,667 --> 00:16:37,567 Reinforcement learning optimizes action policies based on feedback from either simulations or real-world trials. 169 00:16:37,567 --> 00:16:48,917 While advancements in these approaches have been substantial, the main limitation persists: skills are tied to the distributions they are trained on, causing performance degradation under distribution shifts. 170 00:16:48,965 --> 00:16:50,295 Evan: That makes sense. 171 00:16:50,295 --> 00:16:53,995 How does WICL offer a different route compared to these methods? 172 00:16:54,053 --> 00:17:01,773 Ashley: WICL keeps the model’s weights frozen and stores learned knowledge in context, allowing task rehearsal rather than retraining. 173 00:17:01,773 --> 00:17:11,103 This decouples the learned skills from the distribution constraints inherent in trained policies, paving the way for better generalization across different environments. 174 00:17:11,165 --> 00:17:15,615 Evan: What about in-context manipulation, which ICL typically covers? 175 00:17:15,677 --> 00:17:24,287 Ashley: In robotics, in-context learning has largely been used in the form of imitation learning, with methods interpreting demonstrations given at test time. 176 00:17:24,287 --> 00:17:31,397 While one-shot and few-shot methods have shown promise, they still lack the robustness provided by comprehensive world feedback. 177 00:17:31,397 --> 00:17:40,157 Retrieval methods have also been employed, where demonstrations are fetched for trained policies to follow, but these still tie back to trained network weights. 178 00:17:40,205 --> 00:17:46,765 Evan: Have there been efforts to use frozen models as agents or compose APIs and value maps? 179 00:17:46,829 --> 00:17:55,439 Ashley: Yes, recent efforts involve training-free systems that compose APIs, value maps, and keyframes to control robot actions. 180 00:17:55,439 --> 00:17:59,829 Hierarchical systems integrate language models above trained policies. 181 00:17:59,829 --> 00:18:06,769 Embodied harnesses augment this with tools, verification mechanisms, and memory to aid in task execution. 182 00:18:06,769 --> 00:18:13,459 However, these approaches don’t typically distill experience from interaction with the world into reusable memory. 183 00:18:13,577 --> 00:18:19,197 Evan: How does RoboScript and WICL address the missing feedback loop then? 184 00:18:19,253 --> 00:18:24,393 Ashley: RoboScript and WICL fill in this gap by providing the missing world feedback. 185 00:18:24,393 --> 00:18:35,383 WICL complements in-context learning by recording outcomes from rehearsal, articulating iterative improvements, and enabling the agent to generalize across unseen instances. 186 00:18:35,429 --> 00:18:38,969 Evan: What about self-improving agents and world feedback? 187 00:18:38,969 --> 00:18:40,919 How is this relevant to WICL? 188 00:18:40,973 --> 00:18:49,033 Ashley: Self-improving agents, notably language model agents, have shown the ability to improve from interaction while keeping their weights frozen. 189 00:18:49,033 --> 00:18:57,273 Methods like Reflexion, Voyager, and ExpeL build libraries of skills or insights from previous trials, facilitating self-improvement. 190 00:18:57,273 --> 00:19:07,073 In physical manipulation, this principle becomes even more critical, as the learned knowledge needs to transfer across different types of worlds, maintaining generalization. 191 00:19:07,163 --> 00:19:12,793 Evan: Applying these principles to physical robot manipulation must come with additional requirements. 192 00:19:12,793 --> 00:19:13,783 What are those? 193 00:19:13,829 --> 00:19:14,699 Ashley: Indeed. 194 00:19:14,699 --> 00:19:21,979 When applied to physical manipulation, the context must be grounded in observations of a scene that evolves as the robot acts. 195 00:19:21,979 --> 00:19:28,259 The feedback loop includes physical outcomes such as whether an object was successfully grasped or placed. 196 00:19:28,259 --> 00:19:36,779 The context needs to store semantic knowledge independent of specific world appearances, ensuring the learned skills remain universally applicable. 197 00:19:36,845 --> 00:19:43,945 Evan: WICL certainly appears to offer a comprehensive framework addressing key limitations in existing approaches. 198 00:19:43,945 --> 00:19:47,165 What about its feasibility and practical applications? 199 00:19:47,213 --> 00:19:56,223 Ashley: The feasibility of WICL is greatly supported by advances in foundation models capable of long-context interaction and experience accumulation. 200 00:19:56,223 --> 00:20:05,413 Simulators and robot APIs have matured to provide realistic environments for rehearsal and safe feedback, essential for learning accurately and efficiently. 201 00:20:05,413 --> 00:20:13,593 Furthermore, WICL's structured learning pipeline ensures the learnt task memories are ready for both virtual and real-world deployment. 202 00:20:13,637 --> 00:20:19,617 Evan: It's remarkable how WICL integrates various effective strategies to advance robotic manipulation. 203 00:20:19,617 --> 00:20:21,797 That wraps up our related work section. 204 00:20:23,046 --> 00:20:24,536 Evan: Welcome back, listeners. 205 00:20:24,536 --> 00:20:33,386 We've explored the World In-Context Learning (WICL) approach and its various components, experiments, and related methods. 206 00:20:33,386 --> 00:20:38,216 Before we close, let's summarize the key contributions and takeaways from this paper. 207 00:20:38,262 --> 00:20:39,642 Ashley: Certainly, Evan. 208 00:20:39,642 --> 00:20:53,502 This paper introduces WICL, which extends in-context learning by integrating feedback from a world, allowing robots to learn manipulation tasks within a semantic context rather than embedding skills in network weights. 209 00:20:53,502 --> 00:20:58,252 This enables better generalization across different environments without retraining. 210 00:20:58,302 --> 00:21:06,982 Evan: The authors develop RoboScript, a modular infrastructure that implements WICL, ensuring that learned tasks are stored as Task Memories. 211 00:21:06,982 --> 00:21:15,342 This memory-based approach facilitates deployment in varied real-world scenarios and different robot embodiments, all without gradient updates. 212 00:21:15,390 --> 00:21:31,210 Ashley: They conducted extensive experiments using RoboTwin 2.0, which demonstrated RoboScript's ability to achieve an 83.4% mean success rate across clean and randomized settings, significantly outperforming traditional trained policies. 213 00:21:31,210 --> 00:21:39,650 Validation included testing across multiple foundation models and real robot deployment, highlighting its scalability and robustness. 214 00:21:39,762 --> 00:21:52,422 Evan: In essence, WICL leverages the advantages of semantic knowledge embedded in task context, enabling robots to perform complex manipulation tasks with high adaptability and minimal retraining. 215 00:21:52,422 --> 00:21:57,582 This represents a significant advancement in the field of autonomous robotic manipulation. 216 00:21:57,630 --> 00:21:58,760 Ashley: Exactly. 217 00:21:58,760 --> 00:22:09,860 The approach not only enhances generalization and efficiency but also enables recursive self-improvement through reflection, making robotic systems more resilient and capable over time. 218 00:22:09,860 --> 00:22:16,550 The successful real-world application on different robot embodiments further emphasizes its practical potential. 219 00:22:16,614 --> 00:22:19,044 Evan: That's a wrap for today's episode. 220 00:22:19,044 --> 00:22:27,694 Thank you for joining us on Daily Paper Cast as we dissected 'World In-Context Learning for Agentic Robot Manipulation.' 221 00:22:27,750 --> 00:22:30,500 Ashley: We hope you found this discussion insightful. 222 00:22:30,500 --> 00:22:38,590 Make sure to tune in to our future episodes where we continue to bring you the latest research papers and innovations in AI and robotics. 223 00:22:38,696 --> 00:22:44,536 Evan: Until next time, stay curious and keep exploring the fascinating world of AI. 224 00:22:44,536 --> 00:22:45,406 Goodbye! 225 00:22:45,462 --> 00:22:46,482 Ashley: Goodbye!