Vollständiges Transkript von TailSteer: Trajectory-Tail Steering for Drift-Resistant Multi-turn Image Editing

this approach is 'data-free,' meaning it doesn't require manually curated ground-truth editing pairs. 47 00:04:35,846 --> 00:04:41,466 Instead, it leverages the frozen base editor's own outputs to create supervision. 48 00:04:41,522 --> 00:04:47,062 Evan: So, the model essentially learns from its own outputs? 49 00:04:47,114 --> 00:04:48,184 Ashley: Exactly. 50 00:04:48,204 --> 00:04:59,284 To get reliable supervision, the method involves two types of edits: identity editing, where no visual change is requested, and local editing, where a specific localized change is requested. 51 00:04:59,304 --> 00:05:12,054 For identity editing, this provides full-image preservation supervision, and for local editing, it confines the requested modification to a localized region, leaving the remaining content available for partial preservation supervision. 52 00:05:12,168 --> 00:05:16,258 Evan: How do they make sure the supervised edits are reliable? 53 00:05:16,322 --> 00:05:25,102 Ashley: For each edit type, the authors generate rollouts with the frozen base model and then filter them using corresponding identity or locality checks. 54 00:05:25,102 --> 00:05:29,592 Only trajectories that pass these checks are retained and cached for training. 55 00:05:29,592 --> 00:05:32,832 This ensures that the preservation regions are trustworthy. 56 00:05:32,882 --> 00:05:35,672 Evan: Smart way to make sure the model learns accurately! 57 00:05:35,672 --> 00:05:38,642 What’s next after gaining preservation-aware supervision? 58 00:05:38,690 --> 00:05:42,540 Ashley: The next step is testing a one-step terminal intervention. 59 00:05:42,540 --> 00:05:47,480 This is a minimal intervention approach where only the final sampling step is corrected. 60 00:05:47,480 --> 00:05:52,990 The goal here is to see how much improvement can be achieved by modifying just that last step. 61 00:05:53,102 --> 00:05:54,542 Evan: Interesting. 62 00:05:54,542 --> 00:05:57,842 And what does this intervention involve? 63 00:05:57,890 --> 00:06:04,830 Ashley: In this method, they initialize from the original base model and fine-tune it exclusively on the final sampling step. 64 00:06:04,830 --> 00:06:13,810 Starting from the cached pre-terminal state, the model is supervised toward the preservation-aware endpoint, while the preceding time steps remain unchanged. 65 00:06:13,810 --> 00:06:17,760 Essentially, they focus the correction only on the final update. 66 00:06:17,880 --> 00:06:22,490 Evan: How does this final-step correction affect multi-turn stability? 67 00:06:22,538 --> 00:06:32,258 Ashley: When applied, using the refined model for just the final update significantly reduces visible drift and preserves subject identity in long-horizon edits. 68 00:06:32,258 --> 00:06:43,948 However, if the fine-tuned model is applied across all steps, it can lead to desaturation and detail loss, highlighting the importance of preserving the original model’s dynamics in earlier steps. 69 00:06:44,064 --> 00:06:49,404 Evan: So, concentrating all correction in the final update has its limitations. 70 00:06:49,404 --> 00:06:51,424 How does TailSteer address this? 71 00:06:51,482 --> 00:06:57,452 Ashley: This brings us to the full TailSteer framework, which distributes corrections throughout the trajectory tail. 72 00:06:57,452 --> 00:07:09,432 TailSteer initializes from the frozen base model but trains the refined model progressively, focusing on preserving early denoising dynamics and gradually steering the tail towards a preservation-aware endpoint. 73 00:07:09,582 --> 00:07:16,972 Evan: Can you explain how they distribute these corrections through the denoising trajectory? 74 00:07:17,018 --> 00:07:17,998 Ashley: Sure. 75 00:07:17,998 --> 00:07:24,748 TailSteer introduces a concept called 'steering onset,' which is the point in the trajectory where corrections start. 76 00:07:24,768 --> 00:07:29,128 It partitions the trajectory into a prior region and a transition region. 77 00:07:29,218 --> 00:07:39,518 In the prior region, the refined model is trained to reproduce the base model’s dynamics, while in the transition region, the supervision gradually shifts toward the preservation endpoint. 78 00:07:39,578 --> 00:07:43,898 Evan: Does this involve a lot of computational overhead? 79 00:07:43,946 --> 00:07:45,146 Ashley: Not really. 80 00:07:45,166 --> 00:07:53,726 To obtain rollout states from the refined model without replaying the entire trajectory, cached base-model states are used as rollout anchors. 81 00:07:53,726 --> 00:07:56,586 This keeps early rollouts short and efficient. 82 00:07:56,606 --> 00:08:02,996 The key is that all supervision is constructed on states reached by the refined model’s own denoising rollouts. 83 00:08:03,110 --> 00:08:08,840 Evan: And how does TailSteer blend the velocities during these corrections? 84 00:08:08,906 --> 00:08:15,706 Ashley: The framework uses velocity blending, combining the base model’s velocity with the preservation-directed velocity. 85 00:08:15,706 --> 00:08:22,266 This blending shifts from the base model dynamics to the endpoint correction as it progresses through the transition region. 86 00:08:22,266 --> 00:08:31,186 At the final update, the target has the same endpoint-directed form we discussed previously, now evaluated at the state reached by the refined model. 87 00:08:31,250 --> 00:08:36,280 Evan: How do they ensure the model performs effectively at inference time? 88 00:08:36,338 --> 00:08:43,008 Ashley: TailSteer ensures effective inference by allowing the refined model to perform the entire sampling trajectory on its own. 89 00:08:43,008 --> 00:08:49,658 Auxiliary components like the frozen base model, preservation targets, and masks are used only during training. 90 00:08:49,658 --> 00:08:57,538 This makes TailSteer an intrinsic capability of the refined foundation model without requiring model switching or external adapters at inference. 91 00:08:57,602 --> 00:09:02,282 Evan: That

seems like a robust method to handle multi-turn editing. 92 00:09:02,282 --> 00:09:05,782 What practical advantages does TailSteer offer? 93 00:09:05,834 --> 00:09:35,214 Ashley: TailSteer offers three main practical advantages: it significantly improves long-horizon editing robustness while preserving the generative and editing capabilities acquired through earlier training; it generalizes across different model families and input settings since it operates directly on denoising trajectories without relying on specific representations or architectures; and it constructs preservation supervision from source images and the base model's own rollouts without needing curated ground-truth editing pairs. 94 00:09:35,214 --> 00:09:39,234 This scalability further enhances its robustness and reliability. 95 00:09:39,350 --> 00:09:42,900 Evan: It sounds like TailSteer is a versatile solution. 96 00:09:42,900 --> 00:09:45,330 How does it stack up against other methods? 97 00:09:45,386 --> 00:09:56,566 Ashley: Extensive evaluations show that TailSteer reduces cumulative drift in multi-turn editing substantially while retaining base-model performance across various editing tasks and benchmarks. 98 00:09:56,566 --> 00:10:03,946 It has proven to offer stronger long-horizon robustness compared to other methods, setting a new standard for iterative image editing. 99 00:10:04,010 --> 00:10:12,850 Evan: This concludes the Methods section of the paper 'TailSteer: Trajectory-Tail Steering for Drift-Resistant Multi-Turn Image Editing.' 100 00:10:14,115 --> 00:10:20,215 Evan: Let's now delve into the Experiment and Results section of the TailSteer paper. 101 00:10:20,215 --> 00:10:24,595 Ashley, what did the authors focus on in their evaluations? 102 00:10:24,651 --> 00:10:31,321 Ashley: The authors conducted extensive experiments to evaluate TailSteer's performance across multiple generative models. 103 00:10:31,321 --> 00:10:42,601 Specifically, they tested TailSteer on four foundation models from three different model families: FLUX1 Kontext, FLUX2 Klein, SenseNova, and Qwen. 104 00:10:42,651 --> 00:10:45,271 Evan: That’s quite a broad range of models. 105 00:10:45,271 --> 00:10:47,511 How did they go about these evaluations? 106 00:10:47,571 --> 00:10:57,481 Ashley: They collected nearly 12,000 candidate source images from Open Images and MONET datasets, paired each with several local-edit and identity-edit instructions. 107 00:10:57,481 --> 00:11:05,781 The foundation models were trained for 2,000 updates using BF16 precision on eight NVIDIA H100 GPUs. 108 00:11:05,781 --> 00:11:09,541 This ensured a thorough and robust training process for TailSteer. 109 00:11:09,603 --> 00:11:12,683 Evan: That's quite a comprehensive setup. 110 00:11:12,683 --> 00:11:16,453 What kind of benchmarks did they use to assess the performance? 111 00:11:16,515 --> 00:11:20,445 Ashley: They used several benchmarks for different aspects of the evaluation. 112 00:11:20,445 --> 00:11:34,825 For single-turn capability retention, text-to-image generation was assessed on Qwen-Image-Bench and BizGenEval, while single-turn image editing performance was evaluated on GEdit-Bench, RISEBench, and ImgEdit-Bench. 113 00:11:34,935 --> 00:11:39,195 Evan: How did TailSteer perform on these benchmarks? 114 00:11:39,243 --> 00:11:47,653 Ashley: TailSteer largely retained the base models’ capabilities across all evaluated backbones, with performance remaining close to that of the foundation models. 115 00:11:47,653 --> 00:11:58,793 There was minimal degradation in the quality of single-turn image editing, which was a significant achievement compared to other methods like FreqEdit and Consistency LoRA that showed larger declines in performance. 116 00:11:58,851 --> 00:12:00,451 Evan: Interesting. 117 00:12:00,451 --> 00:12:04,031 Did they also look at multi-turn editing specifically? 118 00:12:04,113 --> 00:12:05,113 Ashley: Definitely. 119 00:12:05,113 --> 00:12:10,523 For multi-turn editing evaluation, they used MSE-Bench and EdiVal-Bench. 120 00:12:10,523 --> 00:12:23,553 MSE-Bench measures instruction following and consistency over five editing turns, while EdiVal-Bench assesses reliability as visual context evolves, focusing on agentic, object-centric evaluation. 121 00:12:23,553 --> 00:12:31,763 TailSteer consistently improved content consistency across all settings and achieved higher success rates at selected steering onsets. 122 00:12:31,827 --> 00:12:34,107 Evan: Sounds promising. 123 00:12:34,107 --> 00:12:38,787 Did they introduce any new benchmarks to measure cumulative visual drift? 124 00:12:38,835 --> 00:12:42,395 Ashley: Indeed, they introduced the Add & Preserve benchmark. 125 00:12:42,395 --> 00:12:49,155 This benchmark measures cumulative object accuracy and outside-box drift across ten successive turns. 126 00:12:49,155 --> 00:12:56,195 It allows for a more controlled evaluation that directly distinguishes genuine preservation gains from suppressed edits. 127 00:12:56,259 --> 00:13:00,729 Evan: How did TailSteer fare on the Add & Preserve benchmark? 128 00:13:00,795 --> 00:13:08,805 Ashley: TailSteer showed substantial reductions in cumulative drift while maintaining object accuracy at levels comparable to the base models. 129 00:13:08,805 --> 00:13:23,475 For instance, TailSteer outperformed even GPT-Image-2.5 Sunburst in preservation metrics, such as PSNR, SSIM, and LPIPS, demonstrating its robust capability to manage long-horizon editing. 130 00:13:23,523 --> 00:13:29,163 Evan: That's quite an accomplishment, considering the complexity of multi-turn edits. 131 00:13:29,163 --> 00:13:33,023 Did the authors compare different training strategies for TailSteer? 132 00:13:33,075 --> 00:13:34,455 Ashley: Yes, they did. 133 00:13:34,475 --> 00:13:45,195 They compared various training strategies, including standard supervised fine-tuning, training only the last step, training the trajectory tail, and removing on-policy

training. 134 00:13:45,215 --> 00:13:53,955 Full TailSteer, which includes on-policy training, outperformed all other strategies by better balancing preservation and editing capabilities. 135 00:13:54,063 --> 00:13:58,863 Evan: Did the scale of training data make a difference in TailSteer's performance? 136 00:13:58,923 --> 00:14:03,193 Ashley: They examined how different data scales and types impacted performance. 137 00:14:03,193 --> 00:14:14,203 Larger datasets improved drift resistance significantly, while combining local-edit and identity-edit training data provided a better balance between preservation and editing capabilities. 138 00:14:14,203 --> 00:14:19,673 Their main setting, utilizing around 38,000 trajectories, achieved the best results. 139 00:14:19,731 --> 00:14:25,901 Evan: That surely highlights the importance of extensive and diverse training data. 140 00:14:25,901 --> 00:14:28,841 What are the key takeaways from these experiments? 141 00:14:28,899 --> 00:14:37,969 Ashley: The key takeaways are that TailSteer offers substantial enhancements in long-horizon robustness without compromising the foundational capabilities of the base models. 142 00:14:37,969 --> 00:14:47,909 By concentrating corrections near the denoising trajectory tail, TailSteer effectively reduces cumulative drift and improves visual consistency across multiple edits. 143 00:14:47,909 --> 00:14:54,109 Its versatility across different models and input settings further underscores its scalability and effectiveness. 144 00:14:54,171 --> 00:15:03,231 Evan: This concludes the Experiment section of the paper 'TailSteer: Trajectory-Tail Steering for Drift-Resistant Multi-Turn Image Editing.' 145 00:15:04,493 --> 00:15:13,103 Evan: Moving on to the Related Work section, Ashley, can you provide some context on the existing approaches that TailSteer builds upon? 146 00:15:13,187 --> 00:15:14,037 Ashley: Evan. 147 00:15:14,037 --> 00:15:21,377 The paper discusses several key areas of prior work, focusing on methods that have been employed to tackle multi-turn degradation. 148 00:15:21,377 --> 00:15:31,967 It mentions that despite the growing success of large language models in agentic workflows, long-horizon editing robustness still remains a significant bottleneck for visual creation. 149 00:15:32,081 --> 00:15:38,031 Evan: So, what have researchers tried before TailSteer to handle this issue? 150 00:15:38,093 --> 00:15:49,303 Ashley: There are four main categories of existing methods: representation fidelity, inference-time guidance, feature reuse and visual memory, and multi-turn-specific training. 151 00:15:49,303 --> 00:15:52,743 Each of these approaches has its own strengths and limitations. 152 00:15:52,805 --> 00:15:56,635 Evan: Can you elaborate on representation fidelity? 153 00:15:56,693 --> 00:15:57,713 Ashley: Of course. 154 00:15:57,713 --> 00:16:04,743 Representation fidelity deals with the trade-off between reconstruction fidelity and generative learnability

in latent representations. 155 00:16:04,743 --> 00:16:13,053 Researchers have explored improved representation designs to better support image editing, as noted by Dieleman and Yao in their work from 2025. 156 00:16:13,053 --> 00:16:21,973 These designs aim to mitigate reconstruction errors and improve stability under repeated encoding and decoding, addressing some representation-level limitations. 157 00:16:22,037 --> 00:16:25,957 Evan: And what about inference-time guidance? 158 00:16:26,021 --> 00:16:35,331 Ashley: Inference-time guidance involves using mask-guided constraints or frequency-based corrections to localize edits and mitigate content degradation across turns. 159 00:16:35,331 --> 00:16:38,031 For instance, techniques by Avrahami et al. 160 00:16:38,031 --> 00:16:39,401 and Couairon et al. 161 00:16:39,401 --> 00:16:44,081 provide constraints that guide inversion and sampling to balance editing and preservation. 162 00:16:44,081 --> 00:16:52,021 However, these methods may not generalize well across different foundation models and don’t directly improve the underlying model’s robustness. 163 00:16:52,205 --> 00:16:58,975 Evan: What are the strategies within the category of feature reuse and visual memory? 164 00:16:59,021 --> 00:17:05,211 Ashley: This category includes methods that improve editing consistency through feature reuse and cross-turn memory. 165 00:17:05,211 --> 00:17:11,201 Examples include using context retrieval and test-time memory in long-horizon video generation. 166 00:17:11,201 --> 00:17:18,691 Researchers like Zhu and Kim have worked on these concepts, leveraging scene-consistent memory retrieval to better handle iterative edits. 167 00:17:18,691 --> 00:17:25,401 However, long-horizon consistency still remains limited by the underlying generator’s step-by-step robustness. 168 00:17:25,525 --> 00:17:26,795 Evan: I see. 169 00:17:26,795 --> 00:17:31,505 And finally, multi-turn-specific training approaches? 170 00:17:31,565 --> 00:17:39,825 Ashley: Multi-turn-specific training includes approaches like training editors on video-derived sequences or using cross-turn reinforcement learning. 171 00:17:39,825 --> 00:17:42,225 For example, Qu et al. 172 00:17:42,225 --> 00:17:48,925 proposed VINCIE—a method that focuses on in-context image editing derived from video training data. 173 00:17:48,925 --> 00:17:55,575 Other techniques involve training consistency-oriented LoRA adapters to enhance preservation over successive turns. 174 00:17:55,575 --> 00:18:05,315 However, these methods often require dedicated modules or relearning editing behavior, which TailSteer avoids by refining the existing model’s capabilities. 175 00:18:05,451 --> 00:18:10,151 Evan: So how does TailSteer differ fundamentally from these approaches? 176 00:18:10,205 --> 00:18:17,905 Ashley: TailSteer fundamentally differs by directly refining the foundation model’s innate capabilities to handle multi-turn edits. 177 00:18:17,905 --> 00:18:26,795 It focuses on the critical intervention point—the denoising trajectory tail—without needing auxiliary modules or altering the pretrained prior. 178 00:18:26,795 --> 00:18:34,415 This allows TailSteer to build-in robustness, making it a complementary yet distinct solution compared to existing methods. 179 00:18:34,529 --> 00:18:42,209 Evan: So TailSteer integrates well with existing workflows without introducing additional complexity? 180 00:18:42,269 --> 00:18:43,479 Ashley: Exactly. 181 00:18:43,479 --> 00:18:53,579 By concentrating on corrections near the trajectory tail, TailSteer offers a scalable and efficient approach to significantly improve long-horizon consistency. 182 00:18:53,579 --> 00:19:03,199 Its ability to leverage the model’s own outputs for preservation reference further enhances its effectiveness without the need for curated ground-truth editing pairs. 183 00:19:03,245 --> 00:19:12,805 Evan: This marks the end of the Related Work section for the paper 'TailSteer: Trajectory-Tail Steering for Drift-Resistant Multi-Turn Image Editing'. 184 00:19:14,070 --> 00:19:23,290 Evan: Alright, Ashley, as we approach the end of this episode, let's summarize the key contributions and takeaways of the TailSteer paper. 185 00:19:23,334 --> 00:19:24,564 Ashley: Definitely, Evan. 186 00:19:24,564 --> 00:19:29,304 TailSteer stands out in the field of multi-turn image editing for several reasons. 187 00:19:29,304 --> 00:19:36,984 First and foremost, it identifies the denoising trajectory tail as the key intervention point for addressing cumulative visual drift. 188 00:19:36,984 --> 00:19:44,594 This insight allows for fine-grained corrections precisely where they are needed without broadly altering the foundational model's dynamics. 189 00:19:44,646 --> 00:19:50,286 Evan: That focus on the trajectory tail is really a critical innovation. 190 00:19:50,286 --> 00:19:51,266 What’s next? 191 00:19:51,318 --> 00:20:00,438 Ashley: Secondly, TailSteer introduces a final-stage training framework that leverages the model's own on-policy denoising rollouts. 192 00:20:00,438 --> 00:20:10,468 By constructing preservation supervision from the source image and the base model’s outputs, TailSteer bypasses the need for curated ground-truth editing pairs. 193 00:20:10,468 --> 00:20:13,828 This makes the approach highly scalable and efficient. 194 00:20:13,878 --> 00:20:17,898 Evan: And that scalability is truly impressive. 195 00:20:17,898 --> 00:20:18,948 Is there more? 196 00:20:19,014 --> 00:20:19,954 Ashley: Indeed. 197 00:20:19,954 --> 00:20:25,624 Lastly, the paper provides extensive evaluations demonstrating TailSteer's effectiveness. 198 00:20:25,624 --> 00:20:34,324 It retains the generative and editing capabilities of the base models while significantly reducing cumulative drift across long-horizon editing tasks. 199 00:20:34,324 --> 00:20:40,244 This positions TailSteer as a robust and reliable solution for iterative editing workflows. 200 00:20:40,302 --> 00:20:50,972 Evan: So, TailSteer not only enhances long-term robustness but does so without sacrificing the quality of single-turn edits, which is a big deal. 201 00:20:51,060 --> 00:20:51,950 Ashley: Evan. 202 00:20:51,950 --> 00:20:59,180 The contributions and results of TailSteer showcase its potential to set a new standard in the realm of multi-turn image editing. 203 00:20:59,180 --> 00:21:02,480 It’s efficient, scalable, and highly effective. 204 00:21:02,526 --> 00:21:10,256 Evan: That wraps up our discussion on 'TailSteer: Trajectory-Tail Steering for Drift-Resistant Multi-Turn Image Editing.' 205 00:21:10,302 --> 00:21:14,222 Ashley: We hope you enjoyed this deep dive into today's paper. 206 00:21:14,222 --> 00:21:20,712 Remember to join us again for more insightful discussions on the latest in AI and machine learning research. 207 00:21:20,826 --> 00:21:23,136 Evan: Thank you for tuning in to Daily Paper Cast. 208 00:21:23,136 --> 00:21:26,356 Until next time, stay curious and keep exploring.