1
00:00:03,060 --> 00:00:05,480
Evan: Welcome to Daily Paper Cast.
2
00:00:05,544 --> 00:00:14,724
Ashley: Today we're diving into a paper from the Hugging Face daily paper list dated October 20, 2026, which has received 32 upvotes.
3
00:00:14,784 --> 00:00:20,744
Evan: The paper is titled 'Self-Evolving Coding Agents via Harness-Policy Optimization.'
4
00:00:20,808 --> 00:00:30,128
Ashley: It's authored by Zhepei Wei from Microsoft Research and the University of Virginia, with Wenlin Yao from Microsoft Research as the corresponding author.
5
00:00:30,192 --> 00:00:43,672
Evan: Large language model agents have shown promising potential in coding tasks, driven by two main components: the policy model which defines the agent’s capabilities, and the harness that mediates interaction with the environment.
6
00:00:43,728 --> 00:00:44,498
Ashley: Right.
7
00:00:44,498 --> 00:00:56,808
Typically, research focuses on either optimizing the harness, which includes prompts, workflows, and agent code, or the policy model itself, which learns from successful trajectories or rewards.
8
00:00:56,808 --> 00:01:00,088
However, each approach on its own has limitations.
9
00:01:00,144 --> 00:01:01,214
Evan: Exactly.
10
00:01:01,214 --> 00:01:07,354
Optimizing just the harness leaves the policy model untouched, potentially limiting broader improvements.
11
00:01:07,354 --> 00:01:14,304
On the other hand, policy optimization relies heavily on the stimuli provided by a fixed harness, which can be a bottleneck.
12
00:01:14,352 --> 00:01:25,112
Ashley: The paper addresses these issues by introducing Self-HPO, a recursive self-improving framework for coding agents that alternates between harness and policy optimization.
13
00:01:25,176 --> 00:01:26,376
Evan: Interesting.
14
00:01:26,376 --> 00:01:29,216
How does Self-HPO work exactly?
15
00:01:29,280 --> 00:01:36,110
Ashley: In each round, Self-HPO uses the current policy to propose new harness candidates, known as the H-step.
16
00:01:36,110 --> 00:01:42,150
Then, it updates the policy through reinforcement learning under the optimized harness, known as the P-step.
17
00:01:42,150 --> 00:01:48,120
This dual optimization ensures that harness improvements target the evolving failures of the policy model.
18
00:01:48,228 --> 00:01:56,108
Evan: So the evolving policy essentially becomes its own harness proposer, constantly pushing both components to improve recursively?
19
00:01:56,190 --> 00:01:57,160
Ashley: Precisely.
20
00:01:57,160 --> 00:02:00,990
This dynamic interplay allows for continuous adaptation.
21
00:02:00,990 --> 00:02:12,660
The harness exposes and compensates for weaknesses in the policy, while the policy internalizes the behaviors elicited by the harness and evolves, shifting the failure modes that require attention.
22
00:02:12,780 --> 00:02:16,520
Evan: And this method has been tested across multiple coding benchmarks?
23
00:02:16,584 --> 00:02:17,434
Ashley: Yes.
24
00:02:17,434 --> 00:02:28,224
Self-HPO was evaluated using three coding benchmarks: SWE-Bench Verified, SWE-Bench Multilingual, and SWE-Bench Pro.
25
00:02:28,224 --> 00:02:34,264
It consistently outperformed baselines that optimized either only the harness or only the policy.
26
00:02:34,320 --> 00:02:36,660
Evan: What kind of improvements are we looking at?
27
00:02:36,720 --> 00:02:47,790
Ashley: The average resolve rate of the Qwen3.5-35B-A3B model increased from 57.8% to 64.8%.
28
00:02:47,790 --> 00:02:54,240
This performance boost even generalized to out-of-distribution settings, indicating strong adaptability.
29
00:02:54,288 --> 00:02:55,728
Evan: That's impressive.
30
00:02:55,728 --> 00:02:59,628
What does the analysis say about combining optimized components?
31
00:02:59,718 --> 00:03:10,888
Ashley: Analysis shows that optimized harnesses and policies retain their gains even when paired with their base counterparts, and their joint gain exceeds the sum of their individual improvements.
32
00:03:10,944 --> 00:03:15,444
Evan: Give us a bit more detail about the reliability and capability enhancement.
33
00:03:15,504 --> 00:03:16,614
Ashley: Certainly.
34
00:03:16,614 --> 00:03:27,454
Self-HPO not only improves reliability, demonstrated by its higher pass@1 rates, but also expands the capability frontier, as seen in enhanced pass@k rates.
35
00:03:27,454 --> 00:03:33,244
This means the agent's robustness is improved and it can tackle a broader range of tasks successfully.
36
00:03:33,348 --> 00:03:37,308
Evan: And there's room for even more improvement with stronger external models?
37
00:03:37,368 --> 00:03:44,868
Ashley: Using a more advanced external model like Opus 4.8 as the harness proposer further amplified the performance boost.
38
00:03:44,928 --> 00:03:52,448
Evan: So the framework isn't just limited to inherent policy and harness revisions, but can also leverage external advancements?
39
00:03:52,512 --> 00:03:53,532
Ashley: Exactly.
40
00:03:53,532 --> 00:04:04,092
The robustness of Self-HPO's framework allows it to integrate stronger external models seamlessly, leading to even higher resolve rates and a broader capability frontier.
41
00:04:04,152 --> 00:04:06,582
Evan: That's the end of the Introduction section.
42
00:04:06,582 --> 00:04:11,812
Next, we'll delve into the methods and experimental setup of this innovative framework.
43
00:04:13,118 --> 00:04:20,658
Evan: Let's jump into the methods used in Self-HPO, the heart of how this framework achieves continuous improvement in coding agents.
44
00:04:20,714 --> 00:04:21,794
Ashley: Sure, Evan.
45
00:04:21,794 --> 00:04:31,624
The essence of Self-HPO lies in its two-step optimization process: the Harness-step or H-step, and the Policy-step or P-step.
46
00:04:31,624 --> 00:04:33,734
Let's break down each of these steps.
47
00:04:33,794 --> 00:04:38,374
Evan: Alright, can you explain what happens in the H-step first?
48
00:04:38,426 --> 00:04:46,396
Ashley: In the H-step, the current policy model analyzes its own failures based on the trajectories it generates during task execution.
49
00:04:46,396 --> 00:04:55,866
Essentially, it inspects where and why it did not succeed and uses this information to propose new harness candidates that could potentially address these failures.
50
00:04:55,982 --> 00:05:02,662
Evan: So the policy looks at its own missteps and suggests revisions for the harness to compensate for these weaknesses?
51
00:05:02,714 --> 00:05:03,704
Ashley: Correct.
52
00:05:03,704 --> 00:05:09,984
The current policy model proposes modifications to its harness, creating a set of harness candidates.
53
00:05:09,984 --> 00:05:18,994
These candidates are then evaluated based on their performance on a specific search set, which initially consists of tasks sampled from the training data.
54
00:05:19,118 --> 00:05:23,778
Evan: How does the framework ensure these candidates are effective?
55
00:05:23,864 --> 00:05:25,174
Ashley: Good question.
56
00:05:25,174 --> 00:05:31,274
Each candidate harness is evaluated by running the policy model on the search set tasks again.
57
00:05:31,274 --> 00:05:38,814
The harness that yields the highest performance improvement, in terms of resolve rate, is selected for the next optimization phase.
58
00:05:38,928 --> 00:05:42,928
Evan: And that takes us to the P-step where the policy model is updated.
59
00:05:42,928 --> 00:05:44,978
How does this step work in detail?
60
00:05:45,026 --> 00:05:57,616
Ashley: In the P-step, the previously optimized harness is fixed, and reinforcement learning (RL) techniques are used to update and improve the policy model based on the interactions mediated by this harness.
61
00:05:57,616 --> 00:06:05,886
This involves collecting fresh trajectories where the agent performs coding tasks under the new harness and receiving feedback on its performance.
62
00:06:06,000 --> 00:06:10,010
Evan: Could you elaborate on the RL techniques employed here?
63
00:06:10,088 --> 00:06:10,978
Ashley: Of course.
64
00:06:10,978 --> 00:06:18,608
The paper utilizes Group Relative Policy Optimization (GRPO) and Balanced Adaptive Rollout (BAR).
65
00:06:18,608 --> 00:06:26,668
GRPO helps in optimizing the policy using group-relative advantages, where reward balancing mitigates the issue of reward sparsity.
66
00:06:26,668 --> 00:06:35,618
BAR ensures that additional trajectories are drawn to construct reward-balanced groups, reducing zero-variance groups which provide no useful learning signal.
67
00:06:35,666 --> 00:06:44,506
Evan: So, GRPO and BAR work together to refine the policy model based on its performance under the optimized harness, correct?
68
00:06:44,570 --> 00:06:45,660
Ashley: Exactly.
69
00:06:45,660 --> 00:06:51,980
By doing so, the policy model internalizes the successful behaviors elicited by the new harness.
70
00:06:51,980 --> 00:07:00,030
This updated policy model then shifts the failure modes, implicating different weaknesses to be targeted in subsequent H-steps.
71
00:07:00,074 --> 00:07:06,314
Evan: And with each round, both harness and policy keep improving adaptively, right?
72
00:07:06,362 --> 00:07:07,042
Ashley: Yes.
73
00:07:07,042 --> 00:07:14,422
The evolving policy becomes the new harness proposer for the next H-step, keeping the cycle iterative and recursive.
74
00:07:14,422 --> 00:07:18,312
Additionally, the task set for harness evaluation evolves too.
75
00:07:18,312 --> 00:07:27,982
Hard tasks, which are not solved in the preceding P-step, form the next search set, ensuring the optimization remains aligned with the policy’s latest failure modes.
76
00:07:28,034 --> 00:07:31,714
Evan: That’s a clever way to adapt continuously.
77
00:07:31,714 --> 00:07:36,594
How many rounds of these optimizations were conducted in the experiments?
78
00:07:36,650 --> 00:07:46,320
Ashley: The paper reports conducting two optimization rounds, with each involving six harness candidates per H-step and 50 RL updates per P-step.
79
00:07:46,320 --> 00:07:52,950
This assortment was deemed sufficient to demonstrate the effectiveness of Self-HPO across various benchmarks.
80
00:07:52,994 --> 00:08:03,754
Evan: Now, I know the paper uses Qwen3.5-35B-A3B as the base policy and mini-SWE-agent as the seed harness.
81
00:08:03,754 --> 00:08:06,994
Could you talk more about the datasets and benchmarks used?
82
00:08:07,088 --> 00:08:08,088
Ashley: Definitely.
83
00:08:08,088 --> 00:08:19,358
The training set consists of 2,484 coding tasks sourced from SWE-rebench, SWE-rebench V2, and Scale-SWE datasets.
84
00:08:19,358 --> 00:08:26,408
For evaluation, they used three benchmarks: SWE-Bench Verified, SWE-Bench Multilingual, and SWE-Bench Pro.
85
00:08:26,408 --> 00:08:36,998
These cover a diverse range of coding tasks, including human-validated GitHub issues and tasks spanning multiple programming languages and repository-level challenges.
86
00:08:37,058 --> 00:08:41,298
Evan: And what metrics are used to evaluate the performance?
87
00:08:41,354 --> 00:08:48,104
Ashley: Performance is primarily measured by the resolve rate, which is the percentage of tasks successfully completed by the agent.
88
00:08:48,104 --> 00:08:57,654
Additionally, the paper looks at metrics like pass@1 and pass@k, assessing reliability and capability over a broader range of task attempts.
89
00:08:57,698 --> 00:09:03,498
Evan: How do these metrics help in comparing Self-HPO with other methods?
90
00:09:03,554 --> 00:09:13,634
Ashley: These metrics provide a clear picture of both the reliability of the agent in solving tasks on the first attempt, and its overall capability across multiple attempts.
91
00:09:13,634 --> 00:09:24,174
By comparing these metrics, the paper showcases that Self-HPO outperforms policy-only and harness-only baselines, highlighting its advantage in joint optimization.
92
00:09:24,218 --> 00:09:26,568
Evan: That’s quite comprehensive.
93
00:09:26,568 --> 00:09:31,598
Did the authors perform any ablation studies to validate their method further?
94
00:09:31,658 --> 00:09:32,798
Ashley: Yes, they did.
95
00:09:32,798 --> 00:09:42,708
Ablations examined the impact of replacing RL strategies with fine-tuning methods, reusing fixed search sets, and using a self-proposer versus an external model.
96
00:09:42,708 --> 00:09:55,638
These studies confirmed that RL yields stronger policy updates than fine-tuning, evolving search sets target more relevant tasks, and self-proposal remains effective, though external proposals provide an additional boost.
97
00:09:55,682 --> 00:10:00,382
Evan: This detailed evaluation certainly strengthens the case for Self-HPO.
98
00:10:00,382 --> 00:10:03,302
Is there anything else that stands out in the methods section?
99
00:10:03,392 --> 00:10:11,062
Ashley: Another noteworthy aspect is how the optimized harness not only retains the seed agent's loop but adds targeted interventions.
100
00:10:11,062 --> 00:10:21,002
These include early warnings about budget constraints, checks for empty patches, ensuring API signatures conform to the repository, and fallback mechanisms.
101
00:10:21,002 --> 00:10:27,022
Such specific enhancements help avoid common submission failures, making the agent more robust.
102
00:10:27,134 --> 00:10:33,134
Evan: This kind of targeted control really underpins the agent's adaptive capabilities.
103
00:10:33,134 --> 00:10:37,894
It's fascinating to see how each optimization step builds on the previous one.
104
00:10:37,946 --> 00:10:48,706
Ashley: The interplay between harness and policy modules through consecutive rounds of optimization creates an agent capable of self-improvement in a highly targeted and efficient manner.
105
00:10:48,830 --> 00:10:51,170
Evan: So that's the end of the Methods section.
106
00:10:51,170 --> 00:10:57,350
Next, we'll move on to the experiments and results that showcase the practical impact of Self-HPO.
107
00:10:58,681 --> 00:11:03,931
Evan: Let's dive into the experiments and results that highlight the effectiveness of Self-HPO.
108
00:11:04,017 --> 00:11:04,747
Ashley: Evan.
109
00:11:04,747 --> 00:11:12,647
The experimental evaluation of Self-HPO is thorough, designed to validate its performance across multiple coding benchmarks.
110
00:11:12,647 --> 00:11:21,807
The authors used Qwen3.5-35B-A3B as the base policy and mini-SWE-agent as the seed harness.
111
00:11:21,867 --> 00:11:26,427
Evan: What datasets did they use for training and evaluation?
112
00:11:26,475 --> 00:11:37,265
Ashley: The training set comprised 2,484 tasks sourced from SWE-rebench, SWE-rebench V2, and Scale-SWE datasets.
113
00:11:37,265 --> 00:11:52,335
For evaluation, they employed three benchmarks: SWE-Bench Verified, SWE-Bench Multilingual, and SWE-Bench Pro, assessing diverse coding tasks spanning various programming languages and repository-level challenges.
114
00:11:52,395 --> 00:11:54,535
Evan: How did they structure these experiments?
115
00:11:54,579 --> 00:11:58,999
Ashley: The authors conducted two rounds of harness and policy optimization.
116
00:11:58,999 --> 00:12:05,829
Each round included six harness candidates per H-step and 50 reinforcement learning updates per P-step.
117
00:12:05,829 --> 00:12:11,539
This setup was crucial for demonstrating the iterative improvement capabilities of Self-HPO.
118
00:12:11,625 --> 00:12:14,875
Evan: What were the main performance metrics?
119
00:12:14,931 --> 00:12:21,071
Ashley: The primary performance metric was the resolve rate—the percentage of tasks successfully completed.
120
00:12:21,071 --> 00:12:29,551
Additionally, the authors used pass@1 and pass@k metrics to evaluate reliability and capability across multiple attempts.
121
00:12:29,595 --> 00:12:32,595
Evan: And what did the results show?
122
00:12:32,643 --> 00:12:39,473
Ashley: Self-HPO consistently outperformed baselines that optimized only the policy or only the harness.
123
00:12:39,473 --> 00:12:50,643
For instance, on SWE-Bench Verified, Self-HPO improved the resolve rate from 61.0% with the base model to 69.5%.
124
00:12:50,643 --> 00:13:01,423
Similar gains were observed on SWE-Bench Multilingual and SWE-Bench Pro, with increases of 5.5 and 6.9 percentage points, respectively.
125
00:13:01,467 --> 00:13:03,477
Evan: That's significant.
126
00:13:03,477 --> 00:13:09,727
How does this compare to other methods using external models like Opus 4.8?
127
00:13:09,771 --> 00:13:11,031
Ashley: Great question.
128
00:13:11,031 --> 00:13:17,581
With Opus 4.8 as the harness proposer in the HPO setup, the performance saw further improvements.
129
00:13:17,581 --> 00:13:32,371
Specifically, the resolve rate on SWE-Bench Verified improved to 71.6%, and average scores across benchmarks rose to 66.3%, showing the strong adaptability of Self-HPO.
130
00:13:32,427 --> 00:13:35,827
Evan: Were there any notable insights from the ablation studies?
131
00:13:35,883 --> 00:13:36,593
Ashley: Yes.
132
00:13:36,593 --> 00:13:39,653
The ablation studies highlighted three key insights.
133
00:13:39,653 --> 00:13:51,293
First, reinforcement learning yields stronger policy updates compared to supervised fine-tuning, with resolve rates dropping by 4.4 points when RL was replaced by SFT in the P-step.
134
00:13:51,293 --> 00:14:00,403
Second, evolving the search set to include hard tasks from previous rounds improved performance by 1.5 percentage points over using a fixed set.
135
00:14:00,403 --> 00:14:15,863
Third, while external models like Opus 4.8 provide a performance boost, the self-proposer mechanism in Self-HPO is still highly effective, achieving 69.5% resolve rate on SWE-Bench Verified when used alone.
136
00:14:15,915 --> 00:14:24,255
Evan: It sounds like the adaptability and continuous improvement aspects of Self-HPO are well-supported by these experiments.
137
00:14:24,315 --> 00:14:25,275
Ashley: Indeed.
138
00:14:25,275 --> 00:14:35,485
The authors also cross-evaluated the optimized components, finding that both the enhanced policy and harness retained their improvements, even when paired with their base counterparts.
139
00:14:35,485 --> 00:14:43,335
Combining these optimized components resulted in a joint gain exceeding the sum of their individual gains by 1.6 points.
140
00:14:43,395 --> 00:14:45,605
Evan: What about the pass@k metrics?
141
00:14:45,605 --> 00:14:47,655
Did they show improvements as well?
142
00:14:47,715 --> 00:14:52,055
Ashley: Yes, the pass@k metrics indeed showed significant improvements.
143
00:14:52,055 --> 00:15:02,835
For example, Self-HPO increased the pass@8 rate from 80.0% to 84.8%, demonstrating enhanced capability across multiple attempts.
144
00:15:02,835 --> 00:15:14,435
More notably, pass@1 rates increased for almost all task difficulties, except for some very straightforward tasks where performance slightly dipped, likely due to the focus on harder tasks.
145
00:15:14,499 --> 00:15:22,019
Evan: How does Self-HPO handle scenarios where no valid solution is found in the initial rollouts?
146
00:15:22,113 --> 00:15:36,233
Ashley: During reinforcement learning, Self-HPO reduces zero-success rollout groups—cases where no rollout succeeds—significantly lowering the proportion from around 11% to 17% to just 6.5%.
147
00:15:36,233 --> 00:15:44,083
This reduction is crucial because zero-success groups offer no reward signal, hence mitigating them enhances the learning process.
148
00:15:44,139 --> 00:15:47,019
Evan: That’s a substantial improvement.
149
00:15:47,019 --> 00:15:51,719
Can you give us an example of how the harness optimization works in practice?
150
00:15:51,801 --> 00:15:52,841
Ashley: Of course.
151
00:15:52,841 --> 00:15:56,791
One simple yet impactful example is the 'empty-patch guard'.
152
00:15:56,791 --> 00:16:01,231
Imagine a scenario where a policy submits an empty or malformed patch.
153
00:16:01,231 --> 00:16:08,211
The optimized harness intervenes, prompting the agent to check for any unstaged new files or incorrect submissions.
154
00:16:08,211 --> 00:16:13,991
This simple check ensures that valid patches are submitted, reducing the error rate significantly.
155
00:16:14,043 --> 00:16:15,223
Evan: That makes sense.
156
00:16:15,223 --> 00:16:17,843
Any closing thoughts on the experiment section?
157
00:16:17,907 --> 00:16:25,787
Ashley: Overall, the experiments and results strongly support the effectiveness of Self-HPO in creating self-improving coding agents.
158
00:16:25,787 --> 00:16:34,607
By dynamically optimizing both the harness and policy through iterative rounds, it achieves significant performance gains and enhanced adaptability.
159
00:16:34,659 --> 00:16:37,499
Evan: That brings us to the end of the Experiment section.
160
00:16:37,499 --> 00:16:43,199
In the next segment, we’ll discuss some related work and draw conclusions from this exciting framework.
161
00:16:44,453 --> 00:16:49,773
Evan: Alright, let's explore some of the related work that forms the backdrop for Self-HPO.
162
00:16:49,829 --> 00:16:50,879
Ashley: Sure.
163
00:16:50,879 --> 00:16:59,749
The paper discusses several lines of research that contribute to the broader context of harness and policy optimization for large language model agents.
164
00:16:59,813 --> 00:17:02,823
Evan: Let's start with harness optimization.
165
00:17:02,823 --> 00:17:06,573
What has previous research focused on in this area?
166
00:17:06,629 --> 00:17:16,019
Ashley: Harness optimization involves refining the external scaffolding that determines what the model observes, how its actions are executed, and the feedback it receives.
167
00:17:16,019 --> 00:17:19,669
This spans across prompts, workflows, and agent code.
168
00:17:19,669 --> 00:17:27,329
Studies have shown that optimizing these components without altering the policy model can still lead to significant performance gains.
169
00:17:27,479 --> 00:17:35,209
Evan: Could you give some specific examples of how harness optimization has been applied in past research?
170
00:17:35,261 --> 00:17:44,571
Ashley: For instance, Pryzant et al., 2023, and Khattab et al., 2023, focused on optimizing prompts and programs.
171
00:17:44,571 --> 00:17:52,131
Hu et al., 2024, and Zhang et al., 2025, worked on making workflows more efficient.
172
00:17:52,131 --> 00:17:59,221
Approaches like these have been used to guide agents in a variety of tasks, from software engineering to navigating websites.
173
00:17:59,345 --> 00:18:00,415
Evan: I see.
174
00:18:00,415 --> 00:18:04,845
And what about policy optimization—what's the focus there?
175
00:18:04,901 --> 00:18:13,591
Ashley: Policy optimization typically involves adjusting the policy model itself, often through reinforcement learning or supervised fine-tuning.
176
00:18:13,591 --> 00:18:20,001
This is done to improve the model based on the successful trajectories or rewards it receives during its tasks.
177
00:18:20,001 --> 00:18:31,961
Recent studies, such as those by Guo et al., 2025, and Wei et al., 2025a, have shown considerable advancements by focusing on policy optimization alone.
178
00:18:32,021 --> 00:18:37,241
Evan: So how does Self-HPO blend these two optimization strategies?
179
00:18:37,301 --> 00:18:44,351
Ashley: Self-HPO stands out by alternating between harness optimization and policy optimization within the same framework.
180
00:18:44,351 --> 00:18:50,901
This dual approach is relatively novel compared to prior methods that typically focus on one component at a time.
181
00:18:50,901 --> 00:18:58,961
The paper highlights several recent works that begin to explore this combined optimization, but often rely on an external harness proposer.
182
00:18:59,021 --> 00:19:04,021
Evan: And in the context of Self-HPO, how is this external reliance addressed?
183
00:19:04,085 --> 00:19:11,785
Ashley: Self-HPO mitigates this by using the evolving policy itself as the harness proposer, which is a significant step forward.
184
00:19:11,785 --> 00:19:24,385
For example, Kim et al., 2026, and Chen et al., 2026a, have also worked on optimizing both harness and policy, but they use an external model to suggest harness changes.
185
00:19:24,437 --> 00:19:27,817
Evan: That makes the framework more self-sufficient.
186
00:19:27,817 --> 00:19:32,317
What other aspects set Self-HPO apart in its approach?
187
00:19:32,411 --> 00:19:37,241
Ashley: Another key difference is how Self-HPO evolves the search set for the harness.
188
00:19:37,241 --> 00:19:47,301
While other methods might use a fixed set of tasks for harness evaluation, Self-HPO adapts by using tasks that the policy finds hard to solve in the previous round.
189
00:19:47,301 --> 00:19:53,141
This ensures that the optimization process continuously targets the policy's evolving weaknesses.
190
00:19:53,189 --> 00:19:56,279
Evan: That's a smart way to keep the optimization relevant.
191
00:19:56,279 --> 00:20:01,169
Are there any other notable methods in joint harness–policy optimization?
192
00:20:01,229 --> 00:20:07,799
Ashley: Yes, Luo et al., 2026, proposed HASE, which also uses a self-proposing policy.
193
00:20:07,799 --> 00:20:17,249
However, HASE places harness edits within the RL action space itself, unlike Self-HPO, which alternates between policy learning and harness search.
194
00:20:17,249 --> 00:20:26,969
Also, Xu et al., 2026a, introduced Evo2, which trains a separate harness proposer while keeping the task-solving policy frozen.
195
00:20:27,029 --> 00:20:35,909
Evan: So Self-HPO's method of using the evolving policy as its own harness proposer is fairly unique then.
196
00:20:35,957 --> 00:20:37,097
Ashley: Exactly.
197
00:20:37,097 --> 00:20:45,817
It’s a novel way to ensure that both the harness and policy evolve together, addressing their mutual dependencies directly and efficiently.
198
00:20:45,949 --> 00:20:50,229
Evan: And what about the practical applications of these advancements?
199
00:20:50,229 --> 00:20:52,569
How are they typically utilized?
200
00:20:52,613 --> 00:21:07,673
Ashley: Harness optimizations are employed in a range of applications, from software engineering as seen in the works by Le et al., 2026, to more general use cases like automated computer use described by Agashe et al., 2025.
201
00:21:07,673 --> 00:21:17,913
Joint optimization strategies, especially those addressing evolving tasks and environments, represent a cutting-edge approach to making these systems more robust and generalizable.
202
00:21:17,957 --> 00:21:28,037
Evan: This gives a comprehensive understanding of the related works that inform Self-HPO and the distinct contributions it makes.
203
00:21:28,085 --> 00:21:29,055
Ashley: Indeed.
204
00:21:29,055 --> 00:21:41,985
With Self-HPO's recursive self-improving framework, where the agent itself proposes harness revisions and adapts through reinforcement learning, it pushes the envelope in both harness and policy optimization.
205
00:21:41,985 --> 00:21:45,505
This integrated approach sets a new benchmark in the field.
206
00:21:45,557 --> 00:21:48,957
Evan: That wraps up our discussion on the Related Work section.
207
00:21:50,214 --> 00:21:55,114
Evan: We've covered the introduction, methods, experiments, and related work.
208
00:21:55,114 --> 00:21:59,754
Now, let's summarize the key contributions and takeaways of this paper.
209
00:21:59,814 --> 00:22:00,974
Ashley: Certainly.
210
00:22:00,974 --> 00:22:10,234
The paper introduces Self-HPO, a recursive self-improving framework for coding agents that alternates between harness and policy optimization.
211
00:22:10,278 --> 00:22:18,878
Evan: This approach leverages the evolving policy as its own harness proposer, enabling continuous adaptation and improvement in performance.
212
00:22:18,942 --> 00:22:27,272
Ashley: The framework has been thoroughly tested across three coding benchmarks, demonstrating significant improvements in resolve rates and robustness.
213
00:22:27,272 --> 00:22:33,922
The average resolve rate increased notably from 57.8% to 64.8%.
214
00:22:33,966 --> 00:22:39,876
Evan: One of the standout features is the dynamic interplay between the harness and policy modules.
215
00:22:39,876 --> 00:22:44,326
Both components keep improving adaptively, ensuring sustained gains.
216
00:22:44,382 --> 00:22:45,622
Ashley: Exactly.
217
00:22:45,622 --> 00:22:55,482
The optimized components retain their enhancements even when paired with their base counterparts, and their combined improvement exceeds the sum of individual gains.
218
00:22:55,482 --> 00:22:59,762
This highlights the efficiency of joint harness–policy optimization.
219
00:22:59,814 --> 00:23:14,654
Evan: The paper also provides insights into how reinforcement learning techniques like Group Relative Policy Optimization and Balanced Adaptive Rollout contribute to refining the policy model, making it more effective under the optimized harness.
220
00:23:14,718 --> 00:23:26,898
Ashley: Moreover, the evolving search set ensures that harness optimization targets the latest policy weaknesses, making the agent's performance enhancement continuous and aligned with its evolving capabilities.
221
00:23:27,008 --> 00:23:34,918
Evan: And with the potential to integrate stronger external models, Self-HPO opens up pathways for even greater performance improvements.
222
00:23:34,974 --> 00:23:46,174
Ashley: Overall, Self-HPO represents a significant advancement in creating self-improving coding agents, pushing the boundaries of what these systems can achieve.
223
00:23:46,280 --> 00:23:48,430
Evan: That's all for today's episode.
224
00:23:48,430 --> 00:23:54,470
Thank you for joining us on Daily Paper Cast as we explored the intricacies of Self-HPO.
225
00:23:54,534 --> 00:23:57,254
Ashley: We hope you found the discussion insightful.
226
00:23:57,254 --> 00:24:04,514
Be sure to tune in for our next episode where we delve into another exciting paper from the world of AI and machine learning.
227
00:24:04,596 --> 00:24:08,686
Evan: Until then, stay curious and keep exploring!
228
00:24:08,686 --> 00:24:10,046
Goodbye for now.
229
00:24:10,140 --> 00:24:11,330
Ashley: Goodbye!
00:00:03,060 --> 00:00:05,480
Evan: Welcome to Daily Paper Cast.
2
00:00:05,544 --> 00:00:14,724
Ashley: Today we're diving into a paper from the Hugging Face daily paper list dated October 20, 2026, which has received 32 upvotes.
3
00:00:14,784 --> 00:00:20,744
Evan: The paper is titled 'Self-Evolving Coding Agents via Harness-Policy Optimization.'
4
00:00:20,808 --> 00:00:30,128
Ashley: It's authored by Zhepei Wei from Microsoft Research and the University of Virginia, with Wenlin Yao from Microsoft Research as the corresponding author.
5
00:00:30,192 --> 00:00:43,672
Evan: Large language model agents have shown promising potential in coding tasks, driven by two main components: the policy model which defines the agent’s capabilities, and the harness that mediates interaction with the environment.
6
00:00:43,728 --> 00:00:44,498
Ashley: Right.
7
00:00:44,498 --> 00:00:56,808
Typically, research focuses on either optimizing the harness, which includes prompts, workflows, and agent code, or the policy model itself, which learns from successful trajectories or rewards.
8
00:00:56,808 --> 00:01:00,088
However, each approach on its own has limitations.
9
00:01:00,144 --> 00:01:01,214
Evan: Exactly.
10
00:01:01,214 --> 00:01:07,354
Optimizing just the harness leaves the policy model untouched, potentially limiting broader improvements.
11
00:01:07,354 --> 00:01:14,304
On the other hand, policy optimization relies heavily on the stimuli provided by a fixed harness, which can be a bottleneck.
12
00:01:14,352 --> 00:01:25,112
Ashley: The paper addresses these issues by introducing Self-HPO, a recursive self-improving framework for coding agents that alternates between harness and policy optimization.
13
00:01:25,176 --> 00:01:26,376
Evan: Interesting.
14
00:01:26,376 --> 00:01:29,216
How does Self-HPO work exactly?
15
00:01:29,280 --> 00:01:36,110
Ashley: In each round, Self-HPO uses the current policy to propose new harness candidates, known as the H-step.
16
00:01:36,110 --> 00:01:42,150
Then, it updates the policy through reinforcement learning under the optimized harness, known as the P-step.
17
00:01:42,150 --> 00:01:48,120
This dual optimization ensures that harness improvements target the evolving failures of the policy model.
18
00:01:48,228 --> 00:01:56,108
Evan: So the evolving policy essentially becomes its own harness proposer, constantly pushing both components to improve recursively?
19
00:01:56,190 --> 00:01:57,160
Ashley: Precisely.
20
00:01:57,160 --> 00:02:00,990
This dynamic interplay allows for continuous adaptation.
21
00:02:00,990 --> 00:02:12,660
The harness exposes and compensates for weaknesses in the policy, while the policy internalizes the behaviors elicited by the harness and evolves, shifting the failure modes that require attention.
22
00:02:12,780 --> 00:02:16,520
Evan: And this method has been tested across multiple coding benchmarks?
23
00:02:16,584 --> 00:02:17,434
Ashley: Yes.
24
00:02:17,434 --> 00:02:28,224
Self-HPO was evaluated using three coding benchmarks: SWE-Bench Verified, SWE-Bench Multilingual, and SWE-Bench Pro.
25
00:02:28,224 --> 00:02:34,264
It consistently outperformed baselines that optimized either only the harness or only the policy.
26
00:02:34,320 --> 00:02:36,660
Evan: What kind of improvements are we looking at?
27
00:02:36,720 --> 00:02:47,790
Ashley: The average resolve rate of the Qwen3.5-35B-A3B model increased from 57.8% to 64.8%.
28
00:02:47,790 --> 00:02:54,240
This performance boost even generalized to out-of-distribution settings, indicating strong adaptability.
29
00:02:54,288 --> 00:02:55,728
Evan: That's impressive.
30
00:02:55,728 --> 00:02:59,628
What does the analysis say about combining optimized components?
31
00:02:59,718 --> 00:03:10,888
Ashley: Analysis shows that optimized harnesses and policies retain their gains even when paired with their base counterparts, and their joint gain exceeds the sum of their individual improvements.
32
00:03:10,944 --> 00:03:15,444
Evan: Give us a bit more detail about the reliability and capability enhancement.
33
00:03:15,504 --> 00:03:16,614
Ashley: Certainly.
34
00:03:16,614 --> 00:03:27,454
Self-HPO not only improves reliability, demonstrated by its higher pass@1 rates, but also expands the capability frontier, as seen in enhanced pass@k rates.
35
00:03:27,454 --> 00:03:33,244
This means the agent's robustness is improved and it can tackle a broader range of tasks successfully.
36
00:03:33,348 --> 00:03:37,308
Evan: And there's room for even more improvement with stronger external models?
37
00:03:37,368 --> 00:03:44,868
Ashley: Using a more advanced external model like Opus 4.8 as the harness proposer further amplified the performance boost.
38
00:03:44,928 --> 00:03:52,448
Evan: So the framework isn't just limited to inherent policy and harness revisions, but can also leverage external advancements?
39
00:03:52,512 --> 00:03:53,532
Ashley: Exactly.
40
00:03:53,532 --> 00:04:04,092
The robustness of Self-HPO's framework allows it to integrate stronger external models seamlessly, leading to even higher resolve rates and a broader capability frontier.
41
00:04:04,152 --> 00:04:06,582
Evan: That's the end of the Introduction section.
42
00:04:06,582 --> 00:04:11,812
Next, we'll delve into the methods and experimental setup of this innovative framework.
43
00:04:13,118 --> 00:04:20,658
Evan: Let's jump into the methods used in Self-HPO, the heart of how this framework achieves continuous improvement in coding agents.
44
00:04:20,714 --> 00:04:21,794
Ashley: Sure, Evan.
45
00:04:21,794 --> 00:04:31,624
The essence of Self-HPO lies in its two-step optimization process: the Harness-step or H-step, and the Policy-step or P-step.
46
00:04:31,624 --> 00:04:33,734
Let's break down each of these steps.
47
00:04:33,794 --> 00:04:38,374
Evan: Alright, can you explain what happens in the H-step first?
48
00:04:38,426 --> 00:04:46,396
Ashley: In the H-step, the current policy model analyzes its own failures based on the trajectories it generates during task execution.
49
00:04:46,396 --> 00:04:55,866
Essentially, it inspects where and why it did not succeed and uses this information to propose new harness candidates that could potentially address these failures.
50
00:04:55,982 --> 00:05:02,662
Evan: So the policy looks at its own missteps and suggests revisions for the harness to compensate for these weaknesses?
51
00:05:02,714 --> 00:05:03,704
Ashley: Correct.
52
00:05:03,704 --> 00:05:09,984
The current policy model proposes modifications to its harness, creating a set of harness candidates.
53
00:05:09,984 --> 00:05:18,994
These candidates are then evaluated based on their performance on a specific search set, which initially consists of tasks sampled from the training data.
54
00:05:19,118 --> 00:05:23,778
Evan: How does the framework ensure these candidates are effective?
55
00:05:23,864 --> 00:05:25,174
Ashley: Good question.
56
00:05:25,174 --> 00:05:31,274
Each candidate harness is evaluated by running the policy model on the search set tasks again.
57
00:05:31,274 --> 00:05:38,814
The harness that yields the highest performance improvement, in terms of resolve rate, is selected for the next optimization phase.
58
00:05:38,928 --> 00:05:42,928
Evan: And that takes us to the P-step where the policy model is updated.
59
00:05:42,928 --> 00:05:44,978
How does this step work in detail?
60
00:05:45,026 --> 00:05:57,616
Ashley: In the P-step, the previously optimized harness is fixed, and reinforcement learning (RL) techniques are used to update and improve the policy model based on the interactions mediated by this harness.
61
00:05:57,616 --> 00:06:05,886
This involves collecting fresh trajectories where the agent performs coding tasks under the new harness and receiving feedback on its performance.
62
00:06:06,000 --> 00:06:10,010
Evan: Could you elaborate on the RL techniques employed here?
63
00:06:10,088 --> 00:06:10,978
Ashley: Of course.
64
00:06:10,978 --> 00:06:18,608
The paper utilizes Group Relative Policy Optimization (GRPO) and Balanced Adaptive Rollout (BAR).
65
00:06:18,608 --> 00:06:26,668
GRPO helps in optimizing the policy using group-relative advantages, where reward balancing mitigates the issue of reward sparsity.
66
00:06:26,668 --> 00:06:35,618
BAR ensures that additional trajectories are drawn to construct reward-balanced groups, reducing zero-variance groups which provide no useful learning signal.
67
00:06:35,666 --> 00:06:44,506
Evan: So, GRPO and BAR work together to refine the policy model based on its performance under the optimized harness, correct?
68
00:06:44,570 --> 00:06:45,660
Ashley: Exactly.
69
00:06:45,660 --> 00:06:51,980
By doing so, the policy model internalizes the successful behaviors elicited by the new harness.
70
00:06:51,980 --> 00:07:00,030
This updated policy model then shifts the failure modes, implicating different weaknesses to be targeted in subsequent H-steps.
71
00:07:00,074 --> 00:07:06,314
Evan: And with each round, both harness and policy keep improving adaptively, right?
72
00:07:06,362 --> 00:07:07,042
Ashley: Yes.
73
00:07:07,042 --> 00:07:14,422
The evolving policy becomes the new harness proposer for the next H-step, keeping the cycle iterative and recursive.
74
00:07:14,422 --> 00:07:18,312
Additionally, the task set for harness evaluation evolves too.
75
00:07:18,312 --> 00:07:27,982
Hard tasks, which are not solved in the preceding P-step, form the next search set, ensuring the optimization remains aligned with the policy’s latest failure modes.
76
00:07:28,034 --> 00:07:31,714
Evan: That’s a clever way to adapt continuously.
77
00:07:31,714 --> 00:07:36,594
How many rounds of these optimizations were conducted in the experiments?
78
00:07:36,650 --> 00:07:46,320
Ashley: The paper reports conducting two optimization rounds, with each involving six harness candidates per H-step and 50 RL updates per P-step.
79
00:07:46,320 --> 00:07:52,950
This assortment was deemed sufficient to demonstrate the effectiveness of Self-HPO across various benchmarks.
80
00:07:52,994 --> 00:08:03,754
Evan: Now, I know the paper uses Qwen3.5-35B-A3B as the base policy and mini-SWE-agent as the seed harness.
81
00:08:03,754 --> 00:08:06,994
Could you talk more about the datasets and benchmarks used?
82
00:08:07,088 --> 00:08:08,088
Ashley: Definitely.
83
00:08:08,088 --> 00:08:19,358
The training set consists of 2,484 coding tasks sourced from SWE-rebench, SWE-rebench V2, and Scale-SWE datasets.
84
00:08:19,358 --> 00:08:26,408
For evaluation, they used three benchmarks: SWE-Bench Verified, SWE-Bench Multilingual, and SWE-Bench Pro.
85
00:08:26,408 --> 00:08:36,998
These cover a diverse range of coding tasks, including human-validated GitHub issues and tasks spanning multiple programming languages and repository-level challenges.
86
00:08:37,058 --> 00:08:41,298
Evan: And what metrics are used to evaluate the performance?
87
00:08:41,354 --> 00:08:48,104
Ashley: Performance is primarily measured by the resolve rate, which is the percentage of tasks successfully completed by the agent.
88
00:08:48,104 --> 00:08:57,654
Additionally, the paper looks at metrics like pass@1 and pass@k, assessing reliability and capability over a broader range of task attempts.
89
00:08:57,698 --> 00:09:03,498
Evan: How do these metrics help in comparing Self-HPO with other methods?
90
00:09:03,554 --> 00:09:13,634
Ashley: These metrics provide a clear picture of both the reliability of the agent in solving tasks on the first attempt, and its overall capability across multiple attempts.
91
00:09:13,634 --> 00:09:24,174
By comparing these metrics, the paper showcases that Self-HPO outperforms policy-only and harness-only baselines, highlighting its advantage in joint optimization.
92
00:09:24,218 --> 00:09:26,568
Evan: That’s quite comprehensive.
93
00:09:26,568 --> 00:09:31,598
Did the authors perform any ablation studies to validate their method further?
94
00:09:31,658 --> 00:09:32,798
Ashley: Yes, they did.
95
00:09:32,798 --> 00:09:42,708
Ablations examined the impact of replacing RL strategies with fine-tuning methods, reusing fixed search sets, and using a self-proposer versus an external model.
96
00:09:42,708 --> 00:09:55,638
These studies confirmed that RL yields stronger policy updates than fine-tuning, evolving search sets target more relevant tasks, and self-proposal remains effective, though external proposals provide an additional boost.
97
00:09:55,682 --> 00:10:00,382
Evan: This detailed evaluation certainly strengthens the case for Self-HPO.
98
00:10:00,382 --> 00:10:03,302
Is there anything else that stands out in the methods section?
99
00:10:03,392 --> 00:10:11,062
Ashley: Another noteworthy aspect is how the optimized harness not only retains the seed agent's loop but adds targeted interventions.
100
00:10:11,062 --> 00:10:21,002
These include early warnings about budget constraints, checks for empty patches, ensuring API signatures conform to the repository, and fallback mechanisms.
101
00:10:21,002 --> 00:10:27,022
Such specific enhancements help avoid common submission failures, making the agent more robust.
102
00:10:27,134 --> 00:10:33,134
Evan: This kind of targeted control really underpins the agent's adaptive capabilities.
103
00:10:33,134 --> 00:10:37,894
It's fascinating to see how each optimization step builds on the previous one.
104
00:10:37,946 --> 00:10:48,706
Ashley: The interplay between harness and policy modules through consecutive rounds of optimization creates an agent capable of self-improvement in a highly targeted and efficient manner.
105
00:10:48,830 --> 00:10:51,170
Evan: So that's the end of the Methods section.
106
00:10:51,170 --> 00:10:57,350
Next, we'll move on to the experiments and results that showcase the practical impact of Self-HPO.
107
00:10:58,681 --> 00:11:03,931
Evan: Let's dive into the experiments and results that highlight the effectiveness of Self-HPO.
108
00:11:04,017 --> 00:11:04,747
Ashley: Evan.
109
00:11:04,747 --> 00:11:12,647
The experimental evaluation of Self-HPO is thorough, designed to validate its performance across multiple coding benchmarks.
110
00:11:12,647 --> 00:11:21,807
The authors used Qwen3.5-35B-A3B as the base policy and mini-SWE-agent as the seed harness.
111
00:11:21,867 --> 00:11:26,427
Evan: What datasets did they use for training and evaluation?
112
00:11:26,475 --> 00:11:37,265
Ashley: The training set comprised 2,484 tasks sourced from SWE-rebench, SWE-rebench V2, and Scale-SWE datasets.
113
00:11:37,265 --> 00:11:52,335
For evaluation, they employed three benchmarks: SWE-Bench Verified, SWE-Bench Multilingual, and SWE-Bench Pro, assessing diverse coding tasks spanning various programming languages and repository-level challenges.
114
00:11:52,395 --> 00:11:54,535
Evan: How did they structure these experiments?
115
00:11:54,579 --> 00:11:58,999
Ashley: The authors conducted two rounds of harness and policy optimization.
116
00:11:58,999 --> 00:12:05,829
Each round included six harness candidates per H-step and 50 reinforcement learning updates per P-step.
117
00:12:05,829 --> 00:12:11,539
This setup was crucial for demonstrating the iterative improvement capabilities of Self-HPO.
118
00:12:11,625 --> 00:12:14,875
Evan: What were the main performance metrics?
119
00:12:14,931 --> 00:12:21,071
Ashley: The primary performance metric was the resolve rate—the percentage of tasks successfully completed.
120
00:12:21,071 --> 00:12:29,551
Additionally, the authors used pass@1 and pass@k metrics to evaluate reliability and capability across multiple attempts.
121
00:12:29,595 --> 00:12:32,595
Evan: And what did the results show?
122
00:12:32,643 --> 00:12:39,473
Ashley: Self-HPO consistently outperformed baselines that optimized only the policy or only the harness.
123
00:12:39,473 --> 00:12:50,643
For instance, on SWE-Bench Verified, Self-HPO improved the resolve rate from 61.0% with the base model to 69.5%.
124
00:12:50,643 --> 00:13:01,423
Similar gains were observed on SWE-Bench Multilingual and SWE-Bench Pro, with increases of 5.5 and 6.9 percentage points, respectively.
125
00:13:01,467 --> 00:13:03,477
Evan: That's significant.
126
00:13:03,477 --> 00:13:09,727
How does this compare to other methods using external models like Opus 4.8?
127
00:13:09,771 --> 00:13:11,031
Ashley: Great question.
128
00:13:11,031 --> 00:13:17,581
With Opus 4.8 as the harness proposer in the HPO setup, the performance saw further improvements.
129
00:13:17,581 --> 00:13:32,371
Specifically, the resolve rate on SWE-Bench Verified improved to 71.6%, and average scores across benchmarks rose to 66.3%, showing the strong adaptability of Self-HPO.
130
00:13:32,427 --> 00:13:35,827
Evan: Were there any notable insights from the ablation studies?
131
00:13:35,883 --> 00:13:36,593
Ashley: Yes.
132
00:13:36,593 --> 00:13:39,653
The ablation studies highlighted three key insights.
133
00:13:39,653 --> 00:13:51,293
First, reinforcement learning yields stronger policy updates compared to supervised fine-tuning, with resolve rates dropping by 4.4 points when RL was replaced by SFT in the P-step.
134
00:13:51,293 --> 00:14:00,403
Second, evolving the search set to include hard tasks from previous rounds improved performance by 1.5 percentage points over using a fixed set.
135
00:14:00,403 --> 00:14:15,863
Third, while external models like Opus 4.8 provide a performance boost, the self-proposer mechanism in Self-HPO is still highly effective, achieving 69.5% resolve rate on SWE-Bench Verified when used alone.
136
00:14:15,915 --> 00:14:24,255
Evan: It sounds like the adaptability and continuous improvement aspects of Self-HPO are well-supported by these experiments.
137
00:14:24,315 --> 00:14:25,275
Ashley: Indeed.
138
00:14:25,275 --> 00:14:35,485
The authors also cross-evaluated the optimized components, finding that both the enhanced policy and harness retained their improvements, even when paired with their base counterparts.
139
00:14:35,485 --> 00:14:43,335
Combining these optimized components resulted in a joint gain exceeding the sum of their individual gains by 1.6 points.
140
00:14:43,395 --> 00:14:45,605
Evan: What about the pass@k metrics?
141
00:14:45,605 --> 00:14:47,655
Did they show improvements as well?
142
00:14:47,715 --> 00:14:52,055
Ashley: Yes, the pass@k metrics indeed showed significant improvements.
143
00:14:52,055 --> 00:15:02,835
For example, Self-HPO increased the pass@8 rate from 80.0% to 84.8%, demonstrating enhanced capability across multiple attempts.
144
00:15:02,835 --> 00:15:14,435
More notably, pass@1 rates increased for almost all task difficulties, except for some very straightforward tasks where performance slightly dipped, likely due to the focus on harder tasks.
145
00:15:14,499 --> 00:15:22,019
Evan: How does Self-HPO handle scenarios where no valid solution is found in the initial rollouts?
146
00:15:22,113 --> 00:15:36,233
Ashley: During reinforcement learning, Self-HPO reduces zero-success rollout groups—cases where no rollout succeeds—significantly lowering the proportion from around 11% to 17% to just 6.5%.
147
00:15:36,233 --> 00:15:44,083
This reduction is crucial because zero-success groups offer no reward signal, hence mitigating them enhances the learning process.
148
00:15:44,139 --> 00:15:47,019
Evan: That’s a substantial improvement.
149
00:15:47,019 --> 00:15:51,719
Can you give us an example of how the harness optimization works in practice?
150
00:15:51,801 --> 00:15:52,841
Ashley: Of course.
151
00:15:52,841 --> 00:15:56,791
One simple yet impactful example is the 'empty-patch guard'.
152
00:15:56,791 --> 00:16:01,231
Imagine a scenario where a policy submits an empty or malformed patch.
153
00:16:01,231 --> 00:16:08,211
The optimized harness intervenes, prompting the agent to check for any unstaged new files or incorrect submissions.
154
00:16:08,211 --> 00:16:13,991
This simple check ensures that valid patches are submitted, reducing the error rate significantly.
155
00:16:14,043 --> 00:16:15,223
Evan: That makes sense.
156
00:16:15,223 --> 00:16:17,843
Any closing thoughts on the experiment section?
157
00:16:17,907 --> 00:16:25,787
Ashley: Overall, the experiments and results strongly support the effectiveness of Self-HPO in creating self-improving coding agents.
158
00:16:25,787 --> 00:16:34,607
By dynamically optimizing both the harness and policy through iterative rounds, it achieves significant performance gains and enhanced adaptability.
159
00:16:34,659 --> 00:16:37,499
Evan: That brings us to the end of the Experiment section.
160
00:16:37,499 --> 00:16:43,199
In the next segment, we’ll discuss some related work and draw conclusions from this exciting framework.
161
00:16:44,453 --> 00:16:49,773
Evan: Alright, let's explore some of the related work that forms the backdrop for Self-HPO.
162
00:16:49,829 --> 00:16:50,879
Ashley: Sure.
163
00:16:50,879 --> 00:16:59,749
The paper discusses several lines of research that contribute to the broader context of harness and policy optimization for large language model agents.
164
00:16:59,813 --> 00:17:02,823
Evan: Let's start with harness optimization.
165
00:17:02,823 --> 00:17:06,573
What has previous research focused on in this area?
166
00:17:06,629 --> 00:17:16,019
Ashley: Harness optimization involves refining the external scaffolding that determines what the model observes, how its actions are executed, and the feedback it receives.
167
00:17:16,019 --> 00:17:19,669
This spans across prompts, workflows, and agent code.
168
00:17:19,669 --> 00:17:27,329
Studies have shown that optimizing these components without altering the policy model can still lead to significant performance gains.
169
00:17:27,479 --> 00:17:35,209
Evan: Could you give some specific examples of how harness optimization has been applied in past research?
170
00:17:35,261 --> 00:17:44,571
Ashley: For instance, Pryzant et al., 2023, and Khattab et al., 2023, focused on optimizing prompts and programs.
171
00:17:44,571 --> 00:17:52,131
Hu et al., 2024, and Zhang et al., 2025, worked on making workflows more efficient.
172
00:17:52,131 --> 00:17:59,221
Approaches like these have been used to guide agents in a variety of tasks, from software engineering to navigating websites.
173
00:17:59,345 --> 00:18:00,415
Evan: I see.
174
00:18:00,415 --> 00:18:04,845
And what about policy optimization—what's the focus there?
175
00:18:04,901 --> 00:18:13,591
Ashley: Policy optimization typically involves adjusting the policy model itself, often through reinforcement learning or supervised fine-tuning.
176
00:18:13,591 --> 00:18:20,001
This is done to improve the model based on the successful trajectories or rewards it receives during its tasks.
177
00:18:20,001 --> 00:18:31,961
Recent studies, such as those by Guo et al., 2025, and Wei et al., 2025a, have shown considerable advancements by focusing on policy optimization alone.
178
00:18:32,021 --> 00:18:37,241
Evan: So how does Self-HPO blend these two optimization strategies?
179
00:18:37,301 --> 00:18:44,351
Ashley: Self-HPO stands out by alternating between harness optimization and policy optimization within the same framework.
180
00:18:44,351 --> 00:18:50,901
This dual approach is relatively novel compared to prior methods that typically focus on one component at a time.
181
00:18:50,901 --> 00:18:58,961
The paper highlights several recent works that begin to explore this combined optimization, but often rely on an external harness proposer.
182
00:18:59,021 --> 00:19:04,021
Evan: And in the context of Self-HPO, how is this external reliance addressed?
183
00:19:04,085 --> 00:19:11,785
Ashley: Self-HPO mitigates this by using the evolving policy itself as the harness proposer, which is a significant step forward.
184
00:19:11,785 --> 00:19:24,385
For example, Kim et al., 2026, and Chen et al., 2026a, have also worked on optimizing both harness and policy, but they use an external model to suggest harness changes.
185
00:19:24,437 --> 00:19:27,817
Evan: That makes the framework more self-sufficient.
186
00:19:27,817 --> 00:19:32,317
What other aspects set Self-HPO apart in its approach?
187
00:19:32,411 --> 00:19:37,241
Ashley: Another key difference is how Self-HPO evolves the search set for the harness.
188
00:19:37,241 --> 00:19:47,301
While other methods might use a fixed set of tasks for harness evaluation, Self-HPO adapts by using tasks that the policy finds hard to solve in the previous round.
189
00:19:47,301 --> 00:19:53,141
This ensures that the optimization process continuously targets the policy's evolving weaknesses.
190
00:19:53,189 --> 00:19:56,279
Evan: That's a smart way to keep the optimization relevant.
191
00:19:56,279 --> 00:20:01,169
Are there any other notable methods in joint harness–policy optimization?
192
00:20:01,229 --> 00:20:07,799
Ashley: Yes, Luo et al., 2026, proposed HASE, which also uses a self-proposing policy.
193
00:20:07,799 --> 00:20:17,249
However, HASE places harness edits within the RL action space itself, unlike Self-HPO, which alternates between policy learning and harness search.
194
00:20:17,249 --> 00:20:26,969
Also, Xu et al., 2026a, introduced Evo2, which trains a separate harness proposer while keeping the task-solving policy frozen.
195
00:20:27,029 --> 00:20:35,909
Evan: So Self-HPO's method of using the evolving policy as its own harness proposer is fairly unique then.
196
00:20:35,957 --> 00:20:37,097
Ashley: Exactly.
197
00:20:37,097 --> 00:20:45,817
It’s a novel way to ensure that both the harness and policy evolve together, addressing their mutual dependencies directly and efficiently.
198
00:20:45,949 --> 00:20:50,229
Evan: And what about the practical applications of these advancements?
199
00:20:50,229 --> 00:20:52,569
How are they typically utilized?
200
00:20:52,613 --> 00:21:07,673
Ashley: Harness optimizations are employed in a range of applications, from software engineering as seen in the works by Le et al., 2026, to more general use cases like automated computer use described by Agashe et al., 2025.
201
00:21:07,673 --> 00:21:17,913
Joint optimization strategies, especially those addressing evolving tasks and environments, represent a cutting-edge approach to making these systems more robust and generalizable.
202
00:21:17,957 --> 00:21:28,037
Evan: This gives a comprehensive understanding of the related works that inform Self-HPO and the distinct contributions it makes.
203
00:21:28,085 --> 00:21:29,055
Ashley: Indeed.
204
00:21:29,055 --> 00:21:41,985
With Self-HPO's recursive self-improving framework, where the agent itself proposes harness revisions and adapts through reinforcement learning, it pushes the envelope in both harness and policy optimization.
205
00:21:41,985 --> 00:21:45,505
This integrated approach sets a new benchmark in the field.
206
00:21:45,557 --> 00:21:48,957
Evan: That wraps up our discussion on the Related Work section.
207
00:21:50,214 --> 00:21:55,114
Evan: We've covered the introduction, methods, experiments, and related work.
208
00:21:55,114 --> 00:21:59,754
Now, let's summarize the key contributions and takeaways of this paper.
209
00:21:59,814 --> 00:22:00,974
Ashley: Certainly.
210
00:22:00,974 --> 00:22:10,234
The paper introduces Self-HPO, a recursive self-improving framework for coding agents that alternates between harness and policy optimization.
211
00:22:10,278 --> 00:22:18,878
Evan: This approach leverages the evolving policy as its own harness proposer, enabling continuous adaptation and improvement in performance.
212
00:22:18,942 --> 00:22:27,272
Ashley: The framework has been thoroughly tested across three coding benchmarks, demonstrating significant improvements in resolve rates and robustness.
213
00:22:27,272 --> 00:22:33,922
The average resolve rate increased notably from 57.8% to 64.8%.
214
00:22:33,966 --> 00:22:39,876
Evan: One of the standout features is the dynamic interplay between the harness and policy modules.
215
00:22:39,876 --> 00:22:44,326
Both components keep improving adaptively, ensuring sustained gains.
216
00:22:44,382 --> 00:22:45,622
Ashley: Exactly.
217
00:22:45,622 --> 00:22:55,482
The optimized components retain their enhancements even when paired with their base counterparts, and their combined improvement exceeds the sum of individual gains.
218
00:22:55,482 --> 00:22:59,762
This highlights the efficiency of joint harness–policy optimization.
219
00:22:59,814 --> 00:23:14,654
Evan: The paper also provides insights into how reinforcement learning techniques like Group Relative Policy Optimization and Balanced Adaptive Rollout contribute to refining the policy model, making it more effective under the optimized harness.
220
00:23:14,718 --> 00:23:26,898
Ashley: Moreover, the evolving search set ensures that harness optimization targets the latest policy weaknesses, making the agent's performance enhancement continuous and aligned with its evolving capabilities.
221
00:23:27,008 --> 00:23:34,918
Evan: And with the potential to integrate stronger external models, Self-HPO opens up pathways for even greater performance improvements.
222
00:23:34,974 --> 00:23:46,174
Ashley: Overall, Self-HPO represents a significant advancement in creating self-improving coding agents, pushing the boundaries of what these systems can achieve.
223
00:23:46,280 --> 00:23:48,430
Evan: That's all for today's episode.
224
00:23:48,430 --> 00:23:54,470
Thank you for joining us on Daily Paper Cast as we explored the intricacies of Self-HPO.
225
00:23:54,534 --> 00:23:57,254
Ashley: We hope you found the discussion insightful.
226
00:23:57,254 --> 00:24:04,514
Be sure to tune in for our next episode where we delve into another exciting paper from the world of AI and machine learning.
227
00:24:04,596 --> 00:24:08,686
Evan: Until then, stay curious and keep exploring!
228
00:24:08,686 --> 00:24:10,046
Goodbye for now.
229
00:24:10,140 --> 00:24:11,330
Ashley: Goodbye!
