GIỚI THIỆU VỀ TẬP NÀY
HKUST & Trip.com: RL post-training makes a generative recommender a better ranker but a worse retriever, with recall@500 down 5.1 points on Office as NDCG@10 rises.
Key takeaways:
- Better ranker, worse retriever. Scoring every catalogue item for every test user on five Amazon datasets (Qwen2.5-1.5B-Instruct, semantic IDs), GRPO lifts Office NDCG@10 from 11.59% to 12.55% while hit@500 falls from 52.84% to 47.69%. The loss starts at K≈20 (Office) and K≈50 (Industrial); 14–22% of targets in the SFT top 500 drop out.
- Reordering, not sharpening. Entropy falls from 6.0 to 3.1 nats, but sharpening can't change hit@K; mid-ranked targets are demoted. The exact KL-regularised objective can't lower expected hit@K; the loss comes from how the update is estimated.
- The cause. GRPO (Group Relative Policy Optimization) rewards the logged item in a 16-candidate rollout and penalises the rest, often other users' real next items, and shared weights carry the penalty to similar users. Positive-only advantages remove the loss (and the top gain); negative-only makes it ~3× larger on Office; DPO with beam negatives loses 4.1 points. A pre-RL rollout record predicts who loses most.
- The fix: out-of-rollout anchoring. One KL term keeps the SFT model's order of 16 sampled out-of-rollout neighbours. Office hit@500 returns to 52.14% (4.4 of 5.1 points) with NDCG@10 at 12.64%; the other four datasets reach SFT level. With REINFORCE++ (−14.9 points on Office) the anchor recovers 13.3.
- Our take. Evaluate each funnel stage at the depth it's used. Tests use public benchmarks and a 1.5B model; industrial scale with a ranker in the loop is still open.
Chapters:
00:00 Cold open: better rankers, worse retrievers
02:08 Semantic IDs and how they're deployed
04:07 Why item 500 matters
05:14 Office results: NDCG@10 up, hit@500 down
07:24 Sharpening or reordering?
08:44 Inside GRPO: rollouts and penalties
10:25 Printer-ink example and gating
12:27 Controlled experiments
14:06 The fix: out-of-rollout anchoring
15:59 Recall recovered, top gains kept
17:27 Earlier warnings from industry
18:22 Our take, recap and closing question
Sources:
- Better Rankers, Worse Retrievers: The Hidden Cost of RL Post-Training in Generative Recommendation (Yuheng Chen, Fan Zhang, Yixuan Xu, Chao Yang, Yang Xiang, Can Yang (HKUST / Trip.com Group); arXiv, Nov 17, 2026)
Enjoyed it? Follow Data Science in the Wild (one real production ML/AI system or paper per episode, hosted by Peter Liu) and rate it on Apple Podcasts or Spotify.
Also from Peter: The GenAI Evolution Atlas, Wednesday AI deep dives and Friday news roundups.
Made with AI-generated voices; curated by Peter Liu from the sources above.