TBNAI
Exploration-Preserving Policy Optimization
Publication: cs.AI updates on arXiv.org
Author: Hangzhan jin, Mohammad Hamdaqa, Doina Precup
Source published: 06/10/2026, 04:00:00 UTC
Briefing created: 06/10/2026, 05:06:37 UTC
Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 06/10/2026, 04:18:24 UTC.
Briefings
ExPPO is a reinforcement learning method that uses prompt-relative response surprisal and pass rate to redistribute learning credit, improving reasoning accuracy and coverage under repeated sampling. Experiments show better in-domain and out-of-domain performance compared to standard RL, with strong results at large sampling budgets. The method preserves verifier polarity and maintains total sequence-advantage mass per prompt group.
Key points
- ExPPO redistributes learning credit using prompt-relative response surprisal and prompt pass rate to improve reasoning accuracy.
- Experiments demonstrate improved in-domain and out-of-domain reasoning coverage with higher aggregate response accuracy.
Context and limitations
Analysis characterizes response-level credit allocation and sampled mode updates; specific numerical values for accuracy gains or sampling budget thresholds are not provided in the evidence.
Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 06/10/2026, 05:06:37 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.
View archived briefing on IPFS
bafybeigtn6qmi43yglnkhtdrjra65zst4pyqt2okmcfqmllzw4t6i6lbum