AI

When Is Proof-Step Verification Worth Its Cost? Operating Requirements of Certify-or-Decline Verification for Open-Weight Mathematical Reasoning

Read full article at source ↗

Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 13/11/2026, 07:48:24 UTC.

Briefings

A new verification pipeline using LLM judges to certify or decline proof steps in open-weight mathematical reasoning models was evaluated on GPQA, AIME-2025, and a MATH-500 subset. Certification increased answer-key match precision by 31 percentage points on GPQA and 46 on AIME, but coverage dropped to 35% and 21% respectively. On near-ceiling MATH-500, gains were minimal at 5 points while coverage loss reached 13-27 points. An output-token-matched self-consistency selector achieved 82.9% precision at 35% coverage versus the verifier's 87.6% pooled accuracy. A diagnostic sample showed 52 of 88 completed problems failed proof construction without reaching a judge. External-LLM acceptance of certified chains was 61.0-67.6%, significantly lower than the 87.6% answer-key match rate. These audits lack human or formal-kernel ground truth.

Key points

• Certification raised answer-key match precision by 31 percentage points on the strong GPQA configuration.
• External-LLM acceptance of rendered certified chains was 61.0-67.6%, significantly lower than the 87.6% answer-key match rate.

Context

This study evaluates a proof-step verification pipeline using LLM judges on open-weight models across GPQA, AIME-2025, and a MATH-500 subset to determine if certification is worth its cost.

Limitations

Evidence is limited to a single preprint snapshot from November 2026. Results lack human or formal-kernel ground truth, and external-LLM acceptance rates are not validated against formal verification standards.

Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 13/11/2026, 08:51:44 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.

View archived briefing on IPFS
bafybeiaa7rf4l4pam5vtxangeadakmxkthqrpzn2uw2la2be7ruzupi6ku