AI
Thinking Outside the Box: Can Language Models Rely on External Guidance Selectively?
Publication: Hugging Face Daily Papers
Author: Not available in collected metadata
Source published: 29/09/2026, 20:00:00 UTC
Briefing created: 01/10/2026, 09:07:27 UTC
Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 01/10/2026, 04:18:24 UTC.
Briefings
This study introduces Box^2-Bench, a benchmark designed to isolate how language models regulate reliance on external guidance. By keeping models and tasks fixed while varying workflow reliability, researchers observed that frontier models often benefit from reliable guidance but remain vulnerable when it is misleading or unreliable. The study also found that this selective reliance extends beyond workflows to other external information, such as peer correction and corrupted memory. Training strategies were tested, showing counterfactual supervised fine-tuning improves robustness, while outcome-based reinforcement learning shifts the balance toward greater use of helpful workflows. The findings suggest that agent reliability depends on the ability to distinguish between useful and fallible external information, a factor not captured by task performance alone.
Key points
- Box^2-Bench isolates model reliance on external guidance by varying workflow reliability while keeping models and tasks fixed.
- Frontier models benefit from reliable guidance but remain vulnerable when guidance is misleading or unreliable.
Context and limitations
The benchmark isolates workflow reliability but does not account for all forms of external information or the full spectrum of model capabilities.
Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 01/10/2026, 09:07:27 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.
View archived briefing on IPFS
bafybeid5e2d54dzbpgnpofujvwtz6dspqrftw35qptwez7cy5l5xgxyk4y