TBNAI
Ensemble Routing for Prompt-Adaptive Cost-Aware LLM Inference
Publication: cs.AI updates on arXiv.org
Author: Saul Urso, Emanuele Carlini, Patrizio Dazzi, Matteo Mordacchini
Source published: 10/11/2026, 05:00:00 UTC
Briefing created: 10/11/2026, 10:03:29 UTC
Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 10/11/2026, 08:33:27 UTC.
Briefings
Researchers developed PISA, a prompt-adaptive router that selects a single LLM model per request by estimating individual model capabilities using soft cluster memberships and predictors. PISA-E extends this to ensemble routing, dynamically adjusting model composition and cardinality based on explicit cost-quality preferences. Across four benchmarks, PISA achieved the highest macro-average cost-performance hypervolume in both in-distribution and out-of-distribution settings, while PISA-E outperformed ensemble competitors by 2.09 and 2.87 percentage points in task performance under ID and OOD conditions, respectively.
Key points
- PISA combines soft cluster memberships with model-specific predictors to estimate individual model capabilities for single-model routing.
- PISA-E jointly selects ensemble composition and cardinality under an explicit cost--quality preference without ensemble-specific training.
Context
Large language models increasingly support Web-based services requiring balancing response quality and inference cost across heterogeneous requests. Conventional routing selects one model per prompt, while fixed ensembles cannot adapt their composition and cost to individual requests. This work introduces PISA and PISA-E to address these limitations.
Limitations
Results are based on four LLM routing benchmarks under in-distribution and out-of-distribution conditions. Performance metrics include macro-average cost--performance hypervolume and task performance percentages. No additional background context or author details are provided beyond the abstract.
Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 10/11/2026, 10:03:29 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.
View archived briefing on IPFS
bafybeido2773r2qj7ads7hhrvf7yme7nwfhgsc6pe7mu3j5tk2od4wgzuu