AI

SlimWise: Decoupling Expert Pruning Across Prefill and Decode for Efficient MoE Serving

Read full article at source ↗

Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 07/10/2026, 08:18:24 UTC.

Briefings

SlimWise is a serving framework that decouples expert pruning across prefill and decode phases for MoE models. It performs prefill with the full model and decode with a pruned model that reuses the prefill-generated KV cache. This approach narrows accuracy gaps relative to the full model across two MoE backbones and three pruning criteria, with no training required. The framework was implemented in vLLM and supports both prefill-decode disaggregation and colocated serving.

Key points

• SlimWise performs prefill with the full model and decode with a pruned model that directly reuses the prefill-generated KV cache without conversion.
• SlimWise improves decode throughput by up to 1.81x at 50% expert pruning on Qwen3.6-35B-A3B with minimal accuracy loss.

Context

Mixture-of-experts models activate few experts per token, yet batched decoding can access nearly the entire expert pool, making expert-weight traffic a major bottleneck. Expert pruning reduces this traffic, but conventional approaches also prune compute-bound prefill, sacrificing model quality for little throughput benefit. SlimWise addresses this by tailoring the expert pool to each inference phase.

Limitations

The study covers two MoE backbones and three pruning criteria. Benchmark accuracy can conceal substantial pruning-induced changes in generation length. The framework was implemented in vLLM and supports both prefill-decode disaggregation and PD-colocated serving. No additional background or causal claims are supported beyond the provided evidence.

Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 26/10/2026, 06:31:46 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.

View archived briefing on IPFS
bafybeievzsx3ejjthwtv36lvds574wvrcdsbxnc5ajfcg25jb6mimannoi