AI

FluidPD: In-Place Elasticity for SLO-Aware Prefill-Decode Disaggregated LLM Serving

Read full article at source ↗

Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 08/10/2026, 02:18:23 UTC.

Briefings

FluidPD is an in-place elasticity system for LLM serving that addresses SLO violations caused by prefill-decode phase imbalance. It uses FluidToken to offload transient prefill work to decode workers and FluidRole to reassign running workers between phases without model reload. Evaluated on Azure production traces, it improved SLO attainment by up to 94.6 percentage points compared to static SGLang, proving SLO-aware in-place elasticity improves service quality without additional provisioning.

Key points

• FluidPD introduces two mechanisms: FluidToken for transient imbalance by offloading prefill computation to decode workers, and FluidRole for sustained imbalance by reassigning running workers between phases in place.
• FluidPD uses lightweight pressure indices to expose prefill and decode-side resource pressure before they appear as SLO violations, improving overall SLO attainment by up to 94.6 percentage points over static SGLang on Azure production traces.

Context

LLM serving architectures often separate prefill and decode phases with distinct execution patterns and SLO objectives. Existing systems typically combine a fixed prefill/decode worker ratio with request routing across workers, causing latency SLO violations when workloads shift between phases. FluidPD addresses this by providing SLO-aware in-place elasticity without requiring model reload or engine restart.

Limitations

Evaluation was conducted on Azure production trace workloads only. The system's performance gains are specific to this environment and may not generalize to other cloud providers or workloads. The 94.6 percentage point improvement represents a maximum observed gain across production traces, and the study does not account for long-term stability or failure modes under extreme load conditions.

Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 04/11/2026, 05:31:19 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.

View archived briefing on IPFS
bafybeiemkqwfyc4al2ag5v3agyqhfofp727gh6dyqxrw3n2mlzad6zteh4