AI

BlockLock: Reproducible LLM Inference with Fixed Batch Geometry

Read full article at source ↗

Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 06/11/2026, 05:03:24 UTC.

Briefings

BlockLock is a new method for reproducible LLM inference that fixes the batch geometry of protected requests while preserving shared prefill compute. Evaluated on five dense models and three GPU deployments, BlockLock-Rows reduces latency by 53.5–55.0% and throughput by 115.2–122.2% compared to vLLM Batch-Invariant. The approach constrains only the protected requests, avoiding custom kernel implementations.

Key points

• BlockLock-Rows reduces latency by 53.5–55.0% and throughput by 115.2–122.2% relative to vLLM Batch-Invariant.
• BlockLock uses two modes: BlockLock-Rows with fixed-width rows and BlockLock-Pack with terminal-anchored packing.

Context

This research addresses the need for reproducible LLM inference, where output must remain consistent regardless of other requests in the batch. Existing solutions either constrain every operator or isolate prefill, but BlockLock preserves shared prefill compute while pinning only the protected request's numerical path.

Limitations

Evaluation covers five dense models and three GPU deployments; initial tensor-parallel experiments on two A6000 GPUs are included. Results are based on initial tensor-parallel experiments and may vary with different hardware configurations or model architectures.

Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 06/11/2026, 06:11:14 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.

View archived briefing on IPFS
bafybeibtl3oapdhftxvjx3422xt5lca47enz66xwmvxteus374rgwceuxe