AI

Betelgeuse: Running Qwen 3.8 Flash Next at 22 tok/s sustained (spiking to 48 tok/s) on an RTX 3060 12GB + 16GB DDR4 RAM 100% bit-exact!

Read full article at source ↗

Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 31/10/2026, 15:47:26 UTC.

Briefings

A modified llama.cpp fork named Betelgeuse achieves 16.8 to 48 tok/s on an RTX 3060 12GB + 16GB DDR4, running Qwen 3.8 Flash Next at 100% bit-exact without gate pruning. The system uses a custom in-stream CUDA kernel to resolve expert misses in ~2 microseconds, reducing physical SSD reads from 71 MB/token to 12.43 MB/token. Unlike Strata, which requires 48GB+ RAM for peak speeds, Betelgeuse maintains performance on lower memory by utilizing a dedicated RAM victim cache and exclusive VRAM/RAM allocation.

Key points

• Betelgeuse achieves sustained 16.8 tok/s on warm runs and up to 48 tok/s on peak bursts using Qwen 3.8 Flash Next on an RTX 3060 12GB + 16GB DDR4.
• The system runs at 100% bit-exact with zero gate pruning, verified against golden stock output with 0.000e+00 diff across 28,080 router records and 576 tokens.

Context

This benchmark represents a significant improvement over previous attempts to run large MoE models on consumer hardware. The project demonstrates that streaming + victim-cache architecture can eliminate the 48 per-layer CPU/GPU scheduler splits that previously caused severe thrashing, allowing full bit-exact inference on a 12GB VRAM setup.

Limitations

Performance is heavily dependent on RAM availability; Strata requires 48GB+ for peak speeds while Betelgeuse maintains 16.8-48 tok/s on 16GB RAM. Physical SSD reads remain at 12.43 MB/token, and the system is currently limited to Qwen 3.8 Flash Next; DeepSeek V4.1 testing is ongoing.

Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 31/10/2026, 17:02:41 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.

View archived briefing on IPFS
bafybeih32agwoedrjyan54njcvok4tgemf7zs77awx4p772f2ymu2shznu