AI

Betelgeuse running Qwen 3.8 Flash Next IQ4_XS at 24 tok/s on 2x8GB DDR4 (14 tok/s on a single 16GB stick) + RTX 3060 12GB — 0 dropped experts, 100% byte-exact

Read full article at source ↗

Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 01/11/2026, 08:47:25 UTC.

Briefings

A user tested Betelgeuse AI's IQ4_XS quantization on an RTX 3060 with 16GB total RAM, achieving 14 tok/s on single-channel DDR4 and 24 tok/s on dual-channel 8GB DDR4. The results demonstrate that the exclusive host RAM victim cache architecture allows the GPU's hardware DMA engine to pull victim-cache hits across PCIe at full speed, significantly improving performance on 4-bit models compared to single-channel setups.

Key points

• Betelgeuse AI's IQ4_XS quantization achieved 14 tok/s on a single 16GB DDR4 stick and 24 tok/s on dual-channel 8GB DDR4.
• The performance jump from 14 tok/s to 24 tok/s occurred because the exclusive host RAM victim cache architecture allows the GPU's hardware DMA engine to pull victim-cache hits across PCIe at full speed.

Context

This benchmark tests Betelgeuse AI's streaming + victim-cache architecture over DeepSeek V4.1 Flash (1017) on an RTX 3060 12GB. The exclusive host RAM victim cache + Async Hardware DMA (MOE_DMA=1) between VRAM slot pool and NVMe drive enables full PCIe bandwidth utilization for victim-cache hits, unlike standard dual-channel setups where CPU and PCIe H2D DMA engine compete over a single 64-bit channel.

Limitations

Testing was conducted on a single RTX 3060 12GB with 16GB total system RAM. Results are snapshot-based from a specific GitHub thread and do not account for future hardware upgrades, different RAM speeds, or varying NVMe drive performance. The 24 tok/s figure represents the maximum observed performance on dual-channel 8GB DDR4 under the specific betelgeuse engine configuration.

Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 01/11/2026, 10:25:05 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.

View archived briefing on IPFS
bafybeien2yrmsdv4wqm6o3ludnmaxikt4dqq3bnelkkl7ptozfo3xawmky