TBNAI
$2800 rig with 8x Radeon Pro V620 (256 GB VRAM) + custom vLLM fork = Qwen3.8-Flash-Next at 60 to 100 t/s decode and 3000+ t/s prefill
Publication: top scoring links : multi
Author: /u/_TheWolfOfWalmart_
Source published: 08/10/2026, 17:12:46 UTC
Briefing created: 14/11/2026, 15:41:24 UTC
Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 09/10/2026, 05:47:26 UTC.
Briefings
A user named _TheWolfOfWalmart_ successfully built a custom rig using older Radeon Pro V620 enterprise cards with 32 GB VRAM to run Qwen3.8-Flash-Next via a vLLM fork. The system achieved 60 to 100 tokens per second decode and 3000+ tokens per second prefill, significantly outperforming standard llama.cpp benchmarks. The project involved custom RDNA2 kernel optimization and specific quantization settings (W4A16 for routed experts, BF16 for others) to maximize performance on the constrained hardware.
Key points
- The user _TheWolfOfWalmart_ purchased used Radeon Pro V620 enterprise cards with 32 GB VRAM to build a high-performance LLM rig.
- A custom vLLM fork was developed to run Qwen3.8-Flash-Next, achieving 60 to 100 t/s decode and 3000+ t/s prefill on the custom hardware.
Context
The article describes a personal project where the author attempted to overcome limitations of standard LLM inference libraries like llama.cpp and vLLM on older enterprise GPUs. The success of the vLLM fork and custom kernel optimizations represents a significant achievement in optimizing large language model deployment on constrained, non-RTX hardware.
Limitations
Performance metrics are based on a single user's specific setup with custom quantization and kernel tuning. Results may vary with different hardware configurations or model versions. The author plans to continue testing other models like DeepSeek and GLM-5.3-Flash, but current data only covers Qwen3.8-Flash-Next.
Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 14/11/2026, 15:41:24 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.
View archived briefing on IPFS
bafybeidjzwinmm6eoujl6pnkx3txgo77nshihootw5pyjsgueapurg2rgi