AI

2 gaming PCs + an M4 Pro MacBook over Ethernet + 2 iPhone 17 Pro Maxes over USB-C: Qwen3.8 Flash Next at 37-39 tok/s at 60k context (262k layout in the works)

Read full article at source ↗

Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 18/10/2026, 17:47:26 UTC.

Briefings

A 48-layer Qwen3.8 Flash Next model was distributed across five devices: a gaming PC with an RTX 2060 Super, a MacBook with an M4 Pro, and two iPhones with A19 Pro chips. The system achieved 37-39 tokens per second at 60k context length using pipelined speculation and shared expert caching. The M4 Pro handles the most frequent experts, while the iPhones handle less frequent ones, with communication via Ethernet and USB-C. The setup supports up to 69.6k context length, though the final quality check against a reference distribution remains pending.

Key points

• The Qwen3.8 Flash Next model, a 48-layer mixture-of-experts architecture with 512 experts per layer, was successfully distributed across five devices including a gaming PC, a MacBook, and two iPhones.
• At 60k context length, the system achieved 37-39 tokens per second for writing and 856-895 tokens per second for reading, utilizing pipelined speculation to minimize device waiting times.

Context

This experiment demonstrates a novel hardware-accelerated LLM inference architecture using heterogeneous computing. The setup allows for high-throughput token generation and reading by leveraging the specific strengths of different consumer-grade GPUs and mobile processors, with communication optimized via Ethernet and USB-C to minimize latency.

Limitations

The final quality check comparing the generated output to a reference distribution has not been completed. The system's maximum context length is capped at 69.6k due to memory constraints on the A19 Pro devices. The exact impact of the Mac's expert cache on overall quality remains to be verified.

Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 18/10/2026, 17:56:51 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.

View archived briefing on IPFS
bafybeibbhc7lmrpc7dwcwoh5ivr7ofurmyu2lwdxtene7mvjovg3ypagvm