AI

Meta trained small AI models to read raw bytes instead of tokens. They start out worse, but the paper predicts they end up better

Read full article at source ↗

Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 24/10/2026, 17:47:25 UTC.

Briefings

A 2026 study by DotaMate suggests small AI models trained to read raw bytes instead of tokens can eventually surpass token-based models like Llama 3.2 and Gemma 3. While token models initially perform better, byte models show consistent improvement with more training, eventually outperforming the larger models by a few points. The research indicates byte models require approximately 6x less text to reach comparable performance levels.

Key points

• Small AI models trained to read raw bytes instead of tokens can eventually surpass token-based models like Llama 3.2 and Gemma 3.
• Byte models require approximately 6x less text to reach comparable performance levels compared to token models.

Context

This research addresses the efficiency of reading raw bytes versus tokens in small AI models, with a prediction that byte models will eventually outperform token models despite initial token model superiority.

Limitations

The 'beats Llama and Gemma' claim is a prediction based on extended trend lines rather than actual training data. Token models are slower due to higher processing steps per letter. Scale performance remains unknown as only small models were tested.

Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 24/10/2026, 17:52:00 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.

View archived briefing on IPFS
bafybeigypdrgzlpxvxtmi4cav22grjxgsttznptu4i6dsgth7vlujcdcri