AI

Learning Latent Protein Languages for Autoregressive Generation

Read full article at source ↗

Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 06/10/2026, 17:18:24 UTC.

Briefings

This study introduces two learned latent protein languages, PLL and SLL, designed to improve autoregressive generation for protein sequences and structures. PLL maps amino acid residues to a 4,096-state alphabet using a frozen ESM-2 encoder, while SLL adapts GCP-VQVAE Lite with sequence supervision. Pretraining on these languages yields PLLM and SLLM, which outperform standard amino acid models in compute-scaling efficiency and reduce entropy below 1.5-bit thresholds by 54% during generation. For sequence-to-structure prediction, SLLM reduces validation perplexity by 34% compared to the original tokenizer, and long proteins are generated 1,000 times faster than MSA-based AlphaFold2.

Key points

• PLL maps amino acid residues to a 4,096-state contextual alphabet using a frozen ESM-2 encoder, with one token per residue.
• SLL adapts GCP-VQVAE Lite with auxiliary sequence and confidence supervision to retain decoding to backbone coordinates.

Context

Autoregressive transformers currently struggle with protein sequence and structure generation. This research addresses this by introducing learned latent protein languages that encode residue identities without explicit contextual semantics, addressing a key limitation in current generative models.

Limitations

Results are based on internal measurements and pretraining data; long protein performance is approximate. No additional background context is supplied beyond the provided evidence.

Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 22/10/2026, 14:39:23 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.

View archived briefing on IPFS
bafybeih3mbrsoapktrz7ct2qhsjr3han4oxn7q7b4jrfbqnu4pmug7tnae