AI

What Gradients Add to Text Leakage in Split Language Models, Counted per Token and per Document

Read full article at source ↗

Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 06/10/2026, 10:18:24 UTC.

Briefings

Research reveals that gradient information significantly enhances text leakage in split language models, allowing attackers to reconstruct 97.38% of tokens from activations alone on GPT-2, compared to 94.20% without gradients. Document-level leakage is even more severe, with attackers recovering 37.77% of tokens from 32-token documents using gradients versus 13.71% without them. The attacker's ability to reconstruct text depends on whether they possess the server's weights, and the server's training run length affects model quality and leakage rates.

Key points

• Attackers with publicly released weights of client layers can recover 97.38% of tokens from activations alone on GPT-2, a 3.17 percentage point improvement over 94.20% without gradients.
• Document-level leakage is significantly higher than token-level leakage, with attackers recovering 37.77% of tokens from 32-token documents using gradients versus 13.71% without them.

Context

Split learning allows clients to train language models on servers without sending raw text, sending only output vectors. This study quantifies the security implications of this architecture by measuring reconstruction success rates of client text from server traffic, revealing that gradient information substantially improves an attacker's ability to recover model outputs.

Limitations

Results are limited to GPT-2 and Qwen3-0.6B models; leakage rates vary with the server's training run length and layer configuration. The 95% confidence intervals for token recovery are [2.72, 3.64], and document-level leakage metrics are derived from specific experimental setups rather than generalizable population statistics.

Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 19/10/2026, 19:01:22 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.

View archived briefing on IPFS
bafybeighdinm4ehsyw6evdj7jqhbi3ooyhltoybmjcqzh76u2rm7vzajm4