TBNAI
How a Voice Agent Learns the Rhythm of Conversation — Shawn Wen
Publication: Machine Learning Street Talk (MLST)
Author: Machine Learning Street Talk (MLST)
Source published: 01/10/2026, 20:03:16 UTC
Briefing created: 02/10/2026, 00:37:50 UTC
Based on the episode description, not a full transcript. Source evidence retrieved 01/10/2026, 20:58:24 UTC.
Briefings
Shawn Wen, CTO of PolyAI, explains that voice agents face greater complexity than text agents due to real-time audio adaptation rather than just reasoning. The Audio-native model Dialog-RSN-1 predicts turn-taking signals before generating text, with transcripts generated last for auditability. Training required real noisy calls with added synthetic noise; over-cleaned audio degraded performance. The episode covers latency, regional accents, benchmark limitations, and the shift from content production to cognitive debt management.
Key points
- Shawn Wen is the CTO of PolyAI who explains the challenges of voice agents compared to text agents.
- The Audio-native model Dialog-RSN-1 predicts turn-taking signals before generating text responses.
Context and limitations
The episode covers training data, spectrograms, and synthetic noise; it does not provide specific numerical results or detailed experimental outcomes.
Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 02/10/2026, 00:37:50 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.
View archived briefing on IPFS
bafybeieczdcz3zpzng6xelr2emxc7kfl6n2vwawrnydszevf2zy4oyi7nq