Tracing full-duplex failure modes back to their cause

Averaging a metric over a conversation hides the moments that define it. Two checkpoints with comparable scores can differ entirely in whether the model lets you finish a sentence, and the standard leaderboard number does not separate them.

The failures that mattered in practice were behavioural and gradual: output volume fluctuating across a long reply, noise accumulating as generation continued, hallucination loops where the model talked itself into a corner, and unstable backchannelling that read as either interrupting or inattentive.

The fine-tuned English full-duplex model was benchmarked with Full-Duplex-Bench against the base Moshi model, specifically on overlap handling and conversational turn-taking rather than transcription accuracy alone.

The pipeline then ran at checkpoint level, so a degradation could be located in training time rather than discovered at release. Each pattern was traced back to a cause in the training data or the fine-tuning strategy — which data slice introduced it, which stage amplified it — so the output of evaluation was a specific change to make, not a score to report.

Attribution, not just detection, is the point: a named failure mode with no cause attached does not change the next training run.

Four recurring degradation patterns were isolated, attributed, and turned into targeted fixes in data curation and fine-tuning strategy, with the benchmark comparison against base Moshi providing the reference point for whether each fix held.