Skip to main content

ASR training data generation

Automatic speech recognition (ASR) models need reverberant, noisy speech mixtures paired with exact transcripts. Recording such pairs at scale is slow, expensive, and difficult to control — you rarely get both varied acoustic conditions and precise transcripts from the same recording session.

Audio scene generation produces both from the same recipe: ConversationRules places speech utterances on the timeline with natural turn-taking, the scene convolves them with a real room impulse response, and AudioScene.transcript() reads back the exact text for what actually ended up in the mixture.

Beyond ASR, the same workflow applies to other speech-centric tasks: Speaker Identification & Diarization, Environmental Noise Cancellation (ENC), Speech Enhancement and Denoising models need clean/noisy pairs with controlled SNR, keyword spotting models need varied acoustic conditions, and speaker diarization models need multi-talker scenes with known speaker identities and timing.

Workflow overview​

The workflow has five stages:

  1. Audio datasets — Load a speech dataset and a noise dataset. The speech corpus provides the ASR target utterances; the noise corpus provides background sound placed at separate source positions. For supported formats and loading methods, see the Audio datasets how-to guide.

  2. IR collection — Import a Treble IR collection. Each impulse response describes how sound travels from one source to one receiver in a specific room, giving the scene generator its acoustic diversity.

  3. Scene rules and source groups — Define SceneRules and add SourceGroup entries for speech (tagged as the target) and noise (tagged as background). ConversationRules controls turn-taking behavior; NoiseSourceRules or TransientNoiseRules controls noise placement. For the full pattern, see Source groups in the Scene collections how-to guide; for ConversationRules parameters, see Conversation track generation with explicit rules in the Audio scenes how-to guide.

  4. Listener configuration — Configure ListenerRules with an optional device, microphone self-noise and frequency response via DeviceSpecs, and randomized orientation. For details, see Listener rules in the Scene collections how-to guide.

  5. Scene generation, rendering, and post-processing — Generate a SceneCollection from the rules, run remote device-IR optimization if needed, render audio to disk, and extract transcripts and metadata. For batch rendering details, see Rendering a large collection to disk in the Rendering a scene collection how-to guide.

tip

If you don't already have a speech corpus, use repo_id="treble-technologies/librispeech_asr_sliced" with LibriSpeechAudioLoader — a pre-sliced LibriSpeech variant designed for ConversationRules turn-taking. See Dataset from Hugging Face in the Audio datasets how-to guide for the loading code.

info

The WAV files bundled with this notebook are minimal samples included for demonstration purposes only. They don't provide the diversity needed for real model training — replace them with a larger, more varied corpus for production use.

Loading notebook...