Skip to main content

Training SpatialNet with accurate room acoustic simulations

This page supports our paper presented at Interspeech 2026, Improving multichannel speech enhancement through accurate room-acoustic simulations, available also on arXiv.

The paper shows that training data generated from physics-accurate room acoustic simulation, rather than simplified image-source models, measurably improves multichannel speech enhancement performance with SpatialNet on downstream ASR. The study trains three SpatialNet models with an identical architecture and training protocol, varying only the room acoustic simulation that augments their training data, and evaluates all of them on measured impulse responses. The model trained on high-fidelity hybrid simulations lowers the median word error rate (WER) by 30% on average, and by up to 38% under the most challenging speaker overlap condition, relative to uninformed image-source augmentation.

Median word error rate per overlap condition for SpatialNet models trained on the ISM-U, ISM-M, and Hybrid datasets

Median WER on LibriCSS-EM6 with bootstrapped 95% confidence intervals, across speaker overlap conditions.

The datasets behind the paper​

The study uses three training datasets, each containing about 4800 scenes with up to three temporally overlapping speakers. Two come from the open-source gpuRIR toolbox and one from the Treble SDK. Each dataset trains one SpatialNet-small model at a 16 kHz sampling rate on a six-channel subset of the em32 Eigenmike array. The training target is direct-path speech in all three cases.

Reverberation time distributions of the ISM-U, ISM-M, and Hybrid training datasets

Distribution of the reverberation time T20 across the three training datasets, averaged over octave bands from 63 Hz to 4000 Hz.

ISM-U: uninformed image-source simulation​

ISM-U follows the augmentation setup of the original SpatialNet paper, which doesn't involve the Treble SDK. It simulates impulse responses with gpuRIR, combining image-source simulation in shoebox rooms with a diffuse late reverberation tail that takes over after the first 15 dB of energy decay. Each wall carries a single frequency-independent absorption coefficient. ISM-U samples room dimensions and reverberation times uniformly from fixed intervals, so no prior knowledge about realistic materials or room volumes enters the dataset. It also models the Eigenmike as an open microphone array, which is common practice for gpuRIR augmentation.

ISM-M: matched image-source simulation​

ISM-M uses the same gpuRIR setup as ISM-U, but replaces the uniform sampling with parameters taken from the Hybrid dataset. Room dimensions come from the bounding boxes of the Hybrid room models, source and receiver positions match those of the Hybrid dataset, and each gpuRIR room reproduces the reverberation time of its Hybrid counterpart. The remaining differences to the Hybrid dataset are the scattering objects in the rooms, the simulation paradigm, and the source and receiver modeling.

Hybrid: high-fidelity room acoustics​

The Hybrid dataset comes from the Treble SDK, with all room models taken from the room database and all boundary-condition materials from the material library. It covers 324 furnished room models: 133 living rooms of 40 m³ to 180 m³, 103 classrooms of 90 m³ to 400 m³, and 88 restaurants of 300 m³ to 1600 m³. All materials are frequency-dependent and characterized by complex surface impedances, and they match the surfaces they sit on, such as glass for windows and gypsum or concrete for walls and ceilings.

Each room model holds four randomly placed sources, most of them directive with randomized orientations, and 20 to 30 receivers positioned at least 1 m from any source and 0.5 m from any surface. The wave-based acoustics (DG) solver covers the spectrum up to a crossover frequency between 1 kHz and 2 kHz, depending on room size, and the geometrical acoustics (GA) solver covers the range up to 12 kHz.

A full-wave free-field device-related transfer function (DRTF), simulated up to 12 kHz, represents the Eigenmike. Post-processing renders it onto the 16th-order Ambisonics impulse responses of the hybrid simulations, which accounts for the scattering of the rigid Eigenmike sphere that the open array model of the image-source datasets omits. See Setting up a recording device simulation for the corresponding workflow in the SDK.

Evaluation on measured data​

The study evaluates all three models on LibriCSS-EM6, a dataset of measured six-channel Eigenmike scenes introduced with the paper. Evaluating on measurements rather than on either simulation paradigm keeps the comparison from favoring the training data of any one model.

LibriCSS-EM6 follows the structure of LibriCSS, with 60 sessions and about 5000 utterances spread over six overlap conditions: 0S and 0L, which have no temporal overlap between talkers and separate them by short and long silences respectively, and OV10 to OV40, which vary the temporal overlap from 10% to 40%. The LibriCSS-EM6 setup convolves dry LibriSpeech utterances from the clean test set with measured Eigenmike impulse responses from the Motus and Arni6DoF datasets, then adds multichannel diffuse noise at a signal-to-noise ratio between 0 dB and 20 dB. A Kaldi pipeline, the same one used in the original LibriCSS evaluation, transcribes the enhanced signals, so the reported WER reflects downstream ASR performance rather than an objective signal metric.

Reproduce the results​

Code to reproduce the paper's results will be published in a public GitHub repository. It covers the generation of the ISM training datasets, the evaluation and analysis scripts, and the raw WER values together with the code that turns them into the figures.

info

The repository isn't public yet, and the link will be added here on publication. Until then, get in touch if you need access to the code.

Rerunning the training also requires the Hybrid dataset, which Treble sells rather than distributes publicly. Access to this dataset is arranged directly with our team: get in touch to purchase it.

The repository also includes the full data preparation and dataloader script that feeds the purchased IR collection into the training pipeline, so you can train on the Hybrid dataset as delivered.

Working with Treble datasets and IR collections​

You receive the Hybrid dataset as an IR collection, the same form every Treble dataset takes: a small metadata file describing each impulse response and the acoustic conditions that produced it, with the audio fetched from the cloud on demand. See Where datasets come from for background on how Treble datasets are built and sourced.

Once the collection is loaded, the resources below cover how to manage audio datasets and IR collections.

Audio datasets

Load, construct, and manage audio datasets for use with scene generation.

IR collections

Organize, filter, process, and render impulse responses and their metadata.

Audio scene generation: reproducible and scalable training datasets​

spatial_audio

The repository implements two ways of using the Hybrid dataset for training, so you can compare the approaches directly. The first mirrors the original SpatialNet code, where the dataloader assembles each scene during the training run by convolving impulse responses with speech and noise files on disk. The second uses the Treble SDK audio scene generator, which builds the scenes before training starts.

Generating the scenes up front lets you inspect them before a single epoch runs, reproduce them exactly across runs, and scale to bulk generation, with native support for rule-based conversational turn-taking, configurable signal-to-noise ratios, and device noise simulation. Each scene carries structured metadata, ready for ML training and evaluation pipelines.

See Audio scene generation for the scene model and the available workflows, and Scene collections for the SceneGenerator workflow that turns a set of rules and an IRCollection into a randomized SceneCollection.

Take a look at the Audio scene generation notebook for a complete worked example, or explore the full set of audio scene tutorials to get started.

Two further pages cover closely related material: