Skip to main content

Audio scenes

An AudioScene combines audio tracks, a set of room impulse responses (IRs), and a listener configuration into a single serializable recipe. It represents one capture scenario: audio sources placed at specific positions in a room, recorded at a single receiver using a defined device. No audio is rendered until you call render(), so a scene can be built, inspected, and stored before any computation runs. Once assembled, it can be rendered individually or added to a SceneCollection for batch processing.

The examples on this page show how to build a scene manually. For automated, large-scale scene generation, see the Scene collections how-to.

IR collection​

IR collection loading and filtering​

A scene is built around a single receiver position. Start by loading an IRCollection, then filter it to the receiver you want, leaving one IR per source position in the room.

import polars as pl
from treble_tsdk import treble
import treble_tsdk.scene as sc

tsdk = treble.TSDK()
ir_collection = tsdk.datasets.purchase("Treble10")

# Select a single receiver.
# The resulting IR collection will contain IRs for just one receiver and one simulation.
rec = ir_collection[68].receiver
ir_collection = ir_collection.filter_collection(pl.col("receiver_id") == rec.id)

# Caches IR data to disk so re-running this script reads from disk instead of re-fetching
# from the network every time. Always attach one — see the note below.
data_loader = ir_collection.get_data_loader(work_dir="./cache")

# Only needed for SNR prediction (see "SNR prediction for a single scene" below) — skip
# this if you don't plan to call predict_snr() on the scene.
ir_collection.enrich_with_acoustic_parameters(acoustic_parameters=["spl"])
info

Set a data_loader on every AudioScene you build. IR data can be large, and a data loader caches it to disk on first use. Without one, AudioScene.render() still works, but it logs a warning and re-fetches the IR data remotely on every call — a real cost for anything beyond a single throwaway scene.

Custom metadata columns​

Add metadata columns to the IR collection to simplify source selection. The example below flags elevated sources as noise candidates:

from treble_tsdk.collections.ir_info import IRInfo

# Sources above 1.9 m are treated as noise emitters.
def is_noise_source(ir_info: IRInfo):
    return ir_info.source.z > 1.9

# The new column is attached to the IR collection dataframe and can be in source_selection later.
ir_collection.add_column("IS_NOISE_SOURCE", is_noise_source)

add_column() isn't only for filtering columns like IS_NOISE_SOURCE above — any column added this way also carries through, per track, to a SceneCollection generated later from this IRCollection, since TrackMap.ir.dataframe_row copies the IR collection row verbatim at the time each scene is built. Enrich the IR collection before generating or adding scenes, since a column added afterward won't retroactively appear. See Reading per-IR metadata across a generated collection for reading a value like this back once scenes exist.

Audio datasets​

Dataset loading​

Treble supports loading audio datasets from Hugging Face and from local directories of WAV files. See the Audio datasets how-to for the full set of built-in audio loaders and dataset-loading patterns.

The example below loads three datasets: a speech dataset for foreground tracks, a local music dataset, and a background noise dataset.

from pathlib import Path

# Pre-sliced LibriSpeech — see "Dataset from Hugging Face" in the Audio datasets how-to for why
# this repository is preferred over the raw corpus.
speech_dataset = sc.AudioDataset.from_huggingface(
    repo_id="treble-technologies/librispeech_asr_sliced",
    split="test",
    audio_loader_class=sc.LibriSpeechAudioLoader,
    schema_mapping={"transcript": "text"},
    drop_columns=["audio"],
)

# Change "my_music_wav_dir" below to the path where your local music WAV files are stored.
music_dataset = sc.AudioDataset.from_local_wav_directory(
    wav_dir=Path("my_music_wav_dir")
)

# AudioSet loaded from Hugging Face.
hvac_dataset = sc.AudioDataset.from_huggingface(
    repo_id="agkphysics/AudioSet",
    config="full",
    split="bal_train",
    audio_loader_class=sc.AudioSetAudioLoader,
    schema_mapping={"id": "video_id"},
    drop_columns=["audio"],
    max_parquet_files=1,
)
hvac_dataset = hvac_dataset.filter_collection(
    pl.col("human_labels").list.join(" ").str.to_lowercase().str.contains(r"air conditioning|air conditioner")
)

dog_bark_dataset = treble.scene.AudioDataset.from_huggingface(
    repo_id="437aewuh/dog-dataset",
    config="default",
    split="train",
    audio_loader_class=treble.scene.RowIndexAudioLoader,
)

# Only needed for SNR prediction (see "SNR prediction for a single scene" below)
# This is a rather slow operation, so skip it if you don't use predict_snr().
speech_dataset.enrich_with_spl()
music_dataset.enrich_with_spl()
hvac_dataset.enrich_with_spl()
dog_bark_dataset.enrich_with_spl()

Track generation​

TrackGenerator produces audio tracks from an AudioDataset according to a set of rules. The rules determine the track type and generation behavior:

  • ConversationRules — produces AudioTrack objects with timed AudioBlock references, modeling turn-taking, overlap, and per-block level variation across multiple talkers.
  • NoiseSourceRules — produces RepeatedAudioTrack objects that loop a single audio sample at a fixed free-field level to a defined duration. Use this for continuous background sources such as HVAC or music.
  • TransientNoiseRules — produces AudioTrack objects with one sample scattered at one or more non-overlapping onset times within the duration. Use this for sparse, discrete occurrences such as a door slam or a car horn, rather than a continuous stream.
  • StaticNoiseRules — produces StaticNoiseTrack objects containing synthetically generated frequency-shaped noise. No audio file is required.

TrackGenerator requires an audio_dataset for every rules type except StaticNoiseRules, which needs none. talker_identifier is only used by ConversationRules — it names the column in audio_dataset that groups rows into talkers/identities; the other three rules types ignore it.

The examples below use ConversationRules and TransientNoiseRules. For NoiseSourceRules in the context of automated scene generation, see the Scene collection how-to.

Conversation track generation with explicit rules​

ConversationRules controls conversational timing — turn-taking, overlap, and per-block levels. The result is a set of AudioTrack objects, each containing timed AudioBlock references. Pass a seed for reproducible output.

duration_s = 20

conversation_rules = sc.ConversationRules(
    block_duration_range=(1.5, 5.0),      # duration range for candidate audio blocks in s
    overlap_range=(0.2, 0.5),             # Fractional overlap between consecutive blocks, in [0, 1]
    overlap_probability=0.5,              # probability that consecutive blocks overlap at all
    in_track_level_range_db_spl=(60, 65), # Free-field level range for audio blocks within the same talker in dB SPL
    max_simultaneous_blocks=2,            # polyphony cap across all talkers; defaults to n_tracks if omitted
    samples_per_talker=sc.SamplesPerTalker.reshuffle,  # behavior once a talker's samples run out
)

track_generator = sc.TrackGenerator(
    audio_dataset=speech_dataset,
    rules=conversation_rules,
    talker_identifier="speaker_id",
)
tracks = track_generator.generate_tracks(
    n_tracks=2,
    duration_s=duration_s,
    seed=21,  # fix seed for reproducible track layouts
)

samples_per_talker (a SamplesPerTalker) controls what happens once a talker's own candidate samples are exhausted mid-generation: force_unique uses each sample at most once and raises when they run out, reshuffle (the default) re-shuffles and continues, recycle reuses the same initial shuffled order, and random samples uniformly with replacement for every block.

Call plot_audio_tracks() to inspect the generated timelines before assembling the scene:

sc.plot_audio_tracks(tracks, duration_s)

Audio tracks plot

Conversation tracks with a preset​

For common conversation patterns, use a predefined preset via ConversationRules.from_preset() rather than specifying all parameters manually:

conversation_rules = sc.ConversationRules.from_preset(
    sc.ConversationRulesPresets.sequential_talkers_no_overlap
)

Transient (one-shot) track generation​

Use TransientNoiseRules for sparse, discrete sound events — a dog bark, a door slam, a car horn — rather than a continuous background. Each generated track gets one sample, placed at one or more randomly chosen onset times at least min_gap_s apart:

dog_bark_rules = sc.TransientNoiseRules(
    free_field_level_db_spl=(60, 65),
    n_events_range=(2, 4),  # how many times the sample occurs in the scene
    min_gap_s=1.0,          # minimum silence enforced between placements
)

dog_bark_generator = sc.TrackGenerator(
    audio_dataset=dog_bark_dataset,
    rules=dog_bark_rules,
)
dog_bark_tracks = dog_bark_generator.generate_tracks(n_tracks=1, duration_s=duration_s, seed=7)

Each resulting track is an AudioTrack whose audio_blocks give the exact start_time_s/ end_time_s of every placed occurrence — see Track and IR metadata for post-analysis below for reading these back.

Track and source assignment​

Track-to-source assignment​

Each track is paired with an IR from the collection through a TrackMap, which binds the track to a spatial position in the room. GroupTag categorizes tracks as TARGET, JAMMER, DISTRACTOR, NOISE, or BACKGROUND. Use group_name to group related tracks in the visualization.

# Add two conversation (speech) tracks to the scene, each mapped to a different source position in the room.
track_maps = [
    sc.TrackMap(
        track=tracks[0],
        ir=ir_collection[0],
        group_name="conversation",
        tag=sc.GroupTag.TARGET,
    ),
    sc.TrackMap(
        track=tracks[1],
        ir=ir_collection[2],
        group_name="conversation",
        tag=sc.GroupTag.TARGET,
    ),
]

# Add a transient track to the scene.
track_maps.append(
    sc.TrackMap(
        track=dog_bark_tracks[0],
        ir=ir_collection[4],
        group_name="dog",
        tag=sc.GroupTag.BACKGROUND,
    )
)

Looped background track​

Use RepeatedAudioTrack for sources that should play continuously throughout the scene. The sample is looped to fill the required duration at the specified free-field level.

This is the manual way to build a looped background track — constructing it directly and pairing it with an IR by hand, as shown below. For automated, rule-driven generation of the same track type at scale, use TrackGenerator with NoiseSourceRules instead — see Source groups.

track_maps.append(
    sc.TrackMap(
        track=sc.RepeatedAudioTrack(hvac_dataset[0], free_field_level_db_spl=55, start_time_s=2, duration_s=17),
        ir=ir_collection[3],
        group_name="hvac",
        tag=sc.GroupTag.BACKGROUND,
    )
)

Filtering a track's source content​

Pass filter_definitions to TrackGenerator to shape a group's dry audio before it reaches the room IR — for example, band-limiting a noise source to the range a real emitter would produce. This applies to that TrackGenerator's own tracks only, before any convolution. See Postprocessing for the full set of available FilterDefinition subclasses.

hvac_generator = sc.TrackGenerator(
    audio_dataset=hvac_dataset,
    rules=sc.NoiseSourceRules(free_field_level_db_spl=(55, 56)),
    filter_definitions=[treble.ButterworthFilter(lp_order=2, lp_frequency=2000)],
)
info

This is a source-side filter, scoped to one group's content. A receiver-side filter that shapes what every microphone hears, regardless of source, is set separately via DeviceSpecs.filter_definitions — see Listener rules.

Listener configuration​

Scene listener configuration​

SceneListener defines how the scene is captured at the receiver. It combines a rendering device, a device orientation, optional per-channel noise floors, and optional output filters.

device = tsdk.device_library.get_device_by_name("KEMAR051123_1")

# One StaticNoiseTrack per device channel — models self-noise from each microphone element.
device_noise = [
    sc.StaticNoiseTrack(sc.StaticNoiseProfile.pink_noise(), level_db_spl=40),
    sc.StaticNoiseTrack(sc.StaticNoiseProfile.pink_noise(), level_db_spl=41),
]

listener = sc.SceneListener(
    receiver=rec,
    device=device,
    orientation=treble.Rotation(azimuth=180),  # rotate device 180° around the vertical axis
    noise_definitions=device_noise,
)

With no device set, the receiver renders as a single mono channel — noise_definitions and filter_definitions still apply to that one channel, so a plain mono scene can still carry microphone self-noise or a frequency-response filter without modeling a full device array.

Scene assembly​

Scene assembly and inspection​

An AudioScene brings together the track-to-source mappings and the listener definition. Call plot() to display the track timeline and a 3D room view for quick inspection.

my_scene = sc.AudioScene(
    track_map=track_maps,
    scene_listener=listener,
    duration_s=duration_s,
    data_loader=data_loader,
)
my_scene.plot()

Scene plot

info

At this point my_scene is a serializable recipe — no audio waveforms have been computed yet.

Scene rendering​

Single scene rendering​

Call render() on an AudioScene to produce the mixed audio signal. Set output_separated_tracks=True to also receive per-track rendered outputs.

mixed = my_scene.render(
    sampling_rate=32000,
    render_mode=sc.RenderMode.DEVICE,
)

mixed.plot()
mixed.playback()

Target signal extraction​

Call render_target() to produce the clean target signal. Pass a track index, a list of track indices, or a source group name as target_selection. When a group name or index list is given, each matching track is rendered individually and the results are summed.

Use get_track_indices_by_tag() or get_track_indices_by_group_name() to look up track indices by tag or group name.

# Resolve the first TARGET-tagged track index before passing it to render_target().
target_idx = my_scene.get_track_indices_by_tag(sc.GroupTag.TARGET)[0]

target = my_scene.render_target(
    target_selection=target_idx,
    sampling_rate=32000,
    render_mode=sc.TargetRenderMode.WET_MONO,  # room-convolved, mono output
)

Metadata and transcripts​

Export scene metadata — source positions, receiver properties, device info, track timing, and levels — as a JSON-serializable dictionary with to_struct():

scene_metadata = my_scene.to_struct()

To extract transcripts from speech tracks, use transcript(). Pass track_index=None to retrieve the full scene transcript, or a specific index to retrieve the transcript for one track:

full_transcript   = my_scene.transcript(track_index=None)
target_transcript = my_scene.transcript(track_index=0)

Only ConversationRules-generated blocks contribute a transcript entry; RepeatedAudioTrack and StaticNoiseTrack tracks have none and are silently skipped, even if included in track_index. Each returned line has the form "{start_time_s}s: Track: {track.id}: {transcript}", sorted by start time across every included track.

Track and IR metadata for post-analysis​

Every TrackMap in my_scene.track_map carries what was actually placed in the scene, so you can read positions, timings, and levels back directly instead of inferring them from the rendered audio.

track_map.ir (None for a dry track with no spatial position) exposes:

  • .source/.receiver — each with .label, .location (a Point3d, or the bare .x/.y/.z), and .tags.
  • .simulation — .id, .name, and .get_model() for the room model.
  • .dataframe_row — the full IR collection row for this IR as it stood when the scene was built, including any custom columns added with add_column().

track_map.track fields depend on the concrete type — check with isinstance, since the three types don't share the same attributes:

for tm in my_scene.track_map:
    if isinstance(tm.track, sc.AudioTrack):
        occurrences = [(b.start_time_s, b.end_time_s, b.free_field_level_db_spl) for b in tm.track.audio_blocks]
    elif isinstance(tm.track, sc.RepeatedAudioTrack):
        occurrences = [(tm.track.start_time_s, tm.track.start_time_s + tm.track.duration_s, tm.track.free_field_level_db_spl)]
    else:  # StaticNoiseTrack — exactly one of level_db_spl/mic_snr_db is set
        level = tm.track.level_db_spl if tm.track.level_db_spl is not None else tm.track.mic_snr_db
        occurrences = [(tm.track.start_time_s, tm.track.start_time_s + tm.track.duration_s, level)]

Use these methods to look up tracks by tag, group, or type instead of filtering track_map by hand:

my_scene.get_track_indices_by_tag(sc.GroupTag.BACKGROUND)     # -> list[int]
my_scene.get_track_indices_by_group_name("hvac")              # -> list[int]
my_scene.get_source_group_names()                             # -> list[str]
my_scene.track_info                                           # -> list[TrackSummary(index, tag, group_name, track_type)]