Skip to main content

Collections Library

The CollectionsLibrary is the SDK's high-level interface for bulk simulation and impulse-response workflows with a DataFrame-first approach. Access it via tsdk.collections. It provides:

  • creation of SimulationCollection and IRCollection
  • reconstruction from DataFrames and files
  • an IRDataLoader for deferred IR binary access
  • SceneCollection reconstruction from DataFrames
Key benefits with collections

Getting started​

The documentation below serves as general-purpose documentation. There is also an end-to-end workflow example using the Treble10 dataset.

Create a simulation collection​

p = tsdk.get_or_create_project(
    name="<Your project name>"
)  # reuse existing project or create new project

sims = p.get_simulations(simulation_status=treble.SimulationStatus.completed)
# Report numbers
print("Simulations ", len(sims))
ir_count = sum(len(sim.receivers) * len(sim.sources) for sim in sims)
print("irs ", ir_count)

# add the simulations
coll = tsdk.collections.simulation_collection()
coll.add_simulations(sims)

# Add entire projects to the collection
coll.add_projects([p2, p3, p4])  # or add more sims coll.add_simulations([...])

Adding metadata to the simulation collection​

Metadata can be added to either the simulation collection or the IR collection, depending on what makes sense. For creating train/test splits an option would be do define this on a simulation level.

import hashlib

def split_fn(row):
    # pick a stable identifier attribute from the object
    key = (getattr(row, "id", None))

    h = int(hashlib.md5(str(key).encode("utf-8")).hexdigest(), 16)
    return "A" if (h % 100) < 80 else "B"

coll.add_column("split", split_fn)

Create an IR collection from the simulation collection​

It requires a single line of code to convert a simulation collection in to an IR collection.

ir_coll = coll.get_ir_collection()
# Print the first few lines from the collection
ir_coll.head()

Create an IR collection that inherits custom metadata​

If custom metadata has been added to the simulation collection which should propagate to the IR collection. It is possible to make the IR collection inherit the metadata by using the inherit_columns argument.

ir_coll = coll.get_ir_collection(inherit_columns=[{"split": "split"}])

Append metadata to IR collection​

Spatial and acoustic metadata can be appended to the collection with built-in functions.

The current built-in support for acoustic parameters can be viewed with treble.AcousticParameters.supported_parameters()

It is then straight forward to do the enrichment on the collection.

ir_coll.enrich_with_acoustic_parameters(
    acoustic_parameters=["t20","t30", "c50", "edt"]
)

Spatial metadata can be added with built-in functionalities that extracts information from the underlaying models. One example is to add information regarding line of sight between sources and receivers. Spatial metadata can be added with built-in functionalities that extracts information from the underlying models. One example is to add information regarding line of sight between sources and receivers.

def has_line_of_sight(row) -> bool:
    model = row.simulation.get_model()
    return model.has_line_of_sight(row.source , row.receiver)

ir_coll.add_column(
    "source_receiver_line_of_sight",
    has_line_of_sight,
)

This can also be done with a lambda

ir_coll.add_column(
    "source_receiver_line_of_sight",
    lambda x: x.simulation.get_model().has_line_of_sight(
        x.source, x.receiver
    ),
)

More examples can be found under the spatial metadata section.

Visualization possibilities​

visualize the metadata from the collection​

ir_coll.plot()

distribution-IR-col

Plot a simulation​

The underlying simulation can be visualized

ir_coll[0].simulation.plot()

sim-def-IR-col

Querying / Filtering on a collection​

Define a certain distribution​

# Gaussian distribution of a specific T20 target
parameter = "t20"
t20_target = 0.4
n_samples_gauss = 100000
target_std = 0.1

gaussian = treble.distributions.Gaussian(mean=t20_target, std=target_std)
ir_t20gauss = ir_coll.sample_with_distribution(
    column=parameter, n_samples=n_samples_gauss, distribution=gaussian
)


# Uniform distribution of a specific C50 target
parameter = "c50"
n_samples_uniform = 25000
uniform = treble.distributions.Uniform(low=2, high=20)
ir_subset_c50uniform = ir_coll.sample_with_distribution(
    column=parameter, n_samples=n_samples_uniform, distribution=uniform
)

Simple Filter​

import polars as pl
close_talkers = ir_coll.filter(pl.col("source_receiver_dist") < 2.0)

Saving a collection​

In the SDK​

saved_dataset = tsdk.datasets.save_dataset(
    ir_collection=ir_coll,
    name="My_Enriched_Dataset",
    description="Dataset enriched with metadata"
)

As a parquet file​

# Lets save our collection to disk.
ir_coll.write_parquet(work_dir / 'ir_coll.parquet')

Reconstruct from DataFrame​

sim_df = coll.dataframe
sim_coll2 = tsdk.collections.simulation_collection_from_df(sim_df)
ir_df = ir_coll.dataframe
ir_coll2 = tsdk.collections.ir_collection_from_df(ir_df)

Reconstruct directly from file​

sim_coll = tsdk.collections.simulation_collection_from_file("simulations.parquet")
ir_coll = tsdk.collections.ir_collection_from_file("irs.parquet")

Supported file formats:

  • .parquet
  • .csv
  • .ndjson
  • .json
  • .ipc
  • .arrow

Create an IR data loader​

from treble_tsdk.results.ir_data_loader import IRDataLoaderBehaviour
dl = tsdk.collections.ir_data_loader(
    work_dir="./tmp_ir_cache",
    behaviour=IRDataLoaderBehaviour.lazy,
)