Spatial Audio
The treble SDK is capable of generating high precision and very high order ambisonic impulse responses. Whether creating physically accurate binaural room impulse responses or simulating reverberation time or other acoustic parameters for complex scenario in gaming, our sdk excels at spatial audio.
Quick guide
This quick guide shows how to create an spatial audio auralization using the treble SDK.
The guide does the following steps
- Select a meeting room from the Geometry database in the SDK
- Setup a simulation with 2 talkers and 1 listening position with very high order Ambisonics
- Assign materials to model and run a simulation
- Import a HRTF to device library
- Render BRIR from the HRTF
- Auralize scene
from pathlib import Path
from time import time
# Our working directory for tsdk files.
# Defaults to your home directory /tmp/tsdk, feel free to change it to what you like.
base_dir = Path.home() / "tmp" / "tsdk"
base_dir.mkdir(parents=True, exist_ok=True)
from treble_tsdk.tsdk import TSDK
from treble_tsdk import tsdk_namespace as treble
from treble_tsdk import display_data as dd
from treble_tsdk.postprocessing import DRTF
from scipy.io import wavfile
from scipy import signal
import sounddevice as sd
import numpy as np
tsdk = TSDK()
project_name = "Spatial Audio Tutorial"
project = tsdk.get_or_create_project(
name=project_name,
description="Tutorial demonstrating the usage of spatial audio with Treble SDK",
)
%env TSDK_DASH_PLOTTING=1
Geometry
Let's fetch a Living room from the geomery database
# Lets fetch the meeting room dataset.
rooms = tsdk.geometry_library.get_dataset(dataset="MeetingRoom")
room = rooms[15]
Now we can detailed geometry info and metadata that comes with this curated model database. Let's view position suggestions, as well as list of material assignment layers.
# View detailed geometry info, this shows position suggestions, as well as list of material assignment layers.
dd.as_tree(room)
The model can be visualized, where it is possible to see the different layers and curated positions
# Plotting geometry
room.plot()
Next add listening position and two speech sources using the suggested postions in the dataset
talker1 = room.position_suggestions[8]
talker2 = room.position_suggestions[11]
listening = room.position_suggestions[0]
print(f"Talker1 is {talker1.name}")
print(f"Talker2 is {talker2.name}")
print(f"Listening position is {listening.name}")
Sources
A library of sources and their directivity is major feature of SDK. Here we get the directivity for a speech source
speech_directivity = tsdk.source_directivity_library.query(name="Speech")[0]
Let's view the directivity balloon
speech_directivity.plot_directivity_pattern()
source_properties = treble.SourceProperties(
azimuth_angle=0.0,
elevation_angle=0.0,
source_directivity=speech_directivity
)
source_talker1 = treble.Source(
x=talker1.position[0],
y=talker1.position[1],
z=talker1.position[2],
source_type=treble.SourceType.directive,
label="Speech_source_1",
source_properties=source_properties)
source_properties = treble.SourceProperties(
azimuth_angle=45.0,
elevation_angle=0.0,
source_directivity=speech_directivity
)
source_talker2 = treble.Source(
x=talker2.position[0],
y=talker2.position[1],
z=talker2.position[2],
source_type=treble.SourceType.directive,
label="Speech_source_2",
source_properties=source_properties)
source_list = [source_talker1,source_talker2]
Receiver
Let's add a single listening position to the scene.
receiver_properties = treble.ReceiverProperties(
azimuth_angle=180.0,
ambisonics_order=16
)
receiver = treble.Receiver(
x=listening.position[0],
y=listening.position[1],
z=listening.position[2],
label="listening_position",
receiver_type=treble.ReceiverType.spatial
)
Helper function to plot sabine estimate
import matplotlib.pyplot as plt
def plot_sabine_estimate(sabine_estimate):
frequencies = [63, 125, 250, 500, 1000, 2000, 4000, 8000]
# Create the plot
plt.figure(figsize=(8, 6))
plt.plot(range(len(frequencies)), sabine_estimate, marker="o", linestyle="-")
# Set labels and title
plt.xlabel("Frequencies")
plt.ylabel("RT")
plt.title("Sabine estimate")
# Show x-axis labels at specified frequencies
plt.xticks(range(len(frequencies)), frequencies)
# Display the plot
plt.grid(True)
plt.show()
Material assignment
Let's do a semi random acoustic design of the space. Below we look at the layers in the space and assign a random material that matches the name of the layer. The resulting sabine estimation is plotted in octavebands. Re run this part to get another itaration of the room.
room.layer_names
import random
material_assignment = []
mat_dict = {}
for layer in room.layer_names:
if layer == "Monitor":
mat_list = tsdk.material_library.search("window")
elif layer == "Table":
mat_list = tsdk.material_library.search("wood")
else:
mat_list = tsdk.material_library.search(layer.split()[0])
material_assignment.append(treble.MaterialAssignment(layer,random.choice(mat_list)))
dd.as_table(material_assignment)
sabine_estimate = room.calculate_sabine_estimate(material_assignment)
plot_sabine_estimate(sabine_estimate)
Running a simulation
Here we combine the sources, receivers, model and materil assignment to a simulation definition and add that to the project.
sim_def = treble.SimulationDefinition(
name="spatial_audio_scene",
model=room,
material_assignment=material_assignment,
source_list=source_list,
receiver_list=[receiver],
energy_decay_threshold=35,
crossover_frequency=720,
simulation_type=treble.SimulationType.hybrid
)
Add the simulation to the project and wait for an estimate of the computational resources required
simulation = project.add_simulation(definition=sim_def)
simulation.wait_for_estimate()
# Display the cost estimate for the simulation
dd.as_tree(simulation.estimate())
Let's start the simulation in the cloud
project.start_simulations()
Let's view the live progress
project.as_live_progress()
Setting up an Auralization
Add HRTF from sofa file
The section shows how to import an HRTF into your device library
# Path to .sofa file.
sofa_file = "data/D1_44K_16bit_256tap_FIR_SOFA.sofa"
device_name = f"HRTF_import_{int(time())}"
device_definition = DRTF.create_device_from_sofa_file(
sofa_filepath=sofa_file,
device_name=device_name,
max_ambisonics_order=16
)
device = tsdk.device_library.add_device(device_definition=device_definition)
Or fetch a device/HRTF by name from the device libray
device_name = "HRTF_import"
hrtf_device = tsdk.device_library.get_device_by_name(device_name)
Let's fetch the results
simulation = project.get_simulations()[-1]
results_dir = base_dir / "DeviceDemo"
results = simulation.download_results(destination_directory=results_dir)
Render the BRIR from the spatial IR and HRTF
spatial_ir_talker1 = results.get_spatial_ir(source=simulation.sources[0],receiver=simulation.receivers[0])
# Now we use the spatial IR instance and render it with the HRTF we just uploaded from the .sofa file.
brir_talker1 = spatial_ir_talker1.render_device_ir(device=hrtf_device, azimuth=0.0, elevation=0.0)
spatial_ir_talker2 = results.get_spatial_ir(source=simulation.sources[1],receiver=simulation.receivers[0])
brir_talker2 = spatial_ir_talker2.render_device_ir(device=hrtf_device, azimuth=0.0, elevation=0.0)
BRIR = [brir_talker1, brir_talker2]
Load anechoic audio files
Gain1 = 22
Gain2 = 30 # dB gain
Gain = [Gain1, Gain2]
# Change here if you want to try different signal
wav1 = "data/P1_Edited_shorterversion_HP_32k.wav"
wav2 = "data/P2_Edited_shorterversion_HP_32k.wav"
fs, S1 = wavfile.read(wav1)
fs, S2 = wavfile.read(wav2)
S1 = S1.astype(np.float32) / ((1.0) * (2**15 - 1.0)) # normalize, int16 to float
S2 = S2.astype(np.float32) / ((1.0) * (2**32 - 1.0)) # normalize int32 to float
S1 = np.reshape(S1, [S1.shape[0]]) # Flatten out to 1D
S2= np.reshape(S2, [S2.shape[0]])
length = np.max([S1.shape, S2.shape])
source = np.zeros([length, 2])
source[:length,0] = S1
source[:length,1] = S2
Create output and play
Let's try it out. genereate the scene by convolving the source signals with the BRIR for the speech source and device receiver. Play directly from python, mind the level!
out = np.zeros([length, 2])
for s in range(2):
IR = BRIR[s]
for i in range(2):
out[:, i] += signal.convolve(source[:,s], IR.data[i,:], mode = 'same', method = 'auto') * np.power(10,Gain[s]/20)
sd.play(out, fs)
sd.stop()
Generate wav file of the Receiver scenarios
Generates a wave file with the auralization.
wavfile.write(f"{base_dir}/auralization.wav", fs, out)