Moving auralization (beta)

This page covers how to render and auralize the results of a moving simulation. For how to define moving sources and receivers (trajectories, orientations, and the source/receiver objects themselves), see Moving sources and receivers. For the general post-processing workflow shared by all IR types, see Postprocessing.

info

This page describes the moving auralization workflow, which renders a single moving trajectory on its own. To render a trajectory together with all of the other components that make up a complete audio scene, use the audio scene workflow instead; see Audio scene generation(/treble-sdk/how-to/audio-scene-generation/).

Moving auralization are created from a MovingIR object using its convolve_with_audio_signal() method, which generates a multi-channel rendering of the moving source or receiver convolved with a dry input signal. This function allows the user to specify parameters such as the rendering device, receiver orientation, and movement speed along the trajectory.

For moving sources, the orientation of the static receiver must be specified during rendering. For moving receivers, the orientation along the trajectory must be defined instead. The orientation can be obtained from the associated Trajectory, if it was defined prior to simulation. Additionally, new orientation configurations can be defined at the rendering stage if needed.

Basic examples

Auralise a moving source

An example for moving sources is given as follows.

# Load device
hrtf_device = tsdk.device_library.get_device_by_name("my_hrtf")

# Load mono dry signal
dry_signal = treble.AudioSignal.from_file('this.wav')

# Define the receiver orientation
audio_convolved = moving_irs.convolve_with_audio_signal(
    audio=dry_signal,
    device=hrtf_device,
    speed=0.5,  # m/s
    receiver_orientation=treble.Rotation(-90, 0, 0),
)

Auralise a moving receiver

The following example shows the same pattern for moving receivers:

# Define the receiver orientation
rec_traj = moving_irs.get_trajectory()  # Get trajectory from simulation

orientation_lookahead = (
    rec_traj.look_ahead()
)  # Creates a new orientation looking forward

audio_convolved_lookahead = moving_irs.convolve_with_audio_signal(
    audio=dry_signal,
    device=hrtf_device,
    speed=0.5,  # m/s
    receiver_orientation=orientation_lookahead,
)

You can also define the orientation independently of the trajectory created during simulation definition.

orientation_custom = treble.TrajectoryOrientation(
    {
        0.0: Rotation(azimuth=225, elevation=0, roll=0),
        0.5: Rotation(azimuth=225 - 90, elevation=0, roll=0),
    }
)

audio_convolved_custom = mov_ir_rec.convolve_with_audio_signal(
    audio=dry_signal,
    device=hrtf_device,
    speed=0.5,  # m/s
    receiver_orientation=orientation_custom,
)

As with any convolution, the result is a ConvolvedAudioSignal, here a multi-channel, time-domain signal that reflects the dynamic acoustic behavior along the motion path. It supports the same playback, plotting, and manipulation methods as any other convolved signal (see Audio convolution in the Postprocessing guide):

# Result visualization and playback
audio_convolved.playback()
audio_convolved.plot()

Advanced examples

Customizing movement speed

By default, a moving source or listener travels along the defined path at a constant speed. The movement speed can be adjusted to match the desired scenario, such as a walking person, or any custom motion profile.

When using a moving source or listener, the movement speed can be automatically estimated to match the duration of the input audio signal. This is useful if goal is to traverse the entire trajectory over the course of the audio playback.

speed_to_match_audio = moving_irs.get_speed_to_match_audio(dry_signal)

convolved = moving_irs.convolve_with_audio_signal(
    audio=dry_signal,
    speed=speed_to_match_audio,
    device=hrtf_device,
    receiver_orientation=orientation_custom,
)

The get_speed_to_match_audio() method computes the speed required to complete the trajectory within the duration of the provided audio signal. The returned value can be passed directly to convolve_with_audio_signal() or used as a starting point for further customization.

If more control over the motion is required, the movement can also be defined using a custom speed profile through DynamicPosition, allowing the speed to vary throughout the trajectory over time, enabling acceleration, deceleration, pauses, or any other non-uniform motion. A DynamicPosition can therefore be used whenever a constant speed is not sufficient to accurately represent the desired movement.

There are two transition types available to define the motion profile:

• PositionTransition: position as a function of time. Use this to specify where along the trajectory the source or receiver should be at a given time, interpolating between the start and end times of the transition.
• VelocityTransition: velocity as a function of time. Use this to set the movement speed at a given point in time, enabling acceleration, deceleration, or pauses.

from treble_tsdk.scene import (
    DynamicPosition,
    PositionTransition,
    VelocityTransition,
)

dynamic_speed_pos = DynamicPosition(
    initial_position=0.0,
    duration_s=audio_duration_s,
    frame_rate=100.0,
)

dynamic_speed_pos.add_transition(PositionTransition(target_position=2.0, start_time=0.0, end_time=2.0))
dynamic_speed_pos.add_velocity_transition(VelocityTransition(velocity=3, time=3.0))
dynamic_speed_pos.add_velocity_transition(VelocityTransition(velocity=0.5, time=5.0))
dynamic_speed_pos.add_transition(PositionTransition(target_position=15.0, start_time=6.0, end_time=10.0))
dynamic_speed_pos.plot()

convolved = moving_irs.convolve_with_audio_signal(
    audio=dry_signal,
    speed=dynamic_speed_pos,
    device=hrtf_device,
    receiver_orientation=orientation_lookahead,
)

Controlling orientations on a moving receiver

Unlike moving sources, a moving spatial receiver doesn't require an orientation at creation time. Instead, orientation is typically configured during post-processing, immediately before rendering. This separation allows the same simulated receiver trajectory to be rendered multiple times with different orientations without rerunning the simulation.

There are two ways to define the orientation of a moving spatial receiver:

• TrajectoryOrientation: orientation as a function of position along the trajectory. Used when the listener should face a direction determined by where they are on the path (for example look_ahead(), look_at(), a fixed orientation, or keyframes keyed by trajectory percentage). See Trajectory orientation for how to construct these.
• DynamicOrientation: orientation as a function of time over the duration of the audio signal. Use this when the listener’s head pose should follow an explicit time schedule (for example timed turns, look-at transitions, or jitter), independent of the position in space.

In the moving auralization workflow, both forms can be passed to convolve_with_audio_signal() as receiver_orientation. Scene generation currently expects a time-based DynamicOrientation.

The following example retrieves the trajectory from a completed simulation result and renders it twice with different orientations using the TrajectoryOrientation:

sim = project.get_simulation_by_name('Simulation')
res = sim.get_results_object()

moving_irs = res.get_moving_ir(
    source=sim.sources[0],
    receiver=sim.receivers[0]
)

rec_traj = moving_irs.get_trajectory()

orientation_lookahead = rec_traj.look_ahead()
orientation_custom = treble.TrajectoryOrientation({
    0.0: Rotation(azimuth=225, elevation=0, roll=0),
    0.5: Rotation(azimuth=135, elevation=0, roll=0),
})

# Render the same trajectory with two different orientations
audio_lookahead = moving_irs.convolve_with_audio_signal(
    audio=dry_signal,
    device=hrtf_device,
    speed=0.5,
    receiver_orientation=orientation_lookahead
)
audio_custom = moving_irs.convolve_with_audio_signal(
    audio=dry_signal,
    device=hrtf_device,
    speed=0.5,
    receiver_orientation=orientation_custom
)

When using DynamicOrientation:

from treble_tsdk.scene import DynamicOrientation, OrientationTransition
from treble_tsdk.utility_classes import Rotation

audio_duration_s = dry_signal.duration

orientation_dynamic = DynamicOrientation(
    initial_orientation=Rotation(azimuth=0, elevation=0, roll=0),
    duration_s=audio_duration_s,
    frame_rate=100.0,
)

# Turn to face left between 1 s and 2 s, then turn again between 4 s and 5 s
orientation_dynamic.add_transition(
    OrientationTransition(
        target_orientation=Rotation(azimuth=90, elevation=0, roll=0),
        start_time=1.0,
        end_time=2.0,
    )
)
orientation_dynamic.add_transition(
    OrientationTransition(
        target_orientation=Rotation(azimuth=-45, elevation=0, roll=0),
        start_time=4.0,
        end_time=5.0,
    )
)

audio_dynamic = moving_irs.convolve_with_audio_signal(
    audio=dry_signal,
    device=hrtf_device,
    speed=0.5,
    receiver_orientation=orientation_dynamic,
)

Use DynamicOrientation.from_trajectory_orientation() to build a time-based orientation schedule from a TrajectoryOrientation. This is required for scene generation, and useful when combining a position-based orientation with a custom DynamicPosition speed profile. The conversion samples orientation along the path using the same motion model as auralization (constant speed in m/s, or a DynamicPosition) and returns a baked DynamicOrientation whose duration matches the signal.

from treble_tsdk.scene import DynamicOrientation

orientation_lookahead = rec_traj.look_ahead()

dynamic_orientation = DynamicOrientation.from_trajectory_orientation(
    orientation=orientation_lookahead,
    trajectory=rec_traj,
    position_schedule=dynamic_speed_pos,  # or a constant speed in m/s
    duration_s=audio_duration_s,
    frame_rate=100.0,
)

convolved = moving_irs.convolve_with_audio_signal(
    audio=dry_signal,
    speed=dynamic_speed_pos,
    device=hrtf_device,
    receiver_orientation=dynamic_orientation,
)

info

For more information on working with the results of a moving simulation, see the IR Containers and Postprocessing guides.