Following on from creating immersive and interactive multi-device audio dramas and music experiences and making a production tool so others could experiment with these ideas, .
Placing your friends or family’s phones around the room to share the cockpit with a fighter pilot receiving mysterious radio transmissions, walking between sections of an orchestra in your living room, or joining an immersive album release listening party — these are just some of the experiences created with our toolkit. We initially developed a web-based player to make our experimental production, The Vostok-K Incident, available to audiences at home in 2018. Since then, we’ve made a production tool (Audio Orchestrator, available on MakerBox) and worked with producers to create and publish several pilots on ±«Óãtv Taster (such as Pick A Part and Seeking New Gods).
Using Audio Orchestrator, you can already prototype very flexible multi-device audio experiences without writing any code:
- import your audio tracks;
- add behaviours for how each object should be assigned to the available devices in different conditions; and
- monitor and preview the experience.
With the latest version, even beginner coders can now modify and use their own copy of our open-source template when exporting or previewing their experience.
When designing our pilot productions, we always ended up making some changes to the application exported from Audio Orchestrator. In some cases, we eventually added some of these features to the tool (like calibration mode or displaying timed images) because they were quite generally useful.
However, for Pick-A-Part, we needed to display different images depending on the chosen instrument and grey out options already selected on another device. Likewise, for the Seeking New Gods listening parties, we changed the setup flow to add a countdown stage before letting listeners start the experience. This ensured that everyone would be listening to the same track at the same time even though each group had their own session. These features were quite specific and would have made Audio Orchestrator a lot more complicated, so we could not build them into the tool for everyone. The open-source template now allows independent creators to make similarly specific adjustments to their experiences.
The audio orchestration framework consists of a core library for rendering synchronised audio across multiple web browsers and a template user interface for pairing devices, selecting customisation options, and controlling playback. Advanced developers can edit the template code to build their own user interface components, extend the algorithm used to allocate audio objects to devices, trigger one-off sound effects, or even add entirely new interactions between devices.
We also added support for custom templates to Audio Orchestrator — after configuring it once, it works just like the standard version, including the instant preview. This will also let us (or any other developer) share a template with production partners and greatly simplify the back-and-forth of integrating the latest audio or behaviour changes with any code modifications.
In the ±«Óãtv Research & Development Audio Team, we plan to continue supporting MakerBox users and will continue researching multi-device concepts. For example, we want to develop further ideas on offering more personalised audio experiences and additional content when watching TV or interacting with other media. We’ll also review early work on companion screens now that smartphones, other synchronised devices, and broadband connections are much more ubiquitous.
The open-source template and library, and the new version of Audio Orchestrator, give creators even more flexibility to customise or build entirely new multi-device audio experiences. We’re excited to see what you’ll make with these tools.
