Japan’s Naquyo audio artists harness d&b Soundscape to create immersive Kyoto ‘time warp’.
Japan’s Naquyo collective of world-renowned immersive sound artists took to the stage at Kyoto’s Rohm Theatre for a live performance of field-recorded, synthesised, and live-played sounds designed to transport an audience to Kyoto circa 800AD. Using a d&b audiotechnik sound system and d&b Soundscape technology, they bridged the gap between audio engineer and creative sound artist to merge ancient sounds with futuristic tech in a 360° audio experience.
The project was conceived and performed by ambient composer Kazuya Nagaya and shō (Japanese reed instrument) player Katsuhiko Orii, together with audiovisual artist, Junichi Akagawa, and dancer, Kou Yamamoto (nouseskou).
The key to recreating what a resident of Heian-kyō would hear all those years ago was to immerse an audience within a 360° soundscape. The audience were invited to sit in a circle on stage at the Rohm Theatre, surrounding the performers. To ensure each person heard the ambient sounds, chants and bells with pinpoint accuracy, an object-based sound mixing solution was required, alongside sophisticated room emulation software and an array of loudspeakers designed to immerse the listener.
In the pre-planning stages, Mutek JP introduced the Naquyo creative team to d&b audiotechnik in Yokohama, where they were given a demo by d&b audiotechnik Japan sales and market development’s Shozo Doi and EAS specialist, Yo Kato, together with the team at Japanese d&b audiotechnik rental house, Treasure Island. Together, they designed the system that would form the audio foundations for the performance.
At the heart of the d&b Soundscape system was the DS100 Signal Engine, the audio system based on a Dante-enabled signal matrix. The main system consisted of 16 d&b T-Series T10 2-way speakers with 2x6.5-inch drivers and a 1.4-inch exit compression driver, used in a point source configuration. These were evenly spaced around the audience. In addition, four T10s were hung as ceiling speakers above the audience’s heads.
The subwoofers were d&b V-Series V-SUB 18-inch cardioid subwoofers, one on each of the four sides. These were driven by d&b D20 four-channel amplifiers, and the team also used two DS10 Dante–AES/EBU audio network bridges.
“We wanted to use the same model of speakers for all positions due to the nature of the performance, and we chose the T10 because of its small size and power,” says d&b’s Doi-san. “The V-SUBs were chosen because we wanted to reproduce a natural low end in line with the composer’s expectations.”
The capabilities of d&b Soundscape would fundamentally change the audio artists’ creative approach to the piece. The two Soundscape software modules – sound object positioning tool En-Scene and in-line room emulation tool En-Space – offered them powerful ways to alter and position sound. En-Scene was used throughout the Naquyo performance, including for live performances of the shō, and applied to the sound of Buddhist bells and field recordings to reproduce a natural sound environment.
The sound design was a dynamic, live production that included live-triggered recordings alongside traditional instruments being played live. Selected sounds were programmed to follow dancer nouseskou’s movements, tracked via four sensors on his wrists and ankles. Each sensor was used as a single effect to control the playback position and pitch, using the Ableton Max for Live platform. Based on these movements, TouchDesigner visual programming software for real-time interactive multimedia content calculated the positions of the sounds and sent the coordinates to the DS100 via Open Sound Control (OSC) networking protocol, updating the positions of the sounds within Soundscape.
All the sounds in the Naquyo performance were sent to En-Scene from TouchDesigner, and the team set a delay mode that allowed each sound to find the ideal way to move. The team then used En-Space to select the optimum reverberation for each scene of the work, manipulating the scale of the space as perceived by the audience.
Augmenting the experience was a lighting design that complemented the soundscape and sound triggers, and visuals via four large screens hung to emulate the north, south, east and west. “In one scene, we projected images of those four Kyoto locations onto the screens, while playing back the raw audio recorded in those places,” explains Akagawa-san. “Most of the sounds in the films were triggered and manipulated by our movements in real time, creating a complex but organic chain reaction.”
The workflow for configuring a Soundscape system is to first simulate the venue using d&b ArrayCalc – the simulation tool for d&b line arrays, column and point source loudspeakers as well as subwoofers – and then decide on the speaker requirement, and placement. By using the En-Scene tool the team was able to visualise both the real and perceived acoustical performance in the space.
“This is really useful to allow us to confidently design the best-possible Soundscape system and get the most out of it,” explains d&b’s Kato-san. “Within ArrayCalc, we could also then configure the amplifiers and the DS100 system routing. This design file can then be opened in our remote-control software, R1, and use this to control and monitor the system.”
Mutek Japan is supportive of sound artists using d&b Soundscape. “Technology is an indispensable tool to explore the value of AV content and to constantly evolve,” says Mutek JP. “The fusion of music and technology creates new possibilities for expression and the creation of valuable new experiences.
Naquyo – The Visionary Universe of Heian-kyō was staged as a collaboration between Mutek Japan and the Kyoto Steam World Cultural Exchange Festival, and produced with the support of Uryuyama Gakuen Kyoto University of the Arts Department of Historical Heritage.