Researchers’ Night: the Miller Theater in Augusta, on headphones
On 25 September 2026, in the former Carmine church in Piacenza, we brought The Four Turandot to the European Researchers’ Night: with a joystick and a pair of headphones you could walk through the stalls of the Miller Theater in Augusta while Calaf sings «Nessun dorma», with the hall’s real acoustics. The sound comes from an audio renderer we developed.
On Friday 25 September 2026 the former Carmine church in Piacenza, home of the Laboratorio Aperto, hosted the European Researchers’ Night: stands, workshops and talks by the city’s universities and research centres, as part of the Festival del Pensare Contemporaneo.
The Nicolini Conservatory’s programme included «Attraversare il suono» (Crossing sound), an immersive listening experience for one person at a time, open from 8:30 to 14:00 to young people aged fifteen and over. Inside it was The Four Turandot, the Conservatory’s project for which we hold scientific responsibility. This time the focus was entirely on the audio software.
What you saw
In the three-dimensional reconstruction of the Miller Theater in Augusta, Georgia, built in Unreal Engine 5, Calaf sings «Nessun dorma». He is a MetaHuman animated with the optical motion capture data recorded during the performance. The lip sync is generated directly from the vocal track, and the costume is simulated as fabric, in real time.
What you heard
With a joystick you move freely through the stalls, and the tenor’s voice reaches you with the real acoustics of the Miller Theater. The sound is produced by convolution with the ambisonic impulse responses measured in the hall, the ARIRs, which describe how the voice reflects off the walls, the stage and the stalls. It reaches your headphones in binaural form, and changes with the listener’s position and orientation.
How image and sound talk to each other
The graphics engine talks to an audio renderer we developed, through a dedicated plugin. Unreal sends it the listener’s position up to 60 times per second. The renderer drives the scene time, and on that time animation, lip sync and voice stay in sync.
The result is the feeling of really being inside the theatre, during the opera.
The video
This is the demo video of «Nessun dorma»: Calaf, the stage and the hall are entirely reconstructed. At the Carmine the same moment could be heard from the point in the stalls chosen with the joystick.
The rest of the project, with the performances in Augusta, Milan, Daegu and Hangzhou, is told in The Four Turandot.
