Time travel with augmented reality

‘Vijf jaar zo stil’ (Five Years of Silence) is echoing through the night - a 1000 voices singing. Downtown Utrecht, 10 October - a crowd of people has gathered at Janskerkhof Square for a tradition dating back at least 210 years.

A clock projected with powerful beamers on a monumental building counts down the final seconds to the start of celebrations that have been five years in the making - and waiting. Then it’s showtime. The entire building comes to life, and centuries of history unfold at breakneck speed across its façade.

Our team - all credited below - has been working day and night in collaboration with the student fraternity Utrechtsch Studenten Corps (U.S.C.). This show is a ‘3D projection mapping’: a film projected in a way that takes into account the three-dimensional shape of the building. It creates a powerful optical illusion, making the façade appear to change colour and shape continuously - a flavour of augmented reality.

The story we told traces the deep roots of student and university life, reaching back to 1636. We see the current building being constructed in seconds, occupied and liberated during the Second World War, and even a sing-along moment. The climax? The near-reveal of the theme for the upcoming anniversary year!

It was an inspiring project in many ways:

• A deep dive into the architecture of the buildings involved

• Learning about past themes (discovering what Italian late baroque architecture looked like - and how tall Genghis Khan actually was)

• Thoughtful discussions on how to depict sensitive themes like the Nazi occupation responsibly

• Using AI to animate oil paintings and statues

• Designing a hundred sound effects to make everything feel real

• A fearless on-site team preparing the area and secretly installs the massive projectors

What was most fun:

• Listening to the students and learning what ideals they wanted to express

• Working with our VJs (Ruben, Henk, Teo, Jack) - spread across Europe, even Romania - and watching them combine such wide-ranging skills into new surprises each day

• Getting stuck on creative challenges and always finding a way out (“How do you make Corona look cool?” “How do you recreate a stampede?”)

• The generosity of so many people: our team from ten years ago for reusable assets, the young composer Kamikaze for a powerful rendition of “Bella Ciao” 

Learnings:

• It’s tempting to me to focus on storytelling, but for mapping abstract ‘gallery play’ (abstract animations) is just as powerful. Thankfully, Ruben included this in the finale

• AI is amazing for research, image creation and making old photographs move. For the many nuances in a project like this, ‘old skool’ tools like After Effects and Unreal are still very useful 

• Hardware on site is challenging! With stunts and improvisation, the projection was superb, but calibrating the audio should have been more robust and redundant. (have the option to go above official limits, take considerable  audience noise into account, use copper, not wireless

Watch the full show here: