Dance Monsters uses a variety of technologies to bring its CGI monster avatars to life, some of which have never been used in unscripted television before.
Dance Monsters utilises 3D avatars instead of real-life dancers to give audiences a performance in which they can focus on the dancing first.
To do this, the show has employed various kinds of technologies, like a custom-built performance capture stage, so-called star tracker technology and the help of Epic Games’ Unreal Engine.
How does Netflix’s Dance Monsters work?
If you tuned in to watch Netflix’s newest dance competition show called Dance Monsters for the first time and you were confused about what exactly you were watching, you would not be the only one.
In a world where artificial intelligence and so-called “deepfakes” are becoming more and more prevalent and realistically believable, it can be difficult to determine exactly how this show goes about constructing the “dance monster” performances that allow the amateur dancers who perform on the show, to express themselves without the fear of judgement by the audience for their real, physical appearances.
The innovative technology behind Netflix’s Dance Monsters
In order to do this, Dance Monsters creates lovable “dance monster” CGI avatars for the performers that feature on the show.
While this may seem similar to what other shows like The Masked Dancer have done in the past, the show’s creators have actually used a variety of special production techniques and employed the help of two different production companies, namely, Imaginarium Studios (IMS) and Mo-Sys.
Andrew Jackman, the executive producer, and Sarah Tyekiff, who is the executive producer of Lime Pictures, have revealed that they had to combine many different technologies that had never been used in unscripted television before in order to get the final product that fans have been enjoying from home.
In the end, the formula that Dance Monsters settled on to create their dancing 3D avatars required them to build a custom motion capture stage that matched the actual performance stage that you see on the show, exactly.
They then developed a special rig with multiple different cameras and what the show coined “star tracker technology” to film every performance.
The information from this motion capture stage then allowed them to feed data into a motion-builder through which they could smooth out and correct the renderings of the performance.
As a last step, these smoother renderings would be fed into Epic Games’ Unreal Engine which would essentially recreate the performance with the 3D avatar character, similarly to how the characters in video games are created.
Finally, a rough projection of this performance is shown to the judges and the in-studio audience in real time to get the full effect.
How does the “star tracker technology” work?
In order to get the full 3D picture for every performance, the performance capture stage had 73 Vicon cameras that were capable of tracking small sensors that were placed over the dancers and their head-mounted cameras.
Five main cameras were used to actually film the show and they would move around the stage and the dancers while they performed.
Mo-Sys placed sensors at the top of each of the cameras, which communicated with the “stars” (reflective markers), which were placed on the ceiling of the studio.
This allowed the producers to plot how the cameras moved through the space more accurately by measuring the distances between the sensors and the markers.
Why do the producers edit the footage of the dancers’ performances?
The only editing that the producers did once the footage of the dancers’ performances on the motion capture stage was fed into motion builder, was to correct the way that the renderings interacted with the space.
This includes instances where motions had to be blurred together or smoothed or instances where the monster was interacting strangely with the floor of the stage or any of the other props.
Understanding Epic Games’ Unreal Engine
The final step in how Dance Monsters manages to create the 3D animated performances of the monsters that viewers see when they are watching the show from home is to use Epic Games’ Unreal Engine.
This Unreal Engine is a 3D graphics creation tool which allows creators to access real-time 3D content which ranges from world-building and modelling, to animations.
Unreal Engine is most commonly used to create the video segments and other visual material for popular games like Sackboy: A Big Adventure, Minecraft Dungeons, and of course, Fortnite.
However, since the engine is so beginner-friendly and it is written in C++, the engine is now being used in a variety of industries, including its use in Dance Monsters in the film and television industry.
