24. February 2022. • INTERACTIVE

CASE STUDY: FLIGHT OF THE EAGLE OVER THE DANUBE – VR/360° Multimedia Installation; Interpretation Center Ilok

We participated in the project Flight of the eagle over the Danube, Interpretation Center Ilok, from the very beginning. From the design of the exhibition to multimedia, the task was to multimedially interpret the Eagle Crusader as a famous animal of the region Ilok in Croatia.

The content produced for multimedia installations in museums and interpretation centres can be divided into educational and experiential. Educational content aims to explain some information or tell a story, while experiential content aims to convey an experience or emotion. In this case, from the very start, we wanted to create a special experience, through which the visitors would experience the flight of an eagle over the Danube by flying along with it.

The original idea we considered was, that the eagle flies over the most beautiful locations of the Ilok region. We even filmed a few flights at various locations, but we soon realized that it was much more important to get closer to the atmosphere to create a real experience. For that reason, we decided to shoot an eagle flying over the Danube in one, continuous, slow motion take – at sunset. This resulted with a rather magical experience.

Technical aspects and demands

As we were able to design the creative solution for the interpretation of the content, we immediately decided on a VR/360° experience. The isolation in the VR helmet allows you to focus more on the content, thus have a better experience.

From a technical point of view, this is not an overly complex task: we have a 360° video of drone flight and a combination of a 3D computer-generated and animated eagle. The most important thing is the performance: we have to make a photorealistic view and experience of the eagle’s flight and paste it on a 360° footage of the drone’s flight. First of all, we had to do technical tests of drones and 360° cameras, in order to decide on the best solution.

Drone shooting

For testing, we headed to Savica in Zagreb. There we had enough space to test the drones and the cameras. We had several options for that. It is important to find a good combination of the above, since smaller drones only carry smaller cameras, and the weight of a better 360° camera (with just basic equipment) exceeds 7kg. And since we intended to fly over the river, there was an additional risk for the equipment, so we decided to go with a smaller, more mobile camera. Therefore, the final selection was DJI Mavic 2 i Insta 360x.

The shooting on the site went well. We knew we needed to film at dusk: we arrived the day before and did some light tests that showed we only have one flight available to catch the best light before it gets too dark. One drone flight lasted about 20 minutes, so we deicided to shoot a few takes. Each flight is risky in its own way, because winds over the river appear very quickly, which can create a drone shake, so the recorded material wouldn’t be usable.

Making of the 3D eagle

At the same time, we started with the preparations for the production of the main character – the eagle.

It always starts with a basic acquaintance with the specifics of the animal. Such are usually endemic or rare species , so it is not possible to just buy any model of an eagle. It should be done form scratch: is necessary to make a specific type of eagle, with specific textures. Texturing endemic species is very complex: there are often no good, detailed, well-lit photographs to find, and such textures have to be drawn and processed in Photoshop from existing ones.

When the model is finally good enough, with well applied skin above the textures, we move on to making an animation rig. Rig or the “skeleton” of a character, is the base for animations of all characters. Characters must have well connected body parts, so that moving seems as natural as possible. In the case of animals, it is quite a complex process, and in the case of 3D animation of humans, even more.

To make a high-quality 3D model of an eagle, it was necessary to make a system for laying out feathers. The feathers are quite specific in their movement, so we had to to make a kind of simulation of the movement of the feathers before we proceed to rendering. But even then, it is possible for the layers of feathers to collide unnaturally. In this case, it is necessary to manually adjust and smooth them out.

Animating the eagle

Realistic animation of any animal is a very demanding task. First of all, it is necessary to do good research on how the animal moves. As we mentioned, it is very difficult to find good references. Fortunately, this type of eagle is a very widespread animal and there was enough material to study the exact physiognomy and manner of its movement.

By comparing it with a real flight, an animation of the flight of our 3D eagle was created. At this stage it becomes clear how well the rig was made in the previous phase. Quite often, the rig needs to be further adapted to the specifics of the shot.

The next step is to implement a computer-generated model of an eagle into a live footage recorded as 360° video. For editing purposes, the footage is transferred to an equirectangular display, which is a practical format for processing such materials, because it is possible to insert elements into individual parts of the image. Those elements that we insert into the image need to be converted into equirectangular, in order to merge into perspective.

Compositing

Another big task of post production is deleting the drone from the image. Our eagle is big, but still not as big as a drone, and sometimes as it flutters its wings, the drone can be seen. It was necessary to make a “patch” of the sky where the drone is located and roughly delete it from the image. On such a patched background, a 3D CGI render of the eagle is placed, a color correction is made and it is finalized in such a way that the computer animation and the live footage can be merged as good as possible.

The area of the gaze of each eye needs to be rendered and composed separately, because the perspective somewhat changes. But the depth obtained by the stereoscopic image creates additional wow! effect in the viewer.

Distribution of the content onto the VR headset

The finalization of the material is always done depending on how it will be presented to the viewer. Each VR headset has a different basic resolution, and if you want to make the best product, you need to adjust the output resolution to the chosen headset.

Next, the important thing is the maintenance of a large number of headsets, and control over them, so that visitors would not have the opportunity to exit the application and/or change something within it. In such cases, headset and content control software is used, and it is possible to define, on all devices at the same time, which content will be played, and in what way.

Maintaining multimedia installations is a job inside a job. There is a lot that can be broken over time. Fortunately, with the remote system, it is possible to do software modifications without going to the location.