The hype about Augmented Reality (AR) is slightly going down even though the advantages remain. Especially for broadcasters the advantages of AR can be used in a variety of domains, like material planning, broadcast production, graphic development and marketing. However, AR is still a new territory for editorial and production staff. We, at the IRT, evaluate since a few years now the benefit of AR in the broadcast domain. So far, we focused on studio production with the HoloLens Spectator View and iOS Spectator View.
To develop a broader range of approaches how to use AR reasonable in TV productions the IRT cooperated with the Ludwigs-Maximilians-Universität Munich (LMU). In the 2019 summer semester 15 master students from informatics, media informatics and human computer interaction joint the project.
The scope of the project was to develop an AR application from scratch to prototype
with a reasonable benefit in broadcast workflows. To design the ideas as realistic
and useful as possible the concepts where pitched to representatives of different
broadcast institutions and domains. In discussion with broadcasters the students
could collect feedback and adapt their ideas. All three groups fund a supervisor to
support them with feedback, knowledge and broadcast material. During the summer they
developed their prototypes and demonstrated the approaches on the LMU OpenLabDay
2019.
To start brainstorming, we introduced the students to the basic broadcast workflows
and latest AR technologies. We also demonstrated already realized AR applications,
like the the WDR
Kriegskinder project which was published by the Westdeutscher
Rundfunk in February 2019.
With this input in mind the students started developing their own ideas. By going
over the ideas again and again and constantly presenting them to the plenum we
combined the single approaches and formed three groups, each with joint visions.
As next step the students focused on the technical aspects of their ideas. They
researched related work and tried to figure out which technic to use for the
prototype implementation.
To develop the prototypes the students coded one week together at the LMU. At the
end of the week they demonstrated three AR projects.
Project 1: Interactive 360°
AR
360° content can be produced in a very high quality but there are actually no VR Headsets on the market that can show the content in full quality. Another option to consume the content is a TV screens but it destroys the immersion of the video. To address this problem the Interactive 360° AR team proposed a dual approach. To display the 360° content in its full quality, they use a state-of-the-art TV display. With the help of the Microsoft HoloLens the additional parts of the 360° movie are displayed around the Screen. The TV shows a part of the movie while the rest is displayed as a sphere around the viewer. By selecting a scene out of the sphere, the viewer can actively decide what he wants to see on the TV screen. The team used Unity to synchronize the content on the HoloLens and the screen and Vuforia for the screen-recognition. By clicking on the augmented sphere around the viewer the scene rotates and shows the selected part on the TV. They partnered with the ZDF to get feedback and 360° material for testing. The prototype proves the concept and demonstrates how high quality 360° content can be consumed without the loss of quality or immersion.
Watch the demo video – (graphics by Evelyn Müller) :
Project 2: PlaceholdAR
Efficiency and cost effectiveness are essential parts of broadcast productions. The
PlaceholdAR team focuses with their approach on efficiency during the set
design- and building-process. They partnered with Avirtico and the
Bayerischer Rundfunk to collect feedback about today’s workflows and to
discuss where to set focus. Both partners saw a potential benefit especially for
non-permanent sets.
The PlaceholdAR application includes two modes. In the Set
Designing mode a room can be furnished with virtual production equipment,
like spots, cameras, microphones or deco elements. A Set Designer therefore uses a
tablet to place the virtual objects in the real room. In the Set
Construction mode, the saved virtual objects can be displayed to a
technician. With the help of the Microsoft HoloLens the technician can see the
virtual objects, that are placed in the room, while having both hands free to set up
the physical counterpart. In their prototype the team used Unity for the
scene setup and a Microsoft Azure solution to synchronize and store the
changes between tablet and HoloLens. The prototype demonstrates how virtual objects
are placed, manipulated and stored in a physical environment with a tablet. The
objects are played back on the same physical position on a HoloLens. The PlaceholdAR
team demonstrated successfully an approach to save time and enhance collaboration
during a broadcast production process.
Project 3: Storytelling AR
Every TV show has their fans and loyal viewers and they are always keen to know more.
Fans like to get a look behind the scenes and access to additional material.
Behind-the-Scenes material, like interviews with actors, are used by many
TV shows for marketing reasons. The Storytelling AR team came up with a
context-based application to display additional material. The app shows viewers
augmented content as if they were on a location where the show was recorded. To get
material the Storytelling AR team joined a production of ndF for
the ZDF series “Der Bergdoktor”. This show has a lot of fans, who
visit the film location in the mountains quite frequently. On their main location in
Söll, they recorded for one day the set in 360° to recreate it virtually. The team
used Unity for the scene building, Vuforia to recognize markers
and Blender to build the 3D models. When a marker, like a part of the
set-location, is detected the scene is augmented with virtual objects, like
requisites or production equipment. The application enhances fans of “Der
Bergdoktor”, when they visit the set in Söll, to see AR content through
their smartphones or tablets. The prototype recognizes photographs of the original
location and uses them as markers. The virtual objects are placed around the marker
and additionally audio material and short video sequences can be played back.
The team demonstrated impressively how AR could be used to promote a TV show and
provide additional content to a fanbase.
Watch the demo video (graphics by Franziska Haaf):
These final projects demonstrated the student’s vision in a very high level. The
students where highly motivated and had a lot of fun during the project, which is
reflected in the quality of the prototypes. They gained a lot of experience with AR.
Thanks to their supervisors they collected many insights of the broadcast world,
like filming on a real set in the mountains or working with a film crew.
For the broadcasters these three projects are good examples of how AR can be used in
different broadcast areas, to support storytelling, to create immersive experiences
and to enhance production workflows.
Therefore we start a new round of the project in the summer semester 2020! This time
we focus on storytelling.
So, if you have a great story and want to have it told in AR, let us know and get
in contact with our expert!