Introduction of research on image expression technology at NHK Lab

I would like to introduce two initiatives of NHK Science & Technology Research Laboratories:

Meta Studio

Meta Studio is a compact volumetric TV studio for 3D reconstruction. Meta Studio can be understood as an extension of the conventional studio that can also reconstruct 3D models by means of real-world sensing of the subjects by cameras and sensors for TV programme production use. Whereas a conventional studio outputs only the video images of the subject, Meta Studio realises an integrated 2D-3D production environment that outputs not only video, but also the time-series data of 3D models reconstructed from images captured by multiple cameras. I introduce the principles of the Meta Studio system.

AI robotic camera system

AI robotic camera system automatically captures soccer matches from the main (game-follow) camera position that is roughly aligned with the halfway line. The system is capable of deciding camerawork according to the game situations by implementing a DNN model trained by the situation-dependent framing of skilled camera operators. The framing model is designed to instruct the robotic camera when and where to move while compensating for the time lag in the system thereby enabling immediate reactions. I describe a construct of the system and result of experiment during a professional soccer match.

Short biography

Kota Takahashi joined NHK (Japan Broadcasting Corporation), Japan in 2017. After working in NHK Oita Station, he moved to NHK Science & Technology Research Laboratories (STRL) in 2020. He works on automatic capturing technology for live sports broadcasts and video information presentation technology for volumetric capturing studios.