AI-empowered Point Cloud Video Streaming

2023 Fiscal Year Final Research Report

• Project/Area Number: 20H04174
• Research Category: Grant-in-Aid for Scientific Research (B)
• Allocation Type: Single-year Grants
• Section: 一般
• Review Section: Basic Section 60060:Information network-related
• Research Institution: The University of Electro-Communications
• Principal Investigator: Liu Zhi 電気通信大学, 大学院情報理工学研究科, 准教授 (90750240)
• Co-Investigator(Kenkyū-buntansha): 太田 香 室蘭工業大学, 大学院工学研究科, 文部科学省卓越研究員(教授) (50713971) ; 李 鶴 室蘭工業大学, 大学院工学研究科, 准教授 (40759891) ; Kien Nguyen 千葉大学, 大学院工学研究院, 准教授 (80647222)
• Project Period (FY): 2020-04-01 – 2024-03-31
• Keywords: point cloud / MEC / Video streaming / AI / VR/AR/MR / reinforcement learning / prediction

Outline of Final Research Achievements

This project aims to provide a high quality point cloud video streaming. In particular, we would like to solve the high complexity issue of the video encoding/decoding to facilitate the point cloud video transmission. Then we would like to use AI technologies, such as deep neural networks, to predict future viewing angle to eventually avoid transmitting the unwatched part of the point cloud video. We also would like to use the AI technologies, such as deep reinforcement learning, to manage the network resources to maximize the quality of point cloud video received by users. To provide the computation capacity required for the encoding, transcoding and decoding, AI technologies will also be used to help maximize the usage of the nearby available communication and computation resources. Prototype will be built to verify the performance and help adjust the designed schemes accordingly.

Free Research Field

情報ネットワーク

Academic Significance and Societal Importance of the Research Achievements

ポイントクラウド映像（点群映像）システムは、ユーザーに六自由度の没入型視聴体験を提供し、3Dシーンの任意の視点を自由に選択できるようにします。この技術は次世代の映像技術として期待されており、その卓越した視聴体験と異なる視点からの視聴が可能な特性から、学術界と産業界の両方で注目を集め、多くの応用分野で広く利用されています。
また、このプロジェクトで開発された技術は、この分野の研究課題を解決するだけでなく、他の関連する研究課題の解決にも役立つ可能性があります。