Enhancing Image Quality of the 3D Free-Viewpoint Video in Large-Scale Space Utilizing Player-Billboard Method

• Project/Area Number: 25280056
• Research Category: Grant-in-Aid for Scientific Research (B)
• Allocation Type: Partial Multi-year Fund
• Section: 一般
• Research Field: Perceptual information processing
• Research Institution: University of Tsukuba
• Principal Investigator: OHTA Yuichi 筑波大学, システム情報系(名誉教授), 名誉教授 (50115804)
• Co-Investigator(Kenkyū-buntansha): 亀田 能成 筑波大学, システム情報系, 准教授 (70283637) ; 北原 格 筑波大学, システム情報系, 准教授 (70323277)
• Research Collaborator: SHISHIDO Hidehiko ; KASUYA Nozomu ; SATO Hideaki ; AKECHI Nao ; KASHIWAKUMA Junya ; KAMIYAMA Takashi ; YAMAGIRI Yasushi ; TORIYA Hisatoshi ; SHIZUKU Taisuke ; UEDA Masashi ; SATO Shogo
• Project Period (FY): 2013-04-01 – 2017-03-31
• Project Status: Completed (Fiscal Year 2016)
• Budget Amount: ¥16,900,000 (Direct Cost: ¥13,000,000、Indirect Cost: ¥3,900,000); Fiscal Year 2016: ¥3,120,000 (Direct Cost: ¥2,400,000、Indirect Cost: ¥720,000); Fiscal Year 2015: ¥3,120,000 (Direct Cost: ¥2,400,000、Indirect Cost: ¥720,000); Fiscal Year 2014: ¥3,250,000 (Direct Cost: ¥2,500,000、Indirect Cost: ¥750,000); Fiscal Year 2013: ¥7,410,000 (Direct Cost: ¥5,700,000、Indirect Cost: ¥1,710,000)
• Keywords: 任意視点映像 / ビルボード / 高画質化 / 没入型提示 / 複合現実感 / スポーツ中継 / ネットワークメディア / インタフェース / 人物ビルボード / 大規模空間 / 複合現実型提示 / 映像圧縮

Outline of Final Research Achievements

We realized a technology that enables multiple viewers to observe 3D free-viewpoint video with SDTV image quality via the network arbitrarily choosing their viewpoints.
As the free-viewpoint video generation technology suitable for higher image quality, we made a systematic comparison between the player-billboard method which we originally have developed and the method based on 3D shape reconstruction. We confirmed the player-billboard method is superior to the 3D reconstruction based method for generating high quality video. We have examined the feasibility of our technology to utilize in the live video delivery via network and in the playback type content production. We also developed Mixed-Reality presentation methods such as an immersive display with player's point of view.

Research Products

• [Int'l Joint Research] University of Surrey(英国)
• [Int'l Joint Research] Curtin University(オーストラリア)
• [Int'l Joint Research] Surrey大学(英国)
• [Int'l Joint Research] Curtin大学(オーストラリア)
• [Journal Article] Improvement of Badminton-Player Tracking Applying Image Pixel Compensation (2017)
  • Author(s): Takashi Kamiyama, Yoshinari Kameda, Yuichi Ohta, Itaru Kitahara
  • Journal Title: ITE Transactions on Media Technology and Applications Volume: 5 Pages: 36-41
  • Peer Reviewed / Open Access / Acknowledgement Compliant
• [Journal Article] A Visual Tracking Method of a Badminton Shuttlecock by Utilizing Motion Blur (2015)
  • Author(s): 宍戸英彦，北原 格，亀田能成，大田友一
  • Journal Title: 電子情報通信学会論文誌D 情報・システム Volume: J98-D Issue: 7 Pages: 1083-1097
  • DOI: 10.14923/transinfj.2014JDP7075
  • NAID: 120007136262
  • ISSN: 1880-4535, 1881-0225
  • Year and Date: 2015-07-01
  • Peer Reviewed / Open Access
• [Journal Article] A Trajectory Estimation Method for Badminton Shuttlecock Using Multiple Exposure Time Images (2015)
  • Author(s): 宍戸英彦，北原格，亀田能成，大田友一
  • Journal Title: The Journal of The Institute of Image Information and Television Engineers Volume: 69 Issue: 8 Pages: J237-J244
  • DOI: 10.3169/itej.69.J237
  • NAID: 130005088779
  • ISSN: 1342-6907, 1881-6908
  • Peer Reviewed / Open Access
• [Journal Article] 複合現実感を用いた関心共有による展示物閲覧支援 (2015)
  • Author(s): 雫泰裕，北原格，大田友一
  • Journal Title: 日本バーチャルリアリティ学会論文誌 Volume: 20 Pages: 273-281
  • Peer Reviewed / Open Access
• [Journal Article] サッカー選手を対象とした一人称視点映像のための仮想カメラ運動モデル (2014)
  • Author(s): 糟谷望、北原格、亀田能成、大田友一
  • Journal Title: 電子情報通信学会論文誌D Volume: 97 Pages: 1385-1393
  • Peer Reviewed / Acknowledgement Compliant
• [Journal Article] Mobile Camera Localization Using Aerial-View Images (2014)
  • Author(s): Hisatoshi Toriya, Itaru Kitahara, Yuichi Ohta
  • Journal Title: IPSJ Transactions on Computer Vision and Applications (CVA) Volume: 6 Pages: 111-119
  • NAID: 130004705301
  • Peer Reviewed / Acknowledgement Compliant
• [Journal Article] 鏡像を用いた現実世界と仮想世界の重畳提示における運動視差と前後関係把握精度の関係 (2013)
  • Author(s): 佐藤秀昭，北原格，大田友一
  • Journal Title: 日本バーチャルリアリティ学会論文誌 Volume: 18 Pages: 421-430
  • NAID: 110009657345
  • Peer Reviewed
• [Journal Article] A Trajectory Estimation Method for Badminton Shuttlecock Utilizing Motion Blur (2013)
  • Author(s): Hidehiko Shishido，Itaru Kitahara，Yoshinari Kameda, Yuichi Ohta
  • Journal Title: Springer LNCS 8333 (6th Pacific Rim Symposium on Image and Video Technology (PSIVT2013)) Volume: 1 Pages: 325-336
  • Peer Reviewed
• [Presentation] 大規模空間の自由視点映像ライブ配信に向けて (2017)
  • Author(s): 大田友一
  • Organizer: 第2回PoTS映像学シンポジウム
  • Place of Presentation: 慶応義塾大学日吉キャンパス（神奈川県横浜市）
  • Year and Date: 2017-03-13
  • Invited
• [Presentation] 画像再構成処理を用いた疑似前後視点移動映像の生成 (2016)
  • Author(s): 山中 一希, 亀田 能成, 北原 格, 大田 友一
  • Organizer: 日本バーチャルリアリティ学会複合現実感研究会
  • Place of Presentation: 支笏湖丸駒温泉（北海道千歳市）
  • Year and Date: 2016-10-13
• [Presentation] Full-Scale Visualization of a Person on a Movable Transparent Screen (2016)
  • Author(s): Atsuya Oikawa, Itaru Kitahara, Yoshinari Kameda, and Yuichi Ohta
  • Organizer: The 15th IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR2016)
  • Place of Presentation: Merida, Mexico
  • Year and Date: 2016-09-19
  • Int'l Joint Research
• [Presentation] A Prospective Study About Enhancing Effect of VR in Soccer Training (2016)
  • Author(s): Cesar Daniel Rojas Ferrer, Itaru Kitahara, and Yoshinari Kameda
  • Organizer: 電子情報通信学会MVE研究会
  • Place of Presentation: 東京大学本郷キャンパス（東京都文京区）
  • Year and Date: 2016-07-20
• [Presentation] 白線領域除去によるバドミントン映像からの選手位置解析 (2015)
  • Author(s): 上山嵩, 北原格, 亀田能成, 大田友一
  • Organizer: 電子情報通信学会 技術研究報告MVE
  • Place of Presentation: 小樽観光物産プラザ（北海道小樽市）
  • Year and Date: 2015-10-08
• [Presentation] Image Inpainting処理を用いたバドミントン映像における選手追跡精度の向上 (2015)
  • Author(s): 上山嵩，北原格，亀田能成，大田友一
  • Organizer: 映像情報メディア学会年次大会
  • Place of Presentation: 東京理科大学葛飾キャンパス（東京都葛飾区）
  • Year and Date: 2015-08-26
• [Presentation] Digital Zoom Video of Swimmers by High-Definition Camera (2015)
  • Author(s): Yoshinari Kameda, Itaru Kitahara, Yuichi Ohta, Joe Yoshimi, Keisuke Ishikura, and Hideki Takagi
  • Organizer: 20th Congress of European College of Sport Science (ECSS)
  • Place of Presentation: Malm, Sweden
  • Year and Date: 2015-06-24
  • Int'l Joint Research
• [Presentation] Trajectory Estimation of a Fast and Anomalously Moving Badminton Shuttle (2015)
  • Author(s): Hidehiko Shishido, Itaru Kitahara, Yoshinari Kameda, Yuichi Ohta
  • Organizer: Joint Conference of IWAIT and IFMIA 2015
  • Place of Presentation: Tainan, Taiwan
  • Year and Date: 2015-01-11 – 2015-01-13
• [Presentation] Multi-Resolution Bullet-Time Effect (2014)
  • Author(s): Nao Akechi, Itaru Kitahara, Yoshinari Kameda, Yuichi Ohta
  • Organizer: ACM SIGGRAPH Asia 2014
  • Place of Presentation: ShenZhen, China
  • Year and Date: 2014-12-03 – 2014-12-06
• [Presentation] ズーム操作可能な多視点画像の閲覧方式 (2014)
  • Author(s): 明智那央、北原格、大田友一
  • Organizer: 第13回情報科学技術フォーラム（FIT2014）
  • Place of Presentation: 筑波大学（茨城県）
  • Year and Date: 2014-09-03 – 2014-09-05
• [Presentation] 高速かつ変則的に移動するバドミントンシャトルの軌跡推定法 (2014)
  • Author(s): 宍戸英彦、北原格、亀田能成、大田友一
  • Organizer: 第13回情報科学技術フォーラム（FIT2014）
  • Place of Presentation: 筑波大学（茨城県）
  • Year and Date: 2014-09-03 – 2014-09-05
• [Presentation] MR Simulation for Re-Wallpapering a Room in a Free-Hand Movie (2014)
  • Author(s): Masashi Ueda, Itaru Kitahara, Yuichi Ohta
  • Organizer: The 20th Anniversary International Conference on MultiMedia Modeling (MMM2014)
  • Place of Presentation: Dublin, Ireland
• [Presentation] Augmented-Reality Facial Expression Enhancement for Video Chatting Using Point Distribution Models (2013)
  • Author(s): Shogo Sato, Itaru Kitahara, Yuichi Ohta
  • Organizer: 5th Joint Virtual Reality Conference (JVRC2013)
  • Place of Presentation: Campus Paris Saclay, France
• [Presentation] Body Motion Design for Maneuvering a Virtual Camera in 3D Soccer Game (2013)
  • Author(s): Yasushi Yamagiri, Itaru Kitahara, Yoshinari Kameda, Yuichi Ohta
  • Organizer: The 23rd International Conference on Artificial Reality and Telexistence (ICAT2013)
  • Place of Presentation: 日本科学未来館（東京都）
• [Presentation] A Mobile Camera Localization Method for Updating Topography Map (2013)
  • Author(s): Itaru Kitahara, Hisatoshi Toriya, Yuichi Ohta
  • Organizer: The 3rd International Workshop on Soft Computing and Disaster Control (SocDic 2013)
  • Place of Presentation: Bali, Indonesia
  • Invited
• [Presentation] A Mobile Camera Localization Method Using Aerial-View Images (2013)
  • Author(s): Hisatoshi Toriya, Itaru Kitahara, Yuichi Ohta
  • Organizer: The 2nd Asian Conference on Pattern Recognition (ACPR2013)
  • Place of Presentation: 那覇ロワジールホテル（沖縄県）
• [Presentation] A Virtual Camera Controlling Method Using Multi-Touch Gestures for Capturing Free-viewpoint Video [Best Paper Honorable Mention Award] (2013)
  • Author(s): Junya Kashiwakuma, Itaru Kitahara, Yoshinari Kameda, Yuichi Ohta
  • Organizer: 11th European Interactive TV Conference
  • Place of Presentation: Como, Italy
• [Remarks]
  • URL: http://www.image.iit.tsukuba.ac.jp/
• [Remarks] 筑波大学 画像情報研究室
  • URL: http://www.image.esys.tsukuba.ac.jp
• [Remarks] Computer Vision & Image Media Lab.
  • URL: http://www.image.iit.tsukuba.ac.jp