High-fidelity Photometric 3D Imaging and its Applications

• Project/Area Number: 16H01732
• Research Category: Grant-in-Aid for Scientific Research (A)
• Allocation Type: Single-year Grants
• Section: 一般
• Research Field: Perceptual information processing
• Research Institution: Osaka University
• Principal Investigator: Matsushita Yasuyuki 大阪大学, 情報科学研究科, 教授 (30756507)
• Research Collaborator: TAGAWA SEIICHI ; YAGI YASUSHI ; IKEUCHI KATSUSHI
• Project Period (FY): 2016-04-01 – 2019-03-31
• Project Status: Completed (Fiscal Year 2018)
• Budget Amount: ¥44,460,000 (Direct Cost: ¥34,200,000、Indirect Cost: ¥10,260,000); Fiscal Year 2018: ¥7,020,000 (Direct Cost: ¥5,400,000、Indirect Cost: ¥1,620,000); Fiscal Year 2017: ¥15,340,000 (Direct Cost: ¥11,800,000、Indirect Cost: ¥3,540,000); Fiscal Year 2016: ¥22,100,000 (Direct Cost: ¥17,000,000、Indirect Cost: ¥5,100,000)
• Keywords: コンピュータビジョン / ３次元形状推定 / 3次元形状推定 / 三次元復元 / 3次元復元 / 視覚情報処理 / 3次元形状復元

Outline of Final Research Achievements

3D imaging of real-world objects using a camera is a central problem in computer vision due to its wide application areas such as robot control, object recognition by machines, autonomous driving, and product inspection. It has been understood that 3D imaging with ``photometric'' information enables high-fidelity acquisition of 3D shape. In this work, we advance the photometric 3D imaging technology with an emphasis on enabling (a) 3D measurement of surfaces with diverse reflectances, and (b) an easy-to-use imaging setup. As a result, we have developed a thread of new photometric 3D imaging technologies and achieved the intended goals.

Academic Significance and Societal Importance of the Research Achievements

研究成果の学術的意義は大きく２点ある．一つは，これまで必須と考えられてきた光源強度のキャリブレーションが不要であることを理論的に示し，新たな照度差ステレオ技術を提案した(semi-calibrated photometric stereo)．この結果により，実際のアプリケーションにおいてもキャリブレーションの手間が緩和できることがわかった．二つ目は，機械学習と光学的３次元イメージングの方向性を打ち出したことである．具体的には深層学習を用いてこれまで困難であった反射特性のモデリングをバイパスし，見えと形状のマッピング関数を学習させるというアプローチ世界に先駆けて提案した．

Research Products

• [Int'l Joint Research] KAIST(韓国)
• [Int'l Joint Research] Peking University/HKUST(中国)
• [Journal Article] Semi-Calibrated Photometric Stereo (2018)
  • Author(s): Cho DongHyeon、Matsushita Yasuyuki、Tai Yu Wing、Kweon In So
  • Journal Title: IEEE Transactions on Pattern Analysis and Machine Intelligence Volume: ー Issue: 1 Pages: 1-12
  • DOI: 10.1109/tpami.2018.2873295
• [Journal Article] Numerical shape-from-shading revisited (2018)
  • Author(s): Santo Hiroaki、Samejima Masaki、Matsushita Yasuyuki
  • Journal Title: IPSJ Transactions on Computer Vision and Applications Volume: 10 Issue: 1 Pages: 1-12
  • DOI: 10.1186/s41074-018-0044-9
• [Journal Article] Fast Randomized Singular Value Thresholding for Low-Rank Optimization (2018)
  • Author(s): Oh Tae-Hyun、Matsushita Yasuyuki、Tai Yu-Wing、Kweon In So
  • Journal Title: IEEE Transactions on Pattern Analysis and Machine Intelligence Volume: 40 Issue: 2 Pages: 376-391
  • DOI: 10.1109/tpami.2017.2677440
  • Peer Reviewed / Open Access / Int'l Joint Research
• [Journal Article] Robust Multiview Photometric Stereo Using Planar Mesh Parameterization (2017)
  • Author(s): Park Jaesik、Sinha Sudipta N.、Matsushita Yasuyuki、Tai Yu-Wing、Kweon In So
  • Journal Title: IEEE Transactions on Pattern Analysis and Machine Intelligence Volume: 39 Issue: 8 Pages: 1591-1604
  • DOI: 10.1109/tpami.2016.2608944
  • Peer Reviewed / Open Access / Int'l Joint Research
• [Presentation] Light Structure from Pin Motion - Simple and Accurate Point Light Calibration for Physics-Based Modeling (2018)
  • Author(s): Hiroaki Santo, Michael Waechter, Masaki Samejima, Yusuke Sugano, Yasuyuki Matsushita
  • Organizer: European Conference on Computer Vision (ECCV)
  • Int'l Joint Research
• [Presentation] Uncalibrated Photometric Stereo under Natural Illumination (2018)
  • Author(s): Zhipeng Mo, Boxin Shi, Feng Lu, Sai-Kit Yeung, Yasuyuki Matsushita
  • Organizer: IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  • Int'l Joint Research
• [Presentation] Deep Photometric Stereo Network (2017)
  • Author(s): Hiroaki Santo、Masaki Samejima、Yusuke Sugano、Boxin Shi、Yasuyuki Matsushita
  • Organizer: International Workshop on Physics Based Vision meets Deep Learning (PBDL) in Conjunction with IEEE International Conference on Computer Vision (ICCV)
  • Int'l Joint Research
• [Presentation] Fast General Norm Approximation via Iteratively Reweighted Least Squares (2016)
  • Author(s): Masaki Samejima and Yasuyuki Matsushita
  • Organizer: 4th Workshop on e-Heritage, Asian Conference on Computer Vision
  • Place of Presentation: Taipei, Taiwan
  • Year and Date: 2016-11-20
  • Int'l Joint Research
• [Presentation] Photometric Stereo Under Non-uniform Light Intensities and Exposures (2016)
  • Author(s): Donghyeon Cho, Yasuyuki Matsushita, Yu-Wing Tai, In-So Kweon
  • Organizer: European Conference on Computer Vision
  • Place of Presentation: Amsterdam, Netherlands
  • Year and Date: 2016-10-11
  • Int'l Joint Research