A study of 3D head geometry estimation based on selective use of 2D image and acoustic features

• Project/Area Number: 21700207
• Research Category: Grant-in-Aid for Young Scientists (B)
• Allocation Type: Single-year Grants
• Research Field: Perception information processing/Intelligent robotics
• Research Institution: Waseda University
• Principal Investigator: MAEJIMA Akinobu 早稲田大学, IT研究機構, 研究員 (50454046)
• Project Period (FY): 2009 – 2011
• Project Status: Completed (Fiscal Year 2011)
• Budget Amount: ¥4,160,000 (Direct Cost: ¥3,200,000、Indirect Cost: ¥960,000); Fiscal Year 2011: ¥650,000 (Direct Cost: ¥500,000、Indirect Cost: ¥150,000); Fiscal Year 2010: ¥650,000 (Direct Cost: ¥500,000、Indirect Cost: ¥150,000); Fiscal Year 2009: ¥2,860,000 (Direct Cost: ¥2,200,000、Indirect Cost: ¥660,000)
• Keywords: 画像特徴量 / 音響特徴量 / 顔特徴データベース / 正準相関分析 / ニューラルネットワーク / 3次元頭部形状推定 / 相関分析

Research Abstract

We aimed to realize a 3D geometry estimation method based on selective use of 2D image and acoustic features and obtained four results as follows. (1) We constructed a database which contains 109 range scans and 5 Japanese vowel speeches. (2) We developed a method for fast 3D geometry estimation from a face photograph using prior knowledge of 3D face geometries. (3) We developed a framework to estimate 3D head geometry from a vowel speech using a neural network trained using a pair of acoustic features and 3D head geometries. (4) We developed a registration method for non-rigid object to effectively generate a 3D head model from a range scan, and several achievements have published as journals.

Research Products

• [Journal Article] 人物頭部モデル自動生成システムの実現:最適化局所アフィン変換に基づく人物頭部モデルの自動生成 (2010)
  • Author(s): 前島謙宣
  • Journal Title: 画像ラボ Volume: 21巻 Pages: 29-34
  • NAID: 40017055530
• [Journal Article] 人物頭部モデル自動生成システムの実現最適化局所アフィン変換に基づく人物頭部モデルの自動生成 (2010)
  • Author(s): 前島謙宣, 森島繁生
  • Journal Title: 画像ラボ Volume: 21 Pages: 29-34
  • NAID: 40017055530
• [Journal Article] 最適化局所アフィン変換に基づく正面顔レンジスキャンデータからの頭部モデル自動生成 (2009)
  • Author(s): 前島謙宣, 森島繁生
  • Journal Title: 画像電子学会論文誌 Volume: 38巻 Pages: 404-413
  • NAID: 10025522102
  • Peer Reviewed
• [Journal Article] 最適化局所アフィン変換に基づく正面顔レンジスキャンデータからの頭部モデル自動生成 (2009)
  • Author(s): 前島謙宣、森島繁生
  • Journal Title: 画像電子学会論文誌 38 Pages: 404-413
  • NAID: 10025522102
  • Peer Reviewed
• [Presentation] 単母音に含まれる音響特徴からの3次元頭部形状推定に関する一検討 (2012)
  • Author(s): 前島謙宣, 森島繁生
  • Organizer: Visual Computing/グラフィックスとCAD合同シンポジウム2012
  • Place of Presentation: 早稲田大学
  • Year and Date: 2012-06-24
• [Presentation] 事前知識とStructure-from-Motionを併用した1台のビデオ画像からの3次元顔モデル高速自動生成手法 (2012)
  • Author(s): 原朋也, 久保尋之, 前島謙宣, 森島繁生
  • Organizer: Visual Computing/グラフィックスとCAD合同シンポジウム2012
  • Place of Presentation: 早稲田大学
  • Year and Date: 2012-06-23
• [Presentation] 3D Human Head Geometry Estimation from a Speech (2012)
  • Author(s): Akinobu Maejima, Shigeo Morishima
  • Organizer: ACM SIGGRAPH 2012
  • Place of Presentation: Los Angeles, USA
• [Presentation] Fast-Automatic 3D Face Generation Using a Single Video Camera (2012)
  • Author(s): Tomoya Hara, Hiroyuki Kubo, Akinobu Maejima, Shigeo Morishima
  • Organizer: ACM SIGGRAPH 2012
  • Place of Presentation: Los Angeles, USA
• [Presentation] Automatic 3D Face Generation from Video with Sparse Point Constraint and Dense Deformable Model (2011)
  • Author(s): Tomoya Hara, Akinobu Maejima, Shigeo Morishima
  • Organizer: ACM SIGGRAPHASIA 2011
  • Place of Presentation: China, Hong Kong
  • Year and Date: 2011-12-14
• [Presentation] PC上の顔:デジタルフェイスモデリングの現状と今後 (2011)
  • Author(s): 前島謙宣
  • Organizer: 日本顔学会フォーラム顔学2011
  • Place of Presentation: 日本歯科大学新潟生命歯学部
  • Year and Date: 2011-09-24
• [Presentation] PC上の顔:デジタルフェイスモデリングの現状と今後 (2011)
  • Author(s): 前鳥謙宣
  • Organizer: 日本顔学会フォーラム顔学2011
  • Place of Presentation: 日本歯科大学新潟生命歯学部(招待講演)
  • Year and Date: 2011-09-24
• [Presentation] Realistic Facial Animation by Automatic Individual Head Modeling and Facial Muscle Adjustment (2011)
  • Author(s): Akinobu Maejima, Hiroyuki Kubo, Shigeo Morishima
  • Organizer: HCI International 2011
  • Place of Presentation: USA, Hilton Orlando, Bonnet Creek
  • Year and Date: 2011-07-14
• [Presentation] Realistic Facial Animation by Automatic Individual Head Modeling and Facial Muscle Adjustment (2011)
  • Author(s): Akinobu Maejima, Hiroyuki Kubo, Shigeo Morishima
  • Organizer: HCI International 2011
  • Place of Presentation: USA, Hilton Orlando Bonnet Creek
  • Year and Date: 2011-07-14
• [Presentation] Automatic Generation of Head Models and Facial Animations Considering Personal Characteristics (2010)
  • Author(s): Akinobu Maejima, Hiroto Yarimizu, Hiroyuki Kubo, Shigeo Morishima
  • Organizer: Proceedings of the 17^<th> ACM Symposium on Virtual Reality Software and Technology(VRST2010)
  • Place of Presentation: Hong Kong Poly
  • Year and Date: 2010-11-22
• [Presentation] Automatic Generation of Head Models and Facial Animations Considering Personal Characteristics (2010)
  • Author(s): Akinobu Maejima, Hiroto Yarimizu, Hiroyuki Kubo, Shigeo Morishima
  • Organizer: Proceedings of The 17th ACM Symposium on Virtual Reality Software and Technology (VRST2010)
  • Place of Presentation: HongKong PolyU
  • Year and Date: 2010-11-22
• [Presentation] Facial Animation Reflecting Personal Characteristics by Automatic Head Modeling and Facial Muscle Adjustment (2010)
  • Author(s): Akinobu Maejima, Hiroyuki Kubo, Shigeo Morishima
  • Place of Presentation: 明治大学
  • Year and Date: 2010-10-27
• [Presentation] Facial Animation Reflecting Personal Characteristics by Automatic Head Modeling and Facual Muscle Adjustment (2010)
  • Author(s): Akinobu Maejima, Hiroyuki Kubo, Shigeo Morishima
  • Organizer: 10th International Symposium on Communications and Information Technologies 2010
  • Place of Presentation: 明治大学
  • Year and Date: 2010-10-27
• [Presentation] アクティブスナップショット (2010)
  • Author(s): 前島謙宣, 久保尋之, 森島繁生
  • Organizer: 第15回日本顔学会フォーラム顔学2010
  • Place of Presentation: 東京歯科大学
  • Year and Date: 2010-10-23
• [Presentation] アクティブスナップショット (2010)
  • Author(s): 前島謙宣, 久保尋之, 森島繁生
  • Organizer: 第15回日本顔学会フォーラム顔学2010
  • Place of Presentation: 東京医科歯科大学
  • Year and Date: 2010-10-23
• [Presentation] 顔写真1枚からの3次元顔モデル高速自動生成 (2010)
  • Author(s): 前島謙宣, 森島繁生
  • Organizer: CEDEC2010(CESAデベロッパーズカンファレンス2010)
  • Place of Presentation: パシフィコ横浜会議センター
  • Year and Date: 2010-08-31
• [Presentation] 顔写真1枚からの3次元顔モデル高速自動生成 (2010)
  • Author(s): 前島謙宣, 森島繁生
  • Organizer: CEDEC2010 (CESAデベロッパーズカンファレンス2010)
  • Place of Presentation: パシフィコ横浜会議センター
  • Year and Date: 2010-08-31
• [Presentation] 顔変形モデルと顔形状分布制約に基づく単一顔画像からの3次元顔モデル高速自動生成 (2010)
  • Author(s): 前島謙宣, 森島繁生
  • Organizer: 画像の認識・理解シンポジウム(MIRU2010)
  • Place of Presentation: 釧路市観光国際交流センター
  • Year and Date: 2010-07-28
• [Presentation] 顔形状事前知識に基づく顔画像からの3次元顔モデル高速自動生成 (2010)
  • Author(s): 前島謙宣, 森島繁生
  • Organizer: SSII2010第16回画像センシングシンポジウム
  • Place of Presentation: パシフィコ横浜
  • Year and Date: 2010-06-11
• [Presentation] 顔形状事前知識に基づく顔画像からの3次元顔モデル高速自動生成 (2010)
  • Author(s): 前島謙宣, 森島繁生
  • Organizer: SSII2010 第16回画像センシングシンポジウム
  • Place of Presentation: パシフィコ横浜
  • Year and Date: 2010-06-11
• [Presentation] スナップ写真からの3次元顔モデル高速自動生成 (2010)
  • Author(s): 前島謙宣, 森島繁生
  • Organizer: 電子情報通信学会PRMU研究会
  • Place of Presentation: 鹿児島大学
  • Year and Date: 2010-03-15
• [Presentation] スナップ写真からの3次元顔モデル高速自動生成 (2010)
  • Author(s): 前島謙宣、森島繁生
  • Organizer: 電子情報通信学会PRMU研究会
  • Place of Presentation: 鹿児島大学(鹿児島県)
  • Year and Date: 2010-03-15
• [Presentation] Human Head Modeling based on Fast Automatic Mesh Completion (2009)
  • Author(s): Akinobu Maejima, Shigeo Morishima
  • Organizer: ACM SIGGRAPHASIA2009
  • Place of Presentation: パシフィコ横浜
  • Year and Date: 2009-12-18
• [Presentation] Human Head Modeling based on Fast Automatic Mesh Completion (2009)
  • Author(s): Akinobu Maejima, Shigeo Morishima
  • Organizer: ACM SIGGRAPHASIA 2009
  • Place of Presentation: パシフィコ横浜(神奈川県)
  • Year and Date: 2009-12-18
• [Presentation] 事前知識に基づく正面顔画像からの3次元形状復元 (2009)
  • Author(s): 前島謙宣, 森島繁生
  • Organizer: ビジュアルコンピューティングワークショップ2009
  • Place of Presentation: 辰口温泉
  • Year and Date: 2009-11-13
• [Presentation] 事前知識に基づく正面顔画像からの3次元形状復元 (2009)
  • Author(s): 前島謙宣、森島繁生
  • Organizer: ビジュアルコンピューティングワークショップ2009
  • Place of Presentation: 辰口温泉(石川県)
  • Year and Date: 2009-11-13
• [Presentation] An individual character head modeling for instant casting (2009)
  • Author(s): Akinobu Maejima, Takanori Suzuki, Hiroto Yarimizu, Hiroyuki Kubo, Shigeo Morishima
  • Organizer: The 9^<th> Asian Conference on Computer Vision, In vited workshop1 : vision based human modeling and synthesis
  • Place of Presentation: Xian, China
  • Year and Date: 2009-09-24
• [Presentation] An individual character head modeling for instant casting (2009)
  • Author(s): Akinobu Maejima, Takanori Suzuki, Hiroto Yarimizu, Hiroyuki Kubo, Shigeo Morisima
  • Organizer: The 9^<th> Asian Conference on Computer Vision, Invited Workshop1 : Vision Based Human Modeling and Synthesis
  • Place of Presentation: Xian (China)
  • Year and Date: 2009-09-24
• [Remarks]
  • URL: https://sites.google.com/site/maejimalaboratory
• [Remarks] Youtube : DigiInfo Technology from Japan : Fast, autmatic generation of 3D face models from photos
  • URL: http://www.youtube.com/watch?v=EGMTWL9QihM
• [Remarks] 第13回文化庁メディア芸術祭協賛展,先端技術ショーケース' 10
  • URL: http://www.media.jst.go.jp/research_result/sentan10/leaflet.pdf
• [Remarks]
  • URL: http://www.youtube.com/watch?v=EGMTWL9QihM
• [Remarks]
  • URL: http://www.media.jst.go.jp/research_result/sentan10/leaflet.pdf