Efficient Data Augmentation of Household Behavior with Simulation in Virtual Environments

2021 Fiscal Year Final Research Report

• Project/Area Number: 19K20349
• Research Category: Grant-in-Aid for Early-Career Scientists
• Allocation Type: Multi-year Fund
• Review Section: Basic Section 61030:Intelligent informatics-related
• Research Institution: The University of Tokyo (2020-2021); National Institute of Informatics (2019)
• Principal Investigator: GOUTSU Yusuke 東京大学, 生産技術研究所, 特任研究員 (80816827)
• Project Period (FY): 2019-04-01 – 2022-03-31
• Keywords: 動作認識 / 行動認識 / 言語生成 / 身体動作 / 系列変換 / 敵対的学習 / ニューラルネットワーク

Outline of Final Research Achievements

We have tackled our research related to a fusion of human behavior and language with a sequence-to-sequence translation model, in which human whole-body movement and text are considered as the same sequential data consisting of postures and words respectively, and the main research topic is linguistic description of human motion. Specifically, our approach incorporates a framework for evaluating the validity of entire sequence that has reached the final state through search of sequence elements, and uses the results to train the translation model. This is quite different from previous approaches that evaluate the element-wise prediction sequentially. By reducing the prediction error for the generation of long sequences, not only "walk" but also "walk forwards a few steps" or "walk a quarter circle clockwise" describing observed human motion in detail can be appropriately generated.

Free Research Field

コンピュータビジョン

Academic Significance and Societal Importance of the Research Achievements

人間の動作を詳細に記述できるということは，動作の細かな差異とそれに対応するテキストの関係を捉えられる，即ち行動と言語の高度な表現関係まで取り扱えるようになったことを意味する．これにより，例えばスポーツ解析などにおいて，上級プレイヤーの熟練された動作を予め学習しておくことで初級プレイヤーとの差異を指摘し，更にはどのように動作を修正すれば上達できるかを助言するなどの動作支援への応用に繋がる．このことは，周りに熟練者がいなくてもシステムとのインタラクションを通してコーチングを受けることができるという点で社会的に非常に重要である．