Procedural text generation from execution video

2017 Fiscal Year Final Research Report

• Project/Area Number: 26280084
• Research Category: Grant-in-Aid for Scientific Research (B)
• Allocation Type: Partial Multi-year Fund
• Section: 一般
• Research Field: Intelligent informatics
• Research Institution: Kyoto University
• Principal Investigator: Shinsuke Mori 京都大学, 学術情報メディアセンター, 教授 (90456773)
• Co-Investigator(Kenkyū-buntansha): 飯山 将晃 京都大学, 学術情報メディアセンター, 准教授 (70362415) ; 橋本 敦史 京都大学, 教育学研究科, 助教 (80641753) ; 舩冨 卓哉 奈良先端科学技術大学院大学, 情報科学研究科, 准教授 (20452310)
• Project Period (FY): 2014-04-01 – 2018-03-31
• Keywords: 自然言語生成 / レシピ / 調理映像 / ニューラルネット

Outline of Final Research Achievements

The recipe flow graph corpus ("Flow Graph Corpus from Recipe Texts," Shinsuke Mori, Hirokuni Maeta, Yoko Yamakata, Tetsuro Sasada, LREC, pp.2370-2377, 2014.) and the cooking video record ("KUSK Dataset: Toward a Direct Understanding of Recipe Text and Human Cooking Activity Information," Atsushi Hashimoto, Tetsuro Sasada, Yoko Yamakata, Shinsuke Mori, Michihiko Minoh, CEA, 2014.) that we finished to build last year were used for automatic recipe generation from cooking video as a typical example of our proposal. This result has been presented in a international conference ("Procedural Text Generation from an Execution Video," Atsushi Ushiku, Hayato Hashimoto, Atsushi
Hashimoto, Shinsuke Mori, IJCNLP, 2017.). The method is enough general to be applied to various execution videos. Now we are writing a jounal paper based on these. After having completed the proposal, we are working on its extensions such as multimedia text generation.

Free Research Field

自然言語処理