• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Procedural text generation from execution video

Research Project

Project/Area Number 26280084
Research Category

Grant-in-Aid for Scientific Research (B)

Allocation TypePartial Multi-year Fund
Section一般
Research Field Intelligent informatics
Research InstitutionKyoto University

Principal Investigator

Shinsuke Mori  京都大学, 学術情報メディアセンター, 教授 (90456773)

Co-Investigator(Kenkyū-buntansha) 飯山 将晃  京都大学, 学術情報メディアセンター, 准教授 (70362415)
橋本 敦史  京都大学, 教育学研究科, 助教 (80641753)
舩冨 卓哉  奈良先端科学技術大学院大学, 情報科学研究科, 准教授 (20452310)
Project Period (FY) 2014-04-01 – 2018-03-31
Project Status Completed (Fiscal Year 2017)
Budget Amount *help
¥15,990,000 (Direct Cost: ¥12,300,000、Indirect Cost: ¥3,690,000)
Fiscal Year 2017: ¥1,950,000 (Direct Cost: ¥1,500,000、Indirect Cost: ¥450,000)
Fiscal Year 2016: ¥3,640,000 (Direct Cost: ¥2,800,000、Indirect Cost: ¥840,000)
Fiscal Year 2015: ¥4,680,000 (Direct Cost: ¥3,600,000、Indirect Cost: ¥1,080,000)
Fiscal Year 2014: ¥5,720,000 (Direct Cost: ¥4,400,000、Indirect Cost: ¥1,320,000)
Keywords自然言語生成 / レシピ / 調理映像 / ニューラルネット / 自然言語処理 / 映像処理 / 手順書 / 実施映像 / メディア情報処理 / 画像処理 / 深層学習 / 情報検索 / 文生成 / 人工知能 / 手順文書 / フローグラフ
Outline of Final Research Achievements

The recipe flow graph corpus ("Flow Graph Corpus from Recipe Texts," Shinsuke Mori, Hirokuni Maeta, Yoko Yamakata, Tetsuro Sasada, LREC, pp.2370-2377, 2014.) and the cooking video record ("KUSK Dataset: Toward a Direct Understanding of Recipe Text and Human Cooking Activity Information," Atsushi Hashimoto, Tetsuro Sasada, Yoko Yamakata, Shinsuke Mori, Michihiko Minoh, CEA, 2014.) that we finished to build last year were used for automatic recipe generation from cooking video as a typical example of our proposal. This result has been presented in a international conference ("Procedural Text Generation from an Execution Video," Atsushi Ushiku, Hayato Hashimoto, Atsushi
Hashimoto, Shinsuke Mori, IJCNLP, 2017.). The method is enough general to be applied to various execution videos. Now we are writing a jounal paper based on these. After having completed the proposal, we are working on its extensions such as multimedia text generation.

Report

(5 results)
  • 2017 Annual Research Report   Final Research Report ( PDF )
  • 2016 Annual Research Report
  • 2015 Annual Research Report
  • 2014 Annual Research Report
  • Research Products

    (39 results)

All 2018 2017 2016 2015 2014 Other

All Journal Article (7 results) (of which Int'l Joint Research: 1 results,  Peer Reviewed: 7 results,  Open Access: 7 results,  Acknowledgement Compliant: 7 results) Presentation (21 results) (of which Int'l Joint Research: 9 results,  Invited: 1 results) Remarks (11 results)

  • [Journal Article] 手順文書からの意味構造抽出2017

    • Author(s)
      前田 浩邦, 山肩 洋子, 森 信介
    • Journal Title

      人工知能学会論文誌

      Volume: 32 Pages: 1-8

    • NAID

      130005286766

    • Related Report
      2016 Annual Research Report
    • Peer Reviewed / Open Access / Acknowledgement Compliant
  • [Journal Article] Cooking Recipe Search by Pairs of Ingredient and Action - Word Sequence v.s. Flow-graph Representation -2017

    • Author(s)
      Yoko Yamakata, Hirokuni Maeta, Takuya Kadowaki, Tetsuro Sasada, Shinji Imahori, Shinsuke Mori
    • Journal Title

      Transactions of the Japanese Society for Artificial Intelligenc

      Volume: 32 Pages: 1-9

    • NAID

      130005439848

    • Related Report
      2016 Annual Research Report
    • Peer Reviewed / Open Access / Acknowledgement Compliant
  • [Journal Article] A Comparative Study of Dictionaries and Corpora as Methods for Language Resource Addition2016

    • Author(s)
      Shinsuke Mori, Neubig Graham
    • Journal Title

      LRE

      Volume: 50 Pages: 245-261

    • Related Report
      2016 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research / Acknowledgement Compliant
  • [Journal Article] フローグラフからの手順書の生成2016

    • Author(s)
      山崎 健史, 吉野 幸一郎, 前田 浩邦, 笹田 鉄郎, 橋本 敦史, 舩冨 卓哉, 山肩 洋子, 森 信介
    • Journal Title

      情報処理学会論文誌

      Volume: 57 Pages: 849-862

    • NAID

      170000130900

    • Related Report
      2015 Annual Research Report
    • Peer Reviewed / Open Access / Acknowledgement Compliant
  • [Journal Article] ワークフロー表現を用いたレシピの典型性評価と典型的なレシピの生成2016

    • Author(s)
      山肩 洋子, 笹田 鉄郎, 森 信介
    • Journal Title

      電子通信学会論文誌

      Volume: J99-D Pages: 378-391

    • Related Report
      2015 Annual Research Report
    • Peer Reviewed / Open Access / Acknowledgement Compliant
  • [Journal Article] レシピ用語の定義とその自動認識のためのタグ付与コーパスの構築2015

    • Author(s)
      笹田 鉄郎, 森 信介, 山肩 洋子, 前田 浩邦, 河原 達也
    • Journal Title

      自然言語処理

      Volume: 22 Pages: 107-131

    • NAID

      130005099405

    • Related Report
      2015 Annual Research Report
    • Peer Reviewed / Open Access / Acknowledgement Compliant
  • [Journal Article] 誕生・使用事由によるレシピ検索~生い立ちレシピサーチ~2014

    • Author(s)
      門脇 拓也, 山肩 洋子, 森 信介, 田中 克己
    • Journal Title

      日本データベース学会和文論文誌

      Volume: 13-J Pages: 78-85

    • NAID

      40020275536

    • Related Report
      2014 Annual Research Report
    • Peer Reviewed / Open Access / Acknowledgement Compliant
  • [Presentation] 日本語の単語分割と品詞推定 あるいはKyTeaの話2018

    • Author(s)
      森 信介
    • Organizer
      言語処理学会年次大会ワークショップ
    • Related Report
      2017 Annual Research Report
  • [Presentation] 調理動画からのレシピの自動生成2017

    • Author(s)
      牛 久敦, 橋本 敦史, 森 信介
    • Organizer
      言語処理学会第23回年次大会
    • Place of Presentation
      筑波大学 (つくば市)
    • Year and Date
      2017-03-14
    • Related Report
      2016 Annual Research Report
  • [Presentation] A Comparison of Cooking Recipe Named Entities between Japanese and English2017

    • Author(s)
      Yoko Yamakata, John Carroll, Shinsuke Mori
    • Organizer
      CEA
    • Related Report
      2017 Annual Research Report
    • Int'l Joint Research
  • [Presentation] 教師なし学習による分野特有の固有表現認識2017

    • Author(s)
      友利 涼, 森 信介
    • Organizer
      NLP若手の会 第12回シンポジウム
    • Related Report
      2017 Annual Research Report
  • [Presentation] Procedural Text Generation from an Execution Video2017

    • Author(s)
      Atsushi Ushiku, Hayato Hashimoto, Atsushi Hashimoto, Shinsuke Mori
    • Organizer
      IJCNLP
    • Related Report
      2017 Annual Research Report
    • Int'l Joint Research
  • [Presentation] 分野特有の教師なし固有表現認識2017

    • Author(s)
      友利 涼, 森 信介
    • Organizer
      情報処理学会自然言語処理研究会
    • Related Report
      2017 Annual Research Report
  • [Presentation] Domain Specific Named Entity Recognition Referring to the Real World by Deep Neural Networks2016

    • Author(s)
      Suzushi Tomori, Takashi Ninomiya, Shinsuke Mori
    • Organizer
      ACL
    • Place of Presentation
      Germany (Berlin)
    • Year and Date
      2016-08-07
    • Related Report
      2016 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Design in Everyday Cooking: Challenges for Assisting with Menu Planning and Food Preparation2016

    • Author(s)
      Atsushi Hashimoto, Jun Harashima, Yoko Yamakata, Shinsuke Mori
    • Organizer
      HCII
    • Place of Presentation
      Canada (Toronto)
    • Year and Date
      2016-07-17
    • Related Report
      2016 Annual Research Report
    • Int'l Joint Research
  • [Presentation] 自然言語処理によるテキスト理解に向けて2016

    • Author(s)
      森 信介
    • Organizer
      産業技術総合研究所
    • Place of Presentation
      大阪
    • Year and Date
      2016-02-23
    • Related Report
      2015 Annual Research Report
    • Invited
  • [Presentation] A Method for Extracting Major Workflow composed of Ingredients, Tools, and Actions from Cooking Procedural Text2016

    • Author(s)
      Yoko Yamakata, Shinji Imahori, Hirokuni Maeta, Shinsuke Mori
    • Organizer
      CEA
    • Place of Presentation
      USA (Seattle)
    • Related Report
      2016 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Kusk Object Dataset: Recording Access to Objects in Food Preparation2016

    • Author(s)
      Atsushi Hashimoto, Masaaki Iiyama, Shinsuke Mori, Michihiko Minoh
    • Organizer
      CEA
    • Place of Presentation
      USA (Seattle)
    • Related Report
      2016 Annual Research Report
    • Int'l Joint Research
  • [Presentation] 料理レシピデータセットを用いた料理用語辞書の自動構築-フローグラフから得られる加工履歴に基づく食材名の分布類似度計算に関する検討-2016

    • Author(s)
      山肩 洋子, 難波 英嗣, 大須賀 智子, 原島 純, 但馬 康宏, 森 信介, 大山 敬三
    • Organizer
      電子情報通信学会 食メディア研究会
    • Place of Presentation
      国立情報学研究所 (東京)
    • Related Report
      2016 Annual Research Report
  • [Presentation] KUSK Object Dataset: 調理作業中の物体への接触履歴データセットの作成2015

    • Author(s)
      橋本 敦史, 飯山 将晃, 森 信介, 美濃 導彦
    • Organizer
      電子情報通信学会 データ工学研究会
    • Place of Presentation
      東京
    • Year and Date
      2015-09-25
    • Related Report
      2015 Annual Research Report
  • [Presentation] A Framework for Procedural Text Understanding2015

    • Author(s)
      Hirokuni Maeta, Tetsuro Sasada, Shinsuke Mori
    • Organizer
      IWPT
    • Place of Presentation
      ビルバオ
    • Year and Date
      2015-07-22
    • Related Report
      2015 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Clustering Scenes in Cooking Video Guided by Object Access2015

    • Author(s)
      Yuki Matsumura, Atsushi Hashimoto, Shinsuke Mori, Masayuki Mukunoki, Michihiko Minoh
    • Organizer
      CEA
    • Place of Presentation
      トリノ
    • Year and Date
      2015-07-03
    • Related Report
      2015 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Named Entity Recognizer Trainable from Partially Annotated Data2015

    • Author(s)
      Tetsuro Sasada, Shinsuke Mori, Tatsuya Kawahara and Yoko Yamakata
    • Organizer
      PACLING
    • Place of Presentation
      バリ
    • Year and Date
      2015-05-19
    • Related Report
      2015 Annual Research Report
    • Int'l Joint Research
  • [Presentation] フローグラフからのレシピ文自動生成2014

    • Author(s)
      山崎 健史, 森 信介, 河原 達也
    • Organizer
      情報処理学会自然言語処理研究会
    • Place of Presentation
      東工大(神奈川・横浜)
    • Year and Date
      2014-12-17
    • Related Report
      2014 Annual Research Report
  • [Presentation] A Framework for Recipe Text Interpretation2014

    • Author(s)
      Hirokuni Maeta, Tetsuro Sasada, Shinsuke Mori
    • Organizer
      Cooking and Eating Activity
    • Place of Presentation
      シアトル(USA)
    • Year and Date
      2014-09-14
    • Related Report
      2014 Annual Research Report
  • [Presentation] KUSK Dataset: Toward a Direct Understanding of Recipe Text and Human Cooking Activity Information2014

    • Author(s)
      Atsushi Hashimoto, Tetsuro Sasada, Yoko Yamakata, Shinsuke Mori, Michihiko Minoh
    • Organizer
      Cooking and Eating Activity
    • Place of Presentation
      シアトル(USA)
    • Year and Date
      2014-09-14
    • Related Report
      2014 Annual Research Report
  • [Presentation] FlowGraph2Text: Automatic Sentence Skeleton Compilation for Procedural Text Generation2014

    • Author(s)
      Shinsuke Mori, Hirokuni Maeta, Tetsuro Sasada, Koichiro Yoshino, Atsushi Hashimoto, Takuya Funatomi, Yoko Yamakata
    • Organizer
      International Conference of Natural Language Generation
    • Place of Presentation
      フィラデルフィア(USA)
    • Year and Date
      2014-06-19
    • Related Report
      2014 Annual Research Report
  • [Presentation] Flow Graph Corpus from Recipe Texts2014

    • Author(s)
      Shinsuke Mori, Hirokuni Maeta, Yoko Yamakata, Tetsuro Sasada
    • Organizer
      Language Resource and Evaluation Conference
    • Place of Presentation
      レイキャビク(Iceland)
    • Year and Date
      2014-05-29
    • Related Report
      2014 Annual Research Report
  • [Remarks] 研究室業績リスト

    • URL

      http://www.ar.media.kyoto-u.ac.jp/mori/research/public/main.html

    • Related Report
      2017 Annual Research Report
  • [Remarks] フローグラフコーパス (FG corpus)

    • URL

      http://www.ar.media.kyoto-u.ac.jp/data/recipe/

    • Related Report
      2017 Annual Research Report
  • [Remarks] KUSK Dataset

    • URL

      http://kusk.mm.media.kyoto-u.ac.jp/ja/

    • Related Report
      2017 Annual Research Report
  • [Remarks] レシピ言語処理マニュアル

    • URL

      http://www.ar.media.kyoto-u.ac.jp/how-to/recipe-NLP/

    • Related Report
      2017 Annual Research Report
  • [Remarks] フローグラフコーパス (FG corpus)

    • Related Report
      2016 Annual Research Report
  • [Remarks] レシピ言語処理マニュアル

    • Related Report
      2016 Annual Research Report
  • [Remarks] フローグラフコーパス (FG corpus)

    • URL

      http://plata.ar.media.kyoto-u.ac.jp/data/recipe/

    • Related Report
      2015 Annual Research Report
  • [Remarks] KUSK DATASET

    • URL

      http://kusk.mm.media.kyoto-u.ac.jp/ja/

    • Related Report
      2015 Annual Research Report
  • [Remarks] 固有表現認識器 PWNER

    • URL

      http://plata.ar.media.kyoto-u.ac.jp/tool/PWNER/

    • Related Report
      2015 Annual Research Report
  • [Remarks] レシピ言語処理マニュアル

    • URL

      http://plata.ar.media.kyoto-u.ac.jp/kadowaki/

    • Related Report
      2014 Annual Research Report
  • [Remarks] レシピ フロー グラフ コーパス (r-FG corpus)

    • URL

      http://plata.ar.media.kyoto-u.ac.jp/mori/research/NLR/FGC/main.html

    • Related Report
      2014 Annual Research Report

URL: 

Published: 2014-04-04   Modified: 2019-03-29  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi