• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Understanding Human Activity in Kitchen via Food, Tool, and Action Recognition

Research Project

Project/Area Number 24240030
Research Category

Grant-in-Aid for Scientific Research (A)

Allocation TypeSingle-year Grants
Section一般
Research Field Perception information processing/Intelligent robotics
Research InstitutionKyoto University

Principal Investigator

Minoh Michihiko  京都大学, 学術情報メディアセンター, 教授 (70166099)

Co-Investigator(Kenkyū-buntansha) MUKUNOKI Masayuki  京都大学, 学術情報メディアセンター, 准教授 (20283640)
MORI Shinsuke  京都大学, 学術情報メディアセンター, 准教授 (90456773)
YAMAKATA Yoko  京都大学, 情報学研究科, 准教授 (60423018)
FUNATOMI Takuya  京都大学, 学術情報メディアセンター, 助教 (20452310)
NAKAMURA Kazuaki  京都大学, 法学(政治学)研究科(研究院), 助手 (10584047)
Atsushi Hashimoto  京都大学, 法学(政治学)研究科(研究院), 助手 (80641753)
飯山 将晃  京都大学, 学術情報メディアセンター, 准教授 (70362415)
Co-Investigator(Renkei-kenkyūsha) NAKAMURA Kazuaki  大阪大学, 工学(系)研究科(研究院), 助教 (10584047)
FUNATOMI Takuya  奈良先端科学技術大学院大学, 情報科学研究科, 准教授 (20452310)
Project Period (FY) 2012-04-01 – 2016-03-31
Project Status Completed (Fiscal Year 2015)
Budget Amount *help
¥46,150,000 (Direct Cost: ¥35,500,000、Indirect Cost: ¥10,650,000)
Fiscal Year 2015: ¥11,570,000 (Direct Cost: ¥8,900,000、Indirect Cost: ¥2,670,000)
Fiscal Year 2014: ¥13,000,000 (Direct Cost: ¥10,000,000、Indirect Cost: ¥3,000,000)
Fiscal Year 2013: ¥10,790,000 (Direct Cost: ¥8,300,000、Indirect Cost: ¥2,490,000)
Fiscal Year 2012: ¥10,790,000 (Direct Cost: ¥8,300,000、Indirect Cost: ¥2,490,000)
Keywords映像,文章,音声等認識 / 情報図書館学 / スマートセンサ情報システム / 知能ロボティクス / 行動認識 / 画像,文章,音声等認識 / 画像、文章、音声等認識
Outline of Final Research Achievements

This study aimed to understand human activity in kitchen in which various tools are used and decision-making is required; the activity represents human intelligence that separates us from other animals. In“visible actions” in observations, we focused on object grabbing and releasing actions, and developed a method to recognize completion of “invisible activities” that are linguistically described in recipe text and to forecast the forthcoming action intended by humans. We also developed several methods to extract “invisible activities” in recipe texts as workflow graph with its word class (i.e. food, tools, food-states and so on).

Report

(5 results)
  • 2015 Annual Research Report   Final Research Report ( PDF )
  • 2014 Annual Research Report
  • 2013 Annual Research Report
  • 2012 Annual Research Report
  • Research Products

    (36 results)

All 2016 2015 2014 2012 Other

All Journal Article (6 results) (of which Peer Reviewed: 5 results,  Open Access: 3 results,  Acknowledgement Compliant: 4 results) Presentation (28 results) (of which Int'l Joint Research: 2 results) Remarks (2 results)

  • [Journal Article] フローグラフからの手順書の生成2016

    • Author(s)
      山崎 健史, 吉野 幸一郎, 前田 浩邦, 笹田 鉄郎, 橋本 敦史, 舩冨 卓哉, 山肩 洋子, 森 信介
    • Journal Title

      情報処理学会論文誌

      Volume: 57巻3号 Pages: 849-862

    • NAID

      170000130900

    • Related Report
      2015 Annual Research Report
    • Peer Reviewed / Open Access / Acknowledgement Compliant
  • [Journal Article] Definition of Recipe Terms and Corpus Annotation for their Automatic Recognition2015

    • Author(s)
      笹田 鉄郎, 森 信介, 山肩 洋子, 前田 浩邦, 河原 達也
    • Journal Title

      Journal of Natural Language Processing

      Volume: 22 Issue: 2 Pages: 107-131

    • DOI

      10.5715/jnlp.22.107

    • NAID

      130005099405

    • ISSN
      1340-7619, 2185-8314
    • Related Report
      2015 Annual Research Report
    • Peer Reviewed / Open Access / Acknowledgement Compliant
  • [Journal Article] Development and Evaluation of Near Real-Time Automated System for Measuring Consumption of Seasonings2015

    • Author(s)
      Kazuaki NAKAMURA, Takuya FUNATOMI, Atsushi HASHIMOTO, Mayumi UEDA, Michihiko MINOH
    • Journal Title

      IEICE Transactions on Information and Systems

      Volume: E98.D Issue: 12 Pages: 2229-2241

    • DOI

      10.1587/transinf.2015EDP7072

    • NAID

      130005112334

    • ISSN
      0916-8532, 1745-1361
    • Related Report
      2015 Annual Research Report
    • Peer Reviewed / Open Access / Acknowledgement Compliant
  • [Journal Article] 食材認識のための画像と食材切断時の振動音及び荷重の利用2014

    • Author(s)
      井上仁, 橋本敦史, 中村和晃, 舩冨卓哉, 山肩洋子,上田真由美, 美濃導彦
    • Journal Title

      電子情報通信学会論文誌

      Volume: J97-D No.9 Pages: 1490-1502

    • Related Report
      2014 Annual Research Report
    • Peer Reviewed / Acknowledgement Compliant
  • [Journal Article] 机上物体検出を対象とした接触理由付けによる誤検出棄却2012

    • Author(s)
      橋本 敦史, 舩冨 卓哉, 中村 和晃, 美濃 導彦
    • Journal Title

      電子情報通信学会和文論文誌

      Volume: J95-D Pages: 2113-2123

    • NAID

      110009553853

    • Related Report
      2012 Annual Research Report
    • Peer Reviewed
  • [Journal Article] 自然言語処理における分野適応2012

    • Author(s)
      森 信介
    • Journal Title

      人工知能学会誌

      Volume: 27(4) Pages: 365-372

    • Related Report
      2012 Annual Research Report
  • [Presentation] 経路探索に基づく映像とワークフローの対応付け2016

    • Author(s)
      松村 優樹・橋本 敦史・森 信介・舩冨 卓哉・飯山 将晃・美濃 導彦
    • Organizer
      電子情報通信学会マルチメディア仮想環境基礎研究会
    • Place of Presentation
      名桜大学(沖縄県名護市)
    • Year and Date
      2016-03-07
    • Related Report
      2015 Annual Research Report
  • [Presentation] KUSK Object Dataset:調理作業中の物体への接触履歴データセットの作成2015

    • Author(s)
      橋本 敦史, 飯山 将晃, 森 信介, 美濃 導彦
    • Organizer
      電子情報通信学会データ工学研究会
    • Place of Presentation
      神奈川工科大学 横浜オフィス(神奈川県横浜市西区)
    • Year and Date
      2015-09-25
    • Related Report
      2015 Annual Research Report
  • [Presentation] A Framework for Procedural Text Understanding2015

    • Author(s)
      Hirokuni Maeta, Tetsuro Sasada, Shinsuke Mori
    • Organizer
      The 14th International Conference on Parsing Technologies
    • Place of Presentation
      The Auditorium of the University of the Basque Country, Bilbao, Spain
    • Year and Date
      2015-07-22
    • Related Report
      2015 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Clustering Scenes in Cooking Video Guided by Object Access Guidance?2015

    • Author(s)
      Yuki Matsumura, Atsushi Hashimoto, Shinsuke Mori, Masayuki Mukunoki, Michihiko Minoh
    • Organizer
      7th Workshop on Multimedia for Cooking and Eating Activities
    • Place of Presentation
      Torino Incontra, Centro Congressi, Torino, Italy
    • Year and Date
      2015-07-03
    • Related Report
      2015 Annual Research Report
    • Int'l Joint Research
  • [Presentation] 物体の把持解放を手がかりとした調理映像からの動作区間検出2015

    • Author(s)
      松村優樹, 橋本敦史,椋木雅之,美濃導彦
    • Organizer
      2015年電子情報通信学会総合大会
    • Place of Presentation
      立命館大学びわこ・くさつキャンパス(滋賀県草津市)
    • Year and Date
      2015-03-11
    • Related Report
      2014 Annual Research Report
  • [Presentation] 料理名が同じレシピ間の手順構造マッピングによる動作名オントロジーの導出2014

    • Author(s)
      山肩洋子, 今堀慎治, 笹田鉄郎, 森信介
    • Organizer
      電子情報通信学会 HCGシンポジウム
    • Place of Presentation
      海峡メッセ下関(山口県下関市)
    • Year and Date
      2014-12-19
    • Related Report
      2014 Annual Research Report
  • [Presentation] KUSK Dataset: Toward a Direct Understanding of Recipe Text and Human Cooking Activity2014

    • Author(s)
      Atsushi Hashimoto, Tetsuro Sasada, Yoko Yamakata, Shinsuke Mori, Michihiko Minoh
    • Organizer
      Workshop on Smart Technologyfor Cooking and Eating Activities (in conjunction with UbiComp2014)
    • Place of Presentation
      Motif Seattle Hotel(Seattle, USA)
    • Year and Date
      2014-09-14
    • Related Report
      2014 Annual Research Report
  • [Presentation] A Framework for Recipe Text Interpretation2014

    • Author(s)
      Hirokuni Maeta, Tetsuro Sasada, Shinsuke Mori
    • Organizer
      Workshop on Smart Technologyfor Cooking and Eating Activities (in conjunction with UbiComp2014)
    • Place of Presentation
      Motif Seattle Hotel(Seattle, USA)
    • Year and Date
      2014-09-14
    • Related Report
      2014 Annual Research Report
  • [Presentation] A Method for Detecting Gaze-required action while Cooking for Assisting Video Communication2014

    • Author(s)
      Yoko Yamakata, Asuka Miyazawa, Atsushi Hashimoto, Takuya Funatomi, Michihiko Minoh
    • Organizer
      Workshop on Smart Technologyfor Cooking and Eating Activities (in conjunction with UbiComp2014)
    • Place of Presentation
      Motif Seattle Hotel(Seattle, USA)
    • Year and Date
      2014-09-14
    • Related Report
      2014 Annual Research Report
  • [Presentation] 調理手順文書の自然言語解析結果からの食材・加工からなる作業ツリーの構築2014

    • Author(s)
      山肩洋子, 今堀慎治, 前田浩邦, 森 信介
    • Organizer
      データ工学研究会(DE)
    • Place of Presentation
      国立情報学研究所(東京都千代田区)
    • Year and Date
      2014-09-11
    • Related Report
      2014 Annual Research Report
  • [Presentation] How does User's Access to Object Make HCI Smooth in Recipe Guidance?2014

    • Author(s)
      Atsushi Hashimoto, Jin Inoue, Takuya Funatomi, Michihiko Minoh
    • Organizer
      HCI International 2014
    • Place of Presentation
      Creta Maris Resort and Convention Centre (Crete, Greece)
    • Year and Date
      2014-06-25
    • Related Report
      2014 Annual Research Report 2013 Annual Research Report
  • [Presentation] Language Resource Addition: Dictionary or Corpus?2014

    • Author(s)
      Shinsuke Mori, Graham Neubig
    • Organizer
      The 9th International Conference on Language Resources and Evaluation (LREC)
    • Place of Presentation
      the Harpa Conference Center (Reykjavik, Iceland)
    • Year and Date
      2014-05-29
    • Related Report
      2014 Annual Research Report
  • [Presentation] Flow Graph Corpus from Recipe Texts2014

    • Author(s)
      Shinsuke Mori, Hirokuni Maeta, Yoko Yamakata, Tetsuro Sasada
    • Organizer
      The 9th International Conference on Language Resources and Evaluation (LREC)
    • Place of Presentation
      the Harpa Conference Center (Reykjavik, Iceland)
    • Year and Date
      2014-05-28
    • Related Report
      2014 Annual Research Report
  • [Presentation] Flow Graph Corpus from Recipe Texts

    • Author(s)
      Shinsuke Mori, Hirokuni Maeta, Yoko Yamakata, Tetsuro Sasada
    • Organizer
      The 9th International Conference on Language Resources and Evaluation
    • Place of Presentation
      the Harpa Conference Center (Reykjavik, Iceland)
    • Related Report
      2013 Annual Research Report
  • [Presentation] 誕生・使用事由によるレシピ検索 ~ 生い立ちレシピサーチ ~

    • Author(s)
      門脇拓也, 山肩洋子, 森 信介, 田中克己
    • Organizer
      第6回データ工学と情報マネジメントに関するフォー ラム(DEIM)
    • Place of Presentation
      淡路夢舞台&ウェスティン淡路(兵庫県)
    • Related Report
      2013 Annual Research Report
  • [Presentation] Feature Extraction and Summarization of Recipes Using Flow Graph

    • Author(s)
      Yoko Yamakata, Shinji Imahori, Yuichi Sugiyama, Shinsuke Mori, Katsumi Tanaka
    • Organizer
      The 5th International Conference on Social Informatics
    • Place of Presentation
      京都大学百周年時計台記念館(京都府)
    • Related Report
      2013 Annual Research Report
  • [Presentation] レシピテキストのためのフローグラフの定義

    • Author(s)
      森 信介, 山肩 洋子, 笹田 鉄郎, 前田 浩邦
    • Organizer
      情報処理学会自然言語処理研究会
    • Place of Presentation
      屋久島 環境文化村センター(鹿児島県)
    • Related Report
      2013 Annual Research Report
  • [Presentation] Detecting start and end times of object-handlings on a table by fusion of camera and load sensors

    • Author(s)
      Ryuta Yasuoka, Atsushi Hashimoto, Takuya Funatomi, Michihiko Minoh
    • Organizer
      The 5th Workshop on Multimedia for Cooking and Eating Activities
    • Place of Presentation
      Centre Convencions Internacional Barcelona, Universitat Pompeu Fabra (Barcelona, Spain)
    • Related Report
      2013 Annual Research Report
  • [Presentation] レシピテキストからのフローグラフコーパス作成

    • Author(s)
      前田 浩邦, 山肩 洋子, 森 信介
    • Organizer
      電子情報通信学会 データ工学研究会
    • Place of Presentation
      国立情報学研究所(東京都)
    • Related Report
      2013 Annual Research Report
  • [Presentation] 荷重センサを用いたインタラクション検出

    • Author(s)
      安岡竜太,橋本敦史,舩冨卓哉,美濃導彦
    • Organizer
      電子情報通信学会食メディア研究会2013年8月研究会
    • Place of Presentation
      公立はこだて未来大学(北海道)
    • Related Report
      2013 Annual Research Report
  • [Presentation] 見え特徴と荷重バランス特徴を用いた冷蔵庫における食材管理手法

    • Author(s)
      加茂田 玲奈, 上田 真由美, 舩冨 卓哉, 飯山 将晃, 美濃 導彦
    • Organizer
      第19回画像センシングシンポジウム(SSII2013)
    • Place of Presentation
      パシフィコ横浜(神奈川県)
    • Related Report
      2013 Annual Research Report
  • [Presentation] 調理者の手と容器の位置関係を利用した「かき混ぜる」行動の認識

    • Author(s)
      宮澤 飛鳥, 中村 和晃, 橋本 敦史, 舩冨 卓哉, 美濃 導彦
    • Organizer
      電子情報通信学会 データ工学研究会(DE)
    • Place of Presentation
      国立情報学研究所(東京都)
    • Related Report
      2012 Annual Research Report
  • [Presentation] Grocery Re-identificationusing Load Balance Feature on the Shelffor Monitoring Grocery Inventory

    • Author(s)
      Rena Kamoda, Mayumi Ueda, Takuya Funatomi, Masaaki Iiyama, Michihiko Minoh
    • Organizer
      Proceedings of the ECAI-Workshop on Cooking with Computers
    • Place of Presentation
      University of Science and Letters (Montpellier, France)
    • Related Report
      2012 Annual Research Report
  • [Presentation] A Machine Learning Approach to Recipe Text Processing

    • Author(s)
      Shinsuke Mori, Tetsuro Sasada, Yoko Yamakata, Koichiro Yoshino
    • Organizer
      Proceedings of the ECAI-Workshop on Cooking with Computers
    • Place of Presentation
      University of Science and Letters (Montpellier, France)
    • Related Report
      2012 Annual Research Report
  • [Presentation] TexCutとパッチ型背景モデルの組み合わせによる机上物体検出システム

    • Author(s)
      橋本 敦史, 中村 和晃, 舩冨 卓哉, 美濃 導彦
    • Organizer
      画像の認識・理解シンポジウム
    • Place of Presentation
      福岡国際会議場(福岡県)
    • Related Report
      2012 Annual Research Report
  • [Presentation] Recognizing Ingredients at Cutting Process by Integrating Multimodal Features

    • Author(s)
      Atsushi Hashimoto, Jin Inoue, Kazuaki Nakamura, Takuya Funatomi, Mayumi Ueda, Yoko Yamakata, Michihiko Minoh
    • Organizer
      The 4th Workshop on Multimedia for Cooking and Eating Activities
    • Place of Presentation
      奈良県新公会堂(奈良県)
    • Related Report
      2012 Annual Research Report
  • [Presentation] IwaCam: a Multimedia Processing Platform for Supporting Video-Based Cooking Communication

    • Author(s)
      Hidenori Tsuji, Yoko Yamakata, Takuya Funatomi, Hiromi Hiramatsu, Shinsuke Mori
    • Organizer
      International Conference on Future Generation Communication Technology
    • Place of Presentation
      British Computer Society, London, (United Kingdom)
    • Related Report
      2012 Annual Research Report
  • [Presentation] カメラと荷重センサの統合による机上物体に対するハンドリング開始・終了の検出

    • Author(s)
      安岡 竜太, 橋本 敦史, 舩冨 卓哉, 美濃 導彦
    • Organizer
      電子情報通信学会 マルチメディア・仮想環境基礎研究会
    • Place of Presentation
      福岡工業大学(福岡県)
    • Related Report
      2012 Annual Research Report
  • [Remarks] レシピ フロー グラフ コーパス (r-FG corpus)

    • URL

      http://plata.ar.media.kyoto-u.ac.jp/mori/research/NLR/FGC/main.html

    • Related Report
      2013 Annual Research Report
  • [Remarks] Kyoto University Smart Kitchen Dataset

    • URL

      http://kusk.mm.media.kyoto-u.ac.jp/

    • Related Report
      2013 Annual Research Report

URL: 

Published: 2013-05-15   Modified: 2019-07-29  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi