• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Study on facial expression recognition using thermal image processing and speech recognition in actual environment

Research Project

Project/Area Number 22300077
Research Category

Grant-in-Aid for Scientific Research (B)

Allocation TypeSingle-year Grants
Section一般
Research Field Sensitivity informatics/Soft computing
Research InstitutionKyoto Prefectural University

Principal Investigator

YOSHITOMI Yasunari  京都府立大学, 生命環境科学研究科(系), 教授 (70274769)

Co-Investigator(Kenkyū-buntansha) TABUSE Masayoshi  京都府立大学, 大学院生命環境科学研究科, 准教授 (10244188)
Research Collaborator ASADA Taro  京都府立大学, 大学院生命環境科学研究科, 特任准教授
KATO Ryota  京都府立大学, 大学院生命環境科学研究科, 特任研究員
Project Period (FY) 2010-04-01 – 2015-03-31
Project Status Completed (Fiscal Year 2014)
Budget Amount *help
¥17,940,000 (Direct Cost: ¥13,800,000、Indirect Cost: ¥4,140,000)
Fiscal Year 2013: ¥4,290,000 (Direct Cost: ¥3,300,000、Indirect Cost: ¥990,000)
Fiscal Year 2012: ¥4,030,000 (Direct Cost: ¥3,100,000、Indirect Cost: ¥930,000)
Fiscal Year 2011: ¥4,030,000 (Direct Cost: ¥3,100,000、Indirect Cost: ¥930,000)
Fiscal Year 2010: ¥5,590,000 (Direct Cost: ¥4,300,000、Indirect Cost: ¥1,290,000)
Keywords音声認識 / 表情認識 / 温度画像 / 正面判定 / 母音認識 / 表情強度 / テレビ電話
Outline of Final Research Achievements

We are missing the robot that can recognize facial expressions in taking care a person of advanced years and the robot that can talk with a person loving alone. The purpose of our study is to develop fundamental technologies which must be equipped with these robots for recognizing human state. It is assumed that the robot recognizes the facial expressions using thermal images at the following three timing-positions per utterance; just before speaking, in speaking the first and the last vowels. We have investigated the following issues in order; (1) facial expression recognition in speaking one word, (2) front-view judgment using thermal image, (3) parallel processing system, (4) performance of our system in actual environment, and (5) finding the next issues to be challenged. We have developed several methods and systems useful for designing the robot that can recognize facial expressions using the real-time processing in the natural conversation with human.

Report

(5 results)
  • 2014 Final Research Report ( PDF )
  • 2013 Annual Research Report
  • 2012 Annual Research Report
  • 2011 Annual Research Report
  • 2010 Annual Research Report
  • Research Products

    (43 results)

All 2015 2014 2013 2012 2011 2010 Other

All Journal Article (22 results) (of which Peer Reviewed: 19 results,  Acknowledgement Compliant: 9 results,  Open Access: 4 results) Presentation (15 results) (of which Invited: 2 results) Book (2 results) Remarks (4 results)

  • [Journal Article] Facial Expression Recognition Using Facial Expression Intensity Characteristics of Thermal Image2015

    • Author(s)
      Y.Yoshitomi, T.Asada, R.Kato, and M.Tabuse
    • Journal Title

      Journal of Robotics, Networking and Artificial Life

      Volume: 2

    • Related Report
      2013 Annual Research Report
    • Peer Reviewed / Open Access / Acknowledgement Compliant
  • [Journal Article] Facial Expression Recognition Using Thermal Image Processing and Efficient Preparation of Training-data2015

    • Author(s)
      Y.Nakanishi, Y.Yoshitomi, T.Asada, and M.Tabuse
    • Journal Title

      Journal of Robotics, Networking and Artificial Life

      Volume: 2

    • Related Report
      2013 Annual Research Report
    • Peer Reviewed / Open Access / Acknowledgement Compliant
  • [Journal Article] Facial Expression Recognition Using Facial Expression Intensity Characteristics of Thermal Image2015

    • Author(s)
      Y.Yoshitomi, T.Asada, R.Kato, and M.Tabuse
    • Journal Title

      Proc. of Int. Conf. on Artificial Life and Robotics

      Volume: なし Pages: 298-301

    • Related Report
      2013 Annual Research Report
    • Peer Reviewed / Open Access / Acknowledgement Compliant
  • [Journal Article] Facial Expression Recognition of a Speaker Using Thermal Image Processing and Reject Criteria in Feature Vector Space2014

    • Author(s)
      Y. Nakanishi, Y. Yoshitomi, T. Asada, and M. Tabuse
    • Journal Title

      Journal of Artificial Life and Robotics

      Volume: 19 Issue: 1 Pages: 76-88

    • DOI

      10.1007/s10015-013-0136-7

    • Related Report
      2013 Annual Research Report
    • Peer Reviewed / Acknowledgement Compliant
  • [Journal Article] Method of Facial Expression Analysis Using Video Phone and Thermal Image2014

    • Author(s)
      Y. Yoshitomi, T. Asada, R. Kato, and M. Tabuse
    • Journal Title

      Proc. of Int. Conf. on Artificial Life and Robotics

      Volume: なし Pages: 235-238

    • Related Report
      2013 Annual Research Report
    • Peer Reviewed / Acknowledgement Compliant
  • [Journal Article] Facial Expression Recognition Using Thermal Image Processing and Efficient Preparation of Training-data2014

    • Author(s)
      Y. Nakanishi, Y. Yoshitomi, T. Asada, and M. Tabuse
    • Journal Title

      Proc. of Int. Conf. on Artificial Life and Robotics

      Volume: なし Pages: 239-243

    • Related Report
      2013 Annual Research Report
    • Peer Reviewed / Acknowledgement Compliant
  • [Journal Article] Method of Facial Expression Analysis Using Video Phone and Thermal Image2014

    • Author(s)
      Y.Yoshitomi, T.Asada, R.Kato, and M.Tabuse
    • Journal Title

      Journal of Robotics, Networking and Artificial Life

      Volume: 1 Pages: 7-11

    • Related Report
      2013 Annual Research Report
    • Peer Reviewed / Open Access / Acknowledgement Compliant
  • [Journal Article] 温度画像処理と表情強度を用いた表情認識2014

    • Author(s)
      吉冨康成, 浅田太郎, 加藤亮太, 田伏正佳
    • Journal Title

      ヒューマンインタフェースシンポジウム2014論文集

      Volume: なし Pages: 797-800

    • Related Report
      2013 Annual Research Report
    • Acknowledgement Compliant
  • [Journal Article] 温度画像からの表情認識における学習簡略化の検討2013

    • Author(s)
      中西優, 吉冨康成, 浅田太郎, 田伏正佳
    • Journal Title

      ヒューマンインタフェース シンポジウム2013論文集

      Volume: なし Pages: 489-492

    • Related Report
      2013 Annual Research Report
    • Acknowledgement Compliant
  • [Journal Article] Robust Facial Expression Recognition of a Speaker Using Thermal Image Processing and Updating of Fundamental Training-data2013

    • Author(s)
      Y.Nakanishi, Y.Yoshitomi, T.Asada, and M.Tabuse
    • Journal Title

      Journal of Artificial Life and Robotics

      Volume: 17 Issue: 3-4 Pages: 342-349

    • DOI

      10.1007/s10015-012-0053-1

    • Related Report
      2012 Annual Research Report
    • Peer Reviewed
  • [Journal Article] A System for Facial Expression Recognition of a Speaker Using Thermal Image Processing and Feature Vector Space Characteristics2013

    • Author(s)
      T.Asada, Y.Yoshitomi, Y.Nakanishi and M.Tabuse
    • Journal Title

      Proc. of 18th Int. Symp. on Artificial Life and Robotics

      Volume: なし Pages: 180-185

    • Related Report
      2012 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Facial Expression Recognition of a Speaker Using Thermal Image Processing and Reject Criteria in Feature Vector Space2013

    • Author(s)
      Y.Nakanishi, Y.Yoshitomi, T.Asada, and M.Tabuse
    • Journal Title

      Proc. of 18th Int. Symp. on Artificial Life and Robotics

      Volume: なし Pages: 186-191

    • Related Report
      2012 Annual Research Report
    • Peer Reviewed
  • [Journal Article] A System for Facial Expression Recognition of a Speaker Using Front‐view Face Judgment, Vowel Judgment and Thermal Image Processing2012

    • Author(s)
      T.Asada, Y.Yoshitomi, and M.Tabuse
    • Journal Title

      Journal of Artificial Life and Robotics

      Volume: 17 Issue: 2 Pages: 263-269

    • DOI

      10.1007/s10015-012-0052-2

    • Related Report
      2012 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Facial Expression Recognition of Speaker Using Vowel Judgment and Features of Thermal Face Image2012

    • Author(s)
      Y.Yoshitomi
    • Journal Title

      Proc. of 1st WSEAS Int. Conf. on Information Technology and Computer Networks

      Volume: なし Pages: 139-145

    • Related Report
      2012 Annual Research Report
    • Peer Reviewed
  • [Journal Article] 正面顔判定と母音判定を用いた温度画像からの表情認識システム2012

    • Author(s)
      浅田太郎,吉冨康成,田伏正佳
    • Journal Title

      ヒューマンインタフェース シンポジウム2012論文集

      Volume: なし Pages: 521-522

    • Related Report
      2012 Annual Research Report
  • [Journal Article] A System for Facial Expression Recognition of a Speaker Using Front-view Face Judgment, Vowel Judgment and Thermal Image Processing2012

    • Author(s)
      T.Asada, Y.Yoshitomi, M.Tabuse
    • Journal Title

      Proc.of 17th Int.Symp.on Artificial Life and Robotics

      Pages: 202-209

    • Related Report
      2011 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Robust Facial Expression Recognition of a Speaker Using Thermal Image Processing and Updating of Fundamental Training-data2012

    • Author(s)
      Y.Nakanishi, Y.Yoshitomi, T.Asada, M.Tabuse
    • Journal Title

      Proc.of 17th Int.Symp.on Artificial Life and Robotics

      Pages: 210-215

    • Related Report
      2011 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Facial Expression Recognition of a Speaker Using Vowel Judgment and Thermal Image Processing2011

    • Author(s)
      Y.Yoshitomi, T.Asada, K.Shimada, M.Tabuse
    • Journal Title

      Journal of Artificial Life and Robotics

      Volume: VOL.16 Issue: 3 Pages: 318-323

    • DOI

      10.1007/s10015-011-0939-3

    • Related Report
      2011 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Facial Expression Recognition of a Speaker Using Front-view Face Judgment, Vowel Judgment and Thermal Image Processing2011

    • Author(s)
      T.Fujimura, Y.Yoshitomi, T.Asada, M.Tabuse
    • Journal Title

      Journal of Artificial Life and Robotics

      Volume: VOL.16 Issue: 3 Pages: 411-417

    • DOI

      10.1007/s10015-011-0967-z

    • Related Report
      2011 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Facial Expression Recognition of a Speaker Using Front-view Face Judgment, Vowel Judgment and Thermal Image Processing2011

    • Author(s)
      T.Fujimura, Y.Yoshitomi, T.Asada, M.Tabuse
    • Journal Title

      Proc.of 16th Int.Symp.on Artificial Life and Robotics

      Pages: 219-224

    • Related Report
      2010 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Facial Expression Recognition of a Speaker Using Vowel Judgment and Thermal Image Processing2011

    • Author(s)
      Y.Yoshitomi, T.Asada, K.Shimada, M.Tabuse
    • Journal Title

      Proc.of 16th Int.Symp.on Artificial Life and Robotics

      Pages: 225-230

    • Related Report
      2010 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Facial Expression Recognition for Speaker Using Thermal Image Processing and Speech Recognition System2010

    • Author(s)
      Y.Yoshitomi
    • Journal Title

      Proc.of 10th WSEAS Int.Conf.on Applied Computer Science

      Pages: 182-186

    • Related Report
      2010 Annual Research Report
    • Peer Reviewed
  • [Presentation] Facial Expression Recognition Using Facial Expression Intensity Characteristics of Thermal Image2015

    • Author(s)
      Y.Yoshitomi, T.Asada, R.Kato, and M.Tabuse
    • Organizer
      Int. Conf. on Artificial Life and Robotics
    • Place of Presentation
      ホルトホール(大分市)
    • Year and Date
      2015-01-10 – 2015-01-12
    • Related Report
      2013 Annual Research Report
  • [Presentation] 温度画像処理と表情強度を用いた表情認識2014

    • Author(s)
      吉冨康成, 浅田太郎, 加藤亮太, 田伏正佳
    • Organizer
      ヒューマンインタフェース シンポジウム2014
    • Place of Presentation
      京都工芸繊維大学(松ヶ崎キャンパス)
    • Year and Date
      2014-09-09 – 2014-09-12
    • Related Report
      2013 Annual Research Report
  • [Presentation] Method of Facial Expression Analysis Using Video Phone and Thermal Image2014

    • Author(s)
      Y. Yoshitomi, T. Asada, R. Kato, and M. Tabuse
    • Organizer
      Int. Conf. on Artificial Life and Robotics
    • Place of Presentation
      コンパルホール(大分市)
    • Year and Date
      2014-01-11 – 2014-01-13
    • Related Report
      2013 Annual Research Report
  • [Presentation] Facial Expression Recognition Using Thermal Image Processing and Efficient Preparation of Training-data2014

    • Author(s)
      Y. Nakanishi, Y. Yoshitomi, T. Asada, and M. Tabuse
    • Organizer
      Int. Conf. on Artificial Life and Robotics
    • Place of Presentation
      コンパルホール(大分市)
    • Year and Date
      2014-01-11 – 2014-01-13
    • Related Report
      2013 Annual Research Report
  • [Presentation] 皮膚の赤外線放射特性を活用した表情認識システム2013

    • Author(s)
      吉冨康成
    • Organizer
      生物の優れた機能から着想を得新しいもづくりシンポジウム
    • Place of Presentation
      京都工芸繊維大学(松ヶ崎キャンパス)
    • Year and Date
      2013-12-19
    • Related Report
      2013 Annual Research Report
    • Invited
  • [Presentation] 温度画像からの表情認識における学習簡略化の検討2013

    • Author(s)
      中西優, 吉冨康成, 浅田太郎, 田伏正佳
    • Organizer
      ヒューマンインタフェース シンポジウム2013
    • Place of Presentation
      早稲田大学(西早稲田キャンパス)
    • Year and Date
      2013-09-10 – 2013-09-13
    • Related Report
      2013 Annual Research Report
  • [Presentation] A System for Facial Expression Recognition of a Speaker Using Front-view Face Judgment, Vowel Judgment and Thermal Image Processing2012

    • Author(s)
      T.Asada, Y.Yoshitomi, M.Tabuse
    • Organizer
      17th Int.Symp.on Artificial Life and Robotics
    • Place of Presentation
      B-Con Plaza(大分県)
    • Year and Date
      2012-01-21
    • Related Report
      2011 Annual Research Report
  • [Presentation] Robust Facial Expression Recognition of a Speaker Using Thermal Image Processing and Updating of Fundamental Training-data2012

    • Author(s)
      Y.Nakanishi, Y.Yoshitomi, T.Asada, M.Tabuse
    • Organizer
      17th Int.Symp.on Artificial Life and Robotics
    • Place of Presentation
      B-Con Plaza(大分県)
    • Year and Date
      2012-01-21
    • Related Report
      2011 Annual Research Report
  • [Presentation] Facial Expression Recognition of a Speaker Using Front-view Face Judgment, Vowel Judgment and Thermal Image Processing2011

    • Author(s)
      T.Fujimura, Y.Yoshitomi, T.Asada, M.Tabuse
    • Organizer
      16th Int.Symp.on Artificial Life and Robotics
    • Place of Presentation
      B-Con Plaza(大分県)
    • Year and Date
      2011-01-28
    • Related Report
      2010 Annual Research Report
  • [Presentation] Facial Expression Recognition of a Speaker Using Vowel Judgment and Thermal Image Processing2011

    • Author(s)
      Y.Yoshitomi, T.Asada, K.Shimada, M.Tabuse
    • Organizer
      16th Int.Symp.on Artificial Life and Robotics
    • Place of Presentation
      B-Con Plaza(大分県)
    • Year and Date
      2011-01-28
    • Related Report
      2010 Annual Research Report
  • [Presentation] Facial Expression Recognition for Speaker Using Thermal Image Processing and Speech Recognition System2010

    • Author(s)
      Y.Yoshitomi
    • Organizer
      10th WSEAS International Conference on Applied Computer Science
    • Place of Presentation
      ホテル安比グランド(岩手県)(招待講演)
    • Year and Date
      2010-10-06
    • Related Report
      2010 Annual Research Report
  • [Presentation] Facial Expression Recognition of Speaker Using Vowel Judgment and Features of Thermal Face Image

    • Author(s)
      Y.Yoshitomi
    • Organizer
      1st WSEAS Int. Conf. on Information Technology and Computer Networks
    • Place of Presentation
      Imperial Riding School Renaissance Vienna Hotel (オーストリア、ウイーン市)
    • Related Report
      2012 Annual Research Report
    • Invited
  • [Presentation] A System for Facial Expression Recognition of a Speaker Using Thermal Image Processing and Feature Vector Space Characteristics

    • Author(s)
      T.Asada, Y.Yoshitomi, Y.Nakanishi and M.Tabuse
    • Organizer
      18th Int. Symp. on Artificial Life and Robotics
    • Place of Presentation
      Daejeon Convention Center (韓国、テジョン市)
    • Related Report
      2012 Annual Research Report
  • [Presentation] Facial Expression Recognition of a Speaker Using Thermal Image Processing and Reject Criteria in Feature Vector Space

    • Author(s)
      Y.Nakanishi, Y.Yoshitomi, T.Asada, and M.Tabuse
    • Organizer
      18th Int. Symp. on Artificial Life and Robotics
    • Place of Presentation
      Daejeon Convention Center (韓国、テジョン市)
    • Related Report
      2012 Annual Research Report
  • [Presentation] 正面顔判定と母音判定を用いた温度画像からの表情認識システム

    • Author(s)
      浅田太郎,吉冨康成,田伏正佳
    • Organizer
      ヒューマンインタフェース シンポジウム2012
    • Place of Presentation
      九州大学大橋キャンパス(福岡市)
    • Related Report
      2012 Annual Research Report
  • [Book] Image Processing: Methods, Applications and Challenges2012

    • Author(s)
      Y.Yoshitomi, M.Tabuse, T.Asada et al.
    • Total Pages
      250
    • Publisher
      Nova Science Publisher
    • Related Report
      2012 Annual Research Report
  • [Book] Speech Technology2011

    • Author(s)
      Y.Yoshitomi, T.Asada, M.Tabuse, et al
    • Total Pages
      432
    • Publisher
      InTech
    • Related Report
      2011 Annual Research Report
  • [Remarks] 情報伝達システム学研究室

    • URL

      http://www2.kpu.ac.jp/life_environ/info_com_sys/

    • Related Report
      2013 Annual Research Report
  • [Remarks] 情報伝達システム学研究室

    • URL

      http://www2.kpu.ac.jp/life_environ/info_com_sys/

    • Related Report
      2012 Annual Research Report
  • [Remarks]

    • URL

      http://www2.kpu.ac.jp/life_environ/info_com_sys/

    • Related Report
      2011 Annual Research Report
  • [Remarks]

    • URL

      http://www2.kpu.ac.jp/life_environ/info_com_sys/

    • Related Report
      2010 Annual Research Report

URL: 

Published: 2010-08-23   Modified: 2019-07-29  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi