Intelligent human interaction system based on mental cloning-based software

• Project/Area Number: 20300078
• Research Category: Grant-in-Aid for Scientific Research (B)
• Allocation Type: Single-year Grants
• Section: 一般
• Research Field: Sensitivity informatics/Soft computing
• Research Institution: Iwate Prefectural University
• Principal Investigator: FUJITA Hamido Iwate Prefectural University, ソフトウェア学部, 教授 (30244990)
• Co-Investigator(Kenkyū-buntansha): HAKURA Jun 岩手県立大学, ソフトウェア情報学部, 准教授 (30305289) ; KUREMATSU Masaki 岩手県立大学, ソフトウェア情報学部, 准教授 (00305286) ; ITOU Kenzo 岩手県立大学, ソフトウェア情報学部, 教授 (30305297) ; SASAKI Tamio 岩手県立大学, 社会福祉学部, 教授 (60105070)
• Project Period (FY): 2008 – 2010
• Project Status: Completed (Fiscal Year 2010)
• Budget Amount: ¥19,240,000 (Direct Cost: ¥14,800,000、Indirect Cost: ¥4,440,000); Fiscal Year 2010: ¥5,590,000 (Direct Cost: ¥4,300,000、Indirect Cost: ¥1,290,000); Fiscal Year 2009: ¥5,460,000 (Direct Cost: ¥4,200,000、Indirect Cost: ¥1,260,000); Fiscal Year 2008: ¥8,190,000 (Direct Cost: ¥6,300,000、Indirect Cost: ¥1,890,000)
• Keywords: 感性インタフェース / マルチモーダル / 感性情報学 / ユーザインタフェース

Research Abstract

We have established system that can use a mental model as an engagement for interfacing between man and machine. We have used image and voice of user to extract information to collect emotional characteristic of the user. We try to model user cognitive and mental issues to do. We call it Mental Cloning. This concept is used to achieve the best engagement between human and system. The system is also enrolled as an application that the user would engage through to achieve a mutual engagement with the user. The goal could have a service (e. g., medical) through cognitive based interaction.

Research Products

• [Journal Article] Virtual Doctor System (VDS) : Medical Decision Reasoning Based on Physical and Mental Ontologies (2010)
  • Author(s): Hamido Fujita, Jun Hakura, Masaki Kurematsu
  • Journal Title: IEA/AIE 2010 Part III, LNAI 6098 Vol.3 Pages: 419-428
  • Peer Reviewed
• [Journal Article] Virtual Doctor System (VDS) : Medical Decision Reasoning Based on Physical and Mental Ontologies (2010)
  • Author(s): Hamido Fujita, Jun Hakura, Masaki Kurematsu
  • Journal Title: IEA/AIE 2010 Part III, LNAI 6098 Volume: 3(通巻6098) Pages: 419-428
  • Peer Reviewed
• [Journal Article] Intelligent human interface based on mental cloning-based software (2009)
  • Author(s): Hamido Fujita, Jun HAKURA, Masaki KUREMATSU
  • Journal Title: International Journal on Knowledge-Based Systems Vol.22 Pages: 216-234
  • Peer Reviewed
• [Journal Article] Intelligent human interface based on mental cloning-based software (2009)
  • Author(s): Hamido Fujita, Jun HAKURA, Masaki KUREMATSU
  • Journal Title: International Journal on Knowledge-Based Systems 22(accepted)
  • Peer Reviewed
• [Journal Article] Human Computer Interaction Based on cognitive modeling (2008)
  • Author(s): Fujita Hamido, Hakura Jun, Kurematu Masaki
  • Journal Title: The IPSI BgD Transactions Advanced Research Vol.4 Pages: 16-25
  • Peer Reviewed
• [Journal Article] Cognitive Evolutionary Techniques in User Interface Design Based on Human Aspect Modeling (2008)
  • Author(s): Fujita Hamido, Hakura Jun, Kurematu Masaki
  • Journal Title: NAUN International Journal of Computers Vol.1 Pages: 32-38
  • Peer Reviewed
• [Journal Article] An Extraction of Emotion in Human Speech Using Speech Synthesize and Each Classifier for Each Emotion (2008)
  • Author(s): Masaki Kurematu,Jun Hakura, Hamido Fujita
  • Journal Title: WSEAS TRANSACTIONS on INFORMATION SCIENCE & APPLICATIONS Vol3 Pages: 246-251
  • Peer Reviewed
• [Journal Article] An Extraction of Emotion in Human Speech Using Speech Synthesize and Each Classifier for Each Emotion (2008)
  • Author(s): Masaki Kurematu, Jun Hakura, Hamido Fujita
  • Journal Title: WSEAS TRANSACTIONS on INFORMATION SCIENCE & APPLICATIONS 3 Pages: 246-251
  • Peer Reviewed
• [Journal Article] Cognitive Evolutionary Techniques in User Interface Design Based on Human Aspect Modeling (2008)
  • Author(s): Fujita Hamido, Hakura Jun, Kurematu Masaki
  • Journal Title: NAUN International Journal of Computers 1 Pages: 32-38
  • Peer Reviewed
• [Journal Article] Human Computer Interaction Based on cognitive modeling (2008)
  • Author(s): Fujita Hamido, Hakura Jun, Kurematu Masaki
  • Journal Title: The IPSI BgD Transactions Advanced Research 4 Pages: 16-25
  • Peer Reviewed
• [Presentation] 音声からの感情推定におけるクラス分け指標の提案 (2011)
  • Author(s): 天沼沙織・榑松理樹・羽倉淳・藤田ハミド
  • Organizer: 第73回情報処理全国大会
  • Place of Presentation: 東京
  • Year and Date: 2011-03-04
• [Presentation] 顔画像からのエゴグラム推定手法の構築 (2011)
  • Author(s): 菊地昭文・羽倉淳・榑松理樹・藤田ハミド
  • Organizer: 第73回情報処理全国大会
  • Place of Presentation: 東京
  • Year and Date: 2011-03-03
• [Presentation] 感情表現を意図した音楽理論に基づく音声合成手法の提案 (2011)
  • Author(s): 千葉央輝・榑松理樹・羽倉淳・藤田ハミド
  • Organizer: 第73回情報処理全国大会
  • Place of Presentation: 東京
  • Year and Date: 2011-03-02
• [Presentation] Semantical Case Based Reasoning related to Virtual Doctor System (VDS) (2011)
  • Author(s): Hamido Fujita, Jun Hakura, Masaki Kurematsu
  • Organizer: The 9th WSEAS International Conference on ARTIFICIAL INTELLIGENCE, KNOWLEDGE ENGINEERING and DATA BASES
  • Place of Presentation: イギリス・ケンブリッジ
  • Year and Date: 2011-02-20
• [Presentation] Case-Based Reasoning Framework for Medical Diagnosis for Virtual Doctor System (2011)
  • Author(s): Hamido Fujita
  • Organizer: The 9th WSEAS International Conference on ARTIFICIAL INTELLIGENCE, KNOWLEDGE ENGINEERING and DATA BASES
  • Place of Presentation: イギリス・ケンブリッジ
  • Year and Date: 2011-02-20
• [Presentation] Virtual Doctor System (VDS) : Framework on Reasoning issues Ontology based reasoning for virtual Doctor (2010)
  • Author(s): Hamido Fujita, Jun Hakura, Masaki Kurematsu
  • Organizer: The 2010 Conference on Technologies and Applications of Artificial Intelligence
  • Place of Presentation: 台湾・新竹
  • Year and Date: 2010-11-20
• [Presentation] Multiviews Ontologies Alignment for Medical based Reasoning (2010)
  • Author(s): Hamido Fujita, Jun Hakura, Masaki Kurematsu
  • Organizer: 11th IEEE International Symposium on COMPUTATIONAL INTELLIGENCE
  • Place of Presentation: ハンガリー・ブタペスト
  • Year and Date: 2010-11-20
• [Presentation] An Extraction of Emotion in Human Speech Using Cluster Analysis and a Regression Tree (2010)
  • Author(s): Masaki KUREMATSU, Saori AMANUMA, Jun HAKURA, Hamido FUJITA
  • Organizer: The 10th WSEAS International Conference on APPLIED COMPUTER SCIENCE
  • Place of Presentation: 岩手県八幡平市
  • Year and Date: 2010-10-06
• [Presentation] Virtual Doctor System (VDS) : Framework on Reasoning Issues (2010)
  • Author(s): Hamido Fujita, Jun Hakura, Masaki Kurematsu
  • Organizer: 9th International Conference on Software Methodologies, Tools and Techniques
  • Place of Presentation: 横浜
  • Year and Date: 2010-09-30
• [Presentation] Estimating Interest Level of Person through Postures by Vision System (2010)
  • Author(s): Jun HAKURA, Nobuhiro TAKAHASHI, Masaki KUREMATSU, Hamido FUJITA
  • Organizer: 9th International Conference on Software Methodologies, Tools and Techniques
  • Place of Presentation: 横浜
  • Year and Date: 2010-09-30
• [Presentation] A Framework of Emotional Speech Synthesize Using a Chord and a Scale (2010)
  • Author(s): Masaki KUREMATSU, Hiroki Chiba, Jun HAKURA, Hamido FUJITA
  • Organizer: 9th International Conference on Software Methodologies, Tools and Techniques
  • Place of Presentation: 横浜
  • Year and Date: 2010-09-30
• [Presentation] 音声からの感情推定におけるクラス分けの一考察 (2010)
  • Author(s): 天沼沙織・榑松理樹・羽倉淳・藤田ハミド
  • Organizer: 平成22年度電気関係学会東北支部大会
  • Place of Presentation: 八戸
  • Year and Date: 2010-08-26
• [Presentation] VDS:メンタルクローニング手法を用いたバーチャル・ドクター・システムの研究開発 (2010)
  • Author(s): 藤田ハミド
  • Organizer: ICT推進フェア2010 in東北・セミナー
  • Place of Presentation: 仙台
  • Year and Date: 2010-06-09
• [Presentation] Virtual Doctor System (VDS) : Medical Decision Reasoning Based on Physical and Mental Ontologies (2010)
  • Author(s): Hamido Fujita, Jun Hakura, Masaki Kurematsu
  • Organizer: The Twenty Third International Conference on Industrial, Engineering & Other Applications of Applied Intelligent Systems
  • Place of Presentation: スペイン・コルドバ
  • Year and Date: 2010-06-02
• [Presentation] Human Interaction based Reasoning using Ontology Alignment (2010)
  • Author(s): Hamido Fujita, Jun Hakura, Masaki Kurematsu
  • Organizer: The 9th WSEAS International Conference on APPLICATIONS OF COMPUTER ENGINEERING
  • Place of Presentation: マレーシア・ペナン
  • Year and Date: 2010-03-25
• [Presentation] Human Interaction based Reasoning using Ontology Alignment (2010)
  • Author(s): Hamido Fujita, Jun Hakura, Masaki Kurematsu
  • Organizer: The 9th WSEAS International Conference on APPLICATIONS OF COMPUTER ENGINEERING
  • Place of Presentation: タイ・パタヤ
  • Year and Date: 2010-03-25
• [Presentation] 対面者との相互作用を目的とした表情合成に関する研究 (2010)
  • Author(s): 鷹觜拓也・羽倉淳・榑松理樹・藤田ハミド
  • Organizer: 情報処理学会50周年記念全国大会
  • Place of Presentation: 東京大学
  • Year and Date: 2010-03-11
• [Presentation] 音楽理論に基づく音声合成による感情表出手法 (2010)
  • Author(s): 千葉央輝・槫松理樹・羽倉淳・藤田ハミド
  • Organizer: 情報処理学会50周年記念全国大会
  • Place of Presentation: 東京大学
  • Year and Date: 2010-03-11
• [Presentation] 視線と頭部姿勢に着目したユーザの興味推定 (2010)
  • Author(s): 高橋宣裕・羽倉淳・榑松理樹・藤田ハミド
  • Organizer: 情報処理学会50周年記念全国大会
  • Place of Presentation: 東京大学
  • Year and Date: 2010-03-10
• [Presentation] Virtual Kenji System : Interactive Environment for Subjective Interaction based on Mental Reasoning (2010)
  • Author(s): Hamido Fujita, Jun Hakura, Masaki Kurematsu
  • Organizer: International Conference on Learning Innovation in Science and Technology
  • Place of Presentation: タイ・パタヤ
  • Year and Date: 2010-02-24
• [Presentation] MENTAL CLONING BASE VIRTUAL DIAGNOSTICIAN SYSTEM : Virtual Medical Doctor(VMD) reasoning system (2010)
  • Author(s): Hamido Fujita, Jun Hakura, Masaki Kurematsu
  • Organizer: Third International Conference on Health Informatics
  • Place of Presentation: スペイン・バレンシア
  • Year and Date: 2010-01-22
• [Presentation] VIRTUAL MEDICAL DOCTOR SYSTEMS : Status progress report on Virtual Medical Doctor (VMD) (2010)
  • Author(s): Hamido Fujita, Jun Hakura, Masaki Kurematsu
  • Organizer: Third International Conference on Health Informatics
  • Place of Presentation: スペイン・バレンシア
  • Year and Date: 2010-01-22
• [Presentation] MENTAL CLONING BASE VIRTUAL DIAGNOSTICIAN SYSTEM : Virtual Medical Doctor (VMD) reasoning system (2010)
  • Author(s): Hamido Fujita, Jun Hakura, Masaki Kurematsu
  • Organizer: Third International Conference on Health Informatics
  • Place of Presentation: スペイン・バレンシア
  • Year and Date: 2010-01-22
• [Presentation] Virtual Medical Doctor (VMD) System Reasoning based on Ontology Integration -Virtual Medical Doctor (VMD) reasoning system- (2009)
  • Author(s): Hamido Fujita・Jun Hakura・Masaki Kurematsu
  • Organizer: 電子情報通信学会パターン認識・メディア理解(PRMU)研究会2009年10月研究会
  • Place of Presentation: 広島大学
  • Year and Date: 2009-10-23
• [Presentation] 複数の学習手法を用いた音声からの感情推定 (2009)
  • Author(s): 天沼沙織・槫松理樹・羽倉淳・藤田ハミド
  • Organizer: 電子情報通信学会パターン認識・メディア理解(PRMU)研究会2009年10月研究会
  • Place of Presentation: 広島大学
  • Year and Date: 2009-10-23
• [Presentation] 擬人化システムの表情合成に関する研究 (2009)
  • Author(s): 鷹觜拓也・羽倉淳・榑松理樹・藤田ハミド
  • Organizer: 電子情報通信学会パターン認識・メディア理解(PRMU)研究会2009年10月研究会
  • Place of Presentation: 広島大学
  • Year and Date: 2009-10-23
• [Presentation] Virtual Doctor System based on Mental Cloning (2009)
  • Author(s): Hamid Fujita
  • Organizer: Workshop on Telematics and Robotics for the quality of life of the elderly
  • Place of Presentation: イタリア・ローマ
  • Year and Date: 2009-09-28
• [Presentation] A study on Prediction on Listener Emotion in Speech for Medical Doctor Interface (2009)
  • Author(s): Masaki Kurematsu
  • Organizer: Workshop on Telematics and Robotics for the quality of life of the elderly
  • Place of Presentation: イタリア・ローマ
  • Year and Date: 2009-09-28
• [Presentation] Virtual Medical Doctor Interaction Based on Transactional Analysis (2009)
  • Author(s): Hamido Fujita, Jun Hakura, Masaki Kurematsu
  • Organizer: he 8th International Conference on Software Methodologies, Tools and Techniques
  • Place of Presentation: チェコ・プラハ
  • Year and Date: 2009-09-25
• [Presentation] Facial Expression Invariants for Estimating Mental States of Person (2009)
  • Author(s): Jun Hakura, Masaki Kurematsu, Hamido Fujita
  • Organizer: he 8th International Conference on Software Methodologies, Tools and Techniques
  • Place of Presentation: チェコ・プラハ
  • Year and Date: 2009-09-25
• [Presentation] An Approach to Implement Listeners to Estimate Emotion in Speech (2009)
  • Author(s): Masaki Kurematsu, Marina Ohashi, Orimi Kinosita, Jun Hakura, HamidoFujita
  • Organizer: he 8th International Conference on Software Methodologies, Tools and Techniques
  • Place of Presentation: チェコ・プラハ
  • Year and Date: 2009-09-25
• [Presentation] Virtual Medical Doctor Interaction Based on Transactional Analysis (2009)
  • Author(s): Hamido Fujita, Jun Hakura, Masaki Kurematsu
  • Organizer: he 8th International Conference on Software Met hodologies, Tools and Techniques
  • Place of Presentation: チェコ・プラハ
  • Year and Date: 2009-09-25
• [Presentation] Facial Expression Invariants for Estimating Mental States of Person (2009)
  • Author(s): Jun Hakura, Masaki Kurematsu, Hamido Fujita
  • Organizer: he 8th International Conference on Software Met hodologies, Tools and Techniques
  • Place of Presentation: チェコ・プラハ
  • Year and Date: 2009-09-25
• [Presentation] An Approach to Implement Listeners to Estimate Emotion in Speech (2009)
  • Author(s): Masaki Kurematsu, Marina Ohashi, Orimi Kinosita, Jun Hakura, HamidoFujita
  • Organizer: he 8th International Conference on Software Met hodologies, Tools and Techniques
  • Place of Presentation: チェコ・プラハ
  • Year and Date: 2009-09-25
• [Presentation] An Action Decision Model for Emotions based on Transactional Analysis (2009)
  • Author(s): HAMIDO FUJITA, NATSUMI SAWAI, JUN HAKURA, MASAKI KUREMATSU
  • Organizer: the 8th WSEAS Int.Conf.on ARTIFICIAL INTELLIGENCE, KNOWLEDGE ENGINEERING & DATA BASES
  • Place of Presentation: イギリス・ケンブリッジ
  • Year and Date: 2009-02-21
• [Presentation] Plenary Lecture 1 : Cognitive Reasoning and Recognition for Intelligent Human Interaction based on Mental Cloning (2009)
  • Author(s): HAMIDO FUJITA
  • Organizer: the 8th WSEAS Int.Conf.on ARTIFICIAL INTELLIGENCE, KNOWLEDGE ENGINEERING & DATA BASES
  • Place of Presentation: イギリス・ケンブリッジ
  • Year and Date: 2009-02-21
• [Presentation] An Action Decision Model for Emotions based on Transactional Analysis (2009)
  • Author(s): HAMIDO FUJITA, NATSUMISAWAI, JUN HAKURA, MASAKI KUREMATSU
  • Organizer: the 8th WSEAS Int. Conf. on ARTIFICIAL INTELLIGENCE, KNOWLEDGE ENGINEERING & DATA BASES
  • Place of Presentation: University of Cambridge, Cambridge, UK
  • Year and Date: 2009-02-21
• [Presentation] Plenary Lecture 1 : Cognitive Reasoning and Recognition for Intelligent Human Interaction based on Mental Cloning (2009)
  • Author(s): HAMIDO FUJITA
  • Organizer: the 8th WSEAS Int. Conf. on ARTIFICIAL INTELLIGENCE, KNOWLEDGE ENGINEERING & DATA BASES
  • Place of Presentation: University of Cambridge, Cambridge, UK
  • Year and Date: 2009-02-21
• [Presentation] 交流分析に基づくユーザ感情を用いた行動決定モデルの構築 (2009)
  • Author(s): 澤井夏美, 藤田ハミド, 羽倉淳, 榑松理樹
  • Organizer: 情報処理学会ヒューマンコンピュータインタラクション研究会
  • Place of Presentation: 湯布院公民館
  • Year and Date: 2009-01-28
• [Presentation] 音声合成を用いた音声からの感情推定 (2009)
  • Author(s): 木下おりみ, 榑松理樹, 羽倉 淳, 藤田ハミド
  • Organizer: 電子情報通信学会パターン認識・メディア理解研究会
  • Place of Presentation: 国立情報学研究所・学術総合センター
  • Year and Date: 2009-01-12
• [Presentation] 表情に基づくシチュエーションを用いた感情推定手法 (2009)
  • Author(s): 千田調和, 羽倉 淳, 槫松理樹, 藤田ハミド
  • Organizer: 電子情報通信学会パターン認識・メディア理解研究会
  • Place of Presentation: 国立情報学研究所・学術総合センター
  • Year and Date: 2009-01-12
• [Presentation] 音節に基づく合成音声による感情表出の研究 (2009)
  • Author(s): 大橋真里菜, 榑松理樹, 羽倉 淳, 藤田ハミド
  • Organizer: 電子情報通信学会パターン認識・メディア理解研究会
  • Place of Presentation: 国立情報学研究所・学術総合センター
  • Year and Date: 2009-01-12
• [Presentation] Empirical based Techniques for Human Cognitive Interaction Analysis : Universal Template Design (2008)
  • Author(s): Hamido FUJITA, Jun HAKURA, Masaki KUREMTSU, Shigekazu CHIDA, Yuko ARAKAWA
  • Organizer: The 7th International Conference on Software Methodologies, Tools and Techniques
  • Place of Presentation: アラブ首長国連邦・シャウジャ
  • Year and Date: 2008-10-16
• [Presentation] Automatic Facial Expression Recognition Method Using Situational Information (2008)
  • Author(s): Jun HAKURA, Shigekazu CHIDA, Masaki KUREMATSU, Hamido FUJITA
  • Organizer: The 7th International Conference on Software Methodologies, Tools and Techniques
  • Place of Presentation: アラブ首長国連邦・シャウジャ
  • Year and Date: 2008-10-16
• [Presentation] An Emotion Estimation from Human Speech Using Speech Recognition and Speech Synthesize (2008)
  • Author(s): Masaki KUREMATSU, Marina OHASHI, Orimi KINOSITA, Jun HAKURA, Hamido FUJITA
  • Organizer: The 7th International Conference on Software Methodologies, Tools and Techniques
  • Place of Presentation: アラブ首長国連邦・シャウジャ
  • Year and Date: 2008-10-16
• [Presentation] Empirical based Techniques for Human Cognitive Interaction Analysis : Universal Template Design (2008)
  • Author(s): Hamido FUJITA, Jun HAKUEA, Masaki KUREMTSU, Shigekazu CHIDA, Yuko ARAKAWA
  • Organizer: The 7th International Conference on Software Methodologies, Tools and Techniques
  • Place of Presentation: Millennium Hotel, Sharjah, UAE
  • Year and Date: 2008-10-16
• [Presentation] n Automatic Facial Expression Recognition Method Using Situational Information (2008)
  • Author(s): Jun HAKUEA, Shigekazu CHIDA, Masaki KUREMATSU, Hamido FUJITA
  • Organizer: The 7th International Conference on Software Methodologies, Tools and Techniques
  • Place of Presentation: Millennium Hotel, Sharjah, UAE
  • Year and Date: 2008-10-16
• [Presentation] An Emotion Estimation from Human Speech Using Speech Recognition and Speech Synthesize (2008)
  • Author(s): Masaki KUREMATSU, Marina OHASHI, Orimi KINOSITA, Jun HAKURA, Hamido FUJITA
  • Organizer: The 7th International Conference on Software Methodologies, Tools and Techniques
  • Place of Presentation: Millennium Hotel, Sharjah, UAE
  • Year and Date: 2008-10-16