• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

実時間視聴覚情報統合による複数の人とのマルチモーダル・インタラクションの研究

Research Project

Project/Area Number 16016251
Research Category

Grant-in-Aid for Scientific Research on Priority Areas

Allocation TypeSingle-year Grants
Review Section Science and Engineering
Research InstitutionKyoto University

Principal Investigator

奥乃 博  京都大学, 情報学研究科, 教授 (60318201)

Co-Investigator(Kenkyū-buntansha) 駒谷 和範  京都大学, 情報学研究科, 助手 (40362579)
中臺 一博  (株)ホンダ, リサーチ・インスティテュート・ジャパン, シニア・リサーチャー
Project Period (FY) 2004 – 2005
Project Status Completed (Fiscal Year 2005)
Budget Amount *help
¥14,500,000 (Direct Cost: ¥14,500,000)
Fiscal Year 2005: ¥7,500,000 (Direct Cost: ¥7,500,000)
Fiscal Year 2004: ¥7,000,000 (Direct Cost: ¥7,000,000)
Keywords音環境理解 / 視聴覚情報統合 / ロボット知覚 / GSS / 自動マスク生成 / 文脈的制約 / 空間マッピング / ミッシングフィーチャ / アクティブオーディション / 音と画像の実時間情報統合 / ヒューマノイドロボット / 近接学 / 対人距離による挙動選択 / 肌センサ / 擬音語認識 / 超指向性スピーカ
Research Abstract

最終年度は、ミッシングフィーチャ理論および視聴覚情報統合による複数同時発話認識の洗練化に主としてに取り組んだ。具体的には、マイクロフォンアレイによる音源分離GSSとミッシングフィーチャ理論による音声認識との統合システムの詳細な評価を行うとともに、距離や位置に依存したインタラクションシステムのためにさまざまな設定での評価とその洗練化に取り組んだ。主な成果は以下の通りである。
(1)音源分離にGeometrical Source Separationとmulti-channel post-filterを使用し、後者から得られるチャネル間リーク情報と背景雑音情報を基にマスクを自動作成した。自動生成されたマスクを使用し,マルチバンド版Juliusを用いて認識を行った。ここで、特徴量をスペクトル歪みに強いMSLSとした。同じベンチマークにより、アプリオリマスクの場合と比較し、約62%の性能を達成した。さらに、さまざまな方向と距離に対して評価し、内部パラメータ13個の最適値にあまり規則性がないことが判明し、遺伝的アルゴリズムにより、最適値探索を行い、その有効性を確認した。
(2)人間親密度を空間にマッピングすることにより、複数人とのインタラクションを行うシステムを開発し、被験者による評価実験により有効性を確認した。これによりどの位置に立った人とインタラクションをすべきか、という挙動設計モデルが確立できた。
(3)柔軟な対話戦略を有した音声対話システムの開発するために、対話の進行モデルと履歴の構造モデルという2つの文脈的特徴を使用する手法を開発した。レストラン検索システムにどう手法を実装し、一発話から得られる特徴だけを使用した場合と比較して、意味理解精度が83.4%から92.6%まで向上した。さらに、レストラン検索システムデータの学習で得られた決定木がたの検索システムでも有効であることが分かり、ドメイン非依存な文脈手法を確立できた。

Report

(2 results)
  • 2005 Annual Research Report
  • 2004 Annual Research Report
  • Research Products

    (49 results)

All 2006 2005 2004 2002

All Journal Article (41 results) Book (3 results) Patent(Industrial Property Rights) (5 results)

  • [Journal Article] Common Acoustical Pole Estimation from Multi-Channel Musical Audio Signals2006

    • Author(s)
      Takuya Yoshioka
    • Journal Title

      EICE Trans.on Fundamentals of Electronics, Communications, and Computer Sciences E89-A・1

      Pages: 240-247

    • Related Report
      2005 Annual Research Report
  • [Journal Article] Using Multiple Edit Distances to Automatically Grade Outputs from Machine Translation Systems,2006

    • Author(s)
      Yasuhiro Akiba
    • Journal Title

      IEEE Transactions on Audio, Speech and Language Processing 14・2

      Pages: 393-402

    • Related Report
      2005 Annual Research Report
  • [Journal Article] ミッシングフィーチャ理論を利用した音源分離と音声認識のインターフェースと複数ロボットへの適用2005

    • Author(s)
      山本 俊一
    • Journal Title

      日本ロボット学会誌 23・6

      Pages: 743-751

    • Related Report
      2005 Annual Research Report
  • [Journal Article] ゲーム理論による中心化理論の解体と実言語データに基づく検証.2005

    • Author(s)
      白松 俊
    • Journal Title

      自然言語処理 12・3

      Pages: 91-110

    • Related Report
      2005 Annual Research Report
  • [Journal Article] 非線形振動子による引き込みを利用した仮想空間における歩行2005

    • Author(s)
      小鷹 研理
    • Journal Title

      ヒューマンインタフェース学会論文誌 17・4

      Pages: 26-36

    • Related Report
      2005 Annual Research Report
  • [Journal Article] Pitch-dependent identification of musical instrument sounds2005

    • Author(s)
      Tetsuro Kitahara
    • Journal Title

      Applied Intelligence, 23・3

      Pages: 267-275

    • Related Report
      2005 Annual Research Report
  • [Journal Article] Extracting Multi-Modal Dynamics of Objects using RNNPB2005

    • Author(s)
      Tetsuya Ogata
    • Journal Title

      Journal of Robotics and Mechatropics, 17・6

      Pages: 681-688

    • Related Report
      2005 Annual Research Report
  • [Journal Article] Distance Based Dynamic Interaction of Humanoid Robot with Multiple People2005

    • Author(s)
      Tsuyoshi Tasaki
    • Journal Title

      Lecture Notes in Artificial Intelligence 3533

      Pages: 111-120

    • Related Report
      2005 Annual Research Report
  • [Journal Article] 超音波センサアレイを用いたアクティブセンシングによる3次元物体の位置・形状認識2005

    • Author(s)
      奥乃 博
    • Journal Title

      超音波テクノ 2005・9-10

      Pages: 79-84

    • Related Report
      2005 Annual Research Report
  • [Journal Article] Empirical Verification of Meaning-Game-based Generalization of Centering Theory with Large Japanese2005

    • Author(s)
      Shun Shiramatsu
    • Journal Title

      Proceedings of the 19th Pacific Asia Conference on Language, Information, and Computation (PACLIC 19)

      Pages: 192-210

    • Related Report
      2005 Annual Research Report
  • [Journal Article] INTER : D A Drum Sound Equalizer for Controlling Volume and Timbre of Druams2005

    • Author(s)
      Kazuyoshi Yoshii
    • Journal Title

      Proceedings of 2nd European Workshop on the Integration of Knowledge, Semantic and Digital Media Technologies (EWIMT

      Pages: 205-212

    • Related Report
      2005 Annual Research Report
  • [Journal Article] Walking with Body-sense in Virtual Space Using the Nonlinear Oscillator2005

    • Author(s)
      Kenri Kodaka
    • Journal Title

      Proceedings of the International Conference on Systems, Man and Cybernetics (SIC-2005)

      Pages: 324-329

    • Related Report
      2005 Annual Research Report
  • [Journal Article] INSTRUMENT IDENTIFICATION IN POLYPHONIC MUSIC : FEATURE WEIGHTING WITH MIXED SOUNDS, PITCH-DEPENDENT TIMBRE2005

    • Author(s)
      Tetsuro Kitahara
    • Journal Title

      Proceedings of 6th International Conference on Musical Information Retreival (ISMIR-2005)

      Pages: 558-563

    • Related Report
      2005 Annual Research Report
  • [Journal Article] SINGER IDENTIFICATION BASED ON ACCOMPANIMENT SOUND REDUCTION AND RELIABLE FRAME SELECTION2005

    • Author(s)
      Hiromasa Fujihara
    • Journal Title

      Proceedings of 6th International Conference on Musical Information Retreival (ISMIR-2005)

      Pages: 329-336

    • Related Report
      2005 Annual Research Report
  • [Journal Article] Multiple Moving Speaker Tracking by Microphone Array on Mobile Robot2005

    • Author(s)
      Masamitsu Murase
    • Journal Title

      Proceedings of the Nineth European Conference on Speech Communication and Technology (Interspeech-2005)

      Pages: 249-252

    • Related Report
      2005 Annual Research Report
  • [Journal Article] Contextual Constraints based on Dialogue Models in Database Search Task for Spoken Dialogue Systems2005

    • Author(s)
      Kazunori Komatani
    • Journal Title

      Proceedings of the Nineth European Conference on Speech Communication and Technology (Interspeech-2005)

      Pages: 877-880

    • Related Report
      2005 Annual Research Report
  • [Journal Article] Generating Confirmation to Distinguish Phonologically Confusing Word Pairs in Spoken Dialogue Systems2005

    • Author(s)
      Kazunori Komatani
    • Journal Title

      Proceedings of 4th IJCAI Workshop on Knowledge and Reasoning in Practical Dialogue Systems

      Pages: 40-45

    • Related Report
      2005 Annual Research Report
  • [Journal Article] Making A Robot Recognize Three Simultaneous Sentences in Real-Time2005

    • Author(s)
      Shun'ichi Yamamoto
    • Journal Title

      Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS-2005)

      Pages: 897-892

    • Related Report
      2005 Annual Research Report
  • [Journal Article] Implementation of Active Direction-Pass Filter on Dynamically Reconfigurable Processo2005

    • Author(s)
      Syunsuke Kurotaki
    • Journal Title

      Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS-2005)

      Pages: 515-520

    • Related Report
      2005 Annual Research Report
  • [Journal Article] Spatially Mapping of Friendliness for Human-Robot Interaction2005

    • Author(s)
      Tsuyoshi Tasaki
    • Journal Title

      Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS-2005)

      Pages: 521-526

    • Related Report
      2005 Annual Research Report
  • [Journal Article] Enhanced Robot Speech Recognition Based on Microphone Array Source Separation and Missing Feature Theory2005

    • Author(s)
      Shun'ichi Yamamoto
    • Journal Title

      Proceedings of IEEE RAS International Conference on Robotics and Automation (ICRA-2005)

      Pages: 1489-1494

    • Related Report
      2005 Annual Research Report
  • [Journal Article] A computational model of monkey cortical grating cells2005

    • Author(s)
      Tino Lourens, Hiroshi G.Okuno, Hiroshi Tsujino
    • Journal Title

      Biological Cybernetics 92・1

      Pages: 61-70

    • Related Report
      2004 Annual Research Report
  • [Journal Article] 環境音を対象とした擬音語自動認識2005

    • Author(s)
      石原 一志, 駒谷 和範, 尾形 哲也, 奥乃 博
    • Journal Title

      人工知能学会論文誌 20・3

      Pages: 229-236

    • Related Report
      2004 Annual Research Report
  • [Journal Article] Dynamic Communication of Humanoid Robot with Multiple People Based on Interaction Distance2005

    • Author(s)
      Tsuyoshi Tasaki, Shohei Matsumoto, Hayato Ohba, Shun'ichi Yamamoto, Mitsuhiko Toda, Kazunori Komatani, Tetsuya_Ogata, Hiroshi G.Okuno
    • Journal Title

      人工知能学会論文誌 20・3

      Pages: 209-219

    • NAID

      120005439187

    • Related Report
      2004 Annual Research Report
  • [Journal Article] ミッシングフィーチャ理論を利用した音源分離と音声認識のインターフェースと複数ロボツトへの適用2005

    • Author(s)
      山本 俊一, 中臺 一博, 辻野 広司, 奥乃 博
    • Journal Title

      日本ロボット学会誌 23・4(印刷中)

    • Related Report
      2004 Annual Research Report
  • [Journal Article] Robot Audition : Its Issues and State of the Art (invited talk)2005

    • Author(s)
      Hiroshi G.Okuno
    • Journal Title

      Proceedings of 2nd International Symposium on Life Science (IEMC2005)

      Pages: 13-15

    • Related Report
      2004 Annual Research Report
  • [Journal Article] ロボット聴覚の課題と現状(招待講演)2005

    • Author(s)
      奥乃 博, 中臺 一博
    • Journal Title

      音響学会春季研究発表会,3-7-7

      Pages: 633-636

    • Related Report
      2004 Annual Research Report
  • [Journal Article] Sound and Visual Tracking for Humanoid Robot2004

    • Author(s)
      Hiroshi G.Okuno, Kazuhiro Nakadai, Tino Lourens, Hiroaki Kitano
    • Journal Title

      Applied Intelligence 20・3

      Pages: 253-266

    • Related Report
      2004 Annual Research Report
  • [Journal Article] 音声対話システムにおける適応的な応答生成を行うためのユーザモデル2004

    • Author(s)
      駒谷和範, 上野晋一, 河原達也, 奥乃 博
    • Journal Title

      電子情報通信学会論文誌 87-D2・10

      Pages: 1921-1928

    • NAID

      110003171015

    • Related Report
      2004 Annual Research Report
  • [Journal Article] Effects of increasing modalities in recognizing three simultaneous speeches2004

    • Author(s)
      Hiroshi G.Okuno, Kazuhiro Nakadai, Hiroaki Kitano
    • Journal Title

      Speech Communication 43・4

      Pages: 347-359

    • Related Report
      2004 Annual Research Report
  • [Journal Article] Improvement of Recognition of Simultaneous Speech Signals Using AV Integration and Scattering Theory for Humanoid Robots2004

    • Author(s)
      Kazuhiro Nakadai, Daisuke Matsuura, Hiroshi G.Okuno, Hiroshi Tsujino
    • Journal Title

      Speech Communication 44・1

      Pages: 97-112

    • Related Report
      2004 Annual Research Report
  • [Journal Article] Improvement of Robot Audition by Interfacing Sound Source Separation and Automatic Speech Recognition with Missing Feature Theory2004

    • Author(s)
      Shun'ichi Yamamoto, Kazuhiro Nakadai, Hiroshi Tsujino, Toshio Yokoyama, Hiroshi G.Okuno
    • Journal Title

      Proceedings of IEEE-RAS International Conference on Robots and Automation (ICRA-2004)

      Pages: 1517-1523

    • Related Report
      2004 Annual Research Report
  • [Journal Article] Recognition of Emotional States in Spoken Dialogue with a Robot2004

    • Author(s)
      Kazunori Komatani, Ryosuke Itoh, Tatsuya Kawahara, Hiroshi G.Okuno
    • Journal Title

      Innovations in Applied Artificial Intelligence (IEA/AIE-04) LNA13029

      Pages: 413-423

    • Related Report
      2004 Annual Research Report
  • [Journal Article] Automatic Sound-Imitation Word Recognition from Environmental Sounds focusing on Ambiguity Problem in Determining Phonemes2004

    • Author(s)
      Kazushi Ishihara, Tomohiro Nakatani, Tetsuya Ogata, Hiroshi G.Okuno
    • Journal Title

      PRICAI 2004: Trends in Artificial Intelligence LNA13157

      Pages: 909-918

    • Related Report
      2004 Annual Research Report
  • [Journal Article] Assessment of General Applicability of Robot Audition System by Recognizing Three Simultaneous Speeches2004

    • Author(s)
      Shun'ichi Yamamoto, Kazuhiro Nakadai, Hiroshi Tsujino, Hiroshi G.Okuno
    • Journal Title

      Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS-2004)

      Pages: 2111-2116

    • Related Report
      2004 Annual Research Report
  • [Journal Article] Repeat Recognition for Environmental Sounds2004

    • Author(s)
      Yuya Hattori, Kazushi Ishihara, Kazunori Komatani, Tetsuya Ogata, Hiroshi G.Okuno
    • Journal Title

      Proceedings of IEEE International Workshop on Robot and Human Interaction (Ro-Man-2004)

      Pages: 121-126

    • Related Report
      2004 Annual Research Report
  • [Journal Article] Dynamic Communication of Humanoid Robot with multiple people based on Interaction Distance2004

    • Author(s)
      Tsuyoshi Tasaki, Shohei Matsumoto, Hayato Ohba, Mitsuhiko Toda, Kazunori Komatani, Tetsuya Ogata, Hiroshi G.Okuno
    • Journal Title

      Proceedings of IEEE International Workshop on Robot and Human Interaction (Ro-Man-2004)

      Pages: 81-86

    • NAID

      120005439187

    • Related Report
      2004 Annual Research Report
  • [Journal Article] Disambiguation in Determining Phonemes of Sound-Imitation Words for Environmental Sound Recognition2004

    • Author(s)
      Kazushi Ishihara, Yuya Hattori, Tomohiro Nakatani, Kazunori Komatani, Tetsuya Ogata, Hiroshi G.Okuno
    • Journal Title

      Proceedings of 2004 International Conference on Spoken Language Processing (ICSLP-2004)

      Pages: 1485-1488

    • Related Report
      2004 Annual Research Report
  • [Journal Article] Robot Motion Control using Listener's Back-Channels and Head Gesture Information2004

    • Author(s)
      Tsuyoshi Tasaki, Takeshi Yamaguchi, Kazunoni Komatani, Tetsuya Ogata, Hiroshi G.Okuno
    • Journal Title

      Proceedings of 2004 International Conference on Spoken Language Processing (ICSLP-2004)

      Pages: 1033-1036

    • Related Report
      2004 Annual Research Report
  • [Journal Article] Robot Motion Control using Listener's Back-Channels and Head Gesture Information2004

    • Author(s)
      Tsuyoshi Tasaki, Kazunori Komatani, Tetsuya Ogata, Hiroshi G.Okuno
    • Journal Title

      Proceedings of 2nd international Workshop on Man-Machine Symbiotic Systems

      Pages: 327-338

    • Related Report
      2004 Annual Research Report
  • [Journal Article] Computational Auditory Scene Analysis and Its Application to Robot Audition2004

    • Author(s)
      Hiroshi G.Okuno, Tetsuya Ogata, Kazunori Komatani, Kazuhiro Nakadai
    • Journal Title

      Post-Proceedings of the International Conference on Informatics Research for Development of Knowledge Society Infrastructure

      Pages: 73-80

    • Related Report
      2004 Annual Research Report
  • [Book] 大人のための「ロボット学」2006

    • Author(s)
      PHP研究所(奥乃 博)
    • Total Pages
      251
    • Publisher
      PHP研究所
    • Related Report
      2005 Annual Research Report
  • [Book] 人工知能事典2005

    • Author(s)
      人工知能学会(奥乃 博)
    • Total Pages
      976
    • Publisher
      共立出版
    • Related Report
      2005 Annual Research Report
  • [Book] 人工知能学事典(Lisp)2005

    • Author(s)
      奥乃 博
    • Publisher
      共立出版(印刷中)
    • Related Report
      2004 Annual Research Report
  • [Patent(Industrial Property Rights)] 楽器音認識方法,楽器アノテーション方法,及び楽曲検索方法2006

    • Inventor(s)
      北原鉄朗, 奥乃博
    • Industrial Property Rights Holder
      京都大学
    • Industrial Property Number
      2006-058649
    • Filing Date
      2006-03-03
    • Related Report
      2005 Annual Research Report
  • [Patent(Industrial Property Rights)] ロボット視聴覚システム2004

    • Inventor(s)
      中臺 一博, 奥乃 博, 北野 宏明
    • Industrial Property Rights Holder
      科学技術振興事業団
    • Patent Publication Number
      2004-198656
    • Filing Date
      2004-07-15
    • Related Report
      2004 Annual Research Report
  • [Patent(Industrial Property Rights)] ロボット視聴覚システム2002

    • Inventor(s)
      中臺 一博, 奥乃 博, 北野 宏明
    • Industrial Property Rights Holder
      科学技術振興事業団
    • Filing Date
      2002-12-17
    • Acquisition Date
      2005-01-07
    • Related Report
      2004 Annual Research Report
  • [Patent(Industrial Property Rights)] ロボット視聴覚システム2002

    • Inventor(s)
      中臺 一博, 奥乃 博, 北野 宏明
    • Industrial Property Rights Holder
      科学技術振興事業団
    • Filing Date
      2002-03-01
    • Acquisition Date
      2004-12-17
    • Related Report
      2004 Annual Research Report
  • [Patent(Industrial Property Rights)] 鳴き声による音声ガイドシステム2002

    • Inventor(s)
      中臺 一博, 日台 健一, 奥乃 博, 北野 宏明
    • Industrial Property Rights Holder
      科学技術振興事業団
    • Filing Date
      2002-03-01
    • Acquisition Date
      2004-04-05
    • Related Report
      2004 Annual Research Report

URL: 

Published: 2004-04-01   Modified: 2018-03-28  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi