• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Development of communication support system based on intent recognition for cerebral palsy

Research Project

Project/Area Number 25282053
Research Category

Grant-in-Aid for Scientific Research (B)

Allocation TypePartial Multi-year Fund
Section一般
Research Field Educational technology
Research InstitutionKobe University

Principal Investigator

Takiguchi Tetsuya  神戸大学, 都市安全研究センター, 准教授 (40397815)

Co-Investigator(Kenkyū-buntansha) 有木 康雄  神戸大学, 学内共同利用施設等, 教授 (10135519)
高田 哲  神戸大学, 保健学研究科, 教授 (10216658)
中井 靖  宮崎大学, 教育文化学部, 講師 (80462050)
榎並 直子  神戸大学, 学内共同利用施設等, 助教 (80628925)
中川 誠司  国立研究開発法人産業技術総合研究所, その他部局等, 研究員 (70357614)
Project Period (FY) 2013-04-01 – 2017-03-31
Project Status Completed (Fiscal Year 2016)
Budget Amount *help
¥17,810,000 (Direct Cost: ¥13,700,000、Indirect Cost: ¥4,110,000)
Fiscal Year 2016: ¥3,770,000 (Direct Cost: ¥2,900,000、Indirect Cost: ¥870,000)
Fiscal Year 2015: ¥4,550,000 (Direct Cost: ¥3,500,000、Indirect Cost: ¥1,050,000)
Fiscal Year 2014: ¥5,460,000 (Direct Cost: ¥4,200,000、Indirect Cost: ¥1,260,000)
Fiscal Year 2013: ¥4,030,000 (Direct Cost: ¥3,100,000、Indirect Cost: ¥930,000)
Keywordsヒューマン・インターフェース / 教育工学
Outline of Final Research Achievements

An utterance style of a person with an articulation disorder, such as cerebral palsy, is different from that of physically unimpaired persons, and his/her utterance is often unstable or unclear, which makes it difficult for them to communicate. To develop a communication-support system for them, we propose a new automatic speech recognition (ASR) system using a new acoustic feature extraction technique, a voice conversion (VC) method for articulation disorders that converts unclear utterances to clear utterances, and a multi-modal utterance recognition system using a novel feature integration technique based on a machine-learning approach. Experimental results demonstrated that our ASR, VC, and multimodal recognition methods could improve the speech recognition accuracy, the listening speech quality, and the multimodal (speech and image) recognition accuracy in comparison with conventional approaches, respectively.

Report

(5 results)
  • 2016 Annual Research Report   Final Research Report ( PDF )
  • 2015 Annual Research Report
  • 2014 Annual Research Report
  • 2013 Annual Research Report
  • Research Products

    (48 results)

All 2017 2016 2015 2014 2013

All Journal Article (10 results) (of which Peer Reviewed: 10 results,  Open Access: 4 results) Presentation (38 results) (of which Int'l Joint Research: 9 results,  Invited: 1 results)

  • [Journal Article] Phone Labeling Based on the Probabilistic Representation for Dysarthric Speech Recognition2016

    • Author(s)
      Yuki Takashima, Toru Nakashika, Tetsuya Takiguchi, Yasuo Ariki
    • Journal Title

      American Journal of Signal Processing

      Volume: 6 Pages: 19-23

    • Related Report
      2016 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Multithreading Cascade of SURF for Facial Expression Recognition2016

    • Author(s)
      Jinhui Chen, Zhaojie Luo, Tetsuya Takiguchi, Yasuo Ariki
    • Journal Title

      EURASIP Journal on Image and Video Processing

      Volume: 2016(1) Issue: 1 Pages: 1-13

    • DOI

      10.1186/s13640-016-0140-7

    • NAID

      120005898708

    • Related Report
      2016 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Individuality-preserving Voice Conversion for Articulation Disorders Using Phoneme-categorized Exemplars2015

    • Author(s)
      Ryo Aihara, Tetsuya Takiguchi, Yasuo Ariki
    • Journal Title

      ACM Transactions on Accessible Computing

      Volume: 6 Pages: 1-17

    • Related Report
      2015 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Audio-Visual Speech Recognition Using Convolutive Bottleneck Networks for a Person with Severe Hearing Loss2015

    • Author(s)
      Yuki Takashima, Yasuhiro Kakihara, Ryo Aihara, Tetsuya Takiguchi, Yasuo Ariki, Nobuyuki Mitani, Kiyohiro Omori, Kaoru Nakazono
    • Journal Title

      IPSJ Transactions on Computer Vision and Applications

      Volume: 7 Pages: 64-68

    • NAID

      130005091225

    • Related Report
      2015 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Multimodal voice conversion based on non-negative matrix factorization2015

    • Author(s)
      Kenta Masaka, Ryo Aihara, Tetsuya Takiguchi, Yasuo Ariki
    • Journal Title

      EURASIP Journal on Audio, Speech, and Music Processing

      Volume: 2015:24 Issue: 1 Pages: 1-9

    • DOI

      10.1186/s13636-015-0067-4

    • Related Report
      2015 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Small-parallel exemplar-based voice conversion in noisy environments using affine non-negative matrix factorization2015

    • Author(s)
      Ryo Aihara, Takao Fujii, Toru Nakashika, Tetsuya Takiguchi, Yasuo Ariki
    • Journal Title

      EURASIP Journal on Audio, Speech, and Music Processing

      Volume: 2015:32 Issue: 1 Pages: 1-9

    • DOI

      10.1186/s13636-015-0075-4

    • Related Report
      2015 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Voice Conversion Based on Speaker-dependent Restricted Boltzmann Machines2014

    • Author(s)
      Toru Nakashika, Tetsuya Takiguchi, Yasuo Ariki
    • Journal Title

      IEICE Transactions on Information and Systems

      Volume: E97-D Pages: 1403-1410

    • NAID

      130004841772

    • Related Report
      2014 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Noise-Robust Voice Conversion Based on Sparse Spectral Mapping Using Non-negative Matrix Factorization2014

    • Author(s)
      Ryo Aihara, Ryoichi Takashima, Tetsuya Takiguchi, Yasuo Ariki
    • Journal Title

      IEICE Transactions on Information and Systems

      Volume: E97-D Pages: 1411-1418

    • NAID

      130004841773

    • Related Report
      2014 Annual Research Report
    • Peer Reviewed
  • [Journal Article] A preliminary demonstration of exemplar-based voice conversion for articulation disorders using an individuality-preserving dictionary2014

    • Author(s)
      Ryo Aihara, Ryoichi Takashima, Tetsuya Takiguchi, Yasuo Ariki
    • Journal Title

      EURASIP Journal on Audio, Speech, and Music Processing

      Volume: 2014:5 Issue: 1 Pages: 1-10

    • DOI

      10.1186/1687-4722-2014-5

    • NAID

      120005650137

    • Related Report
      2014 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Exemplar-Based Voice Conversion Using Sparse Representation in Noisy Environments2013

    • Author(s)
      Ryoichi Takashima, Tetsuya Takiguchi, Yasuo Ariki
    • Journal Title

      IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences

      Volume: E96.A Issue: 10 Pages: 1946-1953

    • DOI

      10.1587/transfun.E96.A.1946

    • NAID

      130004519150

    • ISSN
      0916-8508, 1745-1337
    • Related Report
      2013 Annual Research Report
    • Peer Reviewed
  • [Presentation] 適応型Gaussian-Gaussian RBMを用いた構音障害者音声認識2017

    • Author(s)
      高島 悠樹, 中鹿 亘, 滝口 哲也, 有木 康雄
    • Organizer
      日本音響学会2017年春季研究発表会講演論文集
    • Place of Presentation
      明治大学(神奈川県・川崎市)
    • Year and Date
      2017-03-15
    • Related Report
      2016 Annual Research Report
  • [Presentation] 構音障害者のための話者性を維持したHMM音声合成システムの提案2017

    • Author(s)
      上田 怜奈, 滝口 哲也, 有木 康雄
    • Organizer
      日本音響学会2017年春季研究発表会講演論文集
    • Place of Presentation
      明治大学(神奈川県・川崎市)
    • Year and Date
      2017-03-15
    • Related Report
      2016 Annual Research Report
  • [Presentation] 声質変換における非周期性指標の影響とその評価2017

    • Author(s)
      伊藤 大貴, 滝口 哲也, 有木 康雄
    • Organizer
      日本音響学会2017年春季研究発表会講演論文集
    • Place of Presentation
      明治大学(神奈川県・川崎市)
    • Year and Date
      2017-03-15
    • Related Report
      2016 Annual Research Report
  • [Presentation] 最尤変換による唇動画像からの音声生成2017

    • Author(s)
      羅 里奈, 相原 龍, 滝口 哲也, 有木 康雄
    • Organizer
      日本音響学会2017年春季研究発表会講演論文集
    • Place of Presentation
      明治大学(神奈川県・川崎市)
    • Year and Date
      2017-03-15
    • Related Report
      2016 Annual Research Report
  • [Presentation] Factored 3-Way Restricted Boltzmann Machine を用いたマルチモーダル音声認識の検討2016

    • Author(s)
      高島 悠樹, 中鹿 亘, 滝口 哲也, 有木 康雄
    • Organizer
      日本音響学会2016年秋季研究発表会講演論文集
    • Place of Presentation
      富山大学(富山県・富山市)
    • Year and Date
      2016-09-14
    • Related Report
      2016 Annual Research Report
  • [Presentation] 話速補正に基づく話者性を維持した構音障害者のための音声合成システム2016

    • Author(s)
      上田 怜奈, 滝口 哲也, 有木 康雄
    • Organizer
      日本音響学会2016年秋季研究発表会講演論文集
    • Place of Presentation
      富山大学(富山県・富山市)
    • Year and Date
      2016-09-14
    • Related Report
      2016 Annual Research Report
  • [Presentation] 非負値行列因子を用いたマルチモーダル声質変換における画像特徴量の検討2016

    • Author(s)
      羅 里奈, 相原 龍, 滝口 哲也, 有木 康雄
    • Organizer
      日本音響学会2016年秋季研究発表会講演論文集
    • Place of Presentation
      富山大学(富山県・富山市)
    • Year and Date
      2016-09-14
    • Related Report
      2016 Annual Research Report
  • [Presentation] 複素NMFを用いた声質変換の検討2016

    • Author(s)
      李 権俊, 相原 龍, 滝口 哲也, 有木 康雄
    • Organizer
      日本音響学会2016年秋季研究発表会講演論文集
    • Place of Presentation
      富山大学(富山県・富山市)
    • Year and Date
      2016-09-14
    • Related Report
      2016 Annual Research Report
  • [Presentation] Emotional Voice Conversion Using Neural Networks with Different Temporal Scales of F0 based on Wavelet Transform2016

    • Author(s)
      Zhaojie Luo, Tetsuya Takiguchi, Yasuo Ariki, Toru Nakashika
    • Organizer
      9th ISCA Speech Synthesis Workshop
    • Place of Presentation
      サニーベール(米国)
    • Year and Date
      2016-09-13
    • Related Report
      2016 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Audio-Visual Speech Recognition Using Bimodal-Trained Bottleneck Features for a Person with Severe Hearing Loss2016

    • Author(s)
      Yuki Takashima, Ryo Aihara, Tetsuya Takiguchi, Yasuo Ariki, Nobuyuki Mitani, Kiyohiro Omori, Kaoru Nakazono
    • Organizer
      Interspeech
    • Place of Presentation
      サンフランシスコ(米国)
    • Year and Date
      2016-09-08
    • Related Report
      2016 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Lip Reading Using a Dynamic Feature of Lip Images and Convolutional Neural Networks2016

    • Author(s)
      Yiting Li, Yuki Takashima, Tetsuya Takiguchi, Yasuo Ariki
    • Organizer
      IEEE/ACIS International Conference on Computer and Information Science
    • Place of Presentation
      岡山コンベンションセンター(岡山県・岡山市)
    • Year and Date
      2016-06-26
    • Related Report
      2016 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Emotional Voice Conversion Using Deep Neural Networks with MCC and F0 Features2016

    • Author(s)
      Zhaojie Luo, Tetsuya Takiguchi, Yasuo Ariki
    • Organizer
      IEEE/ACIS International Conference on Computer and Information Science
    • Place of Presentation
      岡山コンベンションセンター(岡山県・岡山市)
    • Year and Date
      2016-06-26
    • Related Report
      2016 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Selection of an Optimum Random Matrix Using a Genetic Algorithm for Acoustic Feature Extraction2016

    • Author(s)
      Yuichiro Kataoka, Toru Nakashika, Ryo Aihara, Tetsuya Takiguchi, Yasuo Ariki
    • Organizer
      IEEE/ACIS International Conference on Computer and Information Science
    • Place of Presentation
      岡山コンベンションセンター(岡山県・岡山市)
    • Year and Date
      2016-06-26
    • Related Report
      2016 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Restricted Boltzmann Machine を用いた話者性・雑音を考慮したモデリングの検討2016

    • Author(s)
      高島 悠樹, 中鹿 亘, 滝口 哲也, 有木 康雄
    • Organizer
      日本音響学会2016年春季研究発表会講演論文集
    • Place of Presentation
      桐蔭横浜大学(神奈川県・横浜市)
    • Year and Date
      2016-03-09
    • Related Report
      2015 Annual Research Report
  • [Presentation] Emotional Speech Conversion Using Deep Neural Networks2016

    • Author(s)
      羅 兆杰, 滝口 哲也, 有木 康雄
    • Organizer
      日本音響学会2016年春季研究発表会講演論文集
    • Place of Presentation
      桐蔭横浜大学(神奈川県・横浜市)
    • Year and Date
      2016-03-09
    • Related Report
      2015 Annual Research Report
  • [Presentation] スパースパラレル学習を用いたマルチモーダル声質変換2016

    • Author(s)
      真坂 健太, 相原 龍, 滝口 哲也, 有木 康雄
    • Organizer
      日本音響学会2016年春季研究発表会講演論文集
    • Place of Presentation
      桐蔭横浜大学(神奈川県・横浜市)
    • Year and Date
      2016-03-09
    • Related Report
      2015 Annual Research Report
  • [Presentation] Expression Recognition with Ri-HOG Cascade2016

    • Author(s)
      Jinhui Chen, Zhaojie Luo, Tetsuya Takiguchi, Yasuo Ariki
    • Organizer
      Third Workshop on Computer Vision for Affective Computing
    • Place of Presentation
      台北(台湾)
    • Related Report
      2016 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Dysarthric Speech Modification Using Parallel Utterance Based on Non-negative Temporal Decomposition2016

    • Author(s)
      Ryo Aihara, Tetsuya Takiguchi, and Yasuo Ariki
    • Organizer
      7th Workshop on Speech and Language Processing for Assistive Technologies
    • Place of Presentation
      サンフランシスコ(米国)
    • Related Report
      2016 Annual Research Report
    • Int'l Joint Research
  • [Presentation] 話者正規化学習に基づく潜在的音韻情報を考慮した音声モデリングによる非パラレル声質変換2015

    • Author(s)
      中鹿 亘, 滝口 哲也
    • Organizer
      日本音響学会2015年秋季研究発表会講演論文集
    • Place of Presentation
      会津大学(福島県・会津若松市)
    • Year and Date
      2015-09-16
    • Related Report
      2015 Annual Research Report
  • [Presentation] 構音障害者音声認識のための確率表現に基づく音素ラベリングの検討2015

    • Author(s)
      高島 悠樹, 中鹿 亘, 滝口 哲也, 有木 康雄
    • Organizer
      日本音響学会2015年秋季研究発表会講演論文集
    • Place of Presentation
      会津大学(福島県・会津若松市)
    • Year and Date
      2015-09-16
    • Related Report
      2015 Annual Research Report
  • [Presentation] Individuality-Preserving Spectrum Modification for Articulation Disorders Using Phone Selective Synthesis2015

    • Author(s)
      Reina Ueda, Ryo Aihara, Tetsuya Takiguchi, Yasuo Ariki
    • Organizer
      Workshop on Speech and Language Processing for Assistive Technologies
    • Place of Presentation
      ドレスデン工科大学 (ドレスデン・ドイツ)
    • Year and Date
      2015-09-11
    • Related Report
      2015 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Feature Extraction Using Pre-Trained Convolutive Bottleneck Nets for Dysarthric Speech Recognition2015

    • Author(s)
      Yuki Takashima, Toru Nakashika, Tetsuya Takiguchi, Yasuo Ariki
    • Organizer
      EUSIPCO
    • Place of Presentation
      ニースアクロポリスコンベンションセンター (ニース・フランス)
    • Year and Date
      2015-08-31
    • Related Report
      2015 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Deep Boltzmann Machinesを用いた音素ラベル情報の推定2015

    • Author(s)
      高島悠樹, 中鹿亘, 滝口哲也, 有木康雄
    • Organizer
      日本音響学会2015年春季研究発表会講演論文集
    • Place of Presentation
      中央大学 (東京)
    • Year and Date
      2015-03-16 – 2015-03-18
    • Related Report
      2014 Annual Research Report
  • [Presentation] Normalized Similarity Distance を用いた音声認識の誤り訂正法2015

    • Author(s)
      房安陽平, 滝口哲也, 有木康雄
    • Organizer
      日本音響学会2015年春季研究発表会講演論文集
    • Place of Presentation
      中央大学 (東京)
    • Year and Date
      2015-03-16 – 2015-03-18
    • Related Report
      2014 Annual Research Report
  • [Presentation] Exemplar-based Emotional Voice Conversion Using Non-negative Matrix Factorization2014

    • Author(s)
      Ryo Aihara, Reina Ueda, Tetsuya Takiguchi, and Yasuo Ariki
    • Organizer
      APSIPA
    • Place of Presentation
      Sokha Angkor Resort (カンボジア)
    • Year and Date
      2014-12-09 – 2014-12-12
    • Related Report
      2014 Annual Research Report
  • [Presentation] Error Correction of Automatic Speech Recognition Based on Normalized Web Distance2014

    • Author(s)
      E. Byambakhishig, K. Tanaka, R. Aihara, T. Nakashika, T. Takiguchi, Y. Ariki
    • Organizer
      Interspeech
    • Place of Presentation
      Singapore EXPO (シンガポール)
    • Year and Date
      2014-09-14 – 2014-09-18
    • Related Report
      2014 Annual Research Report
  • [Presentation] Multimodal Exemplar-based Voice Conversion using Lip Features in Noisy Environments2014

    • Author(s)
      Kenta Masaka, Ryo Aihara, Tetsuya Takiguchi, Yasuo Ariki
    • Organizer
      Interspeech
    • Place of Presentation
      Singapore EXPO (シンガポール)
    • Year and Date
      2014-09-14 – 2014-09-18
    • Related Report
      2014 Annual Research Report
  • [Presentation] アクティビティマッピングによる非負値行列因子分解を用いた声質変換2014

    • Author(s)
      相原龍, 滝口哲也, 有木康雄
    • Organizer
      日本音響学会2014年秋季研究発表会講演論文集
    • Place of Presentation
      北海学園大学 (北海道)
    • Year and Date
      2014-09-03 – 2014-09-05
    • Related Report
      2014 Annual Research Report
  • [Presentation] A Robust Learning Algorithm Based on SURF and PSM for Facial Expressions Recognition2014

    • Author(s)
      Jinhui Chen, Tetsuya Takiguchi, Yasuo Ariki
    • Organizer
      画像の認識・理解シンポジウム
    • Place of Presentation
      岡山コンベンションセンター (岡山)
    • Year and Date
      2014-07-28 – 2014-07-31
    • Related Report
      2014 Annual Research Report
  • [Presentation] スパース表現に基づく声質変換と構音障害者への応用2014

    • Author(s)
      滝口哲也
    • Organizer
      電子情報通信学会技術研究報告
    • Place of Presentation
      ホテル花巻 (岩手)
    • Year and Date
      2014-07-24 – 2014-07-26
    • Related Report
      2014 Annual Research Report
    • Invited
  • [Presentation] Individuality-preserving Voice Conversion for Articulation Disorders Using Dictionary Selective Non-negative Matrix Factorization2014

    • Author(s)
      Ryo Aihara, Tetsuya Takiguchi, Yasuo Ariki
    • Organizer
      Workshop on Speech and Language Processing for Assistive Technologies
    • Place of Presentation
      Baltimore Marriott Waterfront (ボルチモア)
    • Year and Date
      2014-06-26
    • Related Report
      2014 Annual Research Report
  • [Presentation] Convolutive Bottleneck Network 特徴量を用いた構音障害者の音声認識2014

    • Author(s)
      吉岡利也, 中鹿亘, 滝口哲也, 有木康雄
    • Organizer
      日本音響学会2014年春季研究発表会
    • Place of Presentation
      東京
    • Related Report
      2013 Annual Research Report
  • [Presentation] 様々なRandom行列を用いた構音障害者の音声特徴量抽出2014

    • Author(s)
      片岡悠一郎, 吉岡利也, 滝口哲也, 有木康雄
    • Organizer
      日本音響学会2014年春季研究発表会
    • Place of Presentation
      東京
    • Related Report
      2013 Annual Research Report
  • [Presentation] Robust Facial Expressions Recognition Using 3D Average Face and Ameliorated AdaBoost2013

    • Author(s)
      Jinhui Chen, Yasuo Ariki, Tetsuya Takiguchi
    • Organizer
      ACM Multimedia
    • Place of Presentation
      バルセロナ(スペイン)
    • Related Report
      2013 Annual Research Report
  • [Presentation] Voice Conversion based on Non-negative Matrix Factorization in Noisy Environments2013

    • Author(s)
      Takao Fujii, Ryo Aihara, Ryoichi Takashima, Tetsuya Takiguchi, Yasuo Ariki
    • Organizer
      IEEE/SICE International Symposium on System Integration
    • Place of Presentation
      神戸
    • Related Report
      2013 Annual Research Report
  • [Presentation] 辞書選択型非負値行列因子分解による構音障害者の声質変換2013

    • Author(s)
      相原 龍,滝口 哲也,有木 康雄
    • Organizer
      電子情報通信学会技術研究報告
    • Place of Presentation
      東京
    • Related Report
      2013 Annual Research Report
  • [Presentation] Convolutional Neural Networksを用いた構音障害者のための音声認識2013

    • Author(s)
      吉岡利也,中鹿亘,滝口哲也,有木康雄
    • Organizer
      日本音響学会2013年秋季研究発表会
    • Place of Presentation
      豊橋
    • Related Report
      2013 Annual Research Report
  • [Presentation] セグメント特徴を考慮したNMFを用いた雑音環境下の声質変換2013

    • Author(s)
      藤井貴生,相原龍,滝口哲也,有木康雄
    • Organizer
      日本音響学会2013年秋季研究発表会
    • Place of Presentation
      豊橋
    • Related Report
      2013 Annual Research Report

URL: 

Published: 2013-05-21   Modified: 2019-07-29  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi