• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Understanding SHITSUKAN recognition mechanisms in speech perception based on concept of amplitude modulation

Publicly Offered Research

Project AreaUnderstanding human recognition of material properties for innovation in SHITSUKAN science and technology
Project/Area Number 18H05004
Research Category

Grant-in-Aid for Scientific Research on Innovative Areas (Research in a proposed research area)

Allocation TypeSingle-year Grants
Research InstitutionJapan Advanced Institute of Science and Technology

Principal Investigator

鵜木 祐史  北陸先端科学技術大学院大学, 先端科学技術研究科, 教授 (00343187)

Project Period (FY) 2018-04-01 – 2020-03-31
Project Status Completed (Fiscal Year 2019)
Budget Amount *help
¥8,320,000 (Direct Cost: ¥6,400,000、Indirect Cost: ¥1,920,000)
Fiscal Year 2019: ¥4,160,000 (Direct Cost: ¥3,200,000、Indirect Cost: ¥960,000)
Fiscal Year 2018: ¥4,160,000 (Direct Cost: ¥3,200,000、Indirect Cost: ¥960,000)
Keywords質感認識 / 変調スペクトル / 変調伝達関数 / 変調フィルタバンク / 振幅変調 / 緊迫感知覚 / 年齢知覚 / 音声の質感認識 / 振幅包絡線 / 聴覚フィルタバンク / 緊迫性
Outline of Annual Research Achievements

本研究では、振幅変調の概念に基づき,音声の質感を認識するメカニズムを理解することを目指す.本研究課題では,(1) 音の振幅包絡線情報の瞬時的な変調スペクトルの分析法の構築,(2)音声の質感認識に関係する物理量の解明,(3) 音声の質感認識における音源と伝送系の質感(場の雰囲気)の関係性の調査の3点を踏まえ,音声の質感認識のメカニズムを検討した.今年度は,課題(2)と課題(3)に取り組んだ.
課題(2)では,音声の質感認識として音の「粗さ」に係わる物理特徴として,音声の振幅包絡線情報を変調スペクトル分析から検討した.音声の非言語情報(感情や個人性)の知覚では変調特性として8 Hz以上が重要であることを,パラ言語情報(緊迫感)の知覚では,変調特性として6 Hz~8 Hzが重要であることを明らかにした.これらの変調周波数は長時間にわたる平均的なものであるが,振幅包絡線情報の反転呈示の聴取実験の結果から,音の質感認識には瞬時的な変調周波数(変調周波数の時間変化)が重要であることも明らかにした.
課題(3)では,音声の質感認識が雑音残響にどのような影響を受けるか,Schroederの室内インパルス応答を利用した残響環境(残響時間0.1, 0.2, 0.5, 1.0, 2.0秒)と白色性ガウス雑音を利用した雑音環境(SN比 20, 15, 10, 5, 0, -5 dB)ならびにこれらを混合した雑音・残響環境下で検討した.その結果,SN比 で10 dB 以上でかつ残響で1.0 秒未満のような条件では,音声の質感認識が影響を受けないことがわかった.
これらを俯瞰的に眺め,音声の質感認識(非言語・パラ言語情報の知覚)を考えると,音声の振幅包絡線情報(変調スペクトル)を特徴として,音声の質感と環境の伝送特性(変調伝達特性)を切り分けて,音の質感を認識していると解釈できる.

Research Progress Status

令和元年度が最終年度であるため、記入しない。

Strategy for Future Research Activity

令和元年度が最終年度であるため、記入しない。

Report

(2 results)
  • 2019 Annual Research Report
  • 2018 Annual Research Report
  • Research Products

    (34 results)

All 2020 2019 2018 Other

All Journal Article (8 results) (of which Int'l Joint Research: 2 results,  Peer Reviewed: 7 results,  Open Access: 6 results) Presentation (23 results) (of which Int'l Joint Research: 7 results,  Invited: 1 results) Remarks (3 results)

  • [Journal Article] Relationship between contributions of temporal amplitude envelope of speech and modulation transfer function in room acoustics to perception of noise-vocoded speech2020

    • Author(s)
      Masashi Unoki and Zhi Zhu
    • Journal Title

      Acoustical Science and Technology

      Volume: 41 Issue: 1 Pages: 233-244

    • DOI

      10.1250/ast.41.233

    • NAID

      130007782607

    • ISSN
      0369-4232, 1346-3969, 1347-5177
    • Year and Date
      2020-01-01
    • Related Report
      2019 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Speech Emotion Recognition Using 3D Convolutions and Attention-Based Sliding Recurrent Networks With Auditory Front-Ends2020

    • Author(s)
      Zhichao Peng, Xingfeng Li, Zhi Zhu, Masashi Unoki, Jianwu Dang, Masato Akagi
    • Journal Title

      IEEE Access

      Volume: 8 Pages: 16560-16572

    • DOI

      10.1109/access.2020.2967791

    • NAID

      120006783353

    • Related Report
      2019 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] 雑音・残響環境における雑音駆動音声の非言語情報の知覚に関する検討2020

    • Author(s)
      朱治,川村美帆,鵜木祐史,
    • Journal Title

      日本音響学会誌

      Volume: 76

    • NAID

      130007948461

    • Related Report
      2019 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Estimates of Transmission Characteristics Related to Perception of Bone-Conducted Speech Using Real Utterances and Transcutaneous Vibration on Larynx2019

    • Author(s)
      Teruki Toya, Peter Birkholz, and Masashi Unoki
    • Journal Title

      Lecture Notes in Computer Sciencebook series (LNCS, volume 11658)

      Volume: 11658 Pages: 491-500

    • Related Report
      2019 Annual Research Report
    • Peer Reviewed / Int'l Joint Research
  • [Journal Article] How the temporal amplitude envelope of speech contributes to urgency perception2019

    • Author(s)
      Masashi Unoki, Miho Kawamura, Maori Kobayashi, Shunsuke Kidani, Masato Akagi
    • Journal Title

      Proceedings of 23rd International Congress on Acoustics

      Volume: - Pages: 1739-1744

    • Related Report
      2019 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Contribution of modulation spectral features on the perception of vocal-emotion using noise-vocoded speech2018

    • Author(s)
      Zhi Zhu, Yukiko Araki, Ryota Miyauchi and Masashi Unoki
    • Journal Title

      Acoustical Science and Technology

      Volume: 39 Issue: 6 Pages: 379-386

    • DOI

      10.1250/ast.39.379

    • NAID

      40021703797

    • ISSN
      0369-4232, 1346-3969, 1347-5177
    • Year and Date
      2018-11-01
    • Related Report
      2018 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Contributions of temporal cue on the perception of speaker individuality and vocal emotion for noise-vocoded speech2018

    • Author(s)
      Zhi Zhu, Yukiko Araki, Ryota Miyauchi and Masashi Unoki
    • Journal Title

      Acoustical Science and Technology

      Volume: 39 Issue: 3 Pages: 234-242

    • DOI

      10.1250/ast.39.234

    • NAID

      130006730824

    • ISSN
      0369-4232, 1346-3969, 1347-5177
    • Year and Date
      2018-05-01
    • Related Report
      2018 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] “Noise Suppression Method Based on Modulation Spectrum Analysis,”2018

    • Author(s)
      Takuto Isoyama and Masashi Unoki
    • Journal Title

      SPECOM2018: Speech and Computer, A. Karpov et al. (Eds.), Springer LNAI

      Volume: 11096 Pages: 234-244

    • DOI

      10.1007/978-3-319-99579-3_25

    • ISBN
      9783319995786, 9783319995793
    • Related Report
      2018 Annual Research Report
  • [Presentation] Pitch perception of noise-vocoded harmonic complex tones mimicking musical instruments2020

    • Author(s)
      Masashi Unoki, Yukina Hosaka, and Shunsuke Kidani
    • Organizer
      Forum Acusticum 2020
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Modeling of sound quality metrics using gammatone and gammachirp filterbanks2020

    • Author(s)
      Takuto Isoyama, Shunsuke Kidani, and Masashi Unoki
    • Organizer
      Forum Acusticum 2020
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Improvement of bone-conducted speech restoration using linear prediction and long short-term memory model2020

    • Author(s)
      Huy Quoc Nguyen and Masashi Unoki
    • Organizer
      Proc. 2020 RISP International Workshop on Nonlinear Circuits, Communications and Signal Processing (NCSP2020)
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research
  • [Presentation] 聴覚フィルタバンクを利用した定常音に対するラウドネスモデルの構築2020

    • Author(s)
      磯山拓都, 木谷俊介, 鵜木祐史
    • Organizer
      電子情報通信学会EA研究会,石垣島
    • Related Report
      2019 Annual Research Report
  • [Presentation] 話者自身が知覚する音声における骨導音声の伝達割合2020

    • Author(s)
      鳥谷輝樹,Peter Birkholz,鵜木祐史
    • Organizer
      日本音響学会2020年春季研究発表会,埼玉大
    • Related Report
      2019 Annual Research Report
  • [Presentation] ガンマトーンフィルタバンクを用いたラウドネスモデルの構築2020

    • Author(s)
      磯山拓都, 水野滉介, 木谷俊介, 鵜木祐史
    • Organizer
      日本音響学会2020年春季研究発表会,埼玉大
    • Related Report
      2019 Annual Research Report
  • [Presentation] 雑音駆動合成における調波複合音のピッチ知覚の検討2020

    • Author(s)
      寳坂友希菜,木谷俊介,鵜木祐史
    • Organizer
      日本音響学会2020年春季研究発表会,埼玉大
    • Related Report
      2019 Annual Research Report
  • [Presentation] 楽音を模した調波複合音の雑音駆動合成音のピッチ知覚の検討2020

    • Author(s)
      寳坂友希菜,木谷俊介,鵜木祐史
    • Organizer
      日本音響学会聴覚研究会,琉球大
    • Related Report
      2019 Annual Research Report
  • [Presentation] 楽音を模した調波複合音の雑音駆動合成音のピッチ知覚の検討2019

    • Author(s)
      寳坂友希菜,木谷俊介,鵜木祐史
    • Organizer
      日本音響学会聴覚研究会,ホテル三日月
    • Related Report
      2019 Annual Research Report
  • [Presentation] 緊迫感知覚に寄与する音声の振幅包絡線情報の検討2019

    • Author(s)
      鵜木祐史,川村美帆,木谷俊介,小林まおり,赤木正人
    • Organizer
      日本音響学会騒音振動研究会,金沢工大
    • Related Report
      2019 Annual Research Report
  • [Presentation] 骨導音声の外耳道内放射特性の推定2019

    • Author(s)
      鳥谷輝樹,Peter Birkholz, 鵜木祐史
    • Organizer
      日本音響学会2019年秋季研究発表会,立命館大
    • Related Report
      2019 Annual Research Report
  • [Presentation] 雑音残響環境における雑音駆動音声の非言語情報知覚の検討2019

    • Author(s)
      朱治,川村美帆,鵜木祐史
    • Organizer
      日本音響学会聴覚研究会,東北大
    • Related Report
      2019 Annual Research Report
  • [Presentation] UnokiStudy on the method for estimating perceptual age using sound quality metrics2019

    • Author(s)
      Tatsuya Hatakeyama and Masashi Unoki
    • Organizer
      Proc. 2019 RISP International Workshop on Nonlinear Circuits, Communications and Signal Processing (NCSP19)
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] 雑音駆動音声の緊迫感知覚の検討2019

    • Author(s)
      川村美帆,鵜木祐史
    • Organizer
      日本音響学会2019年度春季研究発表会,電気通信大学
    • Related Report
      2018 Annual Research Report
  • [Presentation] 振幅包絡線に含まれる緊迫感の知覚,2019

    • Author(s)
      川村美帆,小林まおり,木谷俊介,赤木正人,鵜木祐史
    • Organizer
      日本音響学会聴覚研究会,愛媛大学
    • Related Report
      2018 Annual Research Report
  • [Presentation] Relationship between contributions of temporal amplitude envelope of speech and modulation transfer function in room acoustics to the perception of noise-vocoded speech2018

    • Author(s)
      Masashi Unoki and Zhi Zhu
    • Organizer
      Tohoku Universal Acoustical Communication Month 2018,
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research / Invited
  • [Presentation] Study on the relationship between modulation spectral features and the perception of vocal emotion with noise-vocoded speech2018

    • Author(s)
      Zhu Zhi, Ryota Miyauchi, Yukiko Araki, and Masashi Unoki
    • Organizer
      176th Meeting of the Acoustical Society of America, 2018 Acoustics Week in Canada
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Auditory-inspired end-to-end speech emotion recognition using 3D convolutional recurrent neural networks based on spectral-temporal modulation2018

    • Author(s)
      Zhichao Peng, Zhi Zhu, Masashi Unoki, Jianwu Dang, Masato Akagi
    • Organizer
      IEEE International Conference on Multimedia and Expo (ICME) 2018, San Diego, CA, USA
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] 音質評価指標を用いた年齢知覚に関する検討2018

    • Author(s)
      畠山達也,鵜木祐史
    • Organizer
      日本音響学会聴覚研究会,ホテルこうしゅうえん
    • Related Report
      2018 Annual Research Report
  • [Presentation] 雑音残響環境における雑音駆動音声の個人性及び感情情報の知覚に関する検討2018

    • Author(s)
      朱治,川村美帆,関谷伸一,鵜木祐史
    • Organizer
      日本音響学会2018年度秋季研究発表会,大分大学
    • Related Report
      2018 Annual Research Report
  • [Presentation] 音声の知覚年齢と音質評価指標の関係2018

    • Author(s)
      畠山達也,鵜木祐史
    • Organizer
      第30回電気関係学会北陸支部連合大会,JAIST
    • Related Report
      2018 Annual Research Report
  • [Presentation] 雑音駆動声の 雑音駆動声の個人性・感情知覚における雑音残響環境の影響2018

    • Author(s)
      川村美帆, 朱治, 関谷伸一, 鵜木祐史
    • Organizer
      第30回電気関係学会北陸支部連合大会,JAIST
    • Related Report
      2018 Annual Research Report
  • [Presentation] 雑音残響環境が駆動声の個人性及び感情報の知覚に与える影響2018

    • Author(s)
      川村美帆, 朱治, 関谷伸一, 鵜木祐史
    • Organizer
      電子情報通信学会応用音響研究会,東北学院大学
    • Related Report
      2018 Annual Research Report
  • [Remarks] 多元質感知

    • URL

      http://shitsukan.jp/ISST/

    • Related Report
      2019 Annual Research Report
  • [Remarks] Science Impact

    • URL

      https://www.ingentaconnect.com/content/sil/impact/2020/00002020/00000002/art00008

    • Related Report
      2019 Annual Research Report
  • [Remarks] 多元質感知・公募研究D01-6

    • URL

      http://shitsukan.jp/ISST/advertise/index.html

    • Related Report
      2018 Annual Research Report

URL: 

Published: 2018-04-23   Modified: 2021-01-27  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi