• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Sound event detection method capable of analyzing any environmental sound

Research Project

Project/Area Number 19K20304
Research Category

Grant-in-Aid for Early-Career Scientists

Allocation TypeMulti-year Fund
Review Section Basic Section 61010:Perceptual information processing-related
Research InstitutionDoshisha University (2020-2021)
Ritsumeikan University (2019)

Principal Investigator

Imoto Keisuke  同志社大学, 理工学部, 准教授 (90802116)

Project Period (FY) 2019-04-01 – 2022-03-31
Project Status Completed (Fiscal Year 2021)
Budget Amount *help
¥4,160,000 (Direct Cost: ¥3,200,000、Indirect Cost: ¥960,000)
Fiscal Year 2021: ¥1,040,000 (Direct Cost: ¥800,000、Indirect Cost: ¥240,000)
Fiscal Year 2020: ¥1,430,000 (Direct Cost: ¥1,100,000、Indirect Cost: ¥330,000)
Fiscal Year 2019: ¥1,690,000 (Direct Cost: ¥1,300,000、Indirect Cost: ¥390,000)
Keywords環境音分析 / 音響イベント検出 / 音響シーン分類 / マルチタスク学習 / スパース性 / 不均衡データ / グラフラプラシアン正則化 / ケプストラム / 音響シーン分析
Outline of Research at the Start

本研究課題は,音響イベントには共起性・スパース性がある点や,収録環境(シーン)と発生する音響イベントの種類が関連している点に着目し,これらをモデル学習時の拘束条件に課すことで,少量の環境音データを用いた場合でも高性能な分析を可能とする音響イベント分析法の実現を目指す.
研究期間内に,音響イベントの共起性やスパース性を学習則に組み入れた深層学習法の提案や,音響シーンとイベントのマルチタスク学習法,また,それらを組み合わせた手法の提案を目指す.

Outline of Final Research Achievements

This research project aims to develop a method to achieve reasonable performance in environmental sound analysis, which is one of the most important topics in acoustic processing, even when only a small amount of environmental sound data is available. In particular, we investigated a method for sound event detection (SED) based on the co-occurrence of environmental sounds and the omnipresence of sound events in an acoustic scene. During the research period, we have proposed a method considering the co-occurrence of acoustic events with deep learning methods, a multi-task learning method of SED and acoustic scene classification, and a model learning technique that does not cause performance degradation even when there is a data imbalance between sound event classes, showing that sound events can be detected with high accuracy.

Academic Significance and Societal Importance of the Research Achievements

人間の耳のように,様々な環境音の種類を聞き分ける技術が実現できれば,補聴器などの聴覚補助システムのみならず,公共スペースでの自動監視システム,高齢者や乳幼児の見守りシステム,自動運転の補助,環境の自動モニタリング,知的ロボットなど様々なサービスに広く貢献できる.このように,環境音分析は音響処理の中でも非常に重要な技術として位置づけられる.また,画像/動画の分析などの技術と組み合わせることで,人間の知覚を模した人工知能を実現することも可能となる.

Report

(4 results)
  • 2021 Annual Research Report   Final Research Report ( PDF )
  • 2020 Research-status Report
  • 2019 Research-status Report
  • Research Products

    (47 results)

All 2022 2021 2020 2019

All Journal Article (11 results) (of which Peer Reviewed: 8 results,  Open Access: 4 results) Presentation (35 results) (of which Int'l Joint Research: 15 results,  Invited: 2 results) Patent(Industrial Property Rights) (1 results)

  • [Journal Article] Research Trends in Environmental Sound Analysis and Anomalous Sound Detection2022

    • Author(s)
      井本 桂右, 川口 洋平
    • Journal Title

      IEICE ESS Fundamentals Review

      Volume: 15 Issue: 4 Pages: 268-280

    • DOI

      10.1587/essfr.15.4_268

    • ISSN
      1882-0875
    • Year and Date
      2022-04-01
    • Related Report
      2021 Annual Research Report
    • Open Access
  • [Journal Article] Onoma-to-wave: Environmental Sound Synthesis from Onomatopoeic Words2022

    • Author(s)
      Yuki Okamoto, Keisuke Imoto, Shinnosuke Takamichi, Ryosuke Yamanishi, Takahiro Fukumori, and Yoichi Yamashita
    • Journal Title

      APSIPA Transactions on Signal and Information Processing

      Volume: -

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] 誤検出の深刻さを考慮した音響イベント検出のための評価指標2022

    • Author(s)
      砺波 紀之, 井本 桂右, 岡本 悠希, 福森 隆寛, 山下 洋一
    • Journal Title

      日本音響学会誌

      Volume: -

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Sound Event Detection Guided by Semantic Contexts of Scenes2022

    • Author(s)
      Noriyuki Tonami, Keisuke Imoto, Ryotaro Nagase, Yuki Okamoto, Takahiro Fukumori, and Yoichi Yamashita
    • Journal Title

      Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

      Volume: -

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Joint Analysis of Sound Events and Acoustic Scenes Using Multitask Learning2021

    • Author(s)
      Noriyuki TONAMI, Keisuke IMOTO, Ryosuke YAMANISHI, Yoichi YAMASHITA
    • Journal Title

      IEICE Transactions on Information and Systems

      Volume: E104.D Issue: 2 Pages: 294-301

    • DOI

      10.1587/transinf.2020EDP7036

    • NAID

      130007979514

    • ISSN
      0916-8532, 1745-1361
    • Year and Date
      2021-02-01
    • Related Report
      2020 Research-status Report
    • Peer Reviewed / Open Access
  • [Journal Article] Acoustic Scene Classification Using Multichannel Observation with Partially Missing Channels2021

    • Author(s)
      Imoto Keisuke
    • Journal Title

      Proceedings of European Signal Processing Conference (EUSIPCO)

      Volume: - Pages: 875-879

    • DOI

      10.23919/eusipco54536.2021.9616170

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Impact of Sound Duration and Inactive Frames on Sound Event Detection Performance2021

    • Author(s)
      Imoto Keisuke, Mishima Sakiko, Arai Yumi, Kondo Reishi
    • Journal Title

      Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

      Volume: - Pages: 860-864

    • DOI

      10.1109/icassp39728.2021.9414949

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Sound Event Detection Based on Curriculum Learning Considering Learning Difficulty of Events2021

    • Author(s)
      Tonami Noriyuki、Imoto Keisuke、Okamoto Yuki、Fukumori Takahiro、Yamashita Yoichi
    • Journal Title

      Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

      Volume: - Pages: 875-879

    • DOI

      10.1109/icassp39728.2021.9414184

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Multitask Learning of Acoustic Scenes and Events Using Dynamic Weight Adaptation Based on Multi-Focal Loss2021

    • Author(s)
      Kayo Nada, Keisuke Imoto, Reina Iwamae, and Nobutaka Ono
    • Journal Title

      Proceedings of Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC)

      Volume: - Pages: 1156-1160

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Graph Cepstrum: Spatial Feature Extracted from Partially Connected Microphones2020

    • Author(s)
      IMOTO Keisuke
    • Journal Title

      IEICE Transactions on Information and Systems

      Volume: E103.D Issue: 3 Pages: 631-638

    • DOI

      10.1587/transinf.2019EDP7162

    • NAID

      130007804152

    • ISSN
      0916-8532, 1745-1361
    • Year and Date
      2020-03-01
    • Related Report
      2019 Research-status Report
  • [Journal Article] Recent advances in environmental sound analysis2019

    • Author(s)
      井本 桂右
    • Journal Title

      THE JOURNAL OF THE ACOUSTICAL SOCIETY OF JAPAN

      Volume: 75 Issue: 9 Pages: 512-518

    • DOI

      10.20697/jasj.75.9_512

    • NAID

      130007804084

    • ISSN
      0369-4232, 2432-2040
    • Year and Date
      2019-09-01
    • Related Report
      2019 Research-status Report
    • Open Access
  • [Presentation] Sound Event Detection Guided by Semantic Contexts of Scenes2022

    • Author(s)
      Noriyuki Tonami, Keisuke Imoto, Ryotaro Nagase, Yuki Okamoto, Takahiro Fukumori, and Yoichi Yamashita
    • Organizer
      IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
    • Related Report
      2021 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Fundamentals and Recent Advances in Environmental Sound Analysis2021

    • Author(s)
      Keisuke Imoto
    • Organizer
      Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC)
    • Related Report
      2021 Annual Research Report
    • Int'l Joint Research / Invited
  • [Presentation] Multitask Learning of Acoustic Scenes and Events Using Dynamic Weight Adaptation Based on Multi-Focal Loss2021

    • Author(s)
      Kayo Nada, Keisuke Imoto, Reina Iwamae, and Nobutaka Ono
    • Organizer
      Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC)
    • Related Report
      2021 Annual Research Report
    • Int'l Joint Research
  • [Presentation] coustic Scene Classification Using Multichannel Observation with Partially Missing Channels2021

    • Author(s)
      Keisuke Imoto
    • Organizer
      European Signal Processing Conference (EUSIPCO)
    • Related Report
      2021 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Impact of Sound Duration and Inactive Frames on Sound Event Detection Performance2021

    • Author(s)
      Keisuke Imoto, Sakiko Mishima, Yumi Arai, and Reishi Kondo
    • Organizer
      IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
    • Related Report
      2021 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Sound Event Detection Based on Curriculum Learning Considering Difficulty of Events2021

    • Author(s)
      Noriyuki Tonami, Keisuke Imoto, Yuki Okamoto, Takahiro Fukumori, and Yoichi Yamashita
    • Organizer
      IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
    • Related Report
      2021 Annual Research Report
    • Int'l Joint Research
  • [Presentation] 音響イベントの強ラベル付与におけるアノテーター間のばらつきの分析2021

    • Author(s)
      井本 桂右, 賀谷 采珠, 椿 竣介
    • Organizer
      日本音響学会 2022年春季研究発表会
    • Related Report
      2021 Annual Research Report
  • [Presentation] 音響シーンとイベントが相互に及ぼす影響の調査2021

    • Author(s)
      小松 由佳, 井本 桂右, 小松 達也
    • Organizer
      日本音響学会 2022年春季研究発表会
    • Related Report
      2021 Annual Research Report
  • [Presentation] 弱ラベルを用いた音響シーンとイベントの同時分析2021

    • Author(s)
      椿 竣介, 宇都 瑛祐, 井本 桂右, 小野 順貴
    • Organizer
      日本音響学会 2022年春季研究発表会
    • Related Report
      2021 Annual Research Report
  • [Presentation] 事前定義されていないシーン情報を利用可能な音響イベント検出2021

    • Author(s)
      砺波 紀之, 井本 桂右, 永瀬 亮太郎, 岡本 悠希, 福森 隆寛, 山下 洋一
    • Organizer
      日本音響学会 2022年春季研究発表会
    • Related Report
      2021 Annual Research Report
  • [Presentation] グラフ深層学習を用いた音響イベントとシーンの同時分析2021

    • Author(s)
      髙橋 皓大, 井本 桂右, 土屋 隆生
    • Organizer
      日本音響学会 2022年春季研究発表会
    • Related Report
      2021 Annual Research Report
  • [Presentation] 環境音合成における主観評価手法の検討2021

    • Author(s)
      岡本 悠希, 井本 桂右, 高道 慎之介, 福森 隆寛, 山下 洋一
    • Organizer
      日本音響学会 2022年春季研究発表会
    • Related Report
      2021 Annual Research Report
  • [Presentation] Transformerを用いたオノマトペからの環境音合成2021

    • Author(s)
      岡本 悠希, 井本 桂右, 高道 慎之介, 山西 良典, 福森 隆寛, 山下 洋一
    • Organizer
      日本音響学会 2021年秋季研究発表会
    • Related Report
      2021 Annual Research Report
  • [Presentation] 音響イベントとシーンのマルチタスク学習における評価関数の重みの自動調整2021

    • Author(s)
      岩前 玲那, 白波瀬 壮, 髙橋 皓大, 井本 桂右, 土屋 隆生
    • Organizer
      日本音響学会 2021年春季研究発表会
    • Related Report
      2020 Research-status Report
  • [Presentation] 音響イベント長とイベント非活性区間長の不均衡が検出性能に及ぼす影響2021

    • Author(s)
      井本 桂右, 美島 咲子, 荒井 友督, 近藤 玲史
    • Organizer
      日本音響学会 2021年春季研究発表会
    • Related Report
      2020 Research-status Report
  • [Presentation] Onoma-to-wave: オノマトペからの環境音合成手法の提案2021

    • Author(s)
      岡本 悠希, 井本 桂右, 高道 慎之介, 山西 良典, 福森 隆寛, 山下 洋一
    • Organizer
      日本音響学会 2021年春季研究発表会
    • Related Report
      2020 Research-status Report
  • [Presentation] Experimental Investigation of Robustness of Spatial Cepstrum Features Under Various Recording Conditions2020

    • Author(s)
      Taiga Kawamura, Ryoichi Miyazaki, Keisuke Imoto, and Nobutaka Ono
    • Organizer
      Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC)
    • Related Report
      2020 Research-status Report
    • Int'l Joint Research
  • [Presentation] RWCP-SSD-Onomatopoeia: Onomatopoeic Word Dataset for Environmental Sound Synthesis2020

    • Author(s)
      Yuki Okamoto, Keisuke Imoto, Shinnosuke Takamichi, Ryosuke Yamanishi, Takahiro Fukumori, and Yoichi Yamashita
    • Organizer
      Detection and Classification of Acoustic Scenes and Events (DCASE) Workshop
    • Related Report
      2020 Research-status Report
    • Int'l Joint Research
  • [Presentation] Evaluation Metric of Sound Event Detection Considering Severe Misdetections by Scenes2020

    • Author(s)
      Noriyuki Tonami, Keisuke Imoto, Takahiro Fukumori, and Yoichi Yamashita
    • Organizer
      Detection and Classification of Acoustic Scenes and Events (DCASE) Workshop
    • Related Report
      2020 Research-status Report
    • Int'l Joint Research
  • [Presentation] Sound Event Detection by Multitask Learning of Sound Events and Scenes with Soft Scene Labels2020

    • Author(s)
      Keisuke Imoto, Noriyuki Tonami, Yuma Koizumi, Masahiro Yasuda, Ryosuke Yamanishi, and Yoichi Yamashita
    • Organizer
      IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
    • Related Report
      2020 Research-status Report
    • Int'l Joint Research
  • [Presentation] Scene-dependent Acoustic Event Detection with Scene Conditioning and Fake-scene-condition Loss2020

    • Author(s)
      Tatsuya Komatsu, Keisuke Imoto, and Masahito Togami
    • Organizer
      IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
    • Related Report
      2020 Research-status Report
    • Int'l Joint Research
  • [Presentation] 実環境におけるマイクロホンの移動に対する空間ケプストラムの頑健性の調査2020

    • Author(s)
      河村 泰雅, 宮崎 亮一, 井本 桂右
    • Organizer
      日本音響学会 2020年秋季研究発表会
    • Related Report
      2020 Research-status Report
  • [Presentation] 音響シーンを用いて検出誤りの深刻さを考慮したイベント検出の評価指標,2020

    • Author(s)
      砺波 紀之, 井本 桂右, 福森 隆寛, 山下 洋一
    • Organizer
      日本音響学会 2020年秋季研究発表会
    • Related Report
      2020 Research-status Report
  • [Presentation] 環境音分析ことはじめ2020

    • Author(s)
      井本 桂右
    • Organizer
      日本音響学会 電気音響研究会/電子情報通信学会 応用音響研究会
    • Related Report
      2020 Research-status Report
    • Invited
  • [Presentation] Crow Call Detection Using Gated Convolutional Recurrent Neural Network2020

    • Author(s)
      Yuki Okamoto, Keisuke Imoto, Naoki Tsukahara, Ken Nagata, Koh Sueda, Ryosuke Yamanishi, and Yoichi Yamashita
    • Organizer
      RISP International Workshop on Nonlinear Circuits, Communications and Signal Processing (NCSP)
    • Related Report
      2019 Research-status Report
    • Int'l Joint Research
  • [Presentation] RU Multichannel Domestic Acoustic Scenes 2019: 複数デバイスで構成された分散マイクアレイによる音響シーン分析のための環境音データセット2020

    • Author(s)
      井本 桂右, 小野 順貴
    • Organizer
      日本音響学会 2020年春季研究発表会
    • Related Report
      2019 Research-status Report
  • [Presentation] 音の継続長の違いと検出難度を考慮した音響イベント検出2020

    • Author(s)
      秋山 大知, 井本 桂右, 山西 良典, 山下 洋一
    • Organizer
      日本音響学会 2020年春季研究発表会
    • Related Report
      2019 Research-status Report
  • [Presentation] 環境音分析におけるマルチタスク学習の損失関数に対するアニーリングの検討2020

    • Author(s)
      田中 良樹, 砺波 紀之, 井本 桂右, 山西 良典, 山下 洋一
    • Organizer
      日本音響学会 2020年春季研究発表会
    • Related Report
      2019 Research-status Report
  • [Presentation] オノマトペを用いた環境音合成のためのデータセット構築とその分析2020

    • Author(s)
      岡本 悠希, 井本 桂右, 高道 慎之介, 山西 良典, 山下 洋一
    • Organizer
      日本音響学会 2020年春季研究発表会
    • Related Report
      2019 Research-status Report
  • [Presentation] Sound Event Detection Using Graph Laplacian Regularization Based on Event Co-occurrence2019

    • Author(s)
      Keisuke Imoto and Seisuke Kyochi
    • Organizer
      IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
    • Related Report
      2019 Research-status Report
    • Int'l Joint Research
  • [Presentation] Joint Analysis of Acoustic Events and Scenes Based on Multitask Learning2019

    • Author(s)
      Noriyuki Tonami, Keisuke Imoto, Masahiro Niitsuma, Ryosuke Yamanishi, and Yoichi Yamashita
    • Organizer
      IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA)
    • Related Report
      2019 Research-status Report
    • Int'l Joint Research
  • [Presentation] RU Multichannel Domestic Acoustic Scenes 2019: A Multichannel Dataset Recorded by Distributed Microphones with Various Properties2019

    • Author(s)
      Keisuke Imoto and Nobutaka Ono
    • Organizer
      Detection and Classification of Acoustic Scenes and Events (DCASE) Workshop
    • Related Report
      2019 Research-status Report
    • Int'l Joint Research
  • [Presentation] 音響シーンの知識蒸留を用いた音響イベント検出2019

    • Author(s)
      砺波 紀之, 井本 桂右, 山西 良典, 山下 洋一
    • Organizer
      日本音響学会 2019年秋季研究発表会
    • Related Report
      2019 Research-status Report
  • [Presentation] 多様な環境音の合成と変換のための基礎検討2019

    • Author(s)
      岡本 悠希, 柳生 拓巳, 井本 桂右, 小松 達也, 高道 慎之介, 山西 良典, 山下 洋一
    • Organizer
      日本音響学会 2019年秋季研究発表会
    • Related Report
      2019 Research-status Report
  • [Presentation] Acoustic Event and Scene Analysis: Recent Advances and Challenges2019

    • Author(s)
      Keisuke Imoto
    • Organizer
      日本音響学会 2019年春季研究発表会
    • Related Report
      2019 Research-status Report
  • [Patent(Industrial Property Rights)] 音響モデル生成方法、音響分析方法、演算装置、及び、コンピュータプロ グラム2020

    • Inventor(s)
      井本 桂右,秋山 大知,岡本 悠希,山西 良典,山下 洋一
    • Industrial Property Rights Holder
      井本 桂右,秋山 大知,岡本 悠希,山西 良典,山下 洋一
    • Industrial Property Rights Type
      特許
    • Industrial Property Number
      2020-101291
    • Filing Date
      2020
    • Related Report
      2020 Research-status Report

URL: 

Published: 2019-04-18   Modified: 2023-01-30  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi