• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Development of a Lecture Summarization System Utilizing Social Annotations to Enhance Comprehension and Engagement in Online Education

Research Project

Project/Area Number 21K13641
Research Category

Grant-in-Aid for Early-Career Scientists

Allocation TypeMulti-year Fund
Review Section Basic Section 09070:Educational technology-related
Research InstitutionAnan National College of Technology

Principal Investigator

Ohta Kengo  阿南工業高等専門学校, 創造技術工学科, 准教授 (80712801)

Project Period (FY) 2021-04-01 – 2025-03-31
Project Status Completed (Fiscal Year 2024)
Budget Amount *help
¥4,680,000 (Direct Cost: ¥3,600,000、Indirect Cost: ¥1,080,000)
Fiscal Year 2023: ¥1,690,000 (Direct Cost: ¥1,300,000、Indirect Cost: ¥390,000)
Fiscal Year 2022: ¥1,560,000 (Direct Cost: ¥1,200,000、Indirect Cost: ¥360,000)
Fiscal Year 2021: ¥1,430,000 (Direct Cost: ¥1,100,000、Indirect Cost: ¥330,000)
Keywords音声認識 / 音声要約 / オンライン教育 / ソーシャルアノテーション / 教育工学 / 自由発話 / 講義動画 / オンライン講義 / 講義コンテンツ
Outline of Research at the Start

本研究では、オンライン講義の動画やライブ配信に付与された学生のリアクション(投稿コメントや「いいね」など)と、講師の作成した講義コンテンツ(講義スライドやレジュメなど)を学習データに活用することで、講義音声の音声認識(自動書き起こし)と自動要約(重要な部分の自動抽出)を高精度化する。音声認識の結果を講義動画の字幕として提示したり、自動要約の結果を学生の復習用の教材として提示することにより、オンライン講義コンテンツの再利用性を高め、学生の学びを活性化する。

Outline of Final Research Achievements

This study aimed to enhance the effectiveness of learning in online lectures by developing technologies for automatic speech recognition and summarization of lecture content. The research particularly focused on addressing challenges unique to lecture speech, such as disfluencies, including hesitations and fillers, and the frequent use of domain-specific terminology and complex expressions. To tackle these issues, the study developed customized speech recognition methods capable of handling the spontaneous and technical nature of lecture audio. In addition, it explored techniques for improving the accuracy of identifying key content by incorporating various contextual resources, such as student reactions (e.g., real-time comments and “likes”) and lecture materials like slides and handouts. These technologies are expected to serve as a foundation for more advanced and effective learning support systems in online education environments.

Academic Significance and Societal Importance of the Research Achievements

本研究は、講義音声に含まれる非流暢な発話や専門語彙の認識精度を高める音声処理手法と、講義の重要部分を自動的に抽出する要約手法を組み合わせたものであり、教育工学と音声言語処理の融合領域における新たなアプローチを提示した。オンライン教育が広く普及する中、長時間の講義動画の中から効率的に要点を把握し、復習や要点理解を支援する仕組みは、学習者の自律的な学びを促進し、教育の質の向上や学習機会の拡充に貢献するものと考えられる。

Report

(5 results)
  • 2024 Annual Research Report   Final Research Report ( PDF )
  • 2023 Research-status Report
  • 2022 Research-status Report
  • 2021 Research-status Report
  • Research Products

    (20 results)

All 2024 2023 2022 2021

All Journal Article (1 results) (of which Peer Reviewed: 1 results,  Open Access: 1 results) Presentation (19 results) (of which Int'l Joint Research: 6 results)

  • [Journal Article] Recognition of target domain Japanese speech using language model replacement2024

    • Author(s)
      Mori Daiki、Ohta Kengo、Nishimura Ryota、Ogawa Atsunori、Kitaoka Norihide
    • Journal Title

      EURASIP Journal on Audio, Speech, and Music Processing

      Volume: 2024 Issue: 1 Pages: 1-14

    • DOI

      10.1186/s13636-024-00360-8

    • Related Report
      2024 Annual Research Report
    • Peer Reviewed / Open Access
  • [Presentation] Combining multiple end-to-end speech recognition models based on density ratio approach2023

    • Author(s)
      Hojo Keigo、Mori Daiki、Wakabayashi Yukoh、Ohta Kengo、Ogawa Atsunori、Kitaoka Norihide
    • Organizer
      2023 Asia Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC)
    • Related Report
      2023 Research-status Report
    • Int'l Joint Research
  • [Presentation] Language modeling for spontaneous speech recognition based on disfluency labeling and generation of disfluent text2023

    • Author(s)
      Horii Koharu、Ohta Kengo、Nishimura Ryota、Ogawa Atsunori、Kitaoka Norihide
    • Organizer
      2023 Asia Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC)
    • Related Report
      2023 Research-status Report
    • Int'l Joint Research
  • [Presentation] Wav2vecを用いた音声解析に基づく動画の自動要約手法に関する研究2023

    • Author(s)
      倉渕光希, 太田健吾
    • Organizer
      計測自動制御学会四国支部学術講演会
    • Related Report
      2023 Research-status Report
  • [Presentation] 大規模言語モデルに基づく動画配信における視聴者の立場で応答する雑談対話モデルの構築と評価2023

    • Author(s)
      中田東吾, 太田健吾
    • Organizer
      計測自動制御学会四国支部学術講演会
    • Related Report
      2023 Research-status Report
  • [Presentation] 自発的発話認識のためのBERTによる非流暢文生成に基づく言語モデリング2023

    • Author(s)
      堀井 こはる,太田 健吾,西村 良太,小川 厚徳,北岡 教英
    • Organizer
      日本音響学会講演論文集
    • Related Report
      2022 Research-status Report
  • [Presentation] Implicit language information replace method in Japanese encoder-decode ASR model2022

    • Author(s)
      Daiki Mori, Kengo Ohta, Ryota Nishimura, Norihide Kitaoka
    • Organizer
      The International Conference on Advanced Informatics: Concepts, Theory and Applications
    • Related Report
      2022 Research-status Report
    • Int'l Joint Research
  • [Presentation] End-to-End Spontaneous Speech Recognition Using Disfluency Labeling2022

    • Author(s)
      Koharu Horii, Meiko Fukuda, Kengo Ohta, Ryota Nishimura, Atsunori Ogawa, Norihide Kitaoka
    • Organizer
      Interspeech
    • Related Report
      2022 Research-status Report
    • Int'l Joint Research
  • [Presentation] End-to-End非流暢整形音声認識システムの対話音声による評価2022

    • Author(s)
      堀井こはる, 福田芽衣子, 太田健吾, 西村良太, 小川厚徳, 北岡教英
    • Organizer
      日本音響学会講演論文集
    • Related Report
      2022 Research-status Report
  • [Presentation] ドメイン外音響情報で補強したEncoder-Decoder音声認識モデルの設計2022

    • Author(s)
      森大輝, 太田健吾, 西村良太, 北岡教英
    • Organizer
      日本音響学会講演論文集
    • Related Report
      2022 Research-status Report
  • [Presentation] タスク外音響情報を付加したEnd-to-End音声認識モデルの設計2022

    • Author(s)
      森 大輝,太田 健吾,西村 良太,小川厚徳,北岡 教英
    • Organizer
      日本音響学会春季講演論文集,2-3Q-2
    • Related Report
      2021 Research-status Report
  • [Presentation] 非流暢ラベルを用いた言い淀み整形End-to-End音声認識2022

    • Author(s)
      堀井 こはる,福田 芽衣子,太田 健吾,西村 良太,小川厚徳,北岡 教英
    • Organizer
      日本音響学会春季講演論文集,1-3-5
    • Related Report
      2021 Research-status Report
  • [Presentation] 言い淀みを考慮したEnd-to-End音声認識2021

    • Author(s)
      堀井 こはる,福田 芽衣子,太田 健吾,西村 良太,北岡 教英
    • Organizer
      第19回情報学ワークショップ(WiNF2021),S-5-2
    • Related Report
      2021 Research-status Report
  • [Presentation] Encoder-Decoder音声認識モデルにおける暗黙的言語情報の置換法2021

    • Author(s)
      森 大輝,太田 健吾,西村 良太,北岡 教英
    • Organizer
      第19回情報学ワークショップ(WiNF2021),S-5-5
    • Related Report
      2021 Research-status Report
  • [Presentation] Encoder-Decoder音声認識モデルにおける暗黙的言語情報の置換法2021

    • Author(s)
      森 大輝,太田 健吾,西村 良太,小川 厚徳,北岡 教英
    • Organizer
      日本音響学会秋季講演論文集,1-3-1
    • Related Report
      2021 Research-status Report
  • [Presentation] 言い淀みを考慮した自由発話のEnd-to-End音声認識2021

    • Author(s)
      堀井 こはる,福田 芽衣子,太田 健吾,西村 良太,北岡 教英
    • Organizer
      日本音響学会講演論文集,1-3-3
    • Related Report
      2021 Research-status Report
  • [Presentation] End-to-end音声認識モデルにおける暗黙的言語情報の置換法2021

    • Author(s)
      森大輝,太田健吾,西村良太,小川厚徳,北岡教英
    • Organizer
      音学シンポジウム
    • Related Report
      2021 Research-status Report
  • [Presentation] 言い淀みラベル付けによる非流暢発話のEnd-to-End音声認識2021

    • Author(s)
      堀井こはる,福田芽衣子,太田健吾,西村良太,北岡教英
    • Organizer
      音学シンポジウム
    • Related Report
      2021 Research-status Report
  • [Presentation] Advanced language model fusion method for encoder-decoder model in Japanese speech2021

    • Author(s)
      Daiki Mori,Kengo Ohta,Ryota Nishimura,Atsunori Ogawa,Norihide Kitaoka
    • Organizer
      Proc. APSIPA ASC,pp. 503-510
    • Related Report
      2021 Research-status Report
    • Int'l Joint Research
  • [Presentation] End-to-end spontaneous speech recognition using hesitation labeling2021

    • Author(s)
      Koharu Horii,Meiko Fukuda,Kengo Ohta,Ryota Nishimura,Atsunori Ogawa,Norihide Kitaoka
    • Organizer
      Proc. APSIPA ASC,pp. 1077-1081
    • Related Report
      2021 Research-status Report
    • Int'l Joint Research

URL: 

Published: 2021-04-28   Modified: 2026-01-16  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi