• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Video summarization from a set of videos controlled by text input

Research Project

Project/Area Number 16K16086
Research Category

Grant-in-Aid for Young Scientists (B)

Allocation TypeMulti-year Fund
Research Field Perceptual information processing
Research InstitutionOsaka University

Principal Investigator

Nakashima Yuta  大阪大学, データビリティフロンティア機構, 准教授 (70633551)

Project Period (FY) 2016-04-01 – 2019-03-31
Project Status Completed (Fiscal Year 2018)
Budget Amount *help
¥3,900,000 (Direct Cost: ¥3,000,000、Indirect Cost: ¥900,000)
Fiscal Year 2018: ¥1,170,000 (Direct Cost: ¥900,000、Indirect Cost: ¥270,000)
Fiscal Year 2017: ¥1,430,000 (Direct Cost: ¥1,100,000、Indirect Cost: ¥330,000)
Fiscal Year 2016: ¥1,300,000 (Direct Cost: ¥1,000,000、Indirect Cost: ¥300,000)
Keywords映像要約 / 深層学習 / 畳み込みニューラルネットワーク / 重要領域推定 / 要約映像の評価 / 部分映像検索 / 自然言語処理 / ディープニューラルネットワーク
Outline of Final Research Achievements

This work aims at automating video editing and establishes techniques for automatically generating a video summary from an original video or a set of them. Generally, a video contains various events occurring in various scenes, and it is not obvious that which events should be included in the resulting video summary. In this work, we considered two approaches for video summarization: One approach determines each frame's importance based on certain types of user input. The other approach attempts to reduce the redundancy in the resulting video summary while covering the content of original video as much as possible. For respective approaches, we proposed video summarization methods and experimentally demonstrated their effectiveness. We also reconsider the evaluation of video summarization method and developed a new method for evaluation.

Academic Significance and Societal Importance of the Research Achievements

近年、スマートフォンやデジタルカメラなどで日々大量の映像が撮影されている。本研究は、このような映像に対して短時間でその内容を閲覧可能な映像要約手法を提案しており、映像閲覧時のユーザの負荷の軽減や送受信される映像サイズの削減などの点で高い有用性を持つと考える。加えて、特に映像要約の評価手法については、広く用いられるデータセットで利用される評価手法の問題点を明確にしているという点において、今後の映像要約の研究に大きな影響を与えるものであり、学術的意義も大きいと考える。

Report

(3 results)
  • 2018 Final Research Report ( PDF )
  • 2017 Annual Research Report
  • 2016 Research-status Report
  • Research Products

    (13 results)

All 2018 2017 2016 Other

All Int'l Joint Research (3 results) Journal Article (3 results) (of which Int'l Joint Research: 3 results,  Peer Reviewed: 3 results,  Acknowledgement Compliant: 2 results) Presentation (7 results) (of which Int'l Joint Research: 5 results)

  • [Int'l Joint Research] オウル大学/タンペレ工科大学(フィンランド)

    • Related Report
      2017 Annual Research Report
  • [Int'l Joint Research] University of Oulu(フィンランド)

    • Related Report
      2016 Research-status Report
  • [Int'l Joint Research] University of Valladolid(スペイン)

    • Related Report
      2016 Research-status Report
  • [Journal Article] Summarization of User-Generated Sports Video by Using Deep Action Recognition Features2018

    • Author(s)
      Tejero-de-Pablos Antonio、Nakashima Yuta、Sato Tomokazu、Yokoya Naokazu、Linna Marko、Rahtu Esa
    • Journal Title

      IEEE Transactions on Multimedia

      Volume: 印刷中 Issue: 8 Pages: 2000-2011

    • DOI

      10.1109/tmm.2018.2794265

    • Related Report
      2017 Annual Research Report
    • Peer Reviewed / Int'l Joint Research
  • [Journal Article] Video summarization using textual descriptions for authoring video blogs2016

    • Author(s)
      Mayu Otani, Yuta Nakashima, Tomokazu Sato, and Naokazu Yokoya
    • Journal Title

      Multimedia Tools and Applications

      Volume: - Issue: 9 Pages: 12097-12115

    • DOI

      10.1007/s11042-016-4061-3

    • NAID

      120006457764

    • Related Report
      2017 Annual Research Report 2016 Research-status Report
    • Peer Reviewed / Int'l Joint Research / Acknowledgement Compliant
  • [Journal Article] Flexible human action recognition in depth video sequences using masked joint trajectories2016

    • Author(s)
      Antonio Tejero-de-Pablos, Yuta Nakashima, Naokazu Yokoya, Francisco-Javier Diaz-Pernas, and Mario Martinez-Zarzuela
    • Journal Title

      EURASIP Journal on Image and Video Processing

      Volume: 2016 Issue: 1

    • DOI

      10.1186/s13640-016-0120-y

    • Related Report
      2016 Research-status Report
    • Peer Reviewed / Int'l Joint Research / Acknowledgement Compliant
  • [Presentation] Finding Video Parts with Natural Language2018

    • Author(s)
      Mayu Otani、Yuta Nakashima、Esa Rahtu、Janne Heikkila
    • Organizer
      情報処理学会研究報告コンピュータビジョンとイメージメディア(CVIM)
    • Related Report
      2017 Annual Research Report
  • [Presentation] Fine-grained video retrieval for multi-clip video2017

    • Author(s)
      Mayu Otani、Yuta Nakashima、Esa Rahtu、Janne Heikkila
    • Organizer
      Closing the Loop Between Vision and Language (CLVL) at ICCV
    • Related Report
      2017 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Video Question Answering to Find a Desired Video Segment2017

    • Author(s)
      Mayu Otani、Yuta Nakashima、Esa Rahtu、Janne Heikkila
    • Organizer
      Open Knowledge Base and Question Answering Workshop (OKBQA) at SIGIR
    • Related Report
      2017 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Unsupervised Video Summarization using Deep Video Features2017

    • Author(s)
      Mayu Otani、Yuta Nakashima、Esa Rahtu、Janne Heikkila、Naokazu Yokoya
    • Organizer
      画像の認識・理解シンポジウム(MIRU)
    • Related Report
      2017 Annual Research Report
  • [Presentation] Video summarization using deep semantic features2016

    • Author(s)
      Mayu Otani, Yuta Nakashima, Esa Rahtu, Janne Heikkila, and Naokazu Yokoya
    • Organizer
      13th Asian Conference on Computer Vision
    • Place of Presentation
      Taipei
    • Year and Date
      2016-11-20
    • Related Report
      2016 Research-status Report
    • Int'l Joint Research
  • [Presentation] Human action recognition-based video summarization for RGB-D personal sports video2016

    • Author(s)
      Antonio Tejero-de-Pablos, Yuta Nakashima, Tomokazu Sato, and Naokazu Yokoya
    • Organizer
      2016 IEEE International Conference on Multimedia and Expo
    • Place of Presentation
      Seattle
    • Year and Date
      2016-07-11
    • Related Report
      2016 Research-status Report
    • Int'l Joint Research
  • [Presentation] Learning joint representations of videos and sentences with web image search2016

    • Author(s)
      Mayu Otani, Yuta Nakashima, Esa Rahtu, Janne Heikkila, and Naokazu Yokoya
    • Organizer
      4th Workshop on Web-scale Vision and Social Media (VSM) in conjunction with ECCV 2016
    • Place of Presentation
      Amsterdam
    • Related Report
      2016 Research-status Report
    • Int'l Joint Research

URL: 

Published: 2016-04-21   Modified: 2020-03-30  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi