• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Innovations in TV content analysis using deep learning: face detection and tracking of politicians

Research Project

Project/Area Number 20K20513
Research Category

Grant-in-Aid for Challenging Research (Pioneering)

Allocation TypeMulti-year Fund
Review Section Medium-sized Section 8:Sociology and related fields
Research InstitutionKobe University

Principal Investigator

Kobayashi Tetsuro  神戸大学, 法学研究科, 研究員 (60455194)

Co-Investigator(Kenkyū-buntansha) 松井 勇佑  東京大学, 大学院情報理工学系研究科, 講師 (80780676)
佐藤 真一  国立情報学研究所, コンテンツ科学研究系, 教授 (90249938)
Project Period (FY) 2020-07-30 – 2023-03-31
Project Status Completed (Fiscal Year 2022)
Budget Amount *help
¥25,350,000 (Direct Cost: ¥19,500,000、Indirect Cost: ¥5,850,000)
Fiscal Year 2022: ¥1,690,000 (Direct Cost: ¥1,300,000、Indirect Cost: ¥390,000)
Fiscal Year 2021: ¥10,660,000 (Direct Cost: ¥8,200,000、Indirect Cost: ¥2,460,000)
Fiscal Year 2020: ¥13,000,000 (Direct Cost: ¥10,000,000、Indirect Cost: ¥3,000,000)
Keywords深層学習 / テレビの内容分析 / 顔検出 / 顔トラッキング / 政治コミュニケーション
Outline of Research at the Start

本研究は深層学習の技術をテレビニュースの内容分析に応用し、大量の映像情報を目視に頼ることな、自動的に分析する方法論を確立する。深層学習は、分析対象の分類に有効となる特徴を自ら学び取っていくため、様々な角度や表情で人物が映されるテレビ映像の分析に有効である。本研究の文脈では、事前に特定することが困難な政治家の顔の特徴をアルゴリズムが自律的に学習していくことで、ニュース内における特定の政治家の出現を高い精度で検出することが可能になる。こうした深層学習に基づいた計算アルゴリズムと網羅性の高いニュースアーカイブを組み合わせることで、従来の内容分析では回答できなかった社会科学的問いに答えることを目指す。

Outline of Final Research Achievements

We developed a system capable of automatically analyzing large volumes of news footage that cannot be coded manually, by utilizing deep learning to measure the screen time of political actors. In this process, we proposed a flexible system that does not require training for each detection target by combining face detection, tracking, and clustering. Using this system, we conducted an analysis of over 20 years of NHK News 7 footage, and found that while the Prime Minister's screen time fluctuates, there is no monotonous increase as predicted by the presidentialization hypothesis. Moreover, since 2020, the relative screen time of the ruling party leader on NHK News 7 has increased compared to Hodo Station. These findings were obtained for the first time through the analysis of a large volume of news videos using deep learning, and provided a pioneering example of the application of computer vision technology to social sciences.

Academic Significance and Societal Importance of the Research Achievements

本研究はテレビの内容分析に深層学習の技術を応用した先駆的な研究であり、日本国内では初の試みである。20年以上にもおよぶ大量のニュース映像を目視に頼らずにすべて分析することは、本研究で提案されたシステムを用いて初めて可能になった。この点で、テレビの内容分析に革新的な変化をもたらすという当初の目的は達成された。本研究は社会科学の研究者とコンピュータ科学の研究者の緊密な学際的連携によって可能になったという点において、文理融合の面から学術的意義がある。さらに、提案されたシステムのソースコードはすべて公開しており、アカデミアに限らず世界中の人が利用可能となっており、その点で社会的意義も大きい。

Report

(4 results)
  • 2022 Annual Research Report   Final Research Report ( PDF )
  • 2021 Research-status Report
  • 2020 Research-status Report
  • Research Products

    (15 results)

All 2023 2022 2021

All Journal Article (1 results) (of which Int'l Joint Research: 1 results,  Peer Reviewed: 1 results,  Open Access: 1 results) Presentation (12 results) (of which Int'l Joint Research: 6 results) Book (2 results)

  • [Journal Article] A multimedia document browser based on multilayer networks2021

    • Author(s)
      Benjamin Renoust, Haolin Ren, Guy Melancon, Marie-Luce Viaud, Shin'ichi Satoh
    • Journal Title

      Multim. Tools Appl.

      Volume: 80(15) Issue: 15 Pages: 22551-22588

    • DOI

      10.1007/s11042-020-09872-9

    • Related Report
      2021 Research-status Report 2020 Research-status Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Presentation] KAO: Key Actor detection for TV News2023

    • Author(s)
      Andreu Girbau, Tetsuro Kobayashi, Benjamin Renoust, Yusuke Matsui, Shin'ichi Satoh
    • Organizer
      Asian Polmeth 2023 Annual Meeting
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] 大規模高次元ベクトルデータに対する近似最近傍探索インデックスの分散並列構築2023

    • Author(s)
      小野直樹・松井勇佑
    • Organizer
      パターン認識・メディア理解研究会
    • Related Report
      2022 Annual Research Report
  • [Presentation] Long-Term Analysis of Appearances of Japanese Politicians on TV News: Face Detection and Tracking Using Deep Learning Techniques2022

    • Author(s)
      Tetsuro Kobayashi, Andreu Girbau, Yusuke Matsui, Benjamin Renoust, and Shin'ichi Satoh
    • Organizer
      International Communication Association
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] KAO: Key Actor detection for TV News2022

    • Author(s)
      Andreu Girbau, Tetsuro Kobayashi, Benjamin Renoust, Yusuke Matsui, Shin'ichi Satoh
    • Organizer
      The 25th Meeting on Image Recognition and Understanding
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] データ重複が最近傍探索問題に与える精度変化の分析2022

    • Author(s)
      金海智大・松井勇佑
    • Organizer
      画像の認識・理解シンポジウム
    • Related Report
      2022 Annual Research Report
  • [Presentation] 近似最近傍探索のためのグラフインデックスの分散並列構築2022

    • Author(s)
      小野直樹・松井勇佑
    • Organizer
      パターン認識・メディア理解研究会
    • Related Report
      2022 Annual Research Report
  • [Presentation] KAO: Key Actors DetectiOn for TV News2022

    • Author(s)
      Andreu Girbau, Tetsuro Kobayashi, Benjamin Renoust, Yusuke Matsui, Shin'ichi Satoh
    • Organizer
      パターン認識・メディア理解研究会
    • Related Report
      2022 Annual Research Report
  • [Presentation] Multiple Object Tracking from appearance by hierarchically clustering tracklets2022

    • Author(s)
      Andreu Girbau, Ferran Marques, Shin'ichi Satoh
    • Organizer
      British Machine Vision Conference (BMVC)
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Textual Analysis for Video Memorability Prediction2022

    • Author(s)
      Camille Guinaudeau, Andreu Girbau
    • Organizer
      Multimedia Evaluation workshop
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Long-term analysis of the appearance of Japanese politicians on TV news: Face detection and tracking using deep learning techniques2022

    • Author(s)
      Tetsuro Kobayashi, Andreu Girbau, Yusuke Matsui, Benjamin Renoust, and Shin'ichi Satoh
    • Organizer
      The Japanese Society for Quantitative Political Science
    • Related Report
      2021 Research-status Report
  • [Presentation] Multiple object tracking with mixture density networks for trajectory estimation2021

    • Author(s)
      Andreu Girbau, Xavier Giro-i-Nieto, Ignasi Rius, Ferran Marques
    • Organizer
      CVPR - Robust Video Scene Understanding: Tracking and Video Segmentation (RVSU) Workshop,
    • Related Report
      2021 Research-status Report
    • Int'l Joint Research
  • [Presentation] Assessing movie similarity using a multilayer network model2021

    • Author(s)
      M Lafhel, Y Mourchid, B Renoust, H Cherifi, M El Hassouni
    • Organizer
      FRCCS French Regional Conference on Complex System
    • Related Report
      2021 Research-status Report
  • [Book] Visual Analysis of Multilayer Networks2021

    • Author(s)
      "Fintan McGee, Benjamin Renoust, Daniel Archambault, Mohammad Ghoniem, Andreas Kerren, Bruno Pinaud, Margit Pohl, Benoit Otjacques, Guy Melancon, Tatiana von Landesberger"
    • Total Pages
      150
    • Publisher
      Morgan & Claypool
    • ISBN
      9781636391458
    • Related Report
      2021 Research-status Report
  • [Book] Handling Complex Multilayer Networks: An Approach Based on Visual Network Analytics in Complex Systems, Smart Territories and Mobility2021

    • Author(s)
      Guy Melancon, Benjamin Renoust, Haolin Ren
    • Total Pages
      19
    • Publisher
      Springer
    • ISBN
      9783030593025
    • Related Report
      2021 Research-status Report

URL: 

Published: 2020-08-03   Modified: 2024-01-30  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi