• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Deep Learning for Task-Driven Image Processing and its Application to Distant Pedestrian Detection

Research Project

Project/Area Number 19K12129
Research Category

Grant-in-Aid for Scientific Research (C)

Allocation TypeMulti-year Fund
Section一般
Review Section Basic Section 61030:Intelligent informatics-related
Research InstitutionToyota Technological Institute

Principal Investigator

Ukita Norimichi  豊田工業大学, 工学(系)研究科(研究院), 教授 (20343270)

Co-Investigator(Kenkyū-buntansha) Muhammad Haris  豊田工業大学, 工学(系)研究科(研究院), ポストドクトラル研究員 (60816643)
Project Period (FY) 2019-04-01 – 2022-03-31
Project Status Completed (Fiscal Year 2021)
Budget Amount *help
¥4,290,000 (Direct Cost: ¥3,300,000、Indirect Cost: ¥990,000)
Fiscal Year 2021: ¥1,170,000 (Direct Cost: ¥900,000、Indirect Cost: ¥270,000)
Fiscal Year 2020: ¥1,690,000 (Direct Cost: ¥1,300,000、Indirect Cost: ¥390,000)
Fiscal Year 2019: ¥1,430,000 (Direct Cost: ¥1,100,000、Indirect Cost: ¥330,000)
Keywords画像超解像 / 物体検出 / 画像処理 / 超解像 / 動画超解像 / 車載カメラ
Outline of Research at the Start

深層学習により性能向上した画像処理において,人が画像を見た際の印象向上ではなく,画像処理の「人工知能が画像を認識するための手段」という側面に着目し,画像処理と画像認識が相互に性能向上する統合最適化を研究する.この統合最適化のため「印象向上を目的とした人の主観やピーク信号対雑音比(PSNR)のような画質基準」に加え「画像認識の性能」も評価するタスク指向画像処理のためのend-to-end深層学習を提案する.タスク指向画像処理の汎用性を検証するため,車載カメラ映像中に極めて小さく映る遠方歩行者像の行動認識という困難なタスクを,映像を拡大する超解像画像処理を介して実現する.

Outline of Final Research Achievements

As an examples of task-driven image processing, we put our focus on tiny object detection and image super-resolution as a task and an image processing algorithm, respectively. Different from previous approaches in which these image processing and task are achieved independently, our proposed framework (i) integrates these two sub-problems in a single neural network and (ii) trains this network so that the reconstructed super-resolution image is optimized for improving the tiny object detection task. With this framework, we realize image super-resolution for generating images that are understandable by artificial intelligence instead of image super-resolution for images that are observed by human.

Academic Significance and Societal Importance of the Research Achievements

タスク指向超解像のための基礎技術として,汎用的な画像超解像を研究し,世界的な協議会でも上位入賞する性能を実現した.画像超解像は,人が鑑賞する画像を生成するという従来型のタスクにおいても,記録媒体に保存する画像ファイル容量の圧縮や,Youtubeや遠隔会議における映像配信など,多様な応用において実用的な技術である.
また,本研究で実現した微小物体検出は,車載画像における遠方物体検出などセキュリティや安全を目的にした多様な分野に波及する技術である.

Report

(4 results)
  • 2021 Annual Research Report   Final Research Report ( PDF )
  • 2020 Research-status Report
  • 2019 Research-status Report
  • Research Products

    (29 results)

All 2021 2020 2019 Other

All Int'l Joint Research (4 results) Journal Article (2 results) (of which Int'l Joint Research: 1 results,  Peer Reviewed: 2 results,  Open Access: 2 results) Presentation (17 results) (of which Int'l Joint Research: 13 results,  Invited: 7 results) Remarks (6 results)

  • [Int'l Joint Research] TTI-C(米国)

    • Related Report
      2021 Annual Research Report
  • [Int'l Joint Research] Bukalapak(インドネシア)

    • Related Report
      2021 Annual Research Report
  • [Int'l Joint Research] 豊田工業大学シカゴ校(米国)

    • Related Report
      2020 Research-status Report
  • [Int'l Joint Research] TTI-Chicago(米国)

    • Related Report
      2019 Research-status Report
  • [Journal Article] Multi-modal Data Fusion for Land-subsidence Image Improvement in PSInSAR Analysis2021

    • Author(s)
      Kodai Shimosato and Norimichi Ukita
    • Journal Title

      IEEE Access

      Volume: 9 Pages: 141970-141980

    • DOI

      10.1109/access.2021.3120133

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Deep Back-Projection Networks For Single Image Super-Resolution2021

    • Author(s)
      Muhammad Haris, Greg Shakhnarovich, and Norimichi Ukita
    • Journal Title

      IEEE Transactions on Pattern Analysis and Machine Intelligence

      Volume: 43 Issue: 12 Pages: 4323-4337

    • DOI

      10.1109/tpami.2020.3002836

    • Related Report
      2021 Annual Research Report 2020 Research-status Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Presentation] AVM Image Quality Enhancement by Synthetic Image Learning for Supervised Deblurring2021

    • Author(s)
      Kazutoshi Akita, Masayoshi Hayama, Haruya Kyutoku, Norimichi Ukita
    • Organizer
      the 17th International Conference on Machine Vision Applications
    • Related Report
      2021 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Crack Segmentation for Low-Resolution Images using Joint Learning with Super-Resolution2021

    • Author(s)
      Yuki Kondo, Norimichi Ukita
    • Organizer
      the 17th International Conference on Machine Vision Applications
    • Related Report
      2021 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Distant Bird Detection for Safe Drone Flight and Its Dataset2021

    • Author(s)
      Sanae Fujii, Kazutoshi Akita, Norimichi Ukita
    • Organizer
      the 17th International Conference on Machine Vision Applications
    • Related Report
      2021 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Neural Image Restoration for Time-lapse SPM Images2021

    • Author(s)
      Norimichi Ukita, Fuma Yasue, Kota Shinjo, Yuki Kondo, Kazutoshi Akita, Hibiki Mitsuboshi, and Masamichi Yoshimura
    • Organizer
      29th International Colloquium on Scanning Probe Microscopy
    • Related Report
      2021 Annual Research Report
    • Int'l Joint Research / Invited
  • [Presentation] Task-Driven Super Resolution: Object Detection in Low-resolution Images2021

    • Author(s)
      Muhammad Haris, Greg Shakhnarovich, and Norimichi Ukita
    • Organizer
      the 28th International Conference on Neural Information Processing
    • Related Report
      2021 Annual Research Report
    • Int'l Joint Research
  • [Presentation] AIを用いた超解像2021

    • Author(s)
      浮田宗伯
    • Organizer
      日本学術振興会 167委員会 ナノプローブテクノロジー研究会
    • Related Report
      2020 Research-status Report
    • Invited
  • [Presentation] Space-Time-Aware Multi-Resolution Video Enhancement2020

    • Author(s)
      Muhammad Haris, Greg Shakhnarovich, and Norimichi Ukita
    • Organizer
      IEEE Conference on Computer Vision and Pattern Recognition
    • Related Report
      2020 Research-status Report
    • Int'l Joint Research
  • [Presentation] Region-dependent Scale Proposals for Super-Resolution in Object Detection2020

    • Author(s)
      Kazutoshi Akita, Muhammad Haris, and Norimichi Ukita
    • Organizer
      The Fourth IEEE International Conference on Image Processing, Applications and Systems
    • Related Report
      2020 Research-status Report
    • Int'l Joint Research
  • [Presentation] AIM 2020 Challenge on Video Temporal Super-Resolution2020

    • Author(s)
      Sanghyun Son, et al.
    • Organizer
      Advances in Image Manipulation workshop and challenges on image and video manipulation Workshop
    • Related Report
      2020 Research-status Report
    • Int'l Joint Research
  • [Presentation] AIM 2020 Challenge on Video Extreme Super-Resolution: Methods and Results2020

    • Author(s)
      Dario Fuoli, et al.
    • Organizer
      Advances in Image Manipulation workshop and challenges on image and video manipulation Workshop
    • Related Report
      2020 Research-status Report
    • Int'l Joint Research
  • [Presentation] NTIRE 2020 Challenge on Perceptual Extreme Super-Resolution: Methods and Results2020

    • Author(s)
      Kai Zhang, et al.
    • Organizer
      New Trends in Image Restoration and Enhancement Workshop
    • Related Report
      2020 Research-status Report
    • Int'l Joint Research
  • [Presentation] 機械学習を用いた低解像度画像データの高解像化と電子顕微鏡画像などへの応用2020

    • Author(s)
      浮田宗伯
    • Organizer
      応用物理学会・東海支部 基礎セミナー「AIを用いた画像処理・認識技術の進展」
    • Related Report
      2020 Research-status Report
    • Invited
  • [Presentation] Recurrent Back-Projection Network for Video Super-resolution2019

    • Author(s)
      Muhammad Haris, Greg Shakhnarovich, and Norimichi Ukita
    • Organizer
      IEEE Conference on Computer Vision and Pattern Recognition
    • Related Report
      2019 Research-status Report
    • Int'l Joint Research
  • [Presentation] Image and Video Super-resolution and their Application to Tiny Object Detection2019

    • Author(s)
      Norimichi Ukita
    • Organizer
      3D Computer Vision and Graphics Workshop
    • Related Report
      2019 Research-status Report
    • Int'l Joint Research / Invited
  • [Presentation] 画像の超解像による微小構造解析の可能性2019

    • Author(s)
      浮田宗伯
    • Organizer
      格子欠陥フォーラム「格子欠陥とマテリアルインフォマティクス」
    • Related Report
      2019 Research-status Report
    • Invited
  • [Presentation] Recurrent Back-Projection Network for Video Super-resolution2019

    • Author(s)
      Muhammad Haris, Greg Shakhnarovich, and Norimichi Ukita
    • Organizer
      画像の認識・理解シンポジウム
    • Related Report
      2019 Research-status Report
    • Invited
  • [Presentation] Image and Video Super-resolution for Learning Visual Representations: Application to Tiny Object Detection2019

    • Author(s)
      Norimichi Ukita
    • Organizer
      International Workshop on Symbolic-Neural Learning
    • Related Report
      2019 Research-status Report
    • Int'l Joint Research / Invited
  • [Remarks] 静止画超解像

    • URL

      https://www.toyota-ti.ac.jp/Lab/Denshi/iim/ukita/selection/PAMI2020.sisr.html

    • Related Report
      2020 Research-status Report
  • [Remarks] 動画超解像

    • URL

      https://www.toyota-ti.ac.jp/Lab/Denshi/iim/ukita/selection/CVPR2019.vsr.html

    • Related Report
      2020 Research-status Report
  • [Remarks] 動画時空間超解像

    • URL

      https://www.toyota-ti.ac.jp/Lab/Denshi/iim/ukita/selection/CVPR2020.stsr.html

    • Related Report
      2020 Research-status Report
  • [Remarks] 超解像オンラインデモ

    • URL

      https://133.21.219.252/

    • Related Report
      2020 Research-status Report
  • [Remarks] 静止画超解像

    • URL

      https://www.toyota-ti.ac.jp/Lab/Denshi/iim/ukita/selection/CVPR2018.sr.html

    • Related Report
      2019 Research-status Report
  • [Remarks] 動画像超解像

    • URL

      https://www.toyota-ti.ac.jp/Lab/Denshi/iim/ukita/selection/CVPR2019.vsr.html

    • Related Report
      2019 Research-status Report

URL: 

Published: 2019-04-18   Modified: 2023-01-30  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi