• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Teacher-Student Sequential Re-learning Model in Adaptive Structure Deep Learning

Research Project

Project/Area Number 19K12142
Research Category

Grant-in-Aid for Scientific Research (C)

Allocation TypeMulti-year Fund
Section一般
Review Section Basic Section 61040:Soft computing-related
Research InstitutionPrefectural University of Hiroshima

Principal Investigator

Ichimura Takumi  県立広島大学, 地域創生学部, 教授 (10295842)

Project Period (FY) 2019-04-01 – 2024-03-31
Project Status Completed (Fiscal Year 2023)
Budget Amount *help
¥4,420,000 (Direct Cost: ¥3,400,000、Indirect Cost: ¥1,020,000)
Fiscal Year 2021: ¥1,300,000 (Direct Cost: ¥1,000,000、Indirect Cost: ¥300,000)
Fiscal Year 2020: ¥1,300,000 (Direct Cost: ¥1,000,000、Indirect Cost: ¥300,000)
Fiscal Year 2019: ¥1,820,000 (Direct Cost: ¥1,400,000、Indirect Cost: ¥420,000)
Keywords構造適応型深層学習 / 構造適応型RBM / 構造適応型DBN / Teacher-Student構造適応型深層学習 / AffectNet / 軽度認知症画像判定 / Teacher-Studentモデル / RoadTracer / 避難経路探索 / RBM / DBN / Teacher Student型モデル / ADNI / 深層学習 / 構造適応型学習 / T/S型逐次的再学習モデル / KL情報量 / 顔感情モデル / 敵対的事例 / 制限付ボルツマンマシン / Deep Belief Network / teacher-studentモデル
Outline of Research at the Start

入力データに対して適応的に隠れニューロン数や層の数を自動で生成/消去する構造適応型深層学習法を開発した。開発した手法は,画像や時系列データなどのベンチマークテストでは学習用ではほぼ100%,テスト用では他の深層学習手法より高い分類能力を示した。学習後に未知データが出現した場合,再学習することで,自己組織化的に学習モデルを自ら構造やパラメタ修正するアルゴリズムを研究する。学習済みモデルをTeacherモデル,再学習のためのモデルをStudentモデルとし,KL情報量によって,分類,時系列等の様々なビッグデータセットに対応できる深層学習モデルを構築する。

Outline of Final Research Achievements

An adaptive structural learning method called Adaptive RBM-DBN, which automatically determines the optimal number of RBM neurons and layers for the training data, has shown high classification accuracy. However, the accuracy cannot be expected to improve in the case of ambiguity or inconsistency, such as medical images or emotions. We proposed the Teacher-Student (T/S) adaptive structural deep learning model as an ensemble learning method using two or more models. We applied the model to facial emotion data and dementia data, and found that the classification performance was improved. Furthermore, we proposed a method for real-time discovery of evacuation routes while avoiding roads damaged by landslides using aerial and satellite images during the torrential rain disaster in western Japan, and demonstrated the effectiveness of the proposed method.

Academic Significance and Societal Importance of the Research Achievements

IoTや医療等,データが収集・蓄積されたデータをそのまま画像解析やデータ分析に使用されている。学習時には出現していなかった特徴をもつデータが混在していることがあり,これらを外れ値やノイズとして処理するのではなく,一つの新たな特徴として処理する必要がある。Teacher-Student構造適応型深層学習モデルは実在するデータを無視するのではなく,再学習した知識を蒸留する形で,適切な構造をもつ深層学習法の一つとして提案した。

Report

(6 results)
  • 2023 Annual Research Report   Final Research Report ( PDF )
  • 2022 Research-status Report
  • 2021 Research-status Report
  • 2020 Research-status Report
  • 2019 Research-status Report
  • Research Products

    (40 results)

All 2024 2023 2022 2021 2020 2019

All Journal Article (7 results) (of which Peer Reviewed: 7 results,  Open Access: 2 results) Presentation (31 results) (of which Int'l Joint Research: 8 results) Book (2 results)

  • [Journal Article] 3D Lung Tumor Segmentation System using Adaptive Structural Deep Belief Network2024

    • Author(s)
      Kamada Shin、Ichimura Takumi
    • Journal Title

      Intelligent Systems Reference Library: Advances in Intelligent Disease Diagnosis and Treatment

      Volume: to appear in 2024

    • Related Report
      2023 Annual Research Report
    • Peer Reviewed
  • [Journal Article] A Teacher-Student based Adaptive Structural Deep learning Model and Its Estimating Uncertainty of Image Data2023

    • Author(s)
      Ichimura Takumi、Kamada Shin、Harada Toshihide、Inoue Ken
    • Journal Title

      Handbook of Statistics 49: Artificial Intelligence

      Volume: 49 Pages: 129-149

    • DOI

      10.1016/bs.host.2023.04.001

    • ISBN
      9780443137631
    • Related Report
      2023 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Automatic Extraction of Road Networks by Using Teacher-Student Adaptive Structural Deep Belief Network and Its Application to Landslide Disaster2023

    • Author(s)
      Shin Kamada, Takumi Ichimura
    • Journal Title

      IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing

      Volume: 16 Pages: 6310-6324

    • DOI

      10.1109/jstars.2023.3293593

    • Related Report
      2023 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] A Segmentation Method of Lung Tumor by using Adaptive Structural Deep Belief Network2023

    • Author(s)
      Kamada Shin、Ichimura Takumi
    • Journal Title

      Proc. of The SICE Annual Conference 2023

      Volume: 1 Pages: 1529-1534

    • Related Report
      2023 Annual Research Report
    • Peer Reviewed
  • [Journal Article] An Ensemble Learning Method of Adaptive Structural Deep Belief Network for AffectNet2022

    • Author(s)
      Takumi Ichimura, Shin Kamada
    • Journal Title

      International Journal of Smart Computing and Artificial Intelligence

      Volume: 6 Issue: 1 Pages: 1

    • DOI

      10.52731/ijscai.v6.i1.640

    • ISSN
      2185-9906, 2185-9914
    • Related Report
      2021 Research-status Report
    • Peer Reviewed / Open Access
  • [Journal Article] Adaptive Structural Learning of Deep Belief Network and its Application to Real Time Crack Detection of Concrete Structure using Drone2022

    • Author(s)
      Shin Kamada and Takumi Ichimura
    • Journal Title

      Handbook on Artificial Intelligence-empowered Applied Software Engineering: VOL.2: Smart Software Applications in Cyber-Physical Systems", Springer

      Volume: -

    • Related Report
      2021 Research-status Report
    • Peer Reviewed
  • [Journal Article] A Video Prediction Method by using Long Short Term Memory based Adaptive Structural Learning of Deep Belief Network and its Investigation of Input Sequence Length for Data Structure2021

    • Author(s)
      Shin Kamada, Takumi Ichimura
    • Journal Title

      International Journal of Computational Intelligence Studies

      Volume: -

    • Related Report
      2020 Research-status Report
    • Peer Reviewed
  • [Presentation] マルチモーダル構造適応型深層学習による RAVDESS データセットの 感情分類2024

    • Author(s)
      鎌田真,市村匠
    • Organizer
      第51回知能システムシンポジウム
    • Related Report
      2023 Annual Research Report
  • [Presentation] 構造適応型深層学習を用いた肺腫瘍セグメンテーション手法の開発2022

    • Author(s)
      鎌田真, 市村匠,河原大輔
    • Organizer
      計測自動制御学会第21回コンピューテーショナル・インテリジェンス研究会
    • Related Report
      2022 Research-status Report
  • [Presentation] Teacher-Student型構造適応型深層学習モデルにおける複数GPU計算機の自動計算2022

    • Author(s)
      市村匠,鎌田真
    • Organizer
      2022 IEEE SMC Hiroshima Chapter Young Researchers WorkShop
    • Related Report
      2022 Research-status Report
  • [Presentation] Automatic Extraction of Road Networks from Satellite Images by using Adaptive Structural Deep Belief Network2021

    • Author(s)
      Shin Kamada and Takumi Ichimura
    • Organizer
      Proc. of 2021 Joint 10th International Conference on Informatics, Electronics & Vision (ICIEV) and 2021 5th International Conference on Imaging, Vision & Pattern Recognition (ICIVPR)
    • Related Report
      2021 Research-status Report
    • Int'l Joint Research
  • [Presentation] Image-Based Early Detection of Alzheimer’s Disease by Using Adaptive Structural Deep Learning2021

    • Author(s)
      Kamada Shin、Ichimura Takumi、Harada Toshihide
    • Organizer
      Proc. of 13th International Conference on Intelligent Decision Technologies (KES IDT 2021)
    • Related Report
      2021 Research-status Report
    • Int'l Joint Research
  • [Presentation] Adaptive Structural Deep Learning to Recognize Kinship Using Families in Wild Multimedia2021

    • Author(s)
      Ichimura Takumi、Kamada Shin
    • Organizer
      Proc. of 13th International Conference on Intelligent Decision Technologies (KES IDT 2021)
    • Related Report
      2021 Research-status Report
    • Int'l Joint Research
  • [Presentation] 構造適応型深層学習を用いた道路網認識手法 RoadTracer による土砂検出の試み2021

    • Author(s)
      鎌田真, 市村匠
    • Organizer
      第20回インテリジェントシステムシンポジウム2021(FAN2021)
    • Related Report
      2021 Research-status Report
  • [Presentation] Teacher-Studentモデルに基づく構造適応型深層学習法による認知症MRI画像判定システム2021

    • Author(s)
      市村匠, 鎌田真, 原田俊英, 井上健
    • Organizer
      第33回日本老年医学会 中国地方会
    • Related Report
      2021 Research-status Report
  • [Presentation] Teacher-Student構造適応型深層学習によるアンサンブル学習と認知症MRI画像診断への応用2021

    • Author(s)
      鎌田真, 市村匠, 原田俊英
    • Organizer
      2021 IEEE SMC Hiroshima Chapter Young Researchers WorkShop
    • Related Report
      2021 Research-status Report
  • [Presentation] 構造適応型深層学習法によるRoadTracerの道路検出に対する一考察2021

    • Author(s)
      市村匠, 鎌田真
    • Organizer
      2021 IEEE SMC Hiroshima Chapter Young Researchers WorkShop
    • Related Report
      2021 Research-status Report
  • [Presentation] 構造適応型深層学習を用いた道路網認識手法RoadTracerへの適用2021

    • Author(s)
      鎌田真,市村匠
    • Organizer
      計測自動制御学会第17回コンピューテーショナル・インテリジェンス研究会
    • Related Report
      2020 Research-status Report
  • [Presentation] A Distillation Learning Model of Adaptive Structural Deep Belief Network for AffectNet: Facial Expression Image Database2020

    • Author(s)
      Takumi Ichimura, Shin Kamada
    • Organizer
      Proc. of the 9th International Congress on Advanced Applied Informatics(IIAI AAI 2020)
    • Related Report
      2020 Research-status Report
    • Int'l Joint Research
  • [Presentation] An Embedded System for Image-based Crack Detection by using Fine-Tuning model of Adaptive Structural Learning of Deep Belief Network"2020

    • Author(s)
      Shin Kamada, Takumi Ichimura
    • Organizer
      Proc. of 2020 IEEE Region 10 Conference (TENCON2020)
    • Related Report
      2020 Research-status Report
    • Int'l Joint Research
  • [Presentation] 認知症早期判定のための構造適応型深層学習によるMRI/PET画像の分類2020

    • Author(s)
      鎌田真,市村匠,原田俊英
    • Organizer
      計測自動制御学会 システム・情報部門 学術講演会 2020 (SSI2020)
    • Related Report
      2020 Research-status Report
  • [Presentation] コンクリート構造物のひび割れ検出構造適応型深層学習システムの開発~広島県橋梁・砂防ダム・湾岸施設の検査システム~2020

    • Author(s)
      鎌田真,市村匠,岩崎貴志
    • Organizer
      計測自動制御学会 システム・情報部門 学術講演会 2020 (SSI2020)
    • Related Report
      2020 Research-status Report
  • [Presentation] 構造適応型深層学習の再学習法を用いたAffectNet顔表情画像の分析2020

    • Author(s)
      鎌田真,市村匠
    • Organizer
      2020 IEEE SMC Hiroshima Chapter Young Researchers WorkShop
    • Related Report
      2020 Research-status Report
  • [Presentation] 構造適応型深層学習による親族間の顔画像データベースの学習2020

    • Author(s)
      市村匠,鎌田真
    • Organizer
      2020 IEEE SMC Hiroshima Chapter Young Researchers WorkShop
    • Related Report
      2020 Research-status Report
  • [Presentation] 組み込みPCを用いたコンクリート構造物のひび割れ検出構造適応型深層学習システムの高速化2020

    • Author(s)
      鎌田真,市村匠
    • Organizer
      2020 IEEE SMC Hiroshima Chapter Young Researchers WorkShop
    • Related Report
      2020 Research-status Report
  • [Presentation] 航空写真を用いた深層学習による道路網認識手法RoadTracerの動作パラメタ実験2020

    • Author(s)
      寒竹夏水,鎌田真,市村匠
    • Organizer
      2020 IEEE SMC Hiroshima Chapter Young Researchers WorkShop
    • Related Report
      2020 Research-status Report
  • [Presentation] An Adaptive Structural Learning of Deep Belief Network for Image-based Crack Detection in Concrete Structures Using SDNET20182020

    • Author(s)
      Shin Kamada, Takumi Ichimura, Takashi Iwasaki
    • Organizer
      Proc. of 2020 International Conference on Image Processing and Robotics (ICIPRoB 2020)
    • Related Report
      2019 Research-status Report
    • Int'l Joint Research
  • [Presentation] 構造適応型DBNによる眼底画像データセットOptosの分類2020

    • Author(s)
      市村匠,鎌田真,石川ますみ,小川諒,河野利治
    • Organizer
      計測自動制御学会第47回知能システムシンポジウム
    • Related Report
      2019 Research-status Report
  • [Presentation] 組み込みPCによるコンクリートひび割れ検出構造適応型深層学習システムの開発2020

    • Author(s)
      鎌田真,市村匠
    • Organizer
      計測自動制御学会第47回知能システムシンポジウム
    • Related Report
      2019 Research-status Report
  • [Presentation] An Object Detection by using Adaptive Structural Learning of Deep Belief Network2019

    • Author(s)
      Shin Kamada, Takumi Ichimura
    • Organizer
      Proc. of The International Joint Conference on Neural Networks (IJCNN 2019)
    • Related Report
      2019 Research-status Report
  • [Presentation] Re-learning of Child Model for Misclassified data by using KL Divergence in AffectNet: A Database for Facial Expressio2019

    • Author(s)
      Takumi Ichimura, Shin Kamada
    • Organizer
      Proc. of 2019 IEEE 11th International Workshop on Computational Intelligence and Applications (IWCIA 2019)
    • Related Report
      2019 Research-status Report
    • Int'l Joint Research
  • [Presentation] A Video Recognition Method by using Adaptive Structural Learning of Long Short Term Memory based Deep Belief Network2019

    • Author(s)
      Shin Kamada, Takumi Ichimura
    • Organizer
      Proc. of 2019 IEEE 11th International Workshop on Computational Intelligence and Applications (IWCIA 2019)
    • Related Report
      2019 Research-status Report
    • Int'l Joint Research
  • [Presentation] AffectNet: 顔表情データベースの構造適応型 DBN による感情モデルの KL 情報量を用いた精度向上2019

    • Author(s)
      鎌田真,市村匠
    • Organizer
      計測自動制御学会第15回コンピューテーショナル・インテリジェンス研究会
    • Related Report
      2019 Research-status Report
  • [Presentation] 建築設備図面への画像認識を取り入れた人工知能の適用に関する研究(第1報)人工知能の深層学習手法について2019

    • Author(s)
      多羅尾直,小川潔,水野暁,市村匠,鎌田真
    • Organizer
      令和元年度空気調和・衛生工学会大会
    • Related Report
      2019 Research-status Report
  • [Presentation] 建築設備図面への画像認識を取り入れた人工知能の適用に関する研究 ( 第3報 ) 構造適応型深層学習法による建築設備図面における図面記号の自動認識システムの構築2019

    • Author(s)
      市村匠,鎌田真,多羅尾直,小川潔
    • Organizer
      令和元年度空気調和・衛生工学会大会
    • Related Report
      2019 Research-status Report
  • [Presentation] SDNET2018: コンクリート構造物画像データセットを用いたひび割れ検出構造適応型深層学習システム2019

    • Author(s)
      鎌田真,市村匠
    • Organizer
      計測自動制御学会システム・情報部門学術講演会
    • Related Report
      2019 Research-status Report
  • [Presentation] 構造適応型DBNの物体検出法におけるパラメタ自動調整と建築設備図面における図面記号への適用2019

    • Author(s)
      市村匠,鎌田真,多羅尾直,山口亮
    • Organizer
      計測自動制御学会第16回コンピューテーショナル・インテリジェンス研究会
    • Related Report
      2019 Research-status Report
  • [Presentation] KL情報量に基づく学習後の構造適応型DBNにおけるニューロン付加手法の提案と顔表情画像への適用2019

    • Author(s)
      鎌田真,市村匠
    • Organizer
      計測自動制御学会第16回コンピューテーショナル・インテリジェンス研究会
    • Related Report
      2019 Research-status Report
  • [Book] "A Teacher-Student based Adaptive Structural Deep learning Model and Its Estimating Uncertainty of Image Data", In "Handbook of Statistics Volume 49: Artificial Intelligence"2023

    • Author(s)
      Takumi Ichimura, Shin Kamada, Toshihide Harada and Ken Inoue
    • Total Pages
      20
    • Publisher
      Elsevier
    • ISBN
      9780443137631
    • Related Report
      2022 Research-status Report
  • [Book] Adaptive Structural Learning of Deep Belief Network and its Application to Real Time Crack Detection of Concrete Structure using Drone", Virvou, Tsihrintzis, Bourbakis, Jain eds. "Handbook on Artificial Intelligence-empowered Applied Software Engineering: VOL.2: Smart Software Applications in Cyber-Physical Systems"2022

    • Author(s)
      Shin Kamada and Takumi Ichimura
    • Total Pages
      20
    • Publisher
      Springer
    • ISBN
      9783031076503
    • Related Report
      2022 Research-status Report

URL: 

Published: 2019-04-18   Modified: 2025-01-30  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi