• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

プライバシー保護を考慮した深層学習のための学習可能な画像暗号化法の構築

Research Project

Project/Area Number 23K20933
Project/Area Number (Other) 21H01327 (2021-2023)
Research Category

Grant-in-Aid for Scientific Research (B)

Allocation TypeMulti-year Fund (2024)
Single-year Grants (2021-2023)
Section一般
Review Section Basic Section 21020:Communication and network engineering-related
Research InstitutionTokyo Metropolitan University

Principal Investigator

貴家 仁志  東京都立大学, システムデザイン研究科, 特任教授 (40157110)

Co-Investigator(Kenkyū-buntansha) 今泉 祥子  千葉大学, 大学院情報学研究院, 准教授 (80535013)
塩田 さやか  東京都立大学, システムデザイン研究科, 准教授 (90705039)
Project Period (FY) 2021-04-01 – 2025-03-31
Project Status Granted (Fiscal Year 2024)
Budget Amount *help
¥16,380,000 (Direct Cost: ¥12,600,000、Indirect Cost: ¥3,780,000)
Fiscal Year 2024: ¥2,860,000 (Direct Cost: ¥2,200,000、Indirect Cost: ¥660,000)
Fiscal Year 2023: ¥5,460,000 (Direct Cost: ¥4,200,000、Indirect Cost: ¥1,260,000)
Fiscal Year 2022: ¥5,720,000 (Direct Cost: ¥4,400,000、Indirect Cost: ¥1,320,000)
Fiscal Year 2021: ¥2,340,000 (Direct Cost: ¥1,800,000、Indirect Cost: ¥540,000)
Keywords画像暗号化法 / プライバシー保護 / 深層学習 / 画像暗号化
Outline of Research at the Start

本研究の概要は、学修可能な画像暗号化法によるプライバシー保護が可能な深層学習法を構築することである。深層学習に膨大な量の学習用データを必要とするため、プライバシー保護された十分な学習用データの確保が困難であることである。さらに深層学習には大きなメモリコストと計算コストが必要であることから、ユーザーの多くはクラウド環境を深層学習のために選択する。しかし、クラウド環境におけるセキュリティの信頼性は保証されていないという課題がある。

Outline of Annual Research Achievements

本年度は、深層学習への秘密鍵を用いた暗号化法の適用という本研究テーマにおいて、Vision Trasformer及びConvMixerモデルが持つ組込み構造に着目した昨年度の成果に基礎を置き、暗号化法の適用分野のさらなる拡大について研究を進めた。また同時に、これまでの研究によって明確になった解決すべき課題について取り組んだ。理論及び実験の両面から今年度明らかにすることができた実績を、以下にまとめる。
(a) 組込み法に着目した暗号化法の安全性の強化:モデル本来の性能を低下させないこの暗号化の安全法を各種条件下で評価し、さらなる耐性向上を確認することができた。
(b) 暗号化データを用いた連合学習:複数のユーザが自身のデータを直接公開せずに共同でモデル学習を行う連合学習法が注目されている。暗号化データを用いた連合学習法を研究し、新しい連合学習法を提案した。
(c)敵対的事例攻撃に対する耐性効果:入力データに特殊なノイズを加え、モデルを誤誘導する敵対的事例攻撃に対して、暗号化データの使用した対策を研究した。敵対的事例攻撃には種々のタイプがあり、一般にすべてのタイプの攻撃に有効な対策を施す必要がある。本研究では、暗号化を用いたランダムアンサンブルモデルという新しい視点を提案し、その有効性を評価した。
これらの成果の一部を、6編の学術論文及び6編の国際会議論文として出版した。同時に、電子情報通信学会主催のシンポジウムにおいて、、招待講演を行い、最新の研究成果を広く一般に解説した。

Current Status of Research Progress
Current Status of Research Progress

2: Research has progressed on the whole more than it was originally planned.

Reason

研究成果の一部を6編の学術論文及び6編の国際会議論文として発表することができ、研究は 順調に進んでいる。さらに、国際会議でのスペシャルセッションでの講演やシンポジウムでの招待講演を行うことができ、広く本テーマの重要性や最新の研究成果を一般に公開することができた。特に、近年高性能モデルとして注目を集めているVision Trasformer及びConvMixerに焦点をあてた成果は、画像分類タスク以外にも暗号化データを適用することを可能として、かつモデルの性能に影響を与えない暗号化を可能にすることから、多くの研究者から注目されている。さらに今年度は、さらなる攻撃耐性向上、暗号化データを用いた連合学習法への展開、敵対的事例攻撃に対する耐性効果という、学習可能な暗号化の新しい成果をあげることができた。
今年度の成果によって、高い信頼性を持つ深層学習法の構築のための、秘密鍵を用いた暗号化法の適用という着想の正当性を再確認することができた。同時に、今年度の成果によって、より広い関連分野においてこの暗号化法が新しい視点を与えることが期待できる。

Strategy for Future Research Activity

深層学習への秘密鍵を用いた暗号化法の適用という本研究テーマにおいて、Vision Trasformer及びConvMixerモデルが持つ組込み構造に着目したこれまでの成果によって、暗号化法のプライバシー保護の観点からの有効性に加え、その応用分野のさらなる拡大が示された。最終年度である令和6年度は、これまでの成果を総合的に評価を行い、提案法の有効性や適用限界を明確するための考察と、ジャーナル論文を執筆して成果の公開に努める計画である。具体的には、以下の5点を中心に研究を進める。
(a) 組込み法に着目した暗号化法の安全性の強化法に関する論文執筆。
(b) 暗号化データを用いた連合学習法の構築と論文執筆。
(c) 敵対的事例攻撃に対する耐性効果法に関する論文執筆。
(d) 関連研究の再調査と提案法との客観的比較評価。(e) 残された今後の課題の確認  以上の5点の検討方針を実行を通して、4年間の研究成果をまとめ、広く一般に研究成果を公開する。

Report

(3 results)
  • 2023 Annual Research Report
  • 2022 Annual Research Report
  • 2021 Annual Research Report
  • Research Products

    (43 results)

All 2023 2022 2021

All Journal Article (15 results) (of which Int'l Joint Research: 2 results,  Peer Reviewed: 15 results,  Open Access: 12 results) Presentation (28 results) (of which Int'l Joint Research: 25 results,  Invited: 4 results)

  • [Journal Article] Image and Model Transformation with Secret Key for Vision Transformer2023

    • Author(s)
      KIYA Hitoshi、IIJIMA Ryota、MAUNGMAUNG Aprilpyone、KINOSHITA Yuma
    • Journal Title

      IEICE Transactions on Information and Systems

      Volume: E106.D Issue: 1 Pages: 2-11

    • DOI

      10.1587/transinf.2022MUI0001

    • ISSN
      0916-8532, 1745-1361
    • Year and Date
      2023-01-01
    • Related Report
      2022 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Access Control with Encrypted Feature Maps for Object Detection Models2023

    • Author(s)
      NAGAMORI Teru、ITO Hiroki、MAUNGMAUNG AprilPyone、KIYA Hitoshi
    • Journal Title

      IEICE Transactions on Information and Systems

      Volume: E106.D Issue: 1 Pages: 12-21

    • DOI

      10.1587/transinf.2022MUP0002

    • ISSN
      0916-8532, 1745-1361
    • Year and Date
      2023-01-01
    • Related Report
      2022 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Effects of JPEG Compression on Vision Transformer Image Classification for Encryption-then-Compression Images2023

    • Author(s)
      Hamano Genki、Imaizumi Shoko、Kiya Hitoshi
    • Journal Title

      Sensors

      Volume: 23 Issue: 7 Pages: 3400-3400

    • DOI

      10.3390/s23073400

    • Related Report
      2023 Annual Research Report 2022 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Privacy-Preserving Image Classification Using ConvMixer with Adaptative Permutation Matrix and Block-Wise Scrambled Image Encryption2023

    • Author(s)
      Qi Zheng、MaungMaung AprilPyone、Kiya Hitoshi
    • Journal Title

      Journal of Imaging

      Volume: 9 Issue: 4 Pages: 85-85

    • DOI

      10.3390/jimaging9040085

    • Related Report
      2023 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Missing data completion of multi-channel signals using autoencoder for acoustic scene classification2023

    • Author(s)
      Yuki Shiroma, Yuma Kinoshita, Keisuke Imoto, Sayaka Shiota, Nobutaka Ono, and Hitoshi Kiya
    • Journal Title

      APSIPA Transactions on Signal and Information Processing

      Volume: 12 Issue: 3 Pages: 1-22

    • DOI

      10.1561/116.00000074

    • Related Report
      2023 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] A Jigsaw Puzzle Solver-Based Attack on Image Encryption Using Vision Transformer for Privacy-Preserving DNNs2023

    • Author(s)
      Chuman Tatsuya、Kiya Hitoshi
    • Journal Title

      Information

      Volume: 14 Issue: 6 Pages: 311-311

    • DOI

      10.3390/info14060311

    • Related Report
      2023 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Reversible Data Hiding in Compressible Encrypted Images with Capacity Enhancement2023

    • Author(s)
      Motomura Ryota、Imaizumi Shoko、Kiya Hitoshi
    • Journal Title

      APSIPA Transactions on Signal and Information Processing

      Volume: 12 Issue: 1 Pages: 1-23

    • DOI

      10.1561/116.00000014

    • Related Report
      2023 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Block-Wise Encryption for Reliable Vision Transformer models2023

    • Author(s)
      Hitoshi Kiya, Ryoto Iijima, Teru Nagamori
    • Journal Title

      ECTITransactionsonComputerandInformationTechnology

      Volume: 17 Pages: 409-419

    • Related Report
      2023 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Privacy-Preserving Image Classification Using an Isotropic Network2022

    • Author(s)
      AprilPyone MaungMaung、Kiya Hitoshi
    • Journal Title

      IEEE MultiMedia

      Volume: 29 Issue: 2 Pages: 23-33

    • DOI

      10.1109/mmul.2022.3168441

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed
  • [Journal Article] An Overview of Compressible and Learnable Image Transformation with Secret Key and its Applications2022

    • Author(s)
      Kiya Hitoshi、Maung April Pyone Maung、Kinoshita Yuma、Imaizumi Shoko、Shiota Sayaka
    • Journal Title

      APSIPA Transactions on Signal and Information Processing

      Volume: 11 Issue: 1

    • DOI

      10.1561/116.00000048

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Access Control of Semantic Segmentation Models Using Encrypted Feature Maps2022

    • Author(s)
      Ito Hiroki、AprilPyone MaungMaung、Shiota Sayaka、Kiya Hitoshi
    • Journal Title

      APSIPA Transactions on Signal and Information Processing

      Volume: 11 Issue: 1

    • DOI

      10.1561/116.00000013

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Privacy-Preserving Semantic Segmentation Using Vision Transformer2022

    • Author(s)
      Kiya Hitoshi、Nagamori Teru、Imaizumi Shoko、Shiota Sayaka
    • Journal Title

      Journal of Imaging

      Volume: 8 Issue: 9 Pages: 233-233

    • DOI

      10.3390/jimaging8090233

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] A Reversible Data-Hiding Method with Prediction-Error Expansion in Compressible Encrypted Images2022

    • Author(s)
      Motomura Ryota、Imaizumi Shoko、Kiya Hitoshi
    • Journal Title

      Applied Sciences

      Volume: 12 Issue: 19 Pages: 9418-9418

    • DOI

      10.3390/app12199418

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Image to Perturbation: An Image Transformation Network for Generating Visually Protected Images for Privacy-Preserving Deep Neural Networks2021

    • Author(s)
      Ito Hiroki、Kinoshita Yuma、Aprilpyone Maungmaung、Kiya Hitoshi
    • Journal Title

      IEEE Access

      Volume: 9 Pages: 64629-64638

    • DOI

      10.1109/access.2021.3074968

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] A protection method of trained CNN model with a secret key from unauthorized access2021

    • Author(s)
      Maungmaung AprilPyone、Kiya Hitoshi
    • Journal Title

      APSIPA Transactions on Signal and Information Processing

      Volume: 10 Issue: 1

    • DOI

      10.1017/atsip.2021.9

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed / Open Access
  • [Presentation] A privacy-preserving method using secret key for convolutional neural network-based speech classification2023

    • Author(s)
      Shoko NIWA, Sayaka SHIOTA, Hitoshi KIYA
    • Organizer
      EURASIP European Signal Processing Conference
    • Related Report
      2023 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Security Evaluation of Compressible and Learnable Image Encryption Against Jigsaw Puzzle Solver Attacks2023

    • Author(s)
      Tatsuya CHUMAN, Nobutaka ONO, Hitoshi KIYA
    • Organizer
      IEEE Global Conference on Consumer Electronics
    • Related Report
      2023 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Effects of Data Hiding on Vision Transformer Classification for Encryption-Then-Compression Images2023

    • Author(s)
      Kosuke ABE, Shoko IMAIZUMI, Hitoshi KIY
    • Organizer
      IEEE Global Conference on Consumer Electronics
    • Related Report
      2023 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Enhanced Security With Encrypted Vision Transformer in Federated Learning2023

    • Author(s)
      Rei ASO, Sayaka SHIOTA, Hitoshi KIYA
    • Organizer
      IEEE Global Conference on Consumer Electronics
    • Related Report
      2023 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Enhanced Security Against Adversarial Examples Using a Random Ensemble of Encrypted Vision Transformer Models2023

    • Author(s)
      Ryota IIJIMA, Miki TANAKA, Sayaka SHIOTA, Hitoshi KIYA
    • Organizer
      IEEE Global Conference on Consumer Electronics
    • Related Report
      2023 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Domain Adaptation for Efficiently Fine-Tuning Vision Transformer with Encrypted Images2023

    • Author(s)
      Teru NAGAMORI, Sayaka SHIOTA, Hitoshi KIYA
    • Organizer
      APSIPA Annual Summit and Conference
    • Related Report
      2023 Annual Research Report
    • Int'l Joint Research
  • [Presentation] 人とAIの認知の差異を求めて: プライバシー保護、敵対的事例、連合学習を例にして2023

    • Author(s)
      貴家 仁志
    • Organizer
      電子情報通信学会 PCSJ/IMPS 2023
    • Related Report
      2023 Annual Research Report
    • Invited
  • [Presentation] 信頼できるAIのための乱数を用いた画像変換2023

    • Author(s)
      貴家仁志
    • Organizer
      電子情報通信学会エンリッチメント研究会
    • Related Report
      2022 Annual Research Report
    • Invited
  • [Presentation] Progress and Challenges in Learnable Image Encryption with Secret Key for Reliable Deep Learning2023

    • Author(s)
      Hitoshi Kiya
    • Organizer
      15th International Conference on Knowledge and Smart Technology
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research / Invited
  • [Presentation] A Jigsaw Puzzle Solver-based Attack on Block-wise Image Encryption for Privacy-preserving DNNs2023

    • Author(s)
      Tatsuya CHUMAN, Hitoshi KIYA
    • Organizer
      International Workshop on Advanced Image Technology
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Combined Use of Federated Learning and Image Encryption for Privacy-Preserving Image Classification with Vision Transformer2023

    • Author(s)
      Teru NAGAMORI, Hitoshi KIYA
    • Organizer
      RISP International Workshop on Nonlinear Circuits, Communications and Signal Processing
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Color-NeuraCrypt: Privacy-Preserving Color-Image Classification Using Extended Random Neural Networks2023

    • Author(s)
      Zheng QI, April Pyone MAUNG MAUNG, Hitoshi KIYA
    • Organizer
      RISP International Workshop on Nonlinear Circuits, Communications and Signal Processing
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] A Privacy Preserving Method with a Random Orthogonal Matrix for ConvMixer Models2023

    • Author(s)
      Rei ASO, Tatsuya CHUMAN, Hitoshi KIYA
    • Organizer
      RISP International Workshop on Nonlinear Circuits, Communications and Signal Processing
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] 真実の可視化を操作する:AIを騙す,データ・AIを守る2022

    • Author(s)
      貴家仁志
    • Organizer
      電子情報通信学会信号処理シンポジウム
    • Related Report
      2022 Annual Research Report
    • Invited
  • [Presentation] Privacy-Preserving Image Classification Using Vision Transformer2022

    • Author(s)
      Zheng QI, April Pyone MAUNG MAUNG, Yuma KINOSHITA, Hitoshi KIYA
    • Organizer
      EURASIP European Signal Processing Conference
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] An Access Control Method with Secret Key for Semantic Segmentation Models2022

    • Author(s)
      Teru NAGAMORI, Ryota IIJIMA, Hitoshi KIYA
    • Organizer
      International Conference on Machine Learning and Cybernetics
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] An Encryption Method of ConvMixer Models without Performance Degradation2022

    • Author(s)
      Ryota IIJIMA, Hitoshi KIYA
    • Organizer
      International Conference on Machine Learning and Cybernetics
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] SECURITY EVALUATION OF COMPRESSIBLE IMAGE ENCRYPTION FOR PRIVACY-PRESERVING IMAGE CLASSIFICATION AGAINST CIPHERTEXT-ONLY ATTACKS2022

    • Author(s)
      Tatsuya CHUMAN, Hitoshi KIYA
    • Organizer
      International Conference on Machine Learning and Cybernetics
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Privacy-Preserving Image Classification Using ConvMixer with Adaptive Permutation Matrix2022

    • Author(s)
      Zheng QI, April Pyone MAUNG MAUNG, Hitoshi KIYA
    • Organizer
      IEEE Global Conference on Consumer Electronics
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] StyleGAN Encoder-Based Attack for Block Scrambled Face Images2022

    • Author(s)
      April Pyone MAUNG MAUNG, Hitoshi KIYA
    • Organizer
      APSIPA Annual Summit and Conference
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] On the Transferability of Adversarial Examples between Encrypted Models2022

    • Author(s)
      Miki TANAKA, Isao ECHIZEN, Hitoshi KIYA
    • Organizer
      APSIPA Annual Summit and Conference
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] On the Transferability of Adversarial Examples between Encrypted Models2022

    • Author(s)
      Miki TANAKA, Isao ECHIZEN, Hitoshi KIYA
    • Organizer
      IEEE International Symposium on Intelligent Signal Processing and Communication Systems
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Access Control of Object Detection Models Using Encrypted Feature Maps2022

    • Author(s)
      Teru NAGAMORI, Hiroki ITO, April Pyone MAUNG MAUNG, Hitoshi KIYA,
    • Organizer
      IEEE Global Conference on Life Sciences and Technologies
    • Related Report
      2021 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Adversarial Detector with Robust Classifier2022

    • Author(s)
      Takayuki OSAKABE, April Pyone MAUNG MAUNG, Sayaka SHIOTA, Hitoshi KIYA
    • Organizer
      IEEE Global Conference on Life Sciences and Technologies
    • Related Report
      2021 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Transfer Learning-Based Model Protection With Secret Key,2021

    • Author(s)
      April Pyone MAUNG MAUNG, Hitoshi KIYA,
    • Organizer
      IEEE International Conference on Image Processing
    • Related Report
      2021 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Access Control Using Spatially Invariant Permutation of Feature Maps for Semantic Segmentation Models2021

    • Author(s)
      Hiroki ITO, April Pyone MAUNG MAUNG, Hitoshi KIYA
    • Organizer
      APSIPA Annual Summit and Conference
    • Related Report
      2021 Annual Research Report
    • Int'l Joint Research
  • [Presentation] A Privacy-Preserving Image Retrieval Scheme Using a Codebook Generated From Independent Plain-Image Dataset2021

    • Author(s)
      Kenta IIDA, Hitoshi KIYA
    • Organizer
      APSIPA Annual Summit and Conference
    • Related Report
      2021 Annual Research Report
    • Int'l Joint Research
  • [Presentation] A Protection Method of Trained CNN Model Using Feature Maps Transformed With Secret Key From Unauthorized Access2021

    • Author(s)
      April Pyone MAUNG MAUNG, Hitoshi KIYA
    • Organizer
      APSIPA Annual Summit and Conference
    • Related Report
      2021 Annual Research Report
    • Int'l Joint Research

URL: 

Published: 2021-04-28   Modified: 2024-12-25  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi