Differential Testing Techniques Towards Large-scale Deployment of Deep Learning Systems

• Project/Area Number: 19K24348
• Research Category: Grant-in-Aid for Research Activity Start-up
• Allocation Type: Multi-year Fund
• Review Section: 1001:Information science, computer engineering, and related fields
• Research Institution: Kyushu University
• Principal Investigator: MA LEI 九州大学, システム情報科学研究院, 学術研究員 (70842061)
• Project Period (FY): 2019-08-30 – 2021-03-31
• Project Status: Completed (Fiscal Year 2020)
• Budget Amount: ¥2,860,000 (Direct Cost: ¥2,200,000、Indirect Cost: ¥660,000); Fiscal Year 2020: ¥1,430,000 (Direct Cost: ¥1,100,000、Indirect Cost: ¥330,000); Fiscal Year 2019: ¥1,430,000 (Direct Cost: ¥1,100,000、Indirect Cost: ¥330,000)
• Keywords: 差動テスト / 信頼性と安全性 / 深層学習システム / 深層学習システムの展開 / テスト基準 / ソフトウェア品質保障 / システム開発と展開 / 機械学習工学 / ソフトウェアテスト / 品質保障 / 信頼性

Outline of Research at the Start

深層学習（DL）は、多くの最先端のアプリケーション分野で競争力のある知能と大きな成功を収めてきた。GPU搭載サーバー/クラウドから小型デバイス（携帯電話など）へのDLシステムの導入と適用の需要が急増している。ただし、サーバー/クラウドと小型デバイスの間のDLフレームワーク、プラットフォーム、およびターゲットデバイスのハードウェアの多様性と差異により、DLシステムを小型デバイスに効果的に移行して展開することは非常に困難であり、現在、DLシステムの展開に対して品質保証の手法がまだ欠けている。本研究では、DLシステム展開の品質保証のための自動差動テストフレームワークを構築することが目的としている。

Outline of Final Research Achievements

In this project, we successfully completed the planned research tasks in original research proposal.(1)At an early stage, we made a comprehensive survey to better understand the concrete challenges of deep learning(DL) development and deployment. We found that there are indeed lots of issues causing the deployment quality issues of DL systems.(2)Based on this, we propose multiple differential testing criteria from the uncertainty perspective. We further proposed a differential testing framework named DiffChaser to systematically detect the buggy behavior of DL deployment. We performed systematic evaluations on diverse DL deployment scenarios and found our proposed methods are effective.(3)Furthermore, we conducted in-depth studies on the behavior analysis methods of DL from uncertainty and data distribution perspective with promising results. The results of this project set important foundations on quality assurance of DL deployment for further research and industry applications.

Academic Significance and Societal Importance of the Research Achievements

近年、小型デバイスを目標としたAIチップの急速な進歩は、DLのパワーを小型デバイスにもたらす新たな機会をもたらした。しかし、DLシステムの品質保証技術に関する研究はまだ初期段階である。本研究では、DLの展開段階から多様な小型デバイスへの関連テスト基準、差分テストフレームワーク、および品質向上技術を構築し、重要だが欠けている部分を埋める。高品質のDL手法で小型デバイスを強化することで、応用の範囲がさらに拡大され、知能システムの恩恵が世界中のあらゆる社会にもたらされる。本研究の成果がDLシステム展開のための品質保証を提供し、将来の知能社会の発展を加速するための基盤と応用を築くことが期待される。

Research Products

• [Int'l Joint Research] University of Calgary(カナダ)
• [Int'l Joint Research] Concordia university(カナダ)
• [Int'l Joint Research] Nanyang Technological University(シンガポール)
• [Int'l Joint Research] Peking University(中国)
• [Int'l Joint Research] Nanyang Technological University(シンガポール)
• [Int'l Joint Research] UIUC(米国)
• [Int'l Joint Research] Peking University(中国)
• [Journal Article] Machine Learning Testing: Survey, Landscapes and Horizons (2020)
  • Author(s): Zhang Jie M.、Harman Mark、Ma Lei、Liu Yang
  • Journal Title: IEEE Transactions on Software Engineering Volume: 1 Issue: 1 Pages: 1-37
  • DOI: 10.1109/tse.2019.2962027
  • Peer Reviewed / Int'l Joint Research
• [Presentation] Effective Hybrid System Falsification Using Monte Carlo Tree Search Guided by QB-Robustness (2021)
  • Author(s): Zhenya Zhang, Deyun Lyu, Paolo Arcaini, Lei Ma, Ichiro Hasuo and Jianjun Zhao
  • Organizer: The 33rd International Conference on Computer-Aided Verification, 2021 (CAV 2021, CORE Rank A*)
  • Int'l Joint Research
• [Presentation] A Search-Based Testing Framework for Deep Neural Networks of Source Code Embedding (2021)
  • Author(s): Maryam V. Pour, Li Zhuo, Lei Ma and Hadi Hemmati
  • Organizer: IEEE International Conference on Software Testing, Verification and Validation (ICST 2021, CORE Rank A)
  • Int'l Joint Research
• [Presentation] Decision-Guided Weighted Automata Extraction from Recurrent Neural Networks (2021)
  • Author(s): Xiyue Zhang, Xiaoning Du, Xiaofei Xie, Lei Ma, Yang Liu, Meng Sun
  • Organizer: Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI 2021, CORE Rank A*)
  • Int'l Joint Research
• [Presentation] EfficientDeRain: Learning Pixel-wise Dilation Filtering for High-Efficiency Single-Image (2021)
  • Author(s): Qing Guo, Jingyang Sun, Felix Juefei-Xu, Lei Ma, Xiaofei Xie, Wei Feng, Yang Liu, and Jianjun
  • Organizer: Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI 2021, CORE Rank A*)
  • Int'l Joint Research
• [Presentation] Watch out! Motion is Blurring the Vision of Your Deep Neural Networks (2020)
  • Author(s): Qing Guo, Felix Juefei-Xu, Xiaofei Xie, Lei Ma, Jian Wang, Bing Yu, Wei Feng, Yang Liu
  • Organizer: Thirty-fourth Conference on Neural Information Processing Systems (NeurIPS 2020, CORE Rank A*)
  • Int'l Joint Research
• [Presentation] A first look at the integration of machine learning models in complex autonomous driving systems: a case study on Apollo (2020)
  • Author(s): Zi Peng, Jinqiu Yang, Tse-Hsun Chen, Lei Ma
  • Organizer: The 28th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2020, CORE Rank A*)
  • Int'l Joint Research
• [Presentation] MARBLE: Model-Based Robustness Analysis of Stateful Deep Learning Systems (2020)
  • Author(s): Xiaoning Du, Yi Li, Xiaofei Xie, Lei Ma, Yang Liu, Jianjun Zhao
  • Organizer: The 35th IEEE/ACM International Conference on Automated Software Engineering. (ASE 2020, CORE Rank A)
  • Int'l Joint Research
• [Presentation] Cats Are Not Fish: Deep Learning Testing Calls for Out-Of-Distribution Awareness (2020)
  • Author(s): David Berend, Xiaofei Xie, Lei Ma, Lingjun Zhou, Yang Liu, Chi Xu, Jianjun Zhao
  • Organizer: The 35th IEEE/ACM International Conference on Automated Software Engineering. (ASE 2020, CORE Rank A)
  • Int'l Joint Research
• [Presentation] Few-Shot Guided Mix for DNN Repairing (2020)
  • Author(s): Xuhong Ren, Bing Yu, Hua Qi, Felix Juefei-Xu, Zhuo Li, Wanli Xue, Lei Ma and Jianjun Zhao
  • Organizer: The 36th IEEE International Conference on Software Maintenance and Evolution (CORE Rank A)
  • Int'l Joint Research
• [Presentation] DeepSonar: Towards Effective and Robust Detection of AI-Synthesized Fake Voices (2020)
  • Author(s): Run Wang,Felix Juefei-Xu, Yihao Huang, Qing Guo, Xiaofei Xie, Lei Ma, Yang Liu
  • Organizer: Proceedings of the 28th ACM International Conference on Multimedia (ACM MM, CORE Rank A*)
  • Int'l Joint Research
• [Presentation] Amora: Black-box Adversarial Morphing Attack (2020)
  • Author(s): Run Wang,Felix Juefei-Xu, Yihao Huang, Qing Guo, Xiaofei Xie, Lei Ma, Yang Liu
  • Organizer: Proceedings of the 28th ACM International Conference on Multimedia (ACM MM, CORE Rank A*)
  • Int'l Joint Research
• [Presentation] Towards characterizing adversarial defects of deep learning software from the lens of uncertainty (2020)
  • Author(s): Xiyue Zhang, Xiaofei Xie, Lei Ma, Xiaoning Du, Qiang Hu, Yang Liu, Jianjun Zhao, Meng Sun
  • Organizer: The 42nd International Conference on Software Engineering, 12 pages, 23-29 May 2020, Seoul, South Korea (ICSE’20, CORE Rank A*)
  • Int'l Joint Research
• [Presentation] Generating Adversarial Examples for Holding Robustness of Source Code Processing Models (2020)
  • Author(s): Huangzhao Zhang, Zhuo Li, Ge Li, Lei Ma, Yang Liu, Zhi Jin
  • Organizer: The 34th AAAI Conference on Artificial Intelligence, New York, USA, Feb 7-12, 2020. (AAAI'20, CORE Rank A*)
  • Int'l Joint Research
• [Presentation] Stealthy and Efficient Adversarial Attacks against Deep Reinforcement Learning (2020)
  • Author(s): Jianwen Sun, Tianwei Zhang, Xiaofei Xie, Lei Ma, Yan Zheng, Kangjie Chen, Yang Liu
  • Organizer: The 34th AAAI Conference on Artificial Intelligence, New York, USA, Feb 7-12, 2020. (AAAI'20, CORE Rank A*)
  • Int'l Joint Research
• [Presentation] Generating Adversarial Examples for Holding Robustness of Source Code Processing Models (2020)
  • Author(s): Huangzhao Zhang, Zhuo Li, Ge Li, Lei Ma, Yang Liu, Zhi Jin
  • Organizer: The 34th AAAI Conference on Artificial Intelligence, 8 pages, New York, USA, Feb 7-12, 2020. (AAAI’20, CORE Rank A*)
  • Int'l Joint Research
• [Presentation] Stealthy and Efficient Adversarial Attacks against Deep Reinforcement Learning (2020)
  • Author(s): Jianwen Sun, Tianwei Zhang, Xiaofei Xie, Lei Ma, Yan Zheng, Kangjie Chen, Yang Liu
  • Organizer: The 34th AAAI Conference on Artificial Intelligence, 9 pages, New York, USA, Feb 7-12, 2020. (AAAI’20, CORE Rank A*)
  • Int'l Joint Research
• [Presentation] Towards Characterizing Adversarial Defects of Deep Learning Software from the Lens of Uncertainty (2020)
  • Author(s): Xiyue Zhang, Xiaofei Xie, Lei Ma, Xiaoning Du, Qiang Hu, Yang Liu, Jianjun Zhao, Meng Sun
  • Organizer: The 42nd International Conference on Software Engineering, 12 pages, 23-29 May 2020, Seoul, South Korea (ICSE’20, CORE Rank A*)
  • Int'l Joint Research
• [Presentation] Yan Zheng, Xiaofei Xie, Ting Su, Lei Ma, Jianye Hao, Zhaopeng Meng, Yang Liu, Ruimin Shen, Yinfeng Chen, Changjie Fan (2019)
  • Author(s): Wuji: Automatic Online Combat Game Testing Using Evolutionary Deep Reinforcement Learning
  • Organizer: The 34rd IEEE/ACM International Conference on Automated Software Engineering, pp.772-784, San Diego, California, USA, November 11-15, 2019 (ASE’19, CORE Rank A)
  • Int'l Joint Research
• [Presentation] An Empirical Study towards Characterizing Deep Learning Development and Deployment across Different Frameworks and Platforms (2019)
  • Author(s): Qianyu Guo, Sen Chen, Xiaofei Xie, Lei Ma, Qiang Hu, Hongtao Liu, Yang Liu, Jianjun Zhao, Xiaohong Li
  • Organizer: In Proc. 34th IEEE/ACM Conference on Automated Software Engineering, pp.810-822, San Diego, California, USA, November 11-15, 2019. (ASE’19, CORE Rank A)
  • Int'l Joint Research
• [Presentation] A Quantitative Analysis Framework for Recurrent Neural Network. (2019)
  • Author(s): Xiaoning Du, Xiaofei Xie, Yi Li, Lei Ma, Yang Liu, and Jianjun Zhao
  • Organizer: In Proc. 34th IEEE/ACM Conference on Automated Software Engineering, pp.1062-1065, San Diego, California, USA, November 11-15, 2019. (ASE’19, CORE Rank A)
  • Int'l Joint Research
• [Presentation] Coverage-guided Fuzzing for Feedforward Neural Networks (2019)
  • Author(s): Xiaofei Xie, Hongxu Chen, Yi Li, Lei Ma, Yang Liu, and Jianjun Zhao
  • Organizer: In Proc. 34th IEEE/ACM Conference on Automated Software Engineering, pp.1162-1165, San Diego, California, USA, November 11-15, 2019. (ASE’19, CORE Rank A)
  • Int'l Joint Research
• [Presentation] DeepMutation++: a Mutation Testing Framework for Deep Learning Systems (2019)
  • Author(s): Qiang Hu, Lei Ma, Xiaofei Xie, Bing Yu, Yang Liu, and Jianjun Zhao
  • Organizer: In Proc. 34th IEEE/ACM Conference on Automated Software Engineering, pp.1158-1161, San Diego, California, USA, November 11-15, 2019. (ASE’19, CORE Rank A)
  • Int'l Joint Research
• [Presentation] DeepStellar: Model-Based Quantitative Analysis of Stateful Deep Learning Systems (2019)
  • Author(s): Xiaoning Du, Xiaofei Xie, Yi Li, Lei Ma, Yang Liu and Jianjun Zhao
  • Organizer: The 27th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pp. 477-487, Tallinn, Estonia, August 2019 (FSE’19, CORE A*)
  • Int'l Joint Research
• [Presentation] Secure Deep Learning Engineering: a Road towards Quality Assurance of Intelligent Systems (2019)
  • Author(s): Yang Liu, Lei Ma, Jianjun Zhao
  • Organizer: The 21st International Conference on Formal Engineering Methods, pp.3-15, ShenZhen, China, Nov. 2019 (ICFEM 2019, CORE Rank B)
  • Int'l Joint Research / Invited
• [Presentation] An Empirical Study of Common Challenges in Developing Deep Learning Applications (2019)
  • Author(s): Tianyi Zhang, Cuiyun Gao, Lei Ma, Michael R. Lyu and Miryung Kim
  • Organizer: The 30th International Symposium on Software Reliability Engineering, 12 pages, Oct. 2019, Berlin, Germany (ISSRE’19, CORE Rank A)
  • Int'l Joint Research
• [Presentation] DeepHunter: A Coverage-Guided Fuzz Testing Framework for Deep Neural Networks (2019)
  • Author(s): Xiaofei Xie, Lei Ma [Corresponding Author], Felix Juefei-Xu, Minhui Xue, Hongxu Chen, Yang Liu, Jianjun Zhao, Bo Li, Jianxiong Yin, Simon See
  • Organizer: The 28th International Symposium on Software Testing and Analysis, pp.146-157, Beijing, China, July 2019 (ISSTA’19, CORE Rank A)
  • Int'l Joint Research