Fusion of Object Recognition and Material Recognition for Smarter Image Recognition

• Project/Area Number: 26540078
• Research Category: Grant-in-Aid for Challenging Exploratory Research
• Allocation Type: Multi-year Fund
• Research Field: Perceptual information processing
• Research Institution: The University of Tokyo
• Principal Investigator: Yamasaki Toshihiko 東京大学, 大学院情報理工学系研究科, 准教授 (70376599)
• Research Collaborator: AIZAWA Kiyoharu
• Project Period (FY): 2014-04-01 – 2017-03-31
• Project Status: Completed (Fiscal Year 2016)
• Budget Amount: ¥3,510,000 (Direct Cost: ¥2,700,000、Indirect Cost: ¥810,000); Fiscal Year 2016: ¥1,170,000 (Direct Cost: ¥900,000、Indirect Cost: ¥270,000); Fiscal Year 2015: ¥1,170,000 (Direct Cost: ¥900,000、Indirect Cost: ¥270,000); Fiscal Year 2014: ¥1,170,000 (Direct Cost: ¥900,000、Indirect Cost: ¥270,000)
• Keywords: 物体認識 / 物体検出 / 確信度 / 文脈 / 特徴抽出 / 画像認識 / 機械学習

Outline of Final Research Achievements

There are three major contributions in our research. The first contribution was the confidence analysis for multi-class object recognition using the intermediate values from machine learning algorithms. By using the confidence, it has been made possible to improve the recognition accuracy. The second contribution was the methods to automatically find optimal parameter settings for convolutional neural networks (CNNs) by using an evolutionary algorithm called particle swarm optimization (PSO). We have also developed two candidate pruning algorithms for efficient evolutionary process. The third contribution was taking contextual information into consideration such as the co-occurrence of objects and the location of objects in object detection. We developed candidate pruning and object rescoring methods that leverage contextual information and that can improve the state-of-the-art CNN-based object detection methods such as Fast R-CNN and Faster R-CNN.

Research Products

• [Int'l Joint Research] Panyapiwat Institute of Management(タイ)
• [Journal Article] Confidence Analysis for Multi-Class Object Recognition using the Intermediate Values from Machine Learning Algorithms (2015)
  • Author(s): 山崎俊彦, 大島 辰之輔, 相澤清晴
  • Journal Title: The Journal of The Institute of Image Information and Television Engineers Volume: 69 Issue: 8 Pages: J257-J260
  • DOI: 10.3169/itej.69.J257
  • NAID: 130005088777
  • ISSN: 1342-6907, 1881-6908
  • Peer Reviewed / Open Access / Acknowledgement Compliant
• [Presentation] Efficient Optimization of Convolutional Neural Networks using Particle Swarm Optimization (2017)
  • Author(s): T. Yamasaki, T. Honma, and K. Aizawa
  • Organizer: IEEE International Conference on Multimedia Big Data (BigMM)
  • Place of Presentation: Laguna Hills, California, USA
  • Year and Date: 2017-04-19
  • Int'l Joint Research
• [Presentation] 文脈・状況を考慮した物体検出の高精度化 (2017)
  • Author(s): 井上直人, 古田諒佑, 山崎俊彦, 相澤清晴
  • Organizer: 電子情報通信学会, パターン認識・メディア理解研究会 (PRMU)
  • Place of Presentation: 名城大学 天白キャンパス, 愛知県名古屋市
  • Year and Date: 2017-03-20
• [Presentation] Object Detection Refinement using Markov Random Field Based Pruning and Learning Based Rescoring (2017)
  • Author(s): N. Inoue, R. Furuta, T. Yamasaki, and K. Aizawa
  • Organizer: IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2017)
  • Place of Presentation: New Orleans, USA
  • Year and Date: 2017-03-05
  • Int'l Joint Research
• [Presentation] Support Vector Machineを使い倒す (2016)
  • Author(s): 山崎俊彦
  • Organizer: FIT2016イベント企画, ここから始める情報処理 ～画像、音声、テキスト、検索、学習、一気にまとめてチュートリアル～, FIT2016 (第16回情報科学技術フォーラム)
  • Place of Presentation: 富山大学五幅キャンパス, 富山県富山市
  • Year and Date: 2016-09-07
• [Presentation] Efficient Optimization of Convolutional Neural Networks using Particle Swarm Optimization (2016)
  • Author(s): T. Yamasaki, T. Homma, and T. Aizawa
  • Organizer: MIRU2016 (第19回画像の認識・理解シンポジウム)
  • Place of Presentation: PS2-11, アクトシティ浜松, 静岡県浜松市
  • Year and Date: 2016-08-01
• [Presentation] Markov Random Field Based Pruning and Learning Based Rescoring for Object Detection (2016)
  • Author(s): N. Inoue, R. Furuta, T. Yamasaki, and K. Aizawa
  • Organizer: MIRU2016 (第19回画像の認識・理解シンポジウム)
  • Place of Presentation: PS1-01, アクトシティ浜松, 静岡県浜松市
  • Year and Date: 2016-08-01
• [Presentation] Multi-Label Classification using Class Relations Based on Higher-Order MRF Optimization (2016)
  • Author(s): R. Furuta, Y. Fukushima, T. Yamasaki, and K. Aizawa
  • Organizer: BigVision Workshop in conjunction with CVPR 2016
  • Place of Presentation: Las Vegas, USA
  • Year and Date: 2016-06-26
  • Int'l Joint Research
• [Presentation] Towards Online Impression Prediction of Oral Presentations using Soft Coding (2016)
  • Author(s): T. Yamasaki, Y. Fukushima, R. Furuta, and K. Aizawa
  • Organizer: 1st workshop on Attractiveness Computing in Multimedia (ACM) in conjunction with IEEE International Conference on Multimedia Big Data (BigMM)
  • Place of Presentation: Taipei, Taiwan
  • Year and Date: 2016-04-20
  • Int'l Joint Research
• [Presentation] Towards Online Impression Prediction of Oral Presentations using Soft Coding (2016)
  • Author(s): T. Yamasaki, Y. Fukushima, R. Furuta, and K. Aizawa
  • Organizer: Proc. 1st workshop on Attractiveness Computing in Multimedia (ACM)
  • Place of Presentation: Taipei, Taiwan
  • Year and Date: 2016-04-20
  • Int'l Joint Research
• [Presentation] 類似シーン画像を用いた物体検出のフィルタリング (2016)
  • Author(s): 井上直人, 古田諒佑, 山崎俊彦, 相澤清晴
  • Organizer: 情報処理学会第78回全国大会
  • Place of Presentation: 慶応大学、横浜、神奈川
  • Year and Date: 2016-03-10
• [Presentation] カスケード型識別器と進化計算による自動最適化CNNを用いたハイブリッド画像認識 (2016)
  • Author(s): 本間拓人, 山崎俊彦, 相澤清晴, 川崎順平, 茶谷亮裕, 進藤俊彦
  • Organizer: 電子情報通信学会, 信学技報, 画像工学研究会
  • Place of Presentation: 北海道大学、札幌市、北海道
  • Year and Date: 2016-02-22
• [Presentation] Prediction of User Ratings of Oral Presentations using Label Relations (2015)
  • Author(s): T. Yamasaki, Y. Fukushima, R. Furuta, L. Sun, K. Aizawa, and D. Bollegala
  • Organizer: ACMMM Workshop on Affect and Sentiment in Multimedia (ACMMM ASM)
  • Place of Presentation: Brisbane, Australia
  • Year and Date: 2015-11-30
  • Int'l Joint Research
• [Presentation] 文書・音声特徴によるリアルタイム・プレゼンテーション解析にむけた検討 (2015)
  • Author(s): 山崎俊彦, 古田諒佑, 福島悠介, 相澤清晴
  • Organizer: 2015年度映像メディア処理シンポジウム (IMPS 2015)
  • Place of Presentation: ラフォーレ修善寺, Sizuoka, Shuzenji
  • Year and Date: 2015-11-18
• [Presentation] 高階エネルギーのMRF最適化によるラベル共起を考慮したマルチラベル分類 (2015)
  • Author(s): 古田諒佑, 福島悠介, 山崎俊彦, 相澤清晴
  • Organizer: 2015年度映像メディア処理シンポジウム (IMPS 2015)
  • Place of Presentation: ラフォーレ修善寺, 伊豆市、静岡
  • Year and Date: 2015-11-18
• [Presentation] 畳み込みニューラルネットワークの最適化に関する探索の効率化 (2015)
  • Author(s): 本間拓人, 山崎俊彦, 相澤清晴
  • Organizer: 2015年度映像メディア処理シンポジウム (IMPS 2015)
  • Place of Presentation: ラフォーレ修善寺, 伊豆市、静岡
  • Year and Date: 2015-11-18
• [Presentation] MRF-Based Multi-Label Classification using Label Relations (2015)
  • Author(s): Ryosuke Furuta, Yusuke Fukushima, Toshihiko Yamasaki, Kiyoharu Aizawa
  • Organizer: 情報処理学会・第198回コンピュータビジョンとイメージメディア研究発表会(CVIM)
  • Place of Presentation: 愛媛大学, 松山, 愛媛
  • Year and Date: 2015-09-14
• [Presentation] A Fast Table-Based Approach of Bag-of-Features for Large-Scale Image Classification (2015)
  • Author(s): 邱盟竣, 山崎俊彦, 相澤清晴
  • Organizer: 映像情報メディア学会年次大会
  • Place of Presentation: 東京理科大学葛飾キャンパス, 葛飾区、東京
  • Year and Date: 2015-08-26
• [Presentation] クラス間の共起情報を用いた物体検出結果のフィルタリング (2015)
  • Author(s): 井上直人, 古田諒佑, 山崎俊彦, 相澤清晴
  • Organizer: 映像情報メディア学会年次大会
  • Place of Presentation: 東京理科大学葛飾キャンパス, 葛飾区、東京
  • Year and Date: 2015-08-26
• [Presentation] 粒子群最適化を用いた深層学習のアーキテクチャ自動最適化 (2015)
  • Author(s): 本間 拓人, 山崎 俊彦, 相澤 清晴
  • Organizer: The 18th Meeting on Image Recognition and Understanding (MIRU)
  • Place of Presentation: 阪急エクスポパーク、吹田市、大阪
  • Year and Date: 2015-07-27
• [Presentation] A Fast Method of Visual Words Assignment of Bag-of-Features for Object Recognition (2015)
  • Author(s): Meng-Jiun Chiou, Toshihiko Yamasaki, Kiyoharu Aizawa
  • Organizer: The 18th Meeting on Image Recognition and Understanding (MIRU)
  • Place of Presentation: 阪急エクスポパーク、吹田市、大阪
  • Year and Date: 2015-07-27
• [Presentation] Future Design and User Navigation Using Big Multimedia Data (2015)
  • Author(s): T. Yamasaki
  • Organizer: The 1st International Conference on Advanced Imaging (ICAI)
  • Place of Presentation: 一橋記念講堂、千代田区、東京
  • Year and Date: 2015-06-17
  • Int'l Joint Research / Invited
• [Presentation] 多クラス物体認識のための学習を用いた確信度推定 (2015)
  • Author(s): 大島辰之輔, 山崎俊彦, 相澤清晴
  • Organizer: 電子情報通信学会マルチメディア・仮想環境基礎研究会 (MVE)
  • Place of Presentation: 成蹊大学, 東京
  • Year and Date: 2015-03-03 – 2015-03-04
• [Presentation] 画像パッチ検索を利用したBag-of-Featuresの高速化 (2015)
  • Author(s): 伊藤慧, 相澤清晴, 山崎俊彦
  • Organizer: 電子情報通信学会マルチメディア・仮想環境基礎研究会 (MVE)
  • Place of Presentation: 成蹊大学, 東京
  • Year and Date: 2015-03-03 – 2015-03-04
• [Presentation] 多クラス物体認識のための学習を用いた確信度処理 (2014)
  • Author(s): 大島辰之輔, 山崎俊彦, 相澤清晴
  • Organizer: 映像情報メディア学会冬期大会
  • Place of Presentation: 東京理科大学森戸記念館, Tokyo
  • Year and Date: 2014-12-17 – 2014-12-18
• [Presentation] One-vs-All SVMによる多クラス画像認識における認識結果に対する「確信度」解析 (2014)
  • Author(s): 山崎俊彦
  • Organizer: 映像メディア処理シンポジウム (IMPS 2014)
  • Place of Presentation: ラフォーレ修善寺, Sizuoka, Japan
  • Year and Date: 2014-11-12 – 2014-11-14
• [Presentation] Multi-stage object classification featuring confidence analysis of classifier and inclined local naive Bayes nearest neighbor (2014)
  • Author(s): T. Maeda, T. Yamasaki, and K. Aizawa
  • Organizer: International Conference of Image Processing (ICIP)
  • Place of Presentation: Paris, France
  • Year and Date: 2014-10-27 – 2014-10-30
• [Presentation] SVM is not always confident: telling whether the output from SVM is true or false by analyzing its confidence values (2014)
  • Author(s): T. Yamasaki, T. Maeda, and K. Aizawa
  • Organizer: IEEE International Workshop on Multimedia Signal Processing (MMSP)
  • Place of Presentation: Jakarta, Indonesia
  • Year and Date: 2014-09-22 – 2014-09-24
• [Presentation] Parametric meets inclined local NBNN: 確信度とNBNNを用いたBoFベース画像認識の精度向上 (2014)
  • Author(s): 山崎俊彦, 前田嵩樹, 相澤清晴
  • Organizer: 映像情報メディア学会年次大会
  • Place of Presentation: 大阪大学, 大阪
  • Year and Date: 2014-08-31 – 2014-09-02
• [Presentation] Multi-stage object classification featuring confidence analysis of classifier and inclined local naive bayes nearest neighbor (2014)
  • Author(s): T. Yamasaki, T. Maeda, and K. Aizawa
  • Organizer: Meeting on Image Recognition and Understanding (MIRU2014)
  • Place of Presentation: 岡山コンベンションセンター, Okayama, Japan
  • Year and Date: 2014-07-28 – 2014-07-31
• [Remarks] 山崎俊彦(東京大学大学院情報理工学系研究科電子情報学専攻准教授) 研究プロジェクト概要ページ
  • URL: https://www.hal.t.u-tokyo.ac.jp/~yamasaki/project/index-e.html
• [Remarks]
  • URL: https://www.hal.t.u-tokyo.ac.jp/~yamasaki/project/index-e.html
• [Remarks]
  • URL: https://www.hal.t.u-tokyo.ac.jp/~yamasaki/project/index-e.html