Thery and methods for high dimensional data analysis with internal structure

• Project/Area Number: 26280009
• Research Category: Grant-in-Aid for Scientific Research (B)
• Allocation Type: Partial Multi-year Fund
• Section: 一般
• Research Field: Statistical science
• Research Institution: The Institute of Statistical Mathematics
• Principal Investigator: Fukumizu Kenji 統計数理研究所, 数理・推論研究系, 教授 (60311362)
• Co-Investigator(Kenkyū-buntansha): 鈴木 大慈 東京大学, 大学院情報理工学系研究科, 准教授 (60551372) ; 小林 景 慶應義塾大学, 理工学部(矢上), 准教授 (90465922)
• Project Period (FY): 2014-04-01 – 2019-03-31
• Project Status: Completed (Fiscal Year 2018)
• Budget Amount: ¥12,350,000 (Direct Cost: ¥9,500,000、Indirect Cost: ¥2,850,000); Fiscal Year 2018: ¥2,080,000 (Direct Cost: ¥1,600,000、Indirect Cost: ¥480,000); Fiscal Year 2017: ¥2,600,000 (Direct Cost: ¥2,000,000、Indirect Cost: ¥600,000); Fiscal Year 2016: ¥2,080,000 (Direct Cost: ¥1,600,000、Indirect Cost: ¥480,000); Fiscal Year 2015: ¥2,210,000 (Direct Cost: ¥1,700,000、Indirect Cost: ¥510,000); Fiscal Year 2014: ¥3,380,000 (Direct Cost: ¥2,600,000、Indirect Cost: ¥780,000)
• Keywords: 統計数学 / データ解析 / 高次元 / アルゴリズム / 統計的学習理論 / 高次元データ / 機械学習

Outline of Final Research Achievements

The hub phenomenon has been analysed as an example of special properties of high-dimensional data, and a method of resolving hubs has been proposed. Methods of data analysis have been proposed for non-Euclidean data such as trees and points with skewed distance measures. Additionally, for a new method of density estimation of high-dimensional data, a kernel methods for constructing infinite dimensional exponential families has been proposed, and its estmation has been discussed.

Academic Significance and Societal Importance of the Research Achievements

ビッグデータ時代になり高次元で複雑なデータを扱う必要性が高まったが，そのようなデータの性質や解析法の関して，理論的な知見や有効な方法が成果として得られた．今後，さまざまな分野に現れる高次元データを扱う際にこれらの成果が貢献できると考える．

Research Products

• [Int'l Joint Research] University of Oxford/Imperial College London(英国)
• [Int'l Joint Research] Carnegie Mellon University(米国)
• [Int'l Joint Research] University College London/Oxford University(英国)
• [Int'l Joint Research] Pennsylvania State University(米国)
• [Int'l Joint Research] Mahidol University(タイ)
• [Int'l Joint Research] University of Kentucky/, North Dakota State University,(米国)
• [Int'l Joint Research] University of Novi Sad(Serbia)
• [Int'l Joint Research] Pennsylvania State University(米国)
• [Journal Article] Empirical geodesic graphs and CAT(k) metrics for data analysis (2019)
  • Author(s): Kei Kobayashi and Henry P. Wynn
  • Journal Title: Statistics and Computing Volume: 2019 Feb. Issue: 1 Pages: 1-18
  • DOI: 10.1007/s11222-019-09855-3
  • Peer Reviewed / Int'l Joint Research
• [Journal Article] Post Selection Inference with Incomplete Maximum Mean Discrepancy Estimator (2018)
  • Author(s): Makoto Yamada, Denny Wu, Yao-Hung Hubert Tsai, Hirofumi Ohta, Ruslan Salakhutdinov, Ichiro Takeuchi, Kenji Fukumizu
  • Journal Title: Proceedings of the 7th Seventh International Conference on Learning Representations Volume: 7 Pages: 1-16
  • Peer Reviewed / Open Access / Int'l Joint Research
• [Journal Article] Variational Learning on Aggregate Outputs with Gaussian Processes (2018)
  • Author(s): Ho Chung Law, Dino Sejdinovic, Ewan Cameron, Tim Lucas, Seth Flaxman, Katherine Battle, Kenji Fukumizu
  • Journal Title: Advances in Neural Information Processing Systems Volume: 31
  • Peer Reviewed / Int'l Joint Research
• [Journal Article] Fast learning rate of non-sparse multiple kernel learning and optimal regularization strategies (2018)
  • Author(s): Suzuki Taiji
  • Journal Title: Electronic Journal of Statistics Volume: 12 Issue: 2 Pages: 2141-2192
  • DOI: 10.1214/18-ejs1399
  • Peer Reviewed / Open Access
• [Journal Article] Short-term local weather forecast using dense weather station by deep neural network (2018)
  • Author(s): Yonekura Kazuo、Hattori Hitoshi、Suzuki Taiji
  • Journal Title: 2018 IEEE International Conference on Big Data (Big Data) Volume: 1 Pages: 10-13
  • DOI: 10.1109/bigdata.2018.8622195
  • Peer Reviewed / Open Access
• [Journal Article] Sample Efficient Stochastic Gradient Iterative Hard Thresholding Method for Stochastic Sparse Linear Regression with Limited Attribute Observation (2018)
  • Author(s): Tomoya Murata, and Taiji Suzuki
  • Journal Title: Advances in Neural Information Processing Systems Volume: 31
  • Peer Reviewed / Open Access
• [Journal Article] Adam Induces Implicit Weight Sparsity in Rectifier Neural Networks (2018)
  • Author(s): Atsushi Yaguchi, Taiji Suzuki, Wataru Asano, Shuhei Nitta, Yukinobu Sakata, Akiyuki Tanizawa
  • Journal Title: Proceedings of IEEE 17th International Conference on Machine Learning and Applications (ICMLA 2018) Volume: * Pages: 17-20
  • DOI: 10.1109/icmla.2018.00054
  • Peer Reviewed / Open Access
• [Journal Article] Post Selection Inference with Kernels (2018)
  • Author(s): Makoto Yamada, Yuta Umezu, Kenji Fukumizu, and Ichiro Takeuchi
  • Journal Title: Proceedings of Machine Learning Research: AISTATS 2018 Volume: 84 Pages: 152-160
  • Peer Reviewed / Open Access
• [Journal Article] Kernel Mean Embedding of Distributions: A Review and Beyond (2017)
  • Author(s): Krikamol Muandet, Kenji Fukumizu, Bharath K. Sriperumbudur, Bernhard Schoelkopf
  • Journal Title: Foundations and Trends in Machine Learning Volume: 10 Issue: 1-2 Pages: 1-141
  • DOI: 10.1561/2200000060
  • Peer Reviewed / Open Access / Int'l Joint Research
• [Journal Article] Density Estimation in Infinite Dimensional Exponential Families (2017)
  • Author(s): Bharath Sriperumbudur, Kenji Fukumizu, Arthur Gretton, Aapo Hyvarinen, Revant Kumar
  • Journal Title: Journal of Machine Learning Research Volume: 18(57) Pages: 1-59
  • Peer Reviewed / Open Access / Int'l Joint Research
• [Journal Article] Multilocus phylogenetic analysis with gene tree clustering (2017)
  • Author(s): Ruriko Yoshida, Kenji Fukumizu, Chrysafis Vogiatzis
  • Journal Title: Annals of Operations Research Volume: 印刷中 Issue: 1-2 Pages: 293-313
  • DOI: 10.1007/s10479-017-2456-9
  • Peer Reviewed / Int'l Joint Research
• [Journal Article] A Characterization of Minimum Spanning Tree-like Metric Spaces (2016)
  • Author(s): Hayamizu, M., Endo, H. and Fukumizu, K.
  • Journal Title: IEEE/ACM Transactions on Computational Biology and Bioinformatics Volume: 99 Issue: 2 Pages: 1-1
  • DOI: 10.1109/tcbb.2016.2550431
  • Peer Reviewed / Open Access
• [Journal Article] Unsupervised Group Matching with Application to Cross-lingual Topic Matching without Alignment Information (2016)
  • Author(s): Iwata, T., Kanagawa, M., Hirao, T. and Fukumizu, K.
  • Journal Title: Data Mining and Knowledge Discovery Volume: 2016 Pages: 1-19
  • Peer Reviewed / Open Access
• [Journal Article] Flattening the Density Gradient for Eliminating Spatial Centrality to Reduce Hubness (2016)
  • Author(s): Hara, K., Suzuki, I., Kobayashi, K., Fukumizu, K. and Radovanovic, M.
  • Journal Title: Proc. 30th AAAI Conference on Artificial Intelligence Volume: 1 Pages: 1659-1665
  • Peer Reviewed / Open Access / Int'l Joint Research
• [Journal Article] System Identification and Parameter Estimation in Mathematical Medicine: Examples Demonstrated for Prostate Cancer (2016)
  • Author(s): Yoshito Hirata, Kai Morino, Taiji Suzuki, Qian Guo, Hiroshi Fukuhara, and Kazuyuki Aihara
  • Journal Title: Quantitative Biology Volume: 4(1) Issue: 1 Pages: 13-19
  • DOI: 10.1007/s40484-016-0059-0
  • Peer Reviewed / Open Access / Int'l Joint Research / Acknowledgement Compliant
• [Journal Article] Stochastic Alternating Direction Method of Multipliers for Structured Regularization (2015)
  • Author(s): Taiji Suzuki
  • Journal Title: Journal of Japan Society of Computational Statistics Volume: 28 Pages: 105-124
  • NAID: 130005434004
  • Peer Reviewed
• [Journal Article] Localized Centering: Reducing Hubness in Large-Sample Data (2015)
  • Author(s): K. Hara, I. Suzuki, M. Shimbo, K. Kobayashi, K. Fukumizu, and M. Radomanovic
  • Journal Title: Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence Volume: 1 Pages: 2645-2651
  • Peer Reviewed / Open Access
• [Journal Article] A Consistent Method for Graph Based Anomaly Localization (2015)
  • Author(s): Satoshi Hara, Tetsuro Morimura, Toshihiro Takahashi, Hiroki Yanagisawa, Taiji Suzuki
  • Journal Title: JMLR Workshop and Conference Proceedings Volume: 38
  • Peer Reviewed / Open Access
• [Journal Article] Support Consistency of Direct Sparse-Change Learning in Markov Networks (2015)
  • Author(s): Song Liu, Taiji Suzuki, and Masashi Sugiyama
  • Journal Title: Proceedigs of the Twenty-Ninth AAAI Conference on Artificial Intelligence (AAAI2015) Volume: 1 Pages: 2785-2791
  • NAID: 110009971454
  • Peer Reviewed / Open Access
• [Journal Article] 凸最適化に基づくテンソル分解 (2014)
  • Author(s): 冨岡亮太，鈴木大慈，林浩平，鹿島久嗣
  • Journal Title: 応用数理 Volume: 24 Pages: 16-23
  • NAID: 110009900559
  • Peer Reviewed
• [Journal Article] Stochastic Dual Coordinate Ascent with Alternating Direction Method of Multipliers (2014)
  • Author(s): Taiji Suzuki
  • Journal Title: JMLR Workshop and Conference Proceedings Volume: 32 Pages: 736-744
  • Peer Reviewed / Open Access
• [Presentation] Element-wise shrinkage of positive correlation matrices for increasing sparsity (2018)
  • Author(s): Kei Kobayashi, Henry P. Wynn
  • Organizer: 2018 IMS Annual Meeting on Probability and Statistics
  • Int'l Joint Research
• [Presentation] Monotone shrinkage of correlation matrices for sparsity (2018)
  • Author(s): Kei Kobayashi
  • Organizer: 統計関連学会連合大会
• [Presentation] 2次元アニメーション制作における中割り自動化のためのベクタ形式画像の対応づけ (2018)
  • Author(s): 童 祺俊，小林 景
  • Organizer: 統計関連学会連合大会
• [Presentation] 無限隠れマルコフモデルによるバスケットボールの戦術分析 (2018)
  • Author(s): 竹原 大翼，小林 景
  • Organizer: 統計関連学会連合大会
• [Presentation] MHアルゴリズムによる分散共分散行列のグラフ推定とその応用 (2018)
  • Author(s): 夏 悠寧，小林 景
  • Organizer: 統計関連学会連合大会
• [Presentation] カーネル法に基づく疎な言語表現のための高速計算可能な共起尺度 (2018)
  • Author(s): 横井祥, 小林 颯介, 福水健次, 乾 健太郎
  • Organizer: 2018年度 人工知能学会全国大会（第32回）
• [Presentation] 教師ありクラスタマッチングとその隕石-小惑星分類体系への応用 (2018)
  • Author(s): 福水健次
  • Organizer: 日本分類学会第37回大会
  • Invited
• [Presentation] 統計・機械学習における確率的最適化 (2018)
  • Author(s): 鈴木大慈
  • Organizer: 統計数理研究所公開講座
  • Invited
• [Presentation] 深層学習の統計的学習理論：カーネル法とウェーブレット解析による視点 (2018)
  • Author(s): 鈴木大慈
  • Organizer: 第3回統計・機械学習若手シンポジウム「統計・機械学習の交わりと拡がり」
  • Invited
• [Presentation] 機械学習における構造を利用した確率的最適化技法 (2018)
  • Author(s): 鈴木大慈
  • Organizer: 2018年電子情報通信学会基礎・境界ソサイエティ大会大会
  • Invited
• [Presentation] 機械学習の現状と深層学習の数理 (2018)
  • Author(s): 鈴木大慈
  • Organizer: 山形大学データサイエンス推進室キックオフミーティング
  • Invited
• [Presentation] カーネル法に基づく疎な言語表現のための共起尺度 (2018)
  • Author(s): 横井祥, 小林颯介 , 福水健次, 乾健太郎
  • Organizer: 言語処理学会第24回年次大会
• [Presentation] データ構造を解明する機械学習アプローチ --グラフ構造、幾何構造-- (2017)
  • Author(s): 福水健次
  • Organizer: 第3回WIRPワークショップ
  • Invited
• [Presentation] Local minima and saddle points in hierarchical structure of neural networks (2017)
  • Author(s): Kenji Fukumizu
  • Organizer: Deep Learning: Theory, Algorithms, and Applications
  • Int'l Joint Research / Invited
• [Presentation] 低ランクテンソル推定におけるベイズ推定量の性質 (2015)
  • Author(s): 鈴木大慈
  • Organizer: 第9回日本統計学会春季集会
  • Place of Presentation: 東京
  • Year and Date: 2015-03-08
  • Invited
• [Presentation] スパース推定概観：モデル・理論・応用 (2014)
  • Author(s): 鈴木大慈
  • Organizer: 統計関連学会連合大会
  • Place of Presentation: 東京
  • Year and Date: 2014-09-13 – 2014-09-16
  • Invited
• [Presentation] Risk Bounds of Convex and Bayes Tensor Estimators: Near Optimal Rate without Strong Convexity (2014)
  • Author(s): Taiji Suzuki
  • Organizer: International Workshop on Spatial and Temporal Modeling from Statistical, Machine Learning and Engineering perspectives (STM2014)
  • Place of Presentation: 東京
  • Year and Date: 2014-07-28 – 2014-07-29
  • Invited
• [Presentation] Stochastic Dual Coordinate Ascent with ADMM (2014)
  • Author(s): Taiji Suzuki
  • Organizer: SIAM Conference on Optimization (SIAM-OPT2014)
  • Place of Presentation: San Diego, USA
  • Year and Date: 2014-05-19 – 2014-05-22
• [Remarks]
  • URL: http://www.ism.ac.jp/~fukumizu/
• [Remarks] http://www.ism.ac.jp/~fukumizu/
• [Remarks]
  • URL: http://www.ism.ac.jp/~fukumizu/