Initial value selection and acceleration of the EM algorithm for finite mixture models

• Project/Area Number: 16K00061
• Research Category: Grant-in-Aid for Scientific Research (C)
• Allocation Type: Multi-year Fund
• Section: 一般
• Research Field: Statistical science
• Research Institution: Okayama University of Science
• Principal Investigator: Kuroda Masahiro 岡山理科大学, 経営学部, 教授 (90279042)
• Co-Investigator(Kenkyū-buntansha): 足立 浩平 大阪大学, 人間科学研究科, 教授 (60299055) ; 飯塚 誠也 岡山大学, 全学教育・学生支援機構, 教授 (60322236) ; 森 裕一 岡山理科大学, 経営学部, 教授 (80230085)
• Project Period (FY): 2016-04-01 – 2020-03-31
• Project Status: Completed (Fiscal Year 2019)
• Budget Amount: ¥4,420,000 (Direct Cost: ¥3,400,000、Indirect Cost: ¥1,020,000); Fiscal Year 2019: ¥910,000 (Direct Cost: ¥700,000、Indirect Cost: ¥210,000); Fiscal Year 2018: ¥1,040,000 (Direct Cost: ¥800,000、Indirect Cost: ¥240,000); Fiscal Year 2017: ¥910,000 (Direct Cost: ¥700,000、Indirect Cost: ¥210,000); Fiscal Year 2016: ¥1,560,000 (Direct Cost: ¥1,200,000、Indirect Cost: ¥360,000)
• Keywords: EMアルゴリズム / 加速 / 初期値選択 / 混合モデル / 計算時間の短縮 / ブートストラップ法 / 計算効率 / 加速化 / 非計量データ / 交互最小二乗法 / 収束の加速 / 加速化法

Outline of Final Research Achievements

The EM algorithm is a general and popular algorithm for finding maximum likelihood estimates from incomplete data due to stability in convergence, simplicity in implementation and applicability in practice, while the algorithm only guarantees local and linear convergence. They are the drawbacks when the EM algorithm is applied to finite mixture models. We tried to develop an initial value selection method to select a suitable initial value such that the EM algorithm starting from the selected value can find an estimate maximizing globally the likelihood function. In order to reduce the total computation time and the number of iterations, we developed an algorithm that accelerates the convergence of the EM algorithm. Moreover, we showed an algorithm to improve the speed of computation of the bootstrap method using the EM algorithm.

Academic Significance and Societal Importance of the Research Achievements

画像解析と機械学習などにおいて用いられる大規模データに対して統計モデルを考えるとき，混合モデルを仮定することになる．このモデルのもとでのデータ解析のための数値計算では，短い計算時間で最適な推定値を得ることが求められる．本研究では，統計計算法に用いられるEMアルゴリズムに焦点を当て，計算時間の短縮を図る加速法の開発と，最良な初期値を見つける初期値選択法の開発により，この問題の解決を図る．この研究の応用場面としては，医療診断における画像解析や機械学習等の分野のデータ解析であり，実用価値が高い研究である．

Research Products

• [Journal Article] Initial value selection for the alternating least squares algorithm (2020)
  • Author(s): Kuroda, M., Mori, Y. and Iizuka, M.
  • Journal Title: Advanced Studies in Classification and Data Science（採択済） Volume: －
• [Journal Article] vector ε アルゴリズムによる EM アルゴリズムの収束の加速化とその改良 (2019)
  • Author(s): 黒田正博
  • Journal Title: 計算機統計学（採択済み） Volume: 30
  • NAID: 130007903550
  • Peer Reviewed
• [Journal Article] Fixed factor analysis with clustered factor score constraint (2017)
  • Author(s): Uno, K., Satomura, H. and Adachi, K.
  • Journal Title: Computational Statistics and Data Analysis Volume: 94 Pages: 265-274
  • Peer Reviewed
• [Presentation] Reduced K-Means with nonlinear principal component analysis. (2019)
  • Author(s): Yoshioka, T., Kuroda, M., Mori, Y.
  • Organizer: The 62nd International Statistical Institute 2019 World Statistics Congress (WSC2019)
  • Int'l Joint Research
• [Presentation] Some Applications in Multivariate Methods with Alternating Least Squares. (2019)
  • Author(s): Mori, Y., Katayama, H., Yoshioka, T., Kuroda, M., Iizuka, M.
  • Organizer: Data Science, Statistics & Visualisation 2019 (DSSV2019)
  • Int'l Joint Research
• [Presentation] 尺度混在データに対する次元縮約とクラスタリング. (2019)
  • Author(s): 森裕一, 吉岡嵩紹, 片山浩子, 黒田正博
  • Organizer: 2019年度統計関連学会連合大会
• [Presentation] Improvement of Computation for Nonlinear Multivariate Methods (2018)
  • Author(s): Kuroda, M., Mori, Y., Iizuka, M.
  • Organizer: The 10th Conference of the IASC-ARS/68th Annual NZSA Conference
  • Int'l Joint Research
• [Presentation] Speed-up bootstrap computation for the covariance matrix of MLEs from incomplete data (2018)
  • Author(s): Kuroda, M., Mori, Y
  • Organizer: 2nd International Conference on Econometrics and Statistics EcoSta2018
  • Int'l Joint Research
• [Presentation] Computational efficiency for fuzzy clustering (2018)
  • Author(s): Yoshioka, M., Kuroda, M., Mori, Y.
  • Organizer: The 11th International Conference of the ERCIM WG on Computational and Methodological Statistics (CMStatistics 2018)
  • Int'l Joint Research
• [Presentation] Speed-up of bootstrap computation to incomplete data. (2018)
  • Author(s): Kuroda, M., Mori, Y.
  • Organizer: The IASC-ARS 25th Anniversary Conference & CASC 2nd Annual Conference.
  • Int'l Joint Research / Invited
• [Presentation] Initial value selection for the alternating least squares algorithm. (2017)
  • Author(s): Kuroda, M., Mori, Y., Iizuka, M.
  • Organizer: Conference of the International Federation of Classification Societies 2017
  • Int'l Joint Research
• [Presentation] Improvement of Computation for Nonlinear Multivariate Methods． (2017)
  • Author(s): Kuroda, M., Mori, Y., Iizuka, M.
  • Organizer: The 10th Conference of the IASC-ARS/68th Annual NZSA Conference
  • Int'l Joint Research
• [Presentation] Variable selection for mixed measurement level data in dimension reduction methods and its computation. (2017)
  • Author(s): Mori, Y., Iizuka, M., Kuorda, M.
  • Organizer: The 1st International Conference on Econometrics and Statistics (EcoSta 2017)
  • Int'l Joint Research
• [Presentation] Acceleration of convergence of the alternating least squares algorithm for mixed measurement level multivariate data (2016)
  • Author(s): Kuroda, M., Mori, Y., Iizuka, M. and Sakakihara, M.
  • Organizer: The 10th International Chinese Statistical Association (ICSA) international conference
  • Place of Presentation: 上海（中国）
  • Year and Date: 2016-12-19
  • Int'l Joint Research
• [Presentation] Acceleration of iterative methods for nonnegative matrix factorization (2016)
  • Author(s): Mori, Y., Sakakihara, M., Kuroda, M. and Iizuka, M.
  • Organizer: COMPSTAT 2016
  • Place of Presentation: オビエド（スペイン）
  • Year and Date: 2016-08-23
  • Int'l Joint Research
• [Presentation] Fast estimation using the EM algorithm for Gaussian mixture models (2016)
  • Author(s): Kuroda, M.
  • Organizer: The 4th Institute of Mathematical Statistics Asia Pacific Rim Meeting
  • Place of Presentation: 香港（中国）
  • Year and Date: 2016-06-27
  • Int'l Joint Research
• [Presentation] リスタートを用いた加速化交互最小二乗法による非計量主成分分析の変数選択法につい て (2016)
  • Author(s): 黒田正博, 森 裕一, 飯塚誠也
  • Organizer: 日本計算機統計学会第30回大会
  • Place of Presentation: 京都
  • Year and Date: 2016-05-19
• [Book] 最小二乗法・交互最小二乗法（統計学One Point 3） (2017)
  • Author(s): 森 裕一・黒田 正博・足立 浩平
  • Total Pages: 120
  • Publisher: 共立出版
  • ISBN: 9784320112544
• [Book] Nonlinear Principal Component Analysis and Its Applications (2016)
  • Author(s): Mori. Y., Kuroda, M., Makino, N.
  • Total Pages: 80
  • Publisher: Springer
• [Book] Matrix-Based Introduction to Multivariate Data Analysis (2016)
  • Author(s): Adachi，K.
  • Total Pages: 301
  • Publisher: Springer
• [Remarks] Researchmap
  • URL: https://researchmap.jp/read0096675