Acceleration of the alternating least squares algorithm for multivariate analysis of non-metric data

• Project/Area Number: 24500353
• Research Category: Grant-in-Aid for Scientific Research (C)
• Allocation Type: Multi-year Fund
• Section: 一般
• Research Field: Statistical science
• Research Institution: Okayama University of Science
• Principal Investigator: Kuroda Masahiro 岡山理科大学, 総合情報学部, 教授 (90279042)
• Co-Investigator(Kenkyū-buntansha): MORI YUICHI 岡山理科大学, 総合情報学部, 教授 (80230085) ; SAKAKIHARA MICHIO 岡山理科大学, 総合情報学部, 教授 (70215614) ; NAKAGAWA SHIGEKAZU 倉敷芸術科学大学, 産業科学技術学部, 教授 (90248203) ; ADACHI KOHEI 大阪大学, 人間科学研究科, 教授 (60299055) ; IIZUKA MASAYA 岡山大学, アドミッションセンター, 教授 (60322236)
• Project Period (FY): 2012-04-01 – 2016-03-31
• Project Status: Completed (Fiscal Year 2015)
• Budget Amount: ¥5,200,000 (Direct Cost: ¥4,000,000、Indirect Cost: ¥1,200,000); Fiscal Year 2015: ¥1,170,000 (Direct Cost: ¥900,000、Indirect Cost: ¥270,000); Fiscal Year 2014: ¥1,170,000 (Direct Cost: ¥900,000、Indirect Cost: ¥270,000); Fiscal Year 2013: ¥1,170,000 (Direct Cost: ¥900,000、Indirect Cost: ¥270,000); Fiscal Year 2012: ¥1,690,000 (Direct Cost: ¥1,300,000、Indirect Cost: ¥390,000)
• Keywords: 交互最小二乗法 / 収束の加速 / ベクターイプシロン法 / 非計量主成分分析 / 非計量因子分析 / 非計量データ / 最適尺度化 / 多変量解析 / 主成分分析 / 加速法 / vector epsilon法

Outline of Final Research Achievements

We developed acceleration methods of the alternating least squares (ALS) algorithms for nonlinear principal component analysis (NL-PCA) and factor analyses (NL-FA) of non-metric data. The vector epsilon accelerator is utilized for speeding up the convergence of the ALS algorithm. Numerical experiments demonstrated that the accelerated ALS algorithm provides less expensive computational cost than ordinary ALS algorithm in NL-PCA and NL-FA and greatly reduces the number of iterations and computational time of the ALS algorithm. These speeds of the acceleration algorithm are about 3 to 4 times faster than those of ordinary ALS algorithm.

Research Products

• [Journal Article] Improving the vector epsilon acceleration for the EM algorithm using a re-starting procedure (2015)
  • Author(s): Kuroda, M, Geng, Z. and Sakakihara, M.
  • Journal Title: Computational Statistics Volume: 30 Issue: 4 Pages: 1051-1077
  • DOI: 10.1007/s00180-015-0565-y
  • Peer Reviewed / Int'l Joint Research / Acknowledgement Compliant
• [Journal Article] Performance of acceleration of ALS algorithm in nonlinear PCA (2014)
  • Author(s): Mori, M., Kuroda, M., Izuka, M. and Sakakihara, Y.
  • Journal Title: Proceedings of COMPSTAT 2014 Volume: なし Pages: 257-263
  • Peer Reviewed / Acknowledgement Compliant
• [Journal Article] Alternating least squares in nonlinear principal components (2013)
  • Author(s): Kuroda, M., Mori, Y., Iizuka, M. and Sakakihara, M
  • Journal Title: Computational Statistics Volume: Vol.5 No.6 Issue: 6 Pages: 456-464
  • DOI: 10.1002/wics.1279
  • Peer Reviewed
• [Journal Article] Accelerating and re-starting the alternating least squares algorithm for non-linear principal components analysis (2013)
  • Author(s): 1.Kuroda, M., Mori, Y., Izuka, M. and Sakakihara, M.
  • Journal Title: Proceedings of the 59th World Statistics Congress (ISI2013) Volume: 1 Pages: 5426-5431
  • Peer Reviewed
• [Journal Article] MCMC using Markov bases for computing $p$-values in decomposable log-linear models (2013)
  • Author(s): Kuroda, M., Hashiguchi, H., Nakagawa, S., Geng, Z.
  • Journal Title: Computational Statistics Volume: (掲載確定) Issue: 2 Pages: 831-850
  • DOI: 10.1007/s00180-012-0331-3
  • Peer Reviewed
• [Journal Article] Two-stage acceleration for non-linear PCA (2012)
  • Author(s): Kuroda, M., Sakakihara, M., Mori, Y. and Iizuka, M.
  • Journal Title: Proceedings of COMPSTAT2012 Volume: なし Pages: 461-471
  • Peer Reviewed
• [Journal Article] Acceleration of convergence of the alternating least squares algorithm for nonlinear principal components analysis (2012)
  • Author(s): Kuroda, M., Mori, Y., Iizuka, M., Sakakihara, M.
  • Journal Title: Principal Component Analysis(Sanguansat, P.(Ed.))(InTech Publications) Volume: なし Pages: 129-144
  • DOI: 10.5772/37539
  • ISBN: 9789535101956
  • Peer Reviewed
• [Presentation] Initial value selection of the EM algorithm for Gaussian mixture models (2015)
  • Author(s): Kruoda，M.
  • Organizer: IASC-ARS 2015
  • Place of Presentation: Singapore (Singapore)
  • Year and Date: 2015-12-17
  • Int'l Joint Research
• [Presentation] EMアルゴリズムの加速化と応用 (2015)
  • Author(s): 黒田正博.
  • Organizer: 日本行動計量学会第43回大会
  • Place of Presentation: 首都大学東京 (東京都)
  • Year and Date: 2015-09-01
• [Presentation] 統計計算で用いる反復法の加速 (2014)
  • Author(s): 黒田正博
  • Organizer: 環瀬戸内応用数理研究部会第18回シンポジウム
  • Place of Presentation: 加計学園国際学術交流センター（倉敷）
  • Year and Date: 2014-12-05 – 2014-12-07
• [Presentation] An initial value selection method of the EM algorithm for mixture models (2014)
  • Author(s): Kuroda, M.
  • Organizer: Kyoto International Conference on Modern Statistics in the 21st Century
  • Place of Presentation: Kyoto International Conference Center (Kyoto)
  • Year and Date: 2014-11-17 – 2014-11-18
• [Presentation] 非計量主成分分析における交互最小二乗法とその加速化 (2014)
  • Author(s): 黒田正博，森裕一， 飯塚誠也，榊原道夫
  • Organizer: 2014年度統計関連学会連合大会
  • Place of Presentation: 東京大学（東京）
  • Year and Date: 2014-09-13 – 2014-09-16
• [Presentation] 最小交互二乗法の加速化 (2014)
  • Author(s): 森 裕一，黒田正博，飯塚誠也，榊原道夫
  • Organizer: 日本行動計量学会第42回大会
  • Place of Presentation: 東北大学（仙台）
  • Year and Date: 2014-09-02 – 2014-09-05
• [Presentation] Performance of acceleration of ALS algorithm in nonlinear PCA (2014)
  • Author(s): Mori, M., Kuroda, M., Izuka, M. and Sakakihara, Y.
  • Organizer: COMPSTAT 2014
  • Place of Presentation: International Conference Centre Geneva (Geneva)
  • Year and Date: 2014-08-19 – 2014-08-22
• [Presentation] 非主成分分析の加速化：変数選択への適用 (2013)
  • Author(s): 2.黒田正博，森裕一，飯塚誠也，榊原道夫
  • Organizer: 日本行動計量学会第41回大会
  • Place of Presentation: 東邦大学（千葉）
• [Presentation] re-startを組み込んだ交互最小二乗法の加速法の非計量主成分分析への適用 (2013)
  • Author(s): 2.黒田正博，森裕一，飯塚誠也，榊原道夫
  • Organizer: 日本計算機統計学会第27回大会
  • Place of Presentation: 弘前大学（青森）
• [Presentation] Accelerating and re-starting the alternating least squares algorithm for non-linear principal components analysis (2013)
  • Author(s): 1.Kuroda, M., Mori, Y., Izuka, M. and Sakakihara, M.
  • Organizer: The 59th World Statistics Congress (ISI2013)
  • Place of Presentation: The Hong Kong Convention and Exhibition Centre （Hong Kong）
• [Presentation] Acceleration of the EM algorithm using the vector epsilon accelerator and a re-starting procedure (2013)
  • Author(s): Kuroda, M., Geng, Z and Sakakihara, M.
  • Organizer: The 59th World Statistics Congress (ISI2013)
  • Place of Presentation: The Hong Kong Convention and Exhibition Centre（Hong Kong）
• [Presentation] Acceleration of the EM algorithm to mixture models (2013)
  • Author(s): Kuroda, M. and Geng, Z
  • Organizer: Ishigaki International Conference on Modern Statistics Theories, Practices, and Education in the 21st Century
  • Place of Presentation: ANA InterContinental Ishigaki resort hotel（沖縄）
• [Presentation] 非計量主成分析の加速化－ 実データへ適用 (2012)
  • Author(s): 黒田正博，榊原道夫，飯塚誠也，森 裕一
  • Organizer: 日本行動計量学会第40回大会
  • Place of Presentation: 新潟県立大学（新潟県）
• [Presentation] Graves-Morrisによる交互最小二乗法の加速化 (2012)
  • Author(s): 森 裕一，飯塚誠也，黒田正博，榊原道夫
  • Organizer: 日本計算機統計学会第26回大会
  • Place of Presentation: 香川県社会福祉総合センター（香川県）
• [Presentation] Two-stage acceleration for non-linear PCA (2012)
  • Author(s): Kuroda, M., Sakakihara, M., Mori, Y. and Iizuka, M.
  • Organizer: COMPSTAT2012
  • Place of Presentation: Amathus Beach Hotel (Cyprus)
• [Remarks] Masahiro Kuroda Home Page: Publication
  • URL: http://www.soci.ous.ac.jp/~kuroda/publication.html
• [Remarks] 黒田正博の研究成果
  • URL: https://www.soci.ous.ac.jp/~kuroda/publication.html