The fast solution of least squares problems and its applications.

• Project/Area Number: 15K04768
• Research Category: Grant-in-Aid for Scientific Research (C)
• Allocation Type: Multi-year Fund
• Section: 一般
• Research Field: Computational science
• Research Institution: National Institute of Informatics
• Principal Investigator: Hayami Ken 国立情報学研究所, 情報学プリンシプル研究系, 教授 (20251358)
• Project Period (FY): 2015-04-01 – 2018-03-31
• Project Status: Completed (Fiscal Year 2017)
• Budget Amount: ¥5,070,000 (Direct Cost: ¥3,900,000、Indirect Cost: ¥1,170,000); Fiscal Year 2017: ¥1,690,000 (Direct Cost: ¥1,300,000、Indirect Cost: ¥390,000); Fiscal Year 2016: ¥1,690,000 (Direct Cost: ¥1,300,000、Indirect Cost: ¥390,000); Fiscal Year 2015: ¥1,690,000 (Direct Cost: ¥1,300,000、Indirect Cost: ¥390,000)
• Keywords: 最小二乗問題 / クリロフ部分空間法 / 非負制約付き最小二乗問題 / 画像復元 / 非負値行列因子分解 / 線形計画問題 / 共役勾配法 / 主双対内点法 / 非線形連立方程式 / 反復解法 / active-set法 / 劣決定最小二乗問題 / GMRES法 / 反復法 / 非負制約 / 箱型制約 / 半正定値系

Outline of Final Research Achievements

We developed a method for nonnegative least squares problems by using a modulus transformation which reduces the problem to solving a sequence of unconstrained least squares problems, proved its convergence and showed its superiority. We also applied the method to image restoration problems and showed its effectiveness. Further, we applied it to nonnegative matrix factorization (NMF), which is useful in signal processing etc., and showed its superiority.
We showed that the right preconditioned MINRES method converges without breakdown for least squares problems whose coefficient matrix is symmetric positive semidefinite, and proposed using the Eisenstat-SSOR method for the right preconditioning, and showed its superiority. We applied our inner iteration preconditioned Krylov subspace method to least squares problems arising in each iteration of the primal-dual interior point method for linear programming problems, and showed its effectiveness.

Research Products

• [Int'l Joint Research] Universitat Politecnica de Valencia(スペイン)
• [Int'l Joint Research] Kent State University(米国)
• [Int'l Joint Research] Kent State University(米国)
• [Int'l Joint Research] Universitat Politecnica de Valencia(スペイン)
• [Int'l Joint Research] Tongji University(中国)
• [Int'l Joint Research] 同済大学(中国)
• [Int'l Joint Research] Universitat Politecnica de Valencia(スペイン)
• [Int'l Joint Research] Kent State University(米国)
• [Journal Article] Updating preconditioners for modified least squares problems (2017)
  • Author(s): Marin J.、Mas J.、Guerrero D.、Hayami K.
  • Journal Title: Numerical Algorithms Volume: 75 Issue: 2 Pages: 491-508
  • DOI: 10.1007/s11075-017-0315-z
  • Peer Reviewed / Int'l Joint Research
• [Journal Article] A modulus-based iterative method for constrained Tikhonov regularization (2017)
  • Author(s): Bai, Z.-Z., Buccini, A., Hayami, K., Reichel, L., Yin, J.-F., and Zheng, N.
  • Journal Title: Journal of Computational and Applied Mathematics Volume: 319 Pages: 1-13
  • DOI: 10.1016/j.cam.2016.12.023
  • Peer Reviewed / Int'l Joint Research
• [Journal Article] Modulus-type inner outer iteration methods for nonnegative constrained least squares problems (2016)
  • Author(s): Zheng, N., Hayami, K., and Yin, J.-F.
  • Journal Title: SIAM Journal on Matrix Analysis and Applications Volume: 37 Issue: 3 Pages: 1250-1278
  • DOI: 10.1137/141002220
  • Peer Reviewed / Int'l Joint Research / Acknowledgement Compliant
• [Journal Article] Jacobian-free implicit inner-iterative preconditioner for nonlinear least squares problems (2016)
  • Author(s): Xu, W., Zheng, N., and Hayami, K.,
  • Journal Title: Journal of Scientific Computing Volume: - Issue: 3 Pages: 1-27
  • DOI: 10.1007/s10915-016-0167-z
  • Peer Reviewed / Int'l Joint Research / Acknowledgement Compliant
• [Journal Article] 半正定値系に対するEisenstat SSOR による右前処理MINRES法 (2016)
  • Author(s): 杉原光太, 速水 謙, Ning Zheng
  • Journal Title: 日本応用数理学会論文誌 Volume: - Pages: 1-43
  • NAID: 130005169903
  • Peer Reviewed / Open Access
• [Journal Article] Convergence of inner-iteration GMRES methods for least squares problems (2015)
  • Author(s): Morikuni, K. and Hayami, K.,
  • Journal Title: SIAM Journal on Matrix Analysis and Applications Volume: 36(1) Issue: 1 Pages: 225-250
  • DOI: 10.1137/130946009
  • Peer Reviewed / Acknowledgement Compliant
• [Journal Article] Improvements to the Cluster Newton method for underdetermined inverse problems (2015)
  • Author(s): Gaudreau, P., Hayami, K., Aoki, Y., Safoui, H., and Konagaya, A.
  • Journal Title: Journal of Computational and Applied Mathematics Volume: 283 Pages: 122-141
  • DOI: 10.1016/j.cam.2015.01.014
  • Peer Reviewed / Acknowledgement Compliant
• [Presentation] Fast Solution of Nonnegative Matrix Factorization Via a Matrix-Based Active Set Method, (2018)
  • Author(s): Zheng, N., Hayami, K., and Ono, N.,
  • Organizer: MS95 Matrix Computations with Applications - Part I of II, 18th SIAM Conference on Parallel Processing for Scientific Computing, Waseda University, Tokyo, March 7-10, 2018.
  • Int'l Joint Research
• [Presentation] Fast solution of nonnegative matrix factorization via a matrix-based active set method, (2018)
  • Author(s): Zheng, N., Hayami, K., and Ono, N.,
  • Organizer: SIAM Conference on Applied Linear Algebra (SIAM-ALA18), 4-8 May 2018, Hong Kong Baptist University. http://www.math.hkbu.edu.hk/siam-ala18/
  • Int'l Joint Research
• [Presentation] Convergence of the Right Preconditioned Range Restricted MINRES for Singular Systems, (2018)
  • Author(s): Sugihara, K., Hayami, K., and Zheng, N.,
  • Organizer: SIAM Conference on Applied Linear Algebra (SIAM-ALA18), 4-8 May 2018, Hong Kong Baptist University. http://www.math.hkbu.edu.hk/siam-ala18/
  • Int'l Joint Research
• [Presentation] Stabilizing GMRES Using the Normal Equation Approach for Severely Ill-Conditioned Problems, (2018)
  • Author(s): Liao, Z., Hayami, K., and Morikuni, K.,
  • Organizer: SIAM Conference on Applied Linear Algebra (SIAM-ALA18), 4-8 May 2018, Hong Kong Baptist University. http://www.math.hkbu.edu.hk/siam-ala18/
• [Presentation] A New Matrix-Based Active Set Method for Nonnegative Matrix Factorization (2017)
  • Author(s): Zheng, N., Hayami, K., and Ono, N.,
  • Organizer: Minisymposium M36-2: Numerical methods for large-scale inverse problems (Part II), Applied Inverese Problems 2017, Hangzhou, China, May 29-June 2, 2017
  • Int'l Joint Research
• [Presentation] Numerical computation of tori on delay equations (2017)
  • Author(s): Gimeno, J., Hayami, K., and Jorba, A.
  • Organizer: 第46回数値解析シンポジウム, 講演予稿集, pp. 45-50, 2017, グリーンパーク想い出の森, 滋賀県高島市, 2017年6月28日－30日.
• [Presentation] 対称特異系に対する右前処理MINRES法とMR-2法の収束性 (2017)
  • Author(s): 杉原光太, 速水謙, Ning Zheng
  • Organizer: 日本応用数理学会2017年度 年会, [研究部会OS] 行列・固有値問題の解法とその応用(1), 2017年9月6-8日, 武蔵野大学 有明キャンパス, 予稿集 pp.265-266.
• [Presentation] Stabilized GMRES method using the normal equation approach for highly ill-conditioned problems, (2017)
  • Author(s): Liao, Z. and Hayami, K.,
  • Organizer: 日本応用数理学会「行列・固有値問題の解法とその応用」研究部会 第24回研究会 東京大学工学部, 2017年11月24日
• [Presentation] An Alternating Modulus Nonnegative Least Squares Method for Nonnegative Matrix Factorization (2016)
  • Author(s): Zheng, N., Hayami, K.*, and Ono, N.
  • Organizer: Conference: Numerical Linear Algebra and Applications (NL2A), Centre International de Rencontres Mathematiques (CIRM),
  • Place of Presentation: Luminy, France
  • Year and Date: 2016-10-24
  • Int'l Joint Research / Invited
• [Presentation] An Alternating Modulus Nonnegative Least Squares Method for Nonnegative Matrix Factorization (2016)
  • Author(s): Zheng, N. and Hayami, K.
  • Organizer: 日本応用数理学会2016年度年会 [研究部会OS] 行列・固有値問題の解法とその応用(1)
  • Place of Presentation: 北九州国際会議場(福岡県・北九州市)
  • Year and Date: 2016-09-12
• [Presentation] Implementation of Interior-point Methods for LP Based on Krylov Subspace Iterative Solvers with Inner-Iteration Preconditioning (2016)
  • Author(s): Cui, Y., Morikuni, K., Tsuchiya, T., and Hayami, K.
  • Organizer: 5th IMA Conference on Numerical Linear Algebra and Optimization
  • Place of Presentation: Birmingham, U.K.
  • Year and Date: 2016-09-07
  • Int'l Joint Research
• [Presentation] Right Preconditioned MINRES for Singular Systems (2016)
  • Author(s): Sugihara, K., Zheng, N., and Hayami, K.
  • Organizer: 5th IMA Conference on Numerical Linear Algebra and Optimization
  • Place of Presentation: Birmingham, U.K.
  • Year and Date: 2016-09-07
  • Int'l Joint Research
• [Presentation] Modulus-Based Iteration Methods for Nonnegative and Box Constrained Least Square Problems (2016)
  • Author(s): Zheng, N., Hayami, K.＊, and Yin, J.-F.
  • Organizer: The Sixth China-Japan-Korea Joint Conference on Numerical Mathematics
  • Place of Presentation: Daejeon, Korea
  • Year and Date: 2016-08-22
  • Int'l Joint Research / Invited
• [Presentation] Implementation of Interior-point Methods for LP using Krylov Subspace Methods Preconditioned by Inner Iterations (2016)
  • Author(s): Morikuni, K., Cui, Y., Tsuchiya, T., and Hayami, K.
  • Organizer: Workshop on Advances in Optimization
  • Place of Presentation: TKP品川カンファレンスセンター(東京都・港区）
  • Year and Date: 2016-08-12
  • Int'l Joint Research
• [Presentation] Implementation of Interior-Point Methods for LP using Krylov Subspace Methods Preconditioned by Inner Iterations (2016)
  • Author(s): Morikuni, K., Cui, Y., Tsuchiya, T., and Hayami, K.
  • Organizer: The Fifth International Conference on Continuous Optimization of the Mathematical Optimization Society (ICCOPT)
  • Place of Presentation: 政策研究大学院大学(東京都・港区）
  • Year and Date: 2016-08-06
  • Int'l Joint Research
• [Presentation] Updating preconditioners for least squares problems (2016)
  • Author(s): Marin, J., Mas, J.＊, and Hayami, K.
  • Organizer: Mini Symposium: Recent Advances in the Solution of Least Squares Problems, 20th Conference of the International Linear Algebra Society (ILAS)
  • Place of Presentation: Leuven, Belgium
  • Year and Date: 2016-07-11
  • Int'l Joint Research
• [Presentation] Modulus Iterative Methods for Least Squares Problems with Box Constraints (2016)
  • Author(s): Zheng, N., Hayami, K., and Yin, J.-F.
  • Organizer: Mini Symposium: Recent Advances in the Solution of Least Squares Problems, 20th Conference of the International Linear Algebra Society (ILAS)
  • Place of Presentation: Leuven, Belgium
  • Year and Date: 2016-07-11
  • Int'l Joint Research
• [Presentation] Updating preconditioners for least squares problems (2016)
  • Author(s): Marin, J., Mas, J., and Hayami, K.
  • Organizer: Álgebra Lineal, Análisis Matricial y Aplicaciones (ALAMA2016)
  • Place of Presentation: Leon, Spain
  • Year and Date: 2016-06-20
  • Int'l Joint Research
• [Presentation] Modulus Iterative Methods for Nonnegative Constrained Least Squares Problems Arising from Image Restoration (2016)
  • Author(s): Zheng, N., Hayami, K., Yin, J.
  • Organizer: SIAM Conference on Image Science
  • Place of Presentation: Albuquerque, USA
  • Year and Date: 2016-05-23
  • Int'l Joint Research
• [Presentation] Implementation of interior-point method for LP based on Krylov subspace iterative solver with inner-iteration preconditioning (2016)
  • Author(s): Cui, Y., Morikuni, K., Tsuchiya, T., and Hayami, K.,
  • Organizer: 研究集会「最適化：モデリングとアルゴリズム」
  • Place of Presentation: 政策研究大学院大学
  • Year and Date: 2016-03-22
• [Presentation] Modulus-Type Inner Outer Iteration Methods for Nonnegative Constrained Least Squares Problems (2016)
  • Author(s): Yin, J., Zheng, N., Hayami, K.,
  • Organizer: Structured Matrix Computations with Applications, Tsinghua Sanya International Mathematics Forum
  • Place of Presentation: Tsinghua Sanya International Mathematics Forum, Sanya, China,
  • Year and Date: 2016-03-14
  • Int'l Joint Research / Invited
• [Presentation] Inner-iteration Preconditioning of Normal Equations in an Interior-point Method for LP (2016)
  • Author(s): Cui, Y., Morikuni, K., Tsuchiya, T., and Hayami, K.
  • Organizer: 行列・固有値問題の解法とその応用セッション, 日本応用数理学会 第12回研究部会連合発表会
  • Place of Presentation: 神戸学院大学 ポートアイランドキャンパス
  • Year and Date: 2016-03-04
• [Presentation] 半正定値系に対するEisenstat SSORによる右前処理MINRES法と，特異系への右前処理MINRES法の収束性 (2016)
  • Author(s): 杉原光太, 速水 謙, Zheng Ning,
  • Organizer: 行列・固有値問題の解法とその応用セッション, 日本応用数理学会 第12回研究部会連合発表会
  • Place of Presentation: 神戸学院大学 ポートアイランドキャンパス
  • Year and Date: 2016-03-04
• [Presentation] Hybrid Two-Stage Modulus Iteration Method for Nonnegative Constrained Least Squares Problems (2016)
  • Author(s): Zheng, N., Hayami, K., Yin, J.,
  • Organizer: 行列・固有値問題の解法とその応用セッション, 日本応用数理学会 第12回研究部会連合発表会
  • Place of Presentation: 神戸学院大学 ポートアイランドキャンパス
  • Year and Date: 2016-03-04
• [Presentation] Preconditioning iterative methods for solving least squares problems (2016)
  • Author(s): Mas, J., Marin, J., and Hayami, K.
  • Organizer: Workshop on Numerical Algebra, Algorithms, and Analysis
  • Place of Presentation: 国立情報学研究所
  • Year and Date: 2016-01-11
  • Int'l Joint Research
• [Presentation] Enchanted by singularity (2016)
  • Author(s): Hayami, K.
  • Organizer: Workshop on Numerical Algebra, Algorithms, and Analysis
  • Place of Presentation: 国立情報学研究所
  • Year and Date: 2016-01-11
  • Int'l Joint Research
• [Presentation] Right preconditioned MINRES using Eisenstat SSOR for positive semidefinite systems (2016)
  • Author(s): Sugihara, K., Zheng, N., and Hayami, K.,
  • Organizer: Workshop on Numerical Algebra, Algorithms, and Analysis
  • Place of Presentation: 国立情報学研究所
  • Year and Date: 2016-01-11
  • Int'l Joint Research
• [Presentation] Nonnegative Matrix Factorization via A New Alternating Nonnegative Least Squares Method and Anderson Acceleration (2016)
  • Author(s): Zheng, N., Hayami, K., and Ono, N.
  • Organizer: 日本応用数理学会「行列・固有値問題の解法とその応用」研究部会 第22回研究会
  • Place of Presentation: 東京大学工学部(東京都・文京区）
• [Presentation] A Stabilized GMRES Method for Solving Inconsistent Underdetermined Least Squares Problems (2016)
  • Author(s): Liao, Z., Hayami, K., and Yin, J.-F.
  • Organizer: 日本応用数理学会「行列・固有値問題の解法とその応用」研究部会 第22回研究会
  • Place of Presentation: 東京大学工学部(東京都・文京区）
• [Presentation] Modulus-Type Inner Outer Iteration Methods for Nonnegative and Box Constrained Least Squares Problems and their Application to Image Restoration (2015)
  • Author(s): Zheng, N., Hayami, K.*, and Yin, J.-F.
  • Organizer: The 1st Asia-Pacific Workshop on Image Processing and Applications
  • Place of Presentation: Macquarie University , Sydney, Australia,
  • Year and Date: 2015-12-14
  • Int'l Joint Research
• [Presentation] Modulus Methods for Box Constrained Least Squares Problems (2015)
  • Author(s): Zheng, N., Hayami, K., and Yin, J.-F.,
  • Organizer: 京都大学数理解析研究所 (RIMS) 研究集会: 現象解明に向けた数値解析学の新展開
  • Place of Presentation: 京都大学数理解析研究所
  • Year and Date: 2015-11-18
  • Invited
• [Presentation] 半正定値系に対する Eisenstat SSOR による右前処理 MINRES 法 (2015)
  • Author(s): 杉原光太, 速水謙, Ning Zheng
  • Organizer: 京都大学数理解析研究所 (RIMS) 研究集会: 現象解明に向けた数値解析学の新展開
  • Place of Presentation: 京都大学数理解析研究所
  • Year and Date: 2015-11-18
• [Presentation] Modulus Iterative Methods for Least Squares Problems with Box Constraints (2015)
  • Author(s): Zheng, N., Hayami, K.*, and Yin, J.-F.,
  • Organizer: Minisymposium MS43: The Least Squares Challenge: Modern Methods and Applications, 2015 SIAM Conference on Applied Linear Algebra,
  • Place of Presentation: Atlanta, Georgia
  • Year and Date: 2015-10-26
  • Int'l Joint Research
• [Presentation] Right Preconditioned MINRES using Eisenstat-SSOR for Positive Semidefinite Systems (2015)
  • Author(s): Sugihara, K., Hayami, K., and Zheng, N.,
  • Organizer: 2015 SIAM Conference on Applied Linear Algebra
  • Place of Presentation: Atlanta, Georgia
  • Year and Date: 2015-10-26
  • Int'l Joint Research
• [Presentation] Modulus-Type Inner Outer Iterative Methods for Nonnegative Constrained Least Squares Problems (2015)
  • Author(s): Zheng, N., Hayami, K., and Yin, J.-F.,
  • Organizer: 2015 SIAM Conference on Applied Linear Algebra
  • Place of Presentation: Atlanta, Georgia
  • Year and Date: 2015-10-26
  • Int'l Joint Research
• [Presentation] Modulus Iterative Methods for Box Constrained Least Squares Problems (2015)
  • Author(s): Zheng, N., Hayami, K., and Yin, J.-F.,
  • Organizer: 日本応用数理学会2015年度年会
  • Place of Presentation: 金沢大学
  • Year and Date: 2015-09-09
• [Presentation] 半正定値系に対するEisenstat SSORによる右前処理MINRES法, (2015)
  • Author(s): 杉原 光太, 速水 謙
  • Organizer: 日本応用数理学会2015年度年会
  • Place of Presentation: 金沢大学
  • Year and Date: 2015-09-09
• [Presentation] Modulus Iterative Methods for Box Constrained Least Squares Problems, (2015)
  • Author(s): Zheng, N., Hayami, K., and Yin, J.-F.,
  • Organizer: Minisymposium on Recent Advances in the Solution of Least Squares Problems 2, The International Congress on Industrial and Applied Mathematics (ICIAM2015)
  • Place of Presentation: China National Convention Center, Beijing
  • Year and Date: 2015-08-10
  • Int'l Joint Research
• [Presentation] Application of inner-iteration preconditioning to general least squares problems (2015)
  • Author(s): Morikuni, K. and Hayami, K.,
  • Organizer: Minisymposium on Recent Advances in the Solution of Least Squares Problems 2, The International Congress on Industrial and Applied Mathematics (ICIAM2015)
  • Place of Presentation: China National Convention Center, Beijing
  • Year and Date: 2015-08-10
  • Int'l Joint Research
• [Presentation] Preconditioners for Least Squares Problems, (2015)
  • Author(s): Mas, J., Marin, J., and Hayami, K.,
  • Organizer: Minisymposium on Recent Advances in the Solution of Least Squares Problems 1, The International Congress on Industrial and Applied Mathematics (ICIAM2015)
  • Place of Presentation: China National Convention Center, Beijing
  • Year and Date: 2015-08-10
  • Int'l Joint Research
• [Presentation] Modulus Iterative Method for Least Squares Problems wit Box Constraints (2015)
  • Author(s): Zheng, N., Hayami, K., and Yin, J.-F.,
  • Organizer: Hong Kong-Tokyo Workshop on Scientific Computing
  • Place of Presentation: 国立情報学研究所
  • Year and Date: 2015-04-06
  • Int'l Joint Research / Invited
• [Book] ギルバート・ストラング著, 日本応用数理学会監訳, 今井桂子・岡本久監訳幹事, 世界標準MIT教科書 ストラング：計算理工学, 第7章 大規模連立一次方程式一次方程式の解法 (2016)
  • Author(s): 速水 謙
  • Total Pages: 735
  • Publisher: 近代科学社
• [Remarks]
  • URL: https://researchmap.jp/KenHayami/
• [Remarks]
  • URL: http://research.nii.ac.jp/~hayami/index-j.htm
• [Remarks] 速水研究室ホームページ（ｒｅｓｅａｃｈｍａｐ）
  • URL: http://researchmap.jp/KenHayami/
• [Remarks] 速水研究室ホームページ
  • URL: http://research.nii.ac.jp/~hayami/index-j.htm
• [Remarks]
  • URL: http://researchmap.jp/KenHayami/
• [Remarks]
  • URL: http://research.nii.ac.jp/~hayami/index-j.htm