Machine Learning under Changing Environments

• Project/Area Number: 20680007
• Research Category: Grant-in-Aid for Young Scientists (A)
• Allocation Type: Single-year Grants
• Research Field: Intelligent informatics
• Research Institution: Tokyo Institute of Technology
• Principal Investigator: SUGIYAMA Masashi 東京工業大学, 大学院・情報理工学研究科, 准教授 (90334515)
• Project Period (FY): 2008 – 2011
• Project Status: Completed (Fiscal Year 2011)
• Budget Amount: ¥25,480,000 (Direct Cost: ¥19,600,000、Indirect Cost: ¥5,880,000); Fiscal Year 2011: ¥4,160,000 (Direct Cost: ¥3,200,000、Indirect Cost: ¥960,000); Fiscal Year 2010: ¥6,110,000 (Direct Cost: ¥4,700,000、Indirect Cost: ¥1,410,000); Fiscal Year 2009: ¥6,110,000 (Direct Cost: ¥4,700,000、Indirect Cost: ¥1,410,000); Fiscal Year 2008: ¥9,100,000 (Direct Cost: ¥7,000,000、Indirect Cost: ¥2,100,000)
• Keywords: 学習と知識獲得 / 知能情報処理 / 機械学習 / データマイニング / 非定常環境適応 / 共変量シフト / 重点サンプリング / 重要度重み / ロボット制御 / 強化学習 / 政策反復法 / 政策探索法 / 非定常環境 / 密度比推定

Research Abstract

Existing machine learning research has developed theories and algorithms based on the assumption that data generating environments do not change over time. However, recent applications do not satisfy such a stationarity assumption. In this project, we therefore developed fundamental theories and practical algorithms for coping with changing environments. We further applied the developed algorithms to various real-world applications including robotics, image recognition, brain signal analysis, speech recognition, and natural language processing.

Research Products

• [Journal Article] Lighting condition adaptation for perceived age estimation (2011)
  • Author(s): Ueki, K., Sugiyama, M., & Ihara, Y.
  • Journal Title: Transactions on Information and Systems Volume: vol.E94-D, no.2 Pages: 392-395
  • NAID: 130000453904
  • Peer Reviewed
• [Journal Article] Reward weighted regression with sample reuse for direct policy search in reinforcement learning (2011)
  • Author(s): Hachiya, H., Peters, J., & Sugiyama M.
  • Journal Title: Neural Computation Volume: vol.23, no.11 Pages: 2798-2832
  • Peer Reviewed
• [Journal Article] Reward weighted regression with sample reuse for direct policy search in reinforcement learning (2011)
  • Author(s): Hachiya, H., Peters, J., Sugiyama, M.
  • Journal Title: Neural Computation Volume: 23 Pages: 2798-2832
  • Peer Reviewed
• [Journal Article] Dimensionality reduction for density ratio estimation in high-dimensional spaces (2010)
  • Author(s): Sugiyama, M., Kawanabe, M., & Chui, P. L.
  • Journal Title: Neural Networks Volume: vol.23, no.1 Pages: 44-59
  • NAID: 130008079586
  • Peer Reviewed
• [Journal Article] Semi-supervised speaker identification under covariate shift (2010)
  • Author(s): Yamada, M., Sugiyama, M., & Matsui, T.
  • Journal Title: Signal Processing Volume: vol.90, no.8 Pages: 2353-2361
  • NAID: 130008079628
  • Peer Reviewed
• [Journal Article] Efficient exploration through active learning for value function approximation in reinforcement learning (2010)
  • Author(s): Akiyama, T., Hachiya, H., & Sugiyama, M.
  • Journal Title: Neural Networks Volume: vol.23, no.5 Pages: 639-648
  • NAID: 130008079582
  • Peer Reviewed
• [Journal Article] Application of covariate shift adaptation techniques in brain computer interfaces (2010)
  • Author(s): Li, Y., Kambara, H., Koike, Y., & Sugiyama, M.
  • Journal Title: IEEE Transactions on Biomedical Engineering Volume: vol.57, no.6 Pages: 1318-1324
  • Peer Reviewed
• [Journal Article] Application of covariate shift adaptation techniques in brain computer interfaces. (2010)
  • Author(s): Li, Y., Kambara, H., Koike, Y., Sugiyama, M.
  • Journal Title: IEEE Transactions on Biomedical Engineering vol.57, no.6 Pages: 1318-1324
  • URL: http://sugiyama-www.cs.titech.ac.jp/~sugi/2010/BIWLDA.pdf
• [Journal Article] Efficient exploration through active learning for value function approximation in reinforcement learning. (2010)
  • Author(s): Akiyama, T., Hachiya, H., Sugiyama, M.
  • Journal Title: NeuralNetworks vol.23, no.5 Pages: 639-648
  • NAID: 130008079582
  • URL: http://sugiyama-www.cs.titech.ac.jp/~sugi/2010/API.pdf
• [Journal Article] Semi-supervised speaker identification under covariate shift. (2010)
  • Author(s): Yamada, M., Sugiyama, M., Matsui, T.
  • Journal Title: Signal Processing vol.90, no.8 Pages: 2353-2361
  • NAID: 130008079628
  • URL: http://sugiyama-www.cs.titech.ac.jp/~sugi/2010/IWKLR.pdf
• [Journal Article] Semi supervised speaker identification under covariate shift. (2010)
  • Author(s): Yamada, M., Sugiyama, M., Matsui, T.
  • Journal Title: Signal Processing Volume: 90 Pages: 2353-2361
  • NAID: 130008079628
  • Peer Reviewed
• [Journal Article] Application of covariate shift adaptation techniques in brain computer interfaces. (2010)
  • Author(s): Li, Y., Kambara, H., Koike, Y., Sugiyama, M.
  • Journal Title: IEEE Trans.on Biomedical Engineering Volume: 57 Pages: 1318-1324
  • Peer Reviewed
• [Journal Article] Efficient exploration through active learning for value function approximation in reinforcement learning. (2010)
  • Author(s): Akiyama, T., Hachiya, H., Sugiyama, M.
  • Journal Title: Neural Networks Volume: 23 Pages: 639-648
  • NAID: 130008079582
  • Peer Reviewed
• [Journal Article] Direct density ratio estimation for large-scale covariate shift adaptation (2009)
  • Author(s): Tsuboi, Y., Kashima, H., Hido, S., Bickel, S., & Sugiyama, M.
  • Journal Title: Journal of Information Processing Volume: vol.17 Pages: 138-155
  • NAID: 130000120692
• [Journal Article] Pool-based active learning in approximate linear regression (2009)
  • Author(s): ugiyama, M. & Nakajima, S
  • Journal Title: Machine Learning Volume: vol.75, no.3 Pages: 249-274
  • Peer Reviewed
• [Journal Article] A least-squares approach to direct importance estimation (2009)
  • Author(s): Kanamori, T., Hido, S., & Sugiyama, M.
  • Journal Title: Journal of Machine Learning Research Volume: vol.10 Pages: 1391-1445
  • Peer Reviewed
• [Journal Article] A density-ratio framework for statistical data processing (2009)
  • Author(s): Sugiyama, M., Kanamori, T., Suzuki, T., Hido, S., Sese, J., Takeuchi, I., & Wang, L.
  • Journal Title: IPSJ Transactions on Computer Vision and Applications Volume: vol.1 Pages: 183-208
  • NAID: 130000140299
  • Peer Reviewed
• [Journal Article] Direct importance estimation with Gaussian mixture models (2009)
  • Author(s): Yamada, M. & Sugiyama, M
  • Journal Title: IEICE Transactions on Information and Systems Volume: vol.E92-D, no.10 Pages: 2159-2162
  • NAID: 10026811983
  • Peer Reviewed
• [Journal Article] Adaptive importance sampling for value function approximation in off-policy reinforcement learning (2009)
  • Author(s): Hachiya, H., Akiyama, T., Sugiyama, M., & Peters, J.
  • Journal Title: Neural Networks Volume: vol.22, no.10 Pages: 1399-1410
  • Peer Reviewed
• [Journal Article] Adaptive importance sampling for value function approximation in off-policy reinforcement learning. (2009)
  • Author(s): Hachiya, H., Akiyama, T., Sugiyama, M., Peters, J.
  • Journal Title: Neural Networks vol.22, no.10 Pages: 1399-1410
  • URL: http://sugiyama-www.cs.titech.ac.jp/~sugi/2009/SRPI.pdf
• [Journal Article] Pool-based active learning in approximate linear regression. (2009)
  • Author(s): Sugiyama, M., Nakajima, S.
  • Journal Title: Machine Learning vol.75, no.3 Pages: 249-274
  • URL: http://sugiyama-www.cs.titech.ac.jp/~sugi/2009/PALICE.pdf
• [Journal Article] A least-squares approach to direct importance estimation. (2009)
  • Author(s): Kanamori, T., Hido, S., Sugiyama, M.
  • Journal Title: Journal of Machine Learning Research vol.10 (Jul.) Pages: 1391-1445
  • URL: http://sugiyama-www.cs.titech.ac.jp/~sugi/2009/LSIF.pdf
• [Journal Article] A least-squares approach to direct importance estimation (2009)
  • Author(s): Kanamori, T., Hido, S., Sugiyama, M.
  • Journal Title: Journal of Machine Learning Research 10 Pages: 1391-1445
  • Peer Reviewed
• [Journal Article] A density-ratio framework for statistical data processing (2009)
  • Author(s): Sugiyama, M., Kanamori, T., Suzuki, T., Hido, S., Sese, J., Takeuchi, I., Wang, L.
  • Journal Title: IPSJ Transactions on Computer Vision and Applications 1 Pages: 183-208
  • NAID: 130000140299
  • Peer Reviewed
• [Journal Article] Adaptive importance sampling for value function approximation in off-policy reinforcement learning (2009)
  • Author(s): Hachiya, H., Akiyama, T., Sugiyama, M., Peters, J
  • Journal Title: Neural Networks 22 Pages: 1399-1410
  • Peer Reviewed
• [Journal Article] Direct importance estimation for covariate shift adaptation (2008)
  • Author(s): Sugiyama, M., Suzuki, T., Nakajima, S., Kashima, H., von Bunau, P., & Kawanabe, M.
  • Journal Title: Annals of the Institute of Statistical Mathematics Volume: vol.60, no.4 Pages: 699-746
  • Peer Reviewed
• [Journal Article] A batch ensemble approach to active learning with model selection (2008)
  • Author(s): Sugiyama, M. & Rubens, N.
  • Journal Title: Neural Networks Volume: vol.21, no.9 Pages: 1278-1286
  • Peer Reviewed
• [Journal Article] Direct importance estimation for covariate shift adaptation. (2008)
  • Author(s): Sugiyama, M., Suzuki, T., Nakajima, S., Kashima, H., von Bunau, P., Kawanabe, M.
  • Journal Title: Annals of the Institute of Statistical Mathematics vol.60, no.4 Pages: 699-746
  • URL: http://sugiyama-www.cs.titech.ac.jp/~sugi/2008/KLIEP.pdf
• [Journal Article] Direct importance estimation for covariate shift adaptation. (2008)
  • Author(s): Sugiyama, M., Suzuki, T., Nakajima, S., Kashima, H., von Biinau, P. & Kawanabe, M
  • Journal Title: Annals of the Institute of Statistical Mathematics 60(4) Pages: 699-746
  • Peer Reviewed
• [Journal Article] Adaptive importance sampling for value function approximation in off-policy reinforcement learning
  • Author(s): Hachiya, H., Akiyama, T., Sugiyama, M., & Peters, J
  • Journal Title: Neural Network
  • Peer Reviewed
• [Presentation] Perceived age estimation under lighting condition change by covariate shift adaptation (2010)
  • Author(s): Ueki, K., Sugiyama, M., & Ihara, Y.
  • Organizer: In Proceedings of 20th International Conference on Pattern Recognition(ICPR2010)
  • Place of Presentation: Istanbul, Turkey
• [Presentation] Automatic audio tagging using covariate shift adaptation (2010)
  • Author(s): Wichern, G., Yamada, M., Thornburg, H., Sugiyama, M., & Spanias, A.
  • Organizer: In Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing(ICASSP2010)
  • Place of Presentation: Dallas, Texas, USA
• [Presentation] Perceived age estimation under lighting condition change by covariate shift adaptation. (2010)
  • Author(s): Ueki, K., Sugiyama, M., Ihara, Y.
  • Organizer: 20th International Conferenceon Pattern Recognition (ICPR2010)
  • URL: http://sugiyama-www.cs.titech.ac.jp/~sugi/2010/ICPR2010a.pdf
  • Place of Presentation: Istanbul, Turkey
• [Presentation] Perceived age estimation under lighting condition change by covariate shift adaptation. (2010)
  • Author(s): Ueki, K., Sugiyama, M., Ihara, Y.
  • Organizer: 20th International Conference on Pattern Recognition(ICPR2010)
  • Place of Presentation: イスタンブール,トルコ
• [Presentation] A framework of adaptive brain computer interfaces (2009)
  • Author(s): Li, Y., Koike, Y., & Sugiyama, M.
  • Organizer: In Proceedings of the 2nd International Conference on BioMedical Engineering and Informatics(BMEI09)
  • Place of Presentation: Tianjin, China
• [Presentation] Efficient sample reuse in EM-based policy search (2009)
  • Author(s): Hachiya, H., Peters, J., & Sugiyama, M.
  • Organizer: Presented at the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases(ECML-PKDD2009)
  • Place of Presentation: Berlin, Springer, Bled, Slovenia
• [Presentation] Active policy iteration : Efficient exploration through active learning for value function approximation in reinforcement learning (2009)
  • Author(s): Akiyama, T., Hachiya, H., & Sugiyama, M
  • Organizer: In Proceedings of the Twenty-First International Joint Conference on Artificial Intelligence(IJCAI2009)
  • Place of Presentation: Pasadena, California, USA
• [Presentation] Covariate shift adaptation for semi-supervised speaker identification (2009)
  • Author(s): Yamada, M., Sugiyama, M., & Matsui, T.
  • Organizer: In Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing(ICASSP2009)
  • Place of Presentation: Taipei, Taiwan
• [Presentation] Efficient sample reuse in EM-based policy search. (2009)
  • Author(s): Hachiya, H., Peters, J., Sugiyama, M.
  • Organizer: European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases(ECML-PKDD2009)
  • URL: http://sugiyama-www.cs.titech.ac.jp/~sugi/2009/ECML-PKDD2009.pdf
  • Place of Presentation: Bled, Slovenia
• [Presentation] Active policy iteration: Efficient exploration through active learning for value function approximation in reinforcement learning. (2009)
  • Author(s): Akiyama, T., Hachiya, H., Sugiyama, M.
  • Organizer: Twenty-FirstInternational Joint Conference onArtificial Intelligence (IJCAI2009)
  • URL: http://sugiyama-www.cs.titech.ac.jp/~sugi/2009/IJCAI2009.pdf
  • Place of Presentation: Pasadena, California
• [Presentation] Active policy iteration : Efficient exploration through active learning for value function approximation in reinforcement learning (2009)
  • Author(s): Akiyama, T., Hachiya, H., Sugiyama, M.
  • Organizer: Twenty-First International Joint Conference on Artificial Intelligence(IJCAI2009)
  • Place of Presentation: Pasadena, USA
• [Presentation] Efficient direct density ratio estimation for non-stationarity adaptation and outlier detection (2008)
  • Author(s): Kanamori, T., Hido, S., & Sugiyama, M.
  • Organizer: In D. Koller, D. Schuurmans, Y. Bengio, and L. Botton(Eds.), Advances in Neural Information Processing Systems 21,(Presented at Neural Information Processing Systems(NIPS2008), Vancouver, British Columbia
  • Place of Presentation: Cambridge, MA, MIT Press, Canada, Dec
  • Year and Date: 2008-08-13
• [Presentation] Pool-based agnostic experiment design in linear regression (2008)
  • Author(s): Sugiyama, M. & Nakajima, S
  • Organizer: Presented at the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases(ECML-PKDD2008)
  • Place of Presentation: Berlin, Springer, Antwerp, Belgium
• [Presentation] Adaptive importance sampling with automatic model selection in value function approximation (2008)
  • Author(s): Hachiya, H., Akiyama, T., Sugiyama, M., & Peters, J
  • Organizer: Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence(AAAI2008)
  • Place of Presentation: Chicago, Illinois, USA
• [Presentation] Active learning with model selection in linear regression (2008)
  • Author(s): Sugiyama, M. & Rubens, N
  • Organizer: Proceedings of the Eighth SIAM International Conference on Data Mining(SDM2008)
  • Place of Presentation: Atlanta, Georgia, USA
• [Presentation] Direct density ratio estimation for large-scale covariate shift adaptation (2008)
  • Author(s): Tsuboi, Y., Kashima, H., Hido, S., Bickel, S., & Sugiyama, M.
  • Organizer: Proceedings of the Eighth SIAM International Conference on Data Mining(SDM2008)
  • Place of Presentation: Atlanta, Georgia, USA
• [Presentation] Efficient direct density ratio estimation for non-stationarity adaptation and outlier detection (2008)
  • Author(s): Kanamori, T., Hido, S., Sugiyama, M.
  • Organizer: Neural Information Processing Systems (NIPS2008)
  • URL: http://sugiyama-www.cs.titech.ac.jp/~sugi/2008/NIPS2008.pdf
  • Place of Presentation: Vancouver, Canada
• [Presentation] Semi-supervised speaker identification under covariate shift (2008)
  • Author(s): Yamada, M. & Sugiyama, M
  • Organizer: The Third International Workshop on Data-Mining and Statistical Science (DMSS2008)
  • Place of Presentation: 東京
• [Presentation] Direct importance estimation---A new versatile tool for statisti cal pattern recognition (2008)
  • Author(s): Sugiyama, M., Kanamori, T, Suzuki, T., Hido, S., Sese, J., Takeuchi, I., & Wang, L
  • Organizer: Meeting on Image Recognition and Understanding 2008 (MIRU2008)
  • Place of Presentation: 長野
• [Book] Machine Learning in Non-Stationary Environments : Introduction to Covariate Shift Adaptation (2012)
  • Author(s): Sugiyama, M. & Kawanabe, M
  • Total Pages: 308
  • Publisher: MIT Press, Cambridge, MA, USA
• [Book] Machine Learning in Non-Stationary Environments : Introduction to Covariate Shift Adaptation (2012)
  • Author(s): Sugiyama, M., Kawanabe, M.
  • Total Pages: 308
  • Publisher: MIT Press
• [Book] Dataset Shift in Machine Learning (2009)
  • Author(s): Quinonero-Candela, J., Sugiyama, M., Schwaighofer, A., & Lawrence, N. D.
  • Total Pages: 248
  • Publisher: MIT Press, Cambridge, MA, USA
• [Book] MIT Press, Cambridge, MA, USA (2009)
  • Author(s): Quinonero-Candela, J., Sugiyama, M., Schwaighofer, A., Lawrence, N.D.(Eds.)
  • Publisher: Dataset Shift in Machine Learning
• [Book] Dataset Shift in Machine Learning (2009)
  • Author(s): Quinonero-Candela, J., Sugiyama, M., Schwaighofer, A., & Lawrence, N. D
  • Total Pages: 248
  • Publisher: MIT Press
• [Remarks]
  • URL: http://sugiyama-www.cs.titech.ac.jp/~sugi/
• [Remarks]
  • URL: http://sugiyama-www.cs.titech.ac.jp/~sugi/
• [Remarks] ソフトウェアの公開ページ
  • URL: http://sugiyama-www.cs.titech.ac.jp/~sugi/software/index.html
• [Remarks] ソフトウェアのページ
  • URL: http://sugiyama-www.cs.titech.acjp/~sugi/software/index.html
• [Remarks]
  • URL: http://sugiyama-www.cs.titech.ac.jp/~sugi/software/index.html
• [Remarks] ソフトウェア公開のページ
  • URL: http://sugiyama-www.cs.titech.ac.jp/~sugi/software/KLIEP/index.html