• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

On a study of Markov decision processes with unknown transition matrices

Research Project

Project/Area Number 26400215
Research Category

Grant-in-Aid for Scientific Research (C)

Allocation TypeMulti-year Fund
Section一般
Research Field Foundations of mathematics/Applied mathematics
Research InstitutionKanagawa University

Principal Investigator

HORIGUCHI Masayuki  神奈川大学, 理学部, 教授 (90366401)

Co-Investigator(Kenkyū-buntansha) 中井 達  千葉大学, 教育学部, 教授 (20145808)
Co-Investigator(Renkei-kenkyūsha) YASUDA Masami  千葉大学, 理学研究科, 名誉教授 (00041244)
Research Collaborator Alexey Piunovskiy  
Francois Dufour  
Project Period (FY) 2014-04-01 – 2018-03-31
Project Status Completed (Fiscal Year 2017)
Budget Amount *help
¥3,640,000 (Direct Cost: ¥2,800,000、Indirect Cost: ¥840,000)
Fiscal Year 2017: ¥780,000 (Direct Cost: ¥600,000、Indirect Cost: ¥180,000)
Fiscal Year 2016: ¥780,000 (Direct Cost: ¥600,000、Indirect Cost: ¥180,000)
Fiscal Year 2015: ¥910,000 (Direct Cost: ¥700,000、Indirect Cost: ¥210,000)
Fiscal Year 2014: ¥1,170,000 (Direct Cost: ¥900,000、Indirect Cost: ¥270,000)
Keywordsマルコフ決定過程 / 推移法則未知 / ベイズ学習 / 推移法則が未知の場合
Outline of Final Research Achievements

In this study, we consider the optimization problem of sequential decision processes with unknown transition probabilities. In this model with uncertainty, we formulate an optimization model with interval estimated transition probabilities from the information of observing the states of system. We derived the properties of optimal policies that are based on the representation of interval valued optimization criteria. We also consider Bayesian learning problems as the partially observed optimization problems with uncertain circumstances. In these models, we also deduced the optimization methods for optimal stopping problem and quality control problem in piecewise deterministic processes.

Report

(5 results)
  • 2017 Annual Research Report   Final Research Report ( PDF )
  • 2016 Research-status Report
  • 2015 Research-status Report
  • 2014 Research-status Report
  • Research Products

    (25 results)

All 2018 2017 2016 2015 2014 Other

All Int'l Joint Research (4 results) Journal Article (11 results) (of which Int'l Joint Research: 1 results,  Open Access: 3 results,  Peer Reviewed: 2 results,  Acknowledgement Compliant: 1 results) Presentation (10 results) (of which Int'l Joint Research: 1 results)

  • [Int'l Joint Research] University of Liverpool(英国)

    • Related Report
      2016 Research-status Report
  • [Int'l Joint Research] Universite de Bordeaux(フランス)

    • Related Report
      2016 Research-status Report
  • [Int'l Joint Research] University of Liverpool(英国)

    • Related Report
      2015 Research-status Report
  • [Int'l Joint Research] Universite de Bordeaux(フランス)

    • Related Report
      2015 Research-status Report
  • [Journal Article] 区間型マルコフ決定モデルについて2017

    • Author(s)
      堀口正之
    • Journal Title

      京都大学数理解析研究所講究録

      Volume: 2044

    • Related Report
      2017 Annual Research Report
    • Open Access
  • [Journal Article] 確率的逐次割り当て問題について2017

    • Author(s)
      中井達
    • Journal Title

      京都大学数理解析研究所講究録

      Volume: 2044

    • Related Report
      2017 Annual Research Report
    • Open Access
  • [Journal Article] Optimal Impulsive Control of Piecewise Deterministic Markov Processes2016

    • Author(s)
      Dufour, F., Horiguchi M., and Piunovskiy, A. B
    • Journal Title

      Stochastics

      Volume: 88 Issue: 7 Pages: 1073-1098

    • DOI

      10.1080/17442508.2016.1197925

    • Related Report
      2016 Research-status Report
    • Peer Reviewed / Int'l Joint Research / Acknowledgement Compliant
  • [Journal Article] Interval Bayesian Method for Markov Decision Processes with Unknown Transition Matrices2016

    • Author(s)
      Horiguchi, M.
    • Journal Title

      Proceedings of the 2016 International Conference on Management and Operations Research

      Volume: - Pages: 284-287

    • Related Report
      2016 Research-status Report
    • Peer Reviewed
  • [Journal Article] マルコフ決定過程におけるベイズ管理モデルについて2016

    • Author(s)
      堀口正之
    • Journal Title

      京都大学数理解析研究所講究録

      Volume: 1990 Pages: 73-80

    • Related Report
      2016 Research-status Report
  • [Journal Article] 決定回数が未知の多段決定問題について2016

    • Author(s)
      中井達
    • Journal Title

      京都大学数理解析研究所講究録

      Volume: 1990 Pages: 222-239

    • Related Report
      2016 Research-status Report
  • [Journal Article] 多変量ベイズ管理図の適応手法(II)2015

    • Author(s)
      堀口正之
    • Journal Title

      京都大学数理解析研究所講究録1939「不確実性の下での数理モデルとその周辺」

      Volume: 1939 Pages: 152-161

    • Related Report
      2015 Research-status Report
  • [Journal Article] “Bayesian Inference in Markov Decision Processes.”2015

    • Author(s)
      M. Horiguchi
    • Journal Title

      Modern Trends in Controlled Stochastic Processes: Theory and Applications, Vol. 2 (A.B. Piunovskiy ed.), Luniver Press

      Volume: Vol. 2 Pages: 177-189

    • Related Report
      2015 Research-status Report
  • [Journal Article] マルコフ決定過程における学習プロセスと決定について2015

    • Author(s)
      中井達
    • Journal Title

      京都大学数理解析研究所講究録1939「不確実性の下での数理モデルとその周辺」

      Volume: 1939 Pages: 79-87

    • Related Report
      2015 Research-status Report
  • [Journal Article] 多変量ベイズ管理図の適応手法2014

    • Author(s)
      佐々木稔、堀口正之、蔵野正美
    • Journal Title

      京都大学数理解析研究所講究録

      Volume: 1912 Pages: 181-192

    • Related Report
      2014 Research-status Report
    • Open Access
  • [Journal Article] 確率的凸性と部分観測可能なマルコフ決定過程について2014

    • Author(s)
      中井達
    • Journal Title

      京都大学数理解析研究所講究録

      Volume: 1912 Pages: 193-201

    • Related Report
      2014 Research-status Report
  • [Presentation] Adaptive approach in a multivariate Bayesian control chart2018

    • Author(s)
      堀口正之
    • Organizer
      2018年日本数学会年会
    • Related Report
      2017 Annual Research Report
  • [Presentation] On a multivariate Bayesian control problem in Markov decision processes2017

    • Author(s)
      堀口正之
    • Organizer
      日本数学会
    • Place of Presentation
      首都大学東京
    • Year and Date
      2017-03-25
    • Related Report
      2016 Research-status Report
  • [Presentation] Bayesian control chart with unknown parameter2017

    • Author(s)
      Masayuki HORIGUCHI
    • Organizer
      21st Conference of International Federation Operational Research Societies (IFORS2017)
    • Related Report
      2017 Annual Research Report
    • Int'l Joint Research
  • [Presentation] A Prior Detection Procedure on a Sequential Sampling Problem2016

    • Author(s)
      堀口正之
    • Organizer
      日本数学会
    • Place of Presentation
      関西大学
    • Year and Date
      2016-09-17
    • Related Report
      2016 Research-status Report
  • [Presentation] Optimal Stopping problem in uncertain Markov Decision Processes2016

    • Author(s)
      M. Horiguchi and A.B. Piunovskiy
    • Organizer
      日本数学会2016年度年会
    • Place of Presentation
      筑波大学
    • Year and Date
      2016-03-18
    • Related Report
      2015 Research-status Report
  • [Presentation] 不完備情報マルコフ過程での逐次支出問題について2015

    • Author(s)
      中井達
    • Organizer
      日本オペレーションズ・リサーチ学会 2015年度春期研究発表会
    • Place of Presentation
      東京理科大学
    • Year and Date
      2015-03-27
    • Related Report
      2014 Research-status Report
  • [Presentation] 推移確率行列未知のマルコフ決定過程について2015

    • Author(s)
      堀口正之
    • Organizer
      日本オペレーションズ・リサーチ学会常設研究部会「待ち行列研究部会」(第252回)
    • Place of Presentation
      東京工業大学
    • Year and Date
      2015-02-19
    • Related Report
      2014 Research-status Report
  • [Presentation] Adaptive Markov Control Processesについて2014

    • Author(s)
      堀口正之
    • Organizer
      日本オペレーションズ・リサーチ学会研究部会確率モデルとその応用 (第3回)
    • Place of Presentation
      放送大学 千葉学習センター
    • Year and Date
      2014-09-03
    • Related Report
      2014 Research-status Report
  • [Presentation] 不完備情報マルコフ過程での決定問題と確率的凸性について2014

    • Author(s)
      中井達
    • Organizer
      日本オペレーションズ・リサーチ学会 2014年度秋期研究発表会
    • Place of Presentation
      北海道科学大学
    • Year and Date
      2014-08-28
    • Related Report
      2014 Research-status Report
  • [Presentation] A partially observable Markov decision process under stochastic convexity as an optimal maintenance problem2014

    • Author(s)
      T. Nakai
    • Organizer
      20th Conference of the International Federation of Operational Research Societies
    • Place of Presentation
      Barcelona, Spain
    • Year and Date
      2014-07-17
    • Related Report
      2014 Research-status Report

URL: 

Published: 2014-04-04   Modified: 2022-02-16  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi