• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

通信無し強化学習エージェント群による動的環境への追従

Research Project

Project/Area Number 17J08724
Research Category

Grant-in-Aid for JSPS Fellows

Allocation TypeSingle-year Grants
Section国内
Research Field Intelligent informatics
Research InstitutionThe University of Electro-Communications

Principal Investigator

上野 史  電気通信大学, 情報理工学研究科, 特別研究員(DC1)

Project Period (FY) 2017-04-26 – 2020-03-31
Project Status Completed (Fiscal Year 2019)
Budget Amount *help
¥2,500,000 (Direct Cost: ¥2,500,000)
Fiscal Year 2019: ¥800,000 (Direct Cost: ¥800,000)
Fiscal Year 2018: ¥800,000 (Direct Cost: ¥800,000)
Fiscal Year 2017: ¥900,000 (Direct Cost: ¥900,000)
Keywordsマルチエージェントシステム / 強化学習 / 動的環境 / 通信なし / 動的変化 / 報酬
Outline of Annual Research Achievements

本年度は3年目のテーマである「実問題における適用」に向けて,1)複数種類の動的変化が複合した環境に対する通信なしマルチエージェント強化学習法の提案,および,2)環境変化が断続的に発生する際の適用方法を提案し,最後に,3)提案手法を一般化し,適用可能な実応用問題を定義した.具体的には,1)に関して,エージェント毎の学習範囲と利用する情報を制限して学習することで,環境形状及びエージェント・ゴール数の変化へ追従する手法を提案し,その有効性を示した.また,2)に関して,1)での提案手法において,学習範囲を制限するタイミング,および利用する情報の窓(どれほど昔の情報まで利用するか)をハイパーパラメータとして設定していたものを,環境に合わせて適応的に設定することで,タイミングが異なる断続的な動的変化に追従可能となるように拡張した.これらの成果により,物流システムでいえば,物資の補給地点や目的地,通行路が変わるだけではなく,物資を運ぶロボットの数や目的が増えたとき,そしてそのタイミングが不明であっても,各ロボットが通信を一切行わずに協調行動の学習が可能となり,実問題に対する適用範囲が大きく広がった.そしてこれらの成果は,国際ジャーナルSN Computer Scienceへ掲載が決定しており,英語論文誌SICE JCMSIに掲載済みである.また,国内学会FIT2019,JAWS2019,そして国際会議OptLearnMAS2020にて発表を行った.最後に,3)に関しては,1)と2)の提案手法について,学習範囲の制限により,各エージェントの学習を2体エージェントの協調で分割可能腕あり,提案手法の性能は十分発揮できることを示し,問題に関しては,提案手法がエージェント同士の衝突を加味した上でも性能を発揮することを実験で示した.これにより,実問題として想定していた倉庫ロボットへの適用が可能であることがわかった.この成果は電気学会論文誌Cに掲載済みである.

Research Progress Status

令和元年度が最終年度であるため、記入しない。

Strategy for Future Research Activity

令和元年度が最終年度であるため、記入しない。

Report

(3 results)
  • 2019 Annual Research Report
  • 2018 Annual Research Report
  • 2017 Annual Research Report
  • Research Products

    (52 results)

All 2020 2019 2018 2017

All Journal Article (11 results) (of which Int'l Joint Research: 2 results,  Peer Reviewed: 11 results,  Open Access: 10 results) Presentation (39 results) (of which Int'l Joint Research: 21 results,  Invited: 1 results) Book (1 results) Patent(Industrial Property Rights) (1 results) (of which Overseas: 1 results)

  • [Journal Article] Theoretical Learning Goal Selection for Non-Communicative Multi-Agent Cooperation2020

    • Author(s)
      Uwano Fumito、Takadama Keiki
    • Journal Title

      IEEJ Transactions on Electronics, Information and Systems

      Volume: 140 Issue: 1 Pages: 75-84

    • DOI

      10.1541/ieejeiss.140.75

    • NAID

      130007779190

    • ISSN
      0385-4221, 1348-8155
    • Year and Date
      2020-01-01
    • Related Report
      2019 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Reward Value-based Goal Selection for Agents' Cooperative Route Learning without Communication in Reward and Goal Dynamism2020

    • Author(s)
      Uwano Fumito、Takadama Keiki
    • Journal Title

      SN Computer Science

      Volume: 未定 Issue: 3

    • DOI

      10.1007/s42979-020-00191-2

    • Related Report
      2019 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Utilizing Observed Information for No-Communication Multi-Agent Reinforcement Learning toward Cooperation in Dynamic Environment2019

    • Author(s)
      UWANO Fumito、TAKADAMA Keiki
    • Journal Title

      SICE Journal of Control, Measurement, and System Integration

      Volume: 12 Issue: 5 Pages: 199-208

    • DOI

      10.9746/jcmsi.12.199

    • Related Report
      2019 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Multi-Agent Cooperation Based on Reinforcement Learning with Internal Reward in Maze Problem2018

    • Author(s)
      UWANO Fumito、TATEBE Naoki、TAJIMA Yusuke、NAKATA Masaya、KOVACS Tim、TAKADAMA Keiki
    • Journal Title

      SICE Journal of Control, Measurement, and System Integration

      Volume: 11 Issue: 4 Pages: 321-330

    • DOI

      10.9746/jcmsi.11.321

    • Related Report
      2018 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Weighted Opinion Sharing Model for Cutting Link and Changing Information among Agents as Dynamic Environment2018

    • Author(s)
      UWANO Fumito、SAITO Rei、TAKADAMA Keiki
    • Journal Title

      SICE Journal of Control, Measurement, and System Integration

      Volume: 11 Issue: 4 Pages: 331-340

    • DOI

      10.9746/jcmsi.11.331

    • Related Report
      2018 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Weighted Opinion Sharing Model for Cutting Link and Changing Information among Agents as Dynamic Environment2018

    • Author(s)
      Fumito Uwano、Rei Saito、Keiki Takadama
    • Journal Title

      SICE Journal of Control, Measurement, and System Integration

      Volume: 11

    • Related Report
      2017 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Sleep Stage Estimation Comparing own past heartrate or other's heartrate,2018

    • Author(s)
      Tajima, Y., Uwano, F., Murata, A., Harada, T., and Takadama, K.
    • Journal Title

      '',SICE Journal of Control, Measurement, and System Integration (JCMSI)

      Volume: 11/1 Issue: 1 Pages: 32-39

    • DOI

      10.9746/jcmsi.11.32

    • Related Report
      2017 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] SLIM Spacecraft Location Estimation by Crater Matching Based on Similar Triangles and Its Improvement2018

    • Author(s)
      石井 晴之、福田 盛介、澤井 秀次郎、坂井 真一郎、村田 暁紀、上野 史、辰巳 嵩豊、梅内 祐太、高玉 圭樹、原田 智広、鎌田 弘之、石田 貴行
    • Journal Title

      AEROSPACE TECHNOLOGY JAPAN, THE JAPAN SOCIETY FOR AERONAUTICAL AND SPACE SCIENCES

      Volume: 17 Issue: 0 Pages: 69-78

    • DOI

      10.2322/astj.JSASS-D-17-00011

    • NAID

      130006731199

    • ISSN
      1884-0477
    • Related Report
      2017 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Recovery System Based on Exploration-Biased Genetic Algorithm for Stuck Rover in Planetary Exploration2017

    • Author(s)
      Uwano, F., Tajima, Y., Murata, A., and Takadama, K.
    • Journal Title

      Journal of Robotics and Mechatronics

      Volume: 29 Issue: 5 Pages: 877-886

    • DOI

      10.20965/jrm.2017.p0877

    • NAID

      130007519878

    • ISSN
      0915-3942, 1883-8049
    • Year and Date
      2017-10-20
    • Related Report
      2017 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Comparison Between Reinforcement Learning Methods with Different Goal Selections in Multi-Agent Cooperation2017

    • Author(s)
      Uwano F. and Takadama, K
    • Journal Title

      Journal of Advanced Computational Intelligence and Intelligent Informatics

      Volume: 21 Issue: 5 Pages: 917-929

    • DOI

      10.20965/jaciii.2017.p0917

    • NAID

      130007520191

    • ISSN
      1343-0130, 1883-8014
    • Year and Date
      2017-09-20
    • Related Report
      2017 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Supporting the Exploration of the Learning Goals for a Continuous Learner Toward Creative Learning2017

    • Author(s)
      Okudo, T., Yamaguchi, T., Murata, A., Tatsumi, T., Uwano, F. and Takadama, K.
    • Journal Title

      Journal of Advanced Computational Intelligence and Intelligent Informatics

      Volume: 21 Issue: 5 Pages: 907-916

    • DOI

      10.20965/jaciii.2017.p0907

    • NAID

      130007520194

    • ISSN
      1343-0130, 1883-8014
    • Year and Date
      2017-09-20
    • Related Report
      2017 Annual Research Report
    • Peer Reviewed / Open Access
  • [Presentation] Directionality Reinforcement Learning to Operate Multi-Agent System without Communication2020

    • Author(s)
      Fumito Uwano
    • Organizer
      The 11th International Workshop on Optimization and Learning in Multiagent System
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research
  • [Presentation] エージェント間通信を伴わず環境状態および報酬の包括的動的変化に追従する理論的マルチエージェント強化学習2019

    • Author(s)
      上野史
    • Organizer
      合同エージェントワークショップ&シンポジウム 2019
    • Related Report
      2019 Annual Research Report
  • [Presentation] 非通信マルチエージェント強化学習における獲得報酬値の変動を用いたエージェント数の動的変化への追従2019

    • Author(s)
      上野史
    • Organizer
      第18回情報科学技術フォーラム
    • Related Report
      2019 Annual Research Report
  • [Presentation] How to Select Appropriate Craters to Estimate Location Accurately in Comprehensive Situations for SLIM Project2019

    • Author(s)
      Fumito Uwano
    • Organizer
      The 32nd International Symposium on Space Technology and Science
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research
  • [Presentation] How to Select Appropriate Craters to Estimate Location Accurately in Comprehensive Situations for SLIM Project2019

    • Author(s)
      Fumito Uwano, Takato Tatsumi, Akinori Murata, Keiki Takadama, Hiroyuki Kamata, Takayuki Ishida, Seisuke Fukuda, Shujiro Sawai, and Shinichiro Sakai
    • Organizer
      The 32nd International Symposium on Space Technology and Science and the 9th Nano-Satellite Symposium
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] How to Design Adaptable Agents to Obtain Consensus with Omoiyari2019

    • Author(s)
      Yoshimiki Maekawa, Fumito Uwano, Eiki Kitajima, and Keiki Takadama
    • Organizer
      The 21st International Conference on Human-Computer Interaction
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research / Invited
  • [Presentation] Niche Radius Adaptation in Bat Algorithm for Location Multiple Optima in Multimodal Functions2019

    • Author(s)
      Takuya Iwase, Ryo Takano, Fumito Uwano, Hiroyuki Sato, and Keiki Takadama
    • Organizer
      IEEE Congress on Evolutionary Computation 2019
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Bat Algorithm with Dynamic Niche Radius for Multimodal Optimization2019

    • Author(s)
      Takuya Iwase, Ryo Takano, Fumito Uwano, Hiroyuki Sato, and Keiki Takadama
    • Organizer
      The 3rd International Conference on Intelligent Systems, Metaheuristics & Swarm Intelligence
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Maximum Entropy Inverse Reinforcement Learning with Incomplete Experts2019

    • Author(s)
      Satoshi Hasegawa, Fumito Uwano, and Keiki Takadama
    • Organizer
      The 24th International Symposium on Artificial Life and Robotics
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Novelty Search-based Bat Algorithm: Adjusting Distance among Solutions for Multimodal Optimization2019

    • Author(s)
      Takuya Iwase, Ryo Takano, Fumito Uwano, Hiroyuki Sato, and Keiki Takadama
    • Organizer
      The 22nd Asia Pacific Symposium on Intelligent and Evolutionary Systems
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] 集団適応を導くギャップ補填に基づく「思いやり」2019

    • Author(s)
      前川佳幹,上野史,北島瑛貴,髙玉圭樹
    • Organizer
      第33回人工知能学会全国大会
    • Related Report
      2018 Annual Research Report
  • [Presentation] 故障に対して冗長性を備えた仮想ロボットのニューロ進化による持続可能な行動獲得2019

    • Author(s)
      速水陽平,辰巳嵩豊,上野史,髙玉圭樹
    • Organizer
      第46回知能システムシンポジウム
    • Related Report
      2018 Annual Research Report
  • [Presentation] 好奇心を持つエージェントによる多様性のある情報伝搬シミュレーションモデルの提案2019

    • Author(s)
      北島瑛貴,髙玉圭樹,村田暁紀,上野史
    • Organizer
      HAIシンポジウム2018
    • Related Report
      2018 Annual Research Report
  • [Presentation] Strategy for Learning Cooperative Behavior with Local Information for Multi-agent Systems2018

    • Author(s)
      Fumito Uwano and Keiki Takadama
    • Organizer
      The 21st International Conference on Principles and Practice of Multi-Agent Systems
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Generalizing Rules by Random Forest-based Learning Classifier Systems for High-dimensional Data Mining2018

    • Author(s)
      Fumito Uwano, Koji Dobashi, Keiki Takadama, and Tim Kovacs
    • Organizer
      The Genetic and Evolutionary Computation Conference Companion 2018
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Analyzing Triangle Matching Method Based on Craters for Spacecraft Localization2018

    • Author(s)
      Fumito Uwano, Haruyuki Ishii, Yuta Umenai, Kazuma Matsumoto, Takato Tatsumi, Akinori Murata, and Keiki Takadama
    • Organizer
      The International Symposium on Artificial Intelligence, Robotics and Automation in Space 2018
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Correcting Wrongly Determined Opinions of Agents in Opinion Sharing Model2018

    • Author(s)
      Eiki Kitajima, Caili Zhang, Haruyuki Ishii, Fumito Uwano, and Keiki Takadama
    • Organizer
      The 20th International Conference on Human-Computer Interaction
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Multiple Swarm Intelligence Methods based on Multiple Population with Sharing Best Solution for Drastic Environmental Change2018

    • Author(s)
      Yuta Umenai, Fumito Uwano, Hiroyuki Sato, and Keiki Takadama
    • Organizer
      The Genetic and Evolutionary Computation Conference Companion 2018
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] How to Detect Essential Craters in Camera Shot Image to Increase the Number of Spacecraft Location Estimation while Improving its Accuracy?2018

    • Author(s)
      Haruyuki Ishii, Yuta Umenai, Kazuma Matsumoto, Fumito Uwano, Takato Tatsumi, Keiki Takadama, Hiroyuki Kamata, Takayuki Ishida, Seisuke Fukuda, Shujiro Sawai, and Shinichiro Sakai
    • Organizer
      The International Symposium on Artificial Intelligence, Robotics and Automation in Space 2018
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] 報酬の動的変化に適応する通信無しマルチエージェント協調学習のための公平性に基づく内部報酬設定法2018

    • Author(s)
      上野史,髙玉圭樹
    • Organizer
      計測自動制御学会システム・情報部門学術講演会2018
    • Related Report
      2018 Annual Research Report
  • [Presentation] 包括的な撮影画像パターンに対するSLIM探査機の自己位置推定の評価と精度向上2018

    • Author(s)
      上野史,村田暁紀,辰巳嵩豊,髙玉圭樹,鎌田弘之,石田貴行,福田盛介,澤井秀次郎,坂井真一郎
    • Organizer
      第62回宇宙科学技術連合講演会
    • Related Report
      2018 Annual Research Report
  • [Presentation] 行動系列分割に基づく不完全なエキスパートからの逆強化学習2018

    • Author(s)
      長谷川智,上野史,髙玉圭樹
    • Organizer
      計測自動制御学会システム・情報部門学術講演会2018
    • Related Report
      2018 Annual Research Report
  • [Presentation] 複数解探索を考慮した分散型Bat Algorithm2018

    • Author(s)
      岩瀬拓哉,高野諒,上野史,佐藤寛之,髙玉圭樹
    • Organizer
      計測自動制御学会システム・情報部門学術講演会2018
    • Related Report
      2018 Annual Research Report
  • [Presentation] グリッドネットワーク上の誤報抑制意見共有アルゴリズム2018

    • Author(s)
      北島瑛貴,辰巳嵩豊,村田暁紀,上野史,髙玉圭樹
    • Organizer
      計測自動制御学会システム・情報部門学術講演会2018
    • Related Report
      2018 Annual Research Report
  • [Presentation] 睡眠時無呼吸症候群患者のための無拘束型リアルタイム睡眠段階推定法2018

    • Author(s)
      田島友祐,上野史,原田智広,髙玉圭樹
    • Organizer
      ヘルスケア・医療情報通信技術研究会
    • Related Report
      2018 Annual Research Report
  • [Presentation] Theoretical Analysis of Triangle Matching Method Based on Craters for Spacecraft Localization2018

    • Author(s)
      Fumito Uwano
    • Organizer
      International Symposium on Artificial Intelligence, Robotics and Automation in Space (i-SAIRAS 2018)
    • Related Report
      2017 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Ensemble Heart Rate Extraction Method for Biological Data from Pressure Sensor Sensor2018

    • Author(s)
      Fumito Uwano
    • Organizer
      AAAI Spring Symposium 2018
    • Related Report
      2017 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Multiple Swarm Intelligence Methods based on Multiple Population with Sharing Best Solution for Drastic Environmental Change2018

    • Author(s)
      Yuta Umenai
    • Organizer
      Genetic and Evolutionary Computation Conference (GECCO 2018)
    • Related Report
      2017 Annual Research Report
    • Int'l Joint Research
  • [Presentation] How to Detect Essential Craters in Camera Shot Image to Increase the Number of Spacecraft Location Estimation while Improving its Accuracy?2018

    • Author(s)
      Haruyuki Ishii
    • Organizer
      International Symposium on Artificial Intelligence, Robotics and Automation in Space (i-SAIRAS 2018)
    • Related Report
      2017 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Improving Sleep Stage Estimation Accuracy by Circadian Rhythm Extracted from a Low Frequency Component of Heart Rate2018

    • Author(s)
      Akari Tobaru
    • Organizer
      AAAI Spring Symposium 2018
    • Related Report
      2017 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Correcting Wrongly Determined Opinions of Agents in Opinion Sharing Model2018

    • Author(s)
      Eiki Kitajima
    • Organizer
      International Conference on Human-computer Interaction (HCII 2018)
    • Related Report
      2017 Annual Research Report
    • Int'l Joint Research
  • [Presentation] 負の報酬生成による環境変化に適応可能な逆強化学習2018

    • Author(s)
      Satoshi Hasegawa
    • Organizer
      第45回知能システムシンポジウム
    • Related Report
      2017 Annual Research Report
  • [Presentation] 知識の忘却に基づく迷路形状の変化に追従する非通信マルチエージェント強化学習2017

    • Author(s)
      上野 史
    • Organizer
      計測自動制御学会 システム・情報部門 学術講演会 2017
    • Related Report
      2017 Annual Research Report
  • [Presentation] Strategies to Improve Cuckoo Search Toward Adapting Randomly Changing Environment2017

    • Author(s)
      Yuta Umenai
    • Organizer
      International Conference of Swarm Intelligence (ICSI 2017)
    • Related Report
      2017 Annual Research Report
    • Int'l Joint Research
  • [Presentation] 動的環境適応に向けた粒子群最適化とカッコウ探索の協働のための情報共有方法の検討2017

    • Author(s)
      梅内祐太
    • Organizer
      進化計算シンポジウム 2017
    • Related Report
      2017 Annual Research Report
  • [Presentation] 深層学習による次元圧縮ルールの学習分類子システムにおける初期ルールとしての可能性2017

    • Author(s)
      松本和馬
    • Organizer
      進化計算シンポジウム 2017
    • Related Report
      2017 Annual Research Report
  • [Presentation] SLIM探査機の高度や姿勢の傾きによるクレータ検出位置ずれに対応する自己位置推定法2017

    • Author(s)
      石井晴之
    • Organizer
      第61回宇宙科学技術連合講演会
    • Related Report
      2017 Annual Research Report
  • [Presentation] Searching Multiple Local Optimal Solutions in Multimodal Function by Bat Algorithm based on Novelty Search2017

    • Author(s)
      Takuya Iwase
    • Organizer
      進化計算シンポジウム 2017
    • Related Report
      2017 Annual Research Report
  • [Presentation] 複数解探索を考慮した分散型Bat Algorithm2017

    • Author(s)
      岩瀬拓哉
    • Organizer
      計測自動制御学会 システム・情報部門 学術講演会 2017
    • Related Report
      2017 Annual Research Report
  • [Book] PRIMA 2018: Principles and Practice of Multi-Agent Systems2018

    • Author(s)
      Fumito Uwano and Keiki Takadama
    • Publisher
      Springer
    • ISBN
      9783030030988
    • Related Report
      2018 Annual Research Report
  • [Patent(Industrial Property Rights)] 点群マッチング装置,点群マッチング方法及びプログラム2018

    • Inventor(s)
      髙玉圭樹,石井晴之,上野史
    • Industrial Property Rights Holder
      髙玉圭樹,石井晴之,上野史
    • Industrial Property Rights Type
      特許
    • Industrial Property Number
      2018-106820
    • Filing Date
      2018
    • Related Report
      2018 Annual Research Report
    • Overseas

URL: 

Published: 2017-05-25   Modified: 2024-03-26  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi