• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Quality Control of Human Computation Using Meta-Cognitive Tasks

Research Project

Project/Area Number 15H02782
Research Category

Grant-in-Aid for Scientific Research (B)

Allocation TypeSingle-year Grants
Section一般
Research Field Web informatics, Service informatics
Research InstitutionHokkaido University

Principal Investigator

Oyama Satoshi  北海道大学, 情報科学研究科, 准教授 (30346100)

Co-Investigator(Kenkyū-buntansha) 櫻井 祐子  国立研究開発法人産業技術総合研究所, 情報・人間工学領域, 主任研究員 (10396137)
Research Collaborator TAKASE Tomoumi  
ATARASHI Kyohei  
MORISE Hiroki  
SEKIYA Ryota  
Project Period (FY) 2015-04-01 – 2018-03-31
Project Status Completed (Fiscal Year 2018)
Budget Amount *help
¥12,350,000 (Direct Cost: ¥9,500,000、Indirect Cost: ¥2,850,000)
Fiscal Year 2017: ¥3,770,000 (Direct Cost: ¥2,900,000、Indirect Cost: ¥870,000)
Fiscal Year 2016: ¥3,770,000 (Direct Cost: ¥2,900,000、Indirect Cost: ¥870,000)
Fiscal Year 2015: ¥4,810,000 (Direct Cost: ¥3,700,000、Indirect Cost: ¥1,110,000)
Keywords人工知能 / 認知科学 / メカニズム設計 / 推薦システム / 教育応用 / エージェント
Outline of Final Research Achievements

In this research, we advanced the concept of human computation, which affirmatively uses human's ability in information processing by computer, to utilize human's ability in the quality control of human computation itself. We developed quality control models using human's high cognitive ability such as confidence judgment and multicriteria evaluation as well as high speed and high accuracy calculation ability of computer. In various fields such as image recognition, recommendation systems and education, the effectiveness of the approach was verified by data from actual services and experimental evaluation using real workers.

Academic Significance and Societal Importance of the Research Achievements

現在,機械学習などの人工知能技術による自動化が様々な分野において推し進められているが,依然として人間の判断や評価が必要な問題は多く存在しており,推薦システムや教育など,本質的に人間の介在が不可欠な問題も存在する.本研究はそのような人間を含むシステムにおいて人間の高度な認知能力を活用して品質制御を行う方法を提案しており,発展を続ける人工知能技術と組み合わせることで,様々な分野において製品やサービスの品質向上への貢献が期待できる.

Report

(4 results)
  • 2018 Final Research Report ( PDF )
  • 2017 Annual Research Report
  • 2016 Annual Research Report
  • 2015 Annual Research Report
  • Research Products

    (32 results)

All 2019 2018 2017 2016 2015

All Journal Article (2 results) (of which Peer Reviewed: 1 results,  Open Access: 2 results) Presentation (29 results) (of which Int'l Joint Research: 15 results,  Invited: 1 results) Book (1 results)

  • [Journal Article] Bayesian Probabilistic Tensor Factorization for Recommendation and Rating Aggregation with Multicriteria Evaluation Data2019

    • Author(s)
      Hiroki Morise, Satoshi Oyama, Masahito Kurihara
    • Journal Title

      Expert Systems with Applications (掲載確定)

      Volume: 未定 Pages: 1-8

    • DOI

      10.1016/j.eswa.2019.04.044

    • NAID

      120007151377

    • Related Report
      2017 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] 教育用データ解析コンペティション基盤の設計と実践2018

    • Author(s)
      馬場 雪乃, 高瀬 朝海, 新 恭兵, 小山 聡, 鹿島 久嗣
    • Journal Title

      デジタルプラクティス

      Volume: 9(4) Pages: 859-873

    • NAID

      170000149784

    • Related Report
      2017 Annual Research Report
    • Open Access
  • [Presentation] User-Adaptive Preparation of Mathematical Puzzles Using Item Response Theory and Deep Learning2019

    • Author(s)
      Ryota Sekiya, Satoshi Oyama, Masahito Kurihara
    • Organizer
      32nd International Conference on Industrial, Engineering & Other Applications of Applied Intelligent Systems (IEA/AIE 2019)
    • Related Report
      2017 Annual Research Report
    • Int'l Joint Research
  • [Presentation] 項目反応理論に基づく魔方陣パズルのユーザー適応的自動生成2019

    • Author(s)
      関屋 亮太, 小山 聡, 栗原 正仁
    • Organizer
      第11回データ工学と情報マネジメントに関するフォーラム (DEIM 2019)
    • Related Report
      2017 Annual Research Report
  • [Presentation] 読み手の個性を考慮したニュース記事の感情応答モデルの構築2019

    • Author(s)
      森山 皓未, 新 恭兵, 小山 聡, 栗原 正仁
    • Organizer
      第11回データ工学と情報マネジメントに関するフォーラム (DEIM 2019)
    • Related Report
      2017 Annual Research Report
  • [Presentation] Data Analysis Competition Platform for Educational Purposes: Lessons Learned and Future Challenges2018

    • Author(s)
      Yukino Baba, Tomoumi Takase, Kyohei Atarashi, Satoshi Oyama, Hisashi Kashima
    • Organizer
      Eighth AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI 2018)
    • Related Report
      2017 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Semi-supervised Learning from Crowds Using Deep Generative Models2018

    • Author(s)
      Kyohei Atarashi, Satoshi Oyama, and Masahito Kurihara
    • Organizer
      32nd AAAI Conference on Artificial Intelligence (AAAI 2018)
    • Related Report
      2017 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Shapley Value Regression for Bias Reduction in Answer Aggregation2018

    • Author(s)
      Yuko Sakurai, Satoshi Oyama
    • Organizer
      Disentangling the Relation Between Crowdsourcing and Bias Management (CrowdBias)
    • Related Report
      2017 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Aggregating Crowd Opinions Using Shapley Value Regression2018

    • Author(s)
      Yuko Sakurai, Jun Kawahara, Satoshi Oyama
    • Organizer
      12th Multi-Disciplinary International Conference on Artificial Intelligence (MIWAI 2018)
    • Related Report
      2017 Annual Research Report
    • Int'l Joint Research
  • [Presentation] 多基準評価値を用いた深層学習による推薦と評価値統合2018

    • Author(s)
      森瀬 寛己, 小山 聡, 栗原 正仁
    • Organizer
      情報処理北海道シンポジウム2018
    • Related Report
      2017 Annual Research Report
  • [Presentation] ニュース記事に対する感情応答モデルの構築2018

    • Author(s)
      森山 皓未, 新 恭兵, 小山 聡, 栗原 正仁
    • Organizer
      情報処理北海道シンポジウム2018
    • Related Report
      2017 Annual Research Report
  • [Presentation] 項目応答理論に基づく魔法陣パズルのユーザー適応的自動生成2018

    • Author(s)
      関屋 亮太, 小山 聡, 栗原 正仁
    • Organizer
      情報処理北海道シンポジウム2018
    • Related Report
      2017 Annual Research Report
  • [Presentation] 作業者の「自信」を用いたクラウドソーシングの品質制御2017

    • Author(s)
      小山 聡
    • Organizer
      第1回情報,通信,知識技術に関するシンポジウム
    • Place of Presentation
      兵庫県淡路市 淡路夢舞台国際会議場
    • Year and Date
      2017-02-18
    • Related Report
      2016 Annual Research Report
  • [Presentation] Toward Crowdsourced Mechanism Design2017

    • Author(s)
      Yuko Sakurai, Masafumi Matsuda, Satoshi Oyama
    • Organizer
      Collective Intelligence Conference 2017
    • Related Report
      2017 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Requesters' Personal Values, Just-world beliefs, and Their Choice of Incentive Mechanisms2017

    • Author(s)
      Yuko Sakurai, Masafumi Matsuda, Satoshi Oyama
    • Organizer
      Fifth AAAI Conference on Human Computation and Crowdsourcing (HCOMP 2017)
    • Related Report
      2017 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Crowdsourcing Mechanism Design2017

    • Author(s)
      Yuko Sakurai, Masafumi Matsuda, Masato Shinoda, Satoshi Oyama
    • Organizer
      20th International Conference on Principles and Practice of Multi-Agent Systems (PRIMA 2017)
    • Related Report
      2017 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Collaborative Filtering and Rating Aggregation Based on Multicriteria Rating2017

    • Author(s)
      Hiroki Morise, Satoshi Oyama, Masahito Kurihara
    • Organizer
      First IEEE Workshop on Human-Machine Collaboration in Big Data (IEEE HMData 2017)
    • Related Report
      2017 Annual Research Report
    • Int'l Joint Research
  • [Presentation] クラウドワーカを利用したメカニズム設計のための一考察2017

    • Author(s)
      櫻井 祐子, 松田 昌史, 小山 聡
    • Organizer
      2017年度人工知能学会全国大会
    • Related Report
      2017 Annual Research Report
  • [Presentation] 多基準評価値に基づく協調フィルタリングと評価値統合2017

    • Author(s)
      森瀬 寛己, 小山 聡, 栗原 正仁
    • Organizer
      2017年度人工知能学会全国大会
    • Related Report
      2017 Annual Research Report
  • [Presentation] 行列・テンソル分解による評価値予測と評価値統合2017

    • Author(s)
      森瀬 寛己, 小山 聡, 栗原 正仁
    • Organizer
      情報処理北海道シンポジウム2017
    • Related Report
      2017 Annual Research Report
  • [Presentation] 深層生成モデルによるクラウドソーシングを用いた半教師あり学習2017

    • Author(s)
      新 恭兵, 小山 聡 栗原 正仁
    • Organizer
      情報処理北海道シンポジウム2017
    • Related Report
      2017 Annual Research Report
  • [Presentation] クラウドソーシングを用いた半教師あり学習のための深層生成モデル2017

    • Author(s)
      新 恭兵, 小山 聡, 栗原 正仁
    • Organizer
      第20回情報論的学習理論ワークショップ (IBIS 2017)
    • Related Report
      2017 Annual Research Report
  • [Presentation] ソフトラベルの非確率的解釈と分類学習への応用2017

    • Author(s)
      高瀬 朝海, 小山 聡, 栗原 正仁
    • Organizer
      第20回情報論的学習理論ワークショップ (IBIS 2017)
    • Related Report
      2017 Annual Research Report
  • [Presentation] Quality Control for Crowdsourced Multi-Label Annotation2016

    • Author(s)
      Satoshi Oyama
    • Organizer
      11th Korea-Japan Database Workshop
    • Place of Presentation
      Daegu, South Korea
    • Year and Date
      2016-11-18
    • Related Report
      2016 Annual Research Report
    • Int'l Joint Research / Invited
  • [Presentation] ヒューマンコンピュテーションにおける品質制御とワーカ評価について2016

    • Author(s)
      小山 聡
    • Organizer
      第22回AI知識交流研究会
    • Place of Presentation
      群馬県利根郡みなかみ町 辰巳館
    • Year and Date
      2016-07-30
    • Related Report
      2016 Annual Research Report
  • [Presentation] シャプレイ値に基づくクラウドワーカの能力推定2016

    • Author(s)
      櫻井 祐子, 小山 聡
    • Organizer
      2016年度人工知能学会全国大会
    • Place of Presentation
      福岡県北九州市 北九州国際会議場
    • Year and Date
      2016-06-06
    • Related Report
      2016 Annual Research Report
  • [Presentation] Aggregating Crowd Opinions Using the Shapley Value Regression2016

    • Author(s)
      Yuko Sakurai, Satoshi Oyama
    • Organizer
      Collective Intelligence 2016
    • Place of Presentation
      New York, NY, USA
    • Year and Date
      2016-06-01
    • Related Report
      2016 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Crowdsourced Semantic Matching of Multi-Label Annotations2016

    • Author(s)
      Lei Duan, Satoshi Oyama, Masahito Kurihara, Haruhiko Sato
    • Organizer
      NIPS 2016 Workshop on Crowdsourcing and Machine Learning
    • Place of Presentation
      Barcelona, Spain
    • Related Report
      2016 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Flexible Reward Plans to Elicit Truthful Predictions in Crowdsourcing2015

    • Author(s)
      Yuko Sakurai, Satoshi Oyama, Masato Shinoda, Makoto Yokoo
    • Organizer
      Third AAAI Conference on Human Computation and Crowdsourcing (HCOMP 2015)
    • Place of Presentation
      San Diego, CA, USA
    • Year and Date
      2015-11-08
    • Related Report
      2015 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Flexible Reward Plans for Crowdsourced Tasks2015

    • Author(s)
      Yuko Sakurai, Masato Shinoda, Satoshi Oyama, Makoto Yokoo
    • Organizer
      18th International Conference on Principles and Practice of Multi-Agent Systems (PRIMA 2015)
    • Place of Presentation
      Bertinoro, Italy
    • Year and Date
      2015-10-26
    • Related Report
      2015 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Crowdsourcing for Big Data Analytics (Tutorial)2015

    • Author(s)
      Hisashi Kashima, Satoshi Oyama, Yukino Baba
    • Organizer
      19th Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD 2015)
    • Place of Presentation
      Ho Chi Minh City, Viet Nam
    • Year and Date
      2015-05-19
    • Related Report
      2015 Annual Research Report
    • Int'l Joint Research
  • [Book] ヒューマンコンピュテーションとクラウドソーシング2016

    • Author(s)
      鹿島 久嗣, 小山 聡, 馬場 雪乃
    • Total Pages
      117
    • Publisher
      講談社
    • Related Report
      2015 Annual Research Report

URL: 

Published: 2015-04-16   Modified: 2020-03-30  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi