• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

The evolutionary approach to developing artificial intelligence for understanding human creations

Research Project

Project/Area Number 19H04184
Research Category

Grant-in-Aid for Scientific Research (B)

Allocation TypeSingle-year Grants
Section一般
Review Section Basic Section 61040:Soft computing-related
Research InstitutionOsaka Metropolitan University (2022)
Osaka Prefecture University (2019-2021)

Principal Investigator

Mori Naoki  大阪公立大学, 大学院情報学研究科, 教授 (90295717)

Co-Investigator(Kenkyū-buntansha) 上野 未貴  大阪工業大学, 工学部, 講師 (60757368)
Project Period (FY) 2019-04-01 – 2023-03-31
Project Status Completed (Fiscal Year 2022)
Budget Amount *help
¥17,160,000 (Direct Cost: ¥13,200,000、Indirect Cost: ¥3,960,000)
Fiscal Year 2022: ¥2,730,000 (Direct Cost: ¥2,100,000、Indirect Cost: ¥630,000)
Fiscal Year 2021: ¥3,120,000 (Direct Cost: ¥2,400,000、Indirect Cost: ¥720,000)
Fiscal Year 2020: ¥5,720,000 (Direct Cost: ¥4,400,000、Indirect Cost: ¥1,320,000)
Fiscal Year 2019: ¥5,590,000 (Direct Cost: ¥4,300,000、Indirect Cost: ¥1,290,000)
Keywords創作物理解 / 深層学習 / 進化型計算 / 人工知能 / 機械学習 / 創作と人工知能 / 創作者支援 / Data Augmentation / 創作者と人工知能 / AutoML / 創作物データ / ストーリーデータセット / 物語理解 / コミックデータ / 漫画
Outline of Research at the Start

近年人工知能が注目されているが,人の創作物を理解できる人工知能はまだない.そこで,創作物をデータとして客観視するだけではなく,創作過程で込められた想いを主体的に扱い,創作者と研究者をつなぐ人工知能の構築を目的とする.また,高度な人工知能構築に必須なディープラーニングはいまだに試行錯誤的な域を出ておらず,構築は容易ではない.この問題を解決するために,本研究課題は人の創作物を理解する人工知能を進化的に獲得可能な進化型機械学習を提案する.最終的に,人工知能への適用を前提としたこれまでにない創作物データセットの構築および進化型機械学習により進化的に獲得した人工知能による人の創作物理解を目指す.

Outline of Final Research Achievements

In this project, we aimed to understand the human creation through artificial intelligence. Our research target spanned multiple domains such as comic-based studies (panel recognition, emotion analysis, onomatopoeia comprehension), photo-based research (quantitative photo evaluation, starry sky image generation, insect image recognition), and research on optical illusions (AI-based ambiguous figures recognition, ambiguous figures generation based on image information, and ambiguous figures generation based on linguistic information).
Further, we explored other creative understandings, including calligraphy, fashion, and games, along with research on AutoML (TDGA AutoAugment (TDGA AA), Pareto TDGA AA, tdgaCNN). These achievements have been presented and evaluated at academic conferences, including AAAI. Additionally, we held four organized sessions at the Japanese Society for Artificial Intelligence on the future of creativity by authors and AI, contributing to advancing the field.

Academic Significance and Societal Importance of the Research Achievements

近年人工知能が注目されているが,人の創作物を理解できる人工知能はまだない.そこで,創作物をデータとして客観視するだけではなく,創作過程で込められた想いを主体的に扱い,創作者と研究者をつなぐ人工知能の構築について研究した.また,進化型機械学習として新しいData Augmentation 手法等を提案した.コロナ禍という厳しい状況の中で,4 回のオーガナイズドセッションを企画し,当該分野における人的交流に寄与した.
近年の生成系人工知能の発展によって,創作物に関する研究は今後ますます注目され,積極的に研究がなされていくはずである.本研究課題によって当該分野へ一定の貢献をすることができたと考える.

Report

(5 results)
  • 2022 Annual Research Report   Final Research Report ( PDF )
  • 2021 Annual Research Report
  • 2020 Annual Research Report
  • 2019 Annual Research Report
  • Research Products

    (62 results)

All 2023 2022 2021 2020 2019

All Journal Article (9 results) (of which Peer Reviewed: 9 results,  Open Access: 2 results) Presentation (53 results) (of which Int'l Joint Research: 19 results,  Invited: 1 results)

  • [Journal Article] Constellation identification method using point set data2023

    • Author(s)
      Sayaka Nadamoto, Naoki Mori, Makoto Okada
    • Journal Title

      Artificial Life and Robotics

      Volume: 28 Issue: 2 Pages: 361-366

    • DOI

      10.1007/s10015-023-00860-4

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Automatic Paragraph Segmentation of Novels as Imbalanced Classification2021

    • Author(s)
      飯倉 陸, 岡田 真, 森 直樹
    • Journal Title

      情報処理学会論文誌

      Volume: 62 Issue: 3 Pages: 891-902

    • DOI

      10.20729/00210253

    • NAID

      170000184452

    • Year and Date
      2021-03-15
    • Related Report
      2020 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Paragraph Boundary Recognition in Novels for Story Understanding2021

    • Author(s)
      Iikura Riku、Okada Makoto、Mori Naoki
    • Journal Title

      Applied Sciences

      Volume: 11 Issue: 12 Pages: 5632-5632

    • DOI

      10.3390/app11125632

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] CVAE-Based Complementary Story Generation Considering the Beginning and Ending2021

    • Author(s)
      Iikura Riku、Okada Makoto、Mori Naoki
    • Journal Title

      Distributed Computing and Artificial Intelligence

      Volume: 19 Pages: 22-31

    • DOI

      10.1007/978-3-030-86261-9_3

    • ISBN
      9783030862602, 9783030862619
    • Related Report
      2021 Annual Research Report
    • Peer Reviewed
  • [Journal Article] A Project of “Developing a Dataset and Framework to Analyse Stories in Four-scene Comics Using Deep Neural Networks”2019

    • Author(s)
      上野 未貴
    • Journal Title

      NIHON GAZO GAKKAISHI (Journal of the Imaging Society of Japan)

      Volume: 58 Issue: 5 Pages: 545-551

    • DOI

      10.11370/isj.58.545

    • NAID

      130007726205

    • ISSN
      1344-4425, 1880-4675
    • Year and Date
      2019-10-10
    • Related Report
      2019 Annual Research Report
    • Peer Reviewed
  • [Journal Article] ストーリー解析のための文の分散表現に基づく小説の自動セグメンテーション手法の提案2019

    • Author(s)
      福田清人, 森直樹, 松本啓之亮, 岡田真
    • Journal Title

      芸術科学会論文誌

      Volume: 18 Pages: 63-75

    • Related Report
      2019 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Analysis based on Distributed Representations of Various Parts Images in Four-scene Comics Story Dataset2019

    • Author(s)
      A. Terauchi, N. Mori, M. Ueno
    • Journal Title

      2019 International Conference on Document Analysis and Recognition Workshops

      Volume: 1 Pages: 50-55

    • DOI

      10.1109/icdarw.2019.00014

    • Related Report
      2019 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Evolutionary Approach for Model Compression of Neural Network Using Support Vector Machine with AND/OR Layers2019

    • Author(s)
      K. Ogawa and N. Mori
    • Journal Title

      Proc. of the 18th International Conference on New Trends in Intelligent Software Methodologies, Tools and Techniques

      Volume: 18 Pages: 231-243

    • Related Report
      2019 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Understanding multilingual four-scene comics with deep learning methods2019

    • Author(s)
      J. Chen, R. Iwasaki, N. Mori, M. Okada, M. Ueno
    • Journal Title

      2019 International Conference on Document Analysis and Recognition Workshops

      Volume: 1 Pages: 32-37

    • DOI

      10.1109/icdarw.2019.00011

    • Related Report
      2019 Annual Research Report
    • Peer Reviewed
  • [Presentation] An improved method for multi-label classification methods using deep neural language models with ensemble learning2023

    • Author(s)
      Y. Kusumoto, M. OKADA and N. Mori
    • Organizer
      28th International Symposium on Artificial Life and Robotics
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] A Method to Constract a Masked Knowlege Graph Model using Transformer for Knowledge Graph Reasoning2023

    • Author(s)
      R. Kaneda, M. OKADA and N. Mori
    • Organizer
      The 1st International Knowledge Graph Reasoning Challenge
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Transformer を導入したランク学習に基づく投資戦略の解析2023

    • Author(s)
      坂川 翔祐, 森 直樹
    • Organizer
      第 30 回 人工知能学会 金融情報学研究会
    • Related Report
      2022 Annual Research Report
  • [Presentation] Object Detection of Camouflaged Insect by Artificial Intelligence2022

    • Author(s)
      R. Okamoto and N. Mori
    • Organizer
      The 26th International Congress of Entomology
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Evolutionary Acquisition of CNN Architecture by Thermodynamical Genetic Algorithm2022

    • Author(s)
      R. Okamoto, N. Mori and M. OKADA
    • Organizer
      13th International Congress on Advanced Applied Informatics
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Automatic Astrophotography Editing Method Using Deep Learning2022

    • Author(s)
      S. Nadamoto, N. Mori and M. OKADA
    • Organizer
      13th International Congress on Advanced Applied Informatics
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Neural Ranking Strategy for Portfolio Construction Using Transformers2022

    • Author(s)
      S. Sakagawa and N. Mori
    • Organizer
      13th International Congress on Advanced Applied Informatics
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Vision Transformer と GAN による多義図形認識および生成に関する研究2022

    • Author(s)
      伊藤 薫, 森 直樹, 岡田 真
    • Organizer
      第36回 人工知能学会全国大会
    • Related Report
      2022 Annual Research Report
  • [Presentation] 敵対的生成ネットワークを用いた星空画像生成手法の提案2022

    • Author(s)
      灘本 紗也佳, 森 直樹, 岡田 真
    • Organizer
      第36回 人工知能学会全国大会
    • Related Report
      2022 Annual Research Report
  • [Presentation] 漫画における指示詞の照応関係データセットの構築と機械学習手法を用いた検証2022

    • Author(s)
      大和 秀徳, 岡田 真, 森 直樹
    • Organizer
      第36回 人工知能学会全国大会
    • Related Report
      2022 Annual Research Report
  • [Presentation] Vision Transformer と BERT を用いた漫画のマルチモーダル識別2022

    • Author(s)
      青木 尚人, 森 直樹, 岡田 真
    • Organizer
      第36回 人工知能学会全国大会
    • Related Report
      2022 Annual Research Report
  • [Presentation] 漫画における皮肉データセットの構築とその有効性の検証2022

    • Author(s)
      多田 瑞葵, 岡田 真, 森 直樹
    • Organizer
      第36回 人工知能学会全国大会
    • Related Report
      2022 Annual Research Report
  • [Presentation] Proposal of a Constellation Identification Method Using Point Set Data2022

    • Author(s)
      S. Nadamoto, N. Mori and M. Okada
    • Organizer
      27th International Symposium on Artificial Life and Robotics
    • Related Report
      2021 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Estimating Semantic Relationships between Sentences Using Word Embedding with BERT2021

    • Author(s)
      R. Kaneda, M. Okada and N. Mori
    • Organizer
      10th International Congress on Advanced Applied Informatics
    • Related Report
      2021 Annual Research Report
    • Int'l Joint Research
  • [Presentation] 熱力学的遺伝アルゴリズムおよび Bi-LSTM + VSE に基づくコーディネートスケジュールの最適化2021

    • Author(s)
      林 美衣, 森 直樹
    • Organizer
      第 35 回人工知能学会全国大会
    • Related Report
      2021 Annual Research Report
  • [Presentation] 星図データに基づく星座画像の生成と星座識別手法の提案2021

    • Author(s)
      灘本 紗也佳, 森 直樹, 岡田 真
    • Organizer
      第 35 回人工知能学会全国大会
    • Related Report
      2021 Annual Research Report
  • [Presentation] 潜在変数付き階層型エンコーダ・デコーダモデルに基づく物語の補完的生成手法の提案2021

    • Author(s)
      飯倉 陸, 岡田 真, 森 直樹
    • Organizer
      第 35 回人工知能学会全国大会
    • Related Report
      2021 Annual Research Report
  • [Presentation] Support Vector Machine のカーネルを利用したニューラルネットワークの構築2021

    • Author(s)
      坂川 翔祐, 森 直樹
    • Organizer
      第 35 回人工知能学会全国大会
    • Related Report
      2021 Annual Research Report
  • [Presentation] TDGA AutoAugment を用いた漫画データセットにおけるデータ拡張手法の提案2021

    • Author(s)
      吉田 航, 寺内 光, 森 直樹, 岡田 真
    • Organizer
      第 35 回人工知能学会全国大会
    • Related Report
      2021 Annual Research Report
  • [Presentation] 自由配置型アトラス畳み込みに基づくセマンティックセグメンテーション手法の提案2021

    • Author(s)
      岡本 竜馬, 寺内 光, 森 直樹, 岡田 真
    • Organizer
      第 35 回人工知能学会全国大会
    • Related Report
      2021 Annual Research Report
  • [Presentation] BERT を用いた指示詞の照応関係の推定手法の検討2021

    • Author(s)
      大和 秀徳, 岡田 真, 森 直樹
    • Organizer
      2021 年度 情報処理学会関西支部 支部大会
    • Related Report
      2021 Annual Research Report
  • [Presentation] Improving BERT with Focal Loss for Paragraph Segmentation of Novels2020

    • Author(s)
      R. Iikura, M. Okada, N. Mori
    • Organizer
      Proc. of 17th International Conference of Distributed Computing and Artificial Intelligence
    • Related Report
      2020 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Analysis of Partial Semantic Segmentation for Images of Four-Scene Comics2020

    • Author(s)
      A. Terauchi, N. Mori, M. Ueno
    • Organizer
      Proc. of 17th International Conference of Distributed Computing and Artificial Intelligence
    • Related Report
      2020 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Evolutionary Approach for Autoaugment Using the Thermodynamical Genetic Algorithm2020

    • Author(s)
      A. Terauchi, N. Mori
    • Organizer
      Proc. of 35th AAAI Conference on Artificial Intelligence (AAAI 2021)
    • Related Report
      2020 Annual Research Report
    • Int'l Joint Research
  • [Presentation] 文書要約のための階層型 Attention モデルの提案と Pointer Generator Network のための拡張手法2020

    • Author(s)
      寺西 弘樹, 岡田 真, 森 直樹
    • Organizer
      第 34 回人工知能学会全国大会
    • Related Report
      2020 Annual Research Report
  • [Presentation] Focal Loss を利用した BERT による小説の段落境界推定2020

    • Author(s)
      飯倉 陸, 岡田 真, 森 直樹
    • Organizer
      第 34 回人工知能学会全国大会
    • Related Report
      2020 Annual Research Report
  • [Presentation] 人工知能による多義図形認識手法の提案及び解析2020

    • Author(s)
      堀江 紗世, 森 直樹
    • Organizer
      第 34 回人工知能学会全国大会
    • Related Report
      2020 Annual Research Report
  • [Presentation] 分散表現に基づく漫画の画像と台詞の対応識別2020

    • Author(s)
      寺内 光, 森 直樹, 上野 未貴
    • Organizer
      第 34 回人工知能学会全国大会
    • Related Report
      2020 Annual Research Report
  • [Presentation] ファッションアイテムの分散表現に基づくコーディネート理解2020

    • Author(s)
      林 美衣, 森 直樹
    • Organizer
      第 34 回人工知能学会全国大会
    • Related Report
      2020 Annual Research Report
  • [Presentation] 深層学習に基づく 4 コマ漫画のセリフの感情推定とマルチモーダル化への検討2020

    • Author(s)
      高山 裕成, 岡田 真, 森 直樹
    • Organizer
      情報処理学会関西支部支部大会
    • Related Report
      2020 Annual Research Report
  • [Presentation] 漫画のオブジェクト順位データセットの構築と機械学習による内容理解の提案2020

    • Author(s)
      元山 直輝, 岡田 真, 森 直樹
    • Organizer
      電情報通信学会第 4 回コミック工学研究会子
    • Related Report
      2020 Annual Research Report
  • [Presentation] 深層学習によるだまし絵認識手法の提案および解析2020

    • Author(s)
      堀江 紗世, 森 直樹
    • Organizer
      NICOGRAPH2020
    • Related Report
      2020 Annual Research Report
  • [Presentation] Photograph Classification Based on Main Theme and Multiple Values by Deep Neural Networks2020

    • Author(s)
      Toshinori Aoki, Miki Ueno
    • Organizer
      Advances in Intelligent Systems and Computing
    • Related Report
      2020 Annual Research Report
    • Int'l Joint Research
  • [Presentation] 撮影技術習得を目的とした複数の評価特徴に基づく写真創作の解析2020

    • Author(s)
      青木 俊憲, 上野 未貴
    • Organizer
      人工知能学会全国大会
    • Related Report
      2020 Annual Research Report
  • [Presentation] 深層学習を用いた手書き文字のフォーマルとカジュアルの分類2020

    • Author(s)
      突廻 恵太, 上野 未貴
    • Organizer
      HAIシンポジウム
    • Related Report
      2020 Annual Research Report
  • [Presentation] 写真画像に基づく自動用紙推定の提案2020

    • Author(s)
      平井 喜一, 上野 未貴
    • Organizer
      HAIシンポジウム
    • Related Report
      2020 Annual Research Report
  • [Presentation] A Method to Estimate Request Sentences using LSTM with Self-Attention Mechanism2019

    • Author(s)
      T. Ueda, M. Okada, N. Mori and K. Hashimoto.
    • Organizer
      8th International Congress on Advanced Applied Informatics
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Deep Neural NetworkPretrained by a Support Vector Machine2019

    • Author(s)
      H. Yamamoto and N. Mori
    • Organizer
      Proc. of The 14th International Joint Symposium on Artificial Intelligence and Natural Language Processing (iSAI-NLP 2019),
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research
  • [Presentation] The novel index of the similarity between hand-drawn sketches for machine learning2019

    • Author(s)
      R. Fujii, N. Mori and M. Okada
    • Organizer
      Proc. of The 14th International Joint Symposium on Artificial Intelligence and Natural Language Processing (iSAI-NLP 2019),
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Dialogue Breakdown Detection for Understanding Comics with Deep Learning2019

    • Author(s)
      R. Iwasaki and N. Mori
    • Organizer
      Proc. of The 14th International Joint Symposium on Artificial Intelligence and Natural Language Processing (iSAI-NLP 2019),
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Hierarchical Attention Model for Acquiring Relationships Among Sentences2019

    • Author(s)
      H. Teranishi, M. Okada and N. Mori
    • Organizer
      Proc. of The 14th International Joint Symposium on Artificial Intelligence and Natural Language Processing (iSAI-NLP 2019),
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research
  • [Presentation] 筆順の類似度に関する評価モデルに対する考察2019

    • Author(s)
      藤井涼佑,森 直樹,岡田 真
    • Organizer
      2019 年度人工知能学会全国大会講演論文集
    • Related Report
      2019 Annual Research Report
  • [Presentation] segmented VRAE と遺伝的プログラミングに基づく音楽の創作2019

    • Author(s)
      山本周典,森 直樹
    • Organizer
      2019 年度人工知能学会全国大会講演論文集
    • Related Report
      2019 Annual Research Report
  • [Presentation] 4 コマ漫画ストーリーデータセットにおけるコマ画像の分散表現に基づく解析2019

    • Author(s)
      寺内 光,森 直樹,上野未貴
    • Organizer
      2019 年度人工知能学会全国大会講演論文集
    • Related Report
      2019 Annual Research Report
  • [Presentation] 自然言語処理に基づく4コマ漫画ストーリーデータセットの解析2019

    • Author(s)
      岩崎 凌,森 直樹,上野未貴
    • Organizer
      2019 年度人工知能学会全国大会講演論文集
    • Related Report
      2019 Annual Research Report
  • [Presentation] Long Short-Term Memory に基づく Recurrent Auto-Encoder を用いた文の分散表現獲得手法に対するAttention 機構の導入2019

    • Author(s)
      飯倉 陸, 岡田 真, 森 直樹
    • Organizer
      情報処理学会 第241回 自然言語処理研究会
    • Related Report
      2019 Annual Research Report
  • [Presentation] 座標情報に基づく漫画のオブジェクト順位推定2019

    • Author(s)
      元山直輝, 岡田 真, 森 直樹
    • Organizer
      情報処理学会 第241回 自然言語処理研究会
    • Related Report
      2019 Annual Research Report
  • [Presentation] 進化型機械学習フレームワークに基づくSVM-NNにおける感度解析2019

    • Author(s)
      小川恭子, 森 直樹
    • Organizer
      2019 年電気学会電子・情報・システム部門大会
    • Related Report
      2019 Annual Research Report
  • [Presentation] A Comparative Study of The Corpus for Story Creation System2019

    • Author(s)
      Haruka Takahashi, Miki Ueno, Hitoshi Isahara
    • Organizer
      Distributed Computing and Artificial Intelligence, 16th International Conference, Advances in Intelligent Systems and Computing
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Considering Creative Stories from Interaction Perspective2019

    • Author(s)
      Miki Ueno
    • Organizer
      7th International Conference on Human-Agent Interaction, iHAI2019
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research / Invited
  • [Presentation] 創作者と人工知能が創る創作の未来2019

    • Author(s)
      上野 未貴
    • Organizer
      人工知能学会
    • Related Report
      2019 Annual Research Report
  • [Presentation] 創作支援を目的とした類似度調節によるストーリー生成システムの構築2019

    • Author(s)
      高橋 遼, 上野 未貴, 井佐原 均
    • Organizer
      人工知能学会
    • Related Report
      2019 Annual Research Report
  • [Presentation] 作中役割を考慮した物語文の類似文検索による創作者の支援と感性の解析2019

    • Author(s)
      葛井 健文, 上野 未貴, 井佐原 均
    • Organizer
      人工知能学会
    • Related Report
      2019 Annual Research Report

URL: 

Published: 2019-04-18   Modified: 2024-01-30  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi