• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Building World Knowledge by Grounding Language and Multimedia

Research Project

Project/Area Number 19H04166
Research Category

Grant-in-Aid for Scientific Research (B)

Allocation TypeSingle-year Grants
Section一般
Review Section Basic Section 61030:Intelligent informatics-related
Research InstitutionThe University of Tokyo

Principal Investigator

Hideki Nakayama  東京大学, 大学院情報理工学系研究科, 准教授 (00643305)

Project Period (FY) 2019-04-01 – 2023-03-31
Project Status Completed (Fiscal Year 2022)
Budget Amount *help
¥17,160,000 (Direct Cost: ¥13,200,000、Indirect Cost: ¥3,960,000)
Fiscal Year 2022: ¥3,510,000 (Direct Cost: ¥2,700,000、Indirect Cost: ¥810,000)
Fiscal Year 2021: ¥4,030,000 (Direct Cost: ¥3,100,000、Indirect Cost: ¥930,000)
Fiscal Year 2020: ¥4,030,000 (Direct Cost: ¥3,100,000、Indirect Cost: ¥930,000)
Fiscal Year 2019: ¥5,590,000 (Direct Cost: ¥4,300,000、Indirect Cost: ¥1,290,000)
Keywords画像認識 / 自然言語処理 / 知識獲得 / 知識グラフ / 世界知識 / マルチメディア / マルチモーダル / 人工知能 / ゼロショット認識 / 未来予測 / 少量データ学習 / グラウンディング / 深層学習 / シーングラフ / ストーリー生成 / 教師なし学習
Outline of Research at the Start

言語を理解し操るための背景となる常識的知識(世界知識)をいかにしてコンピュータに与えるかは、自然言語処理における古くからの難問である。本研究では、画像・動画像等のマルチメディアをマイニングし、世界知識を構築する新しいアプローチを提案する。画像や動画像は実世界の観察そのものであり、陽にテキスト化されにくい日常的な物事の時空間的な共起関係や因果関係に関する豊富な情報を含んでいるため、テキストマイニングで得られる世界知識を強力に補完するものになると期待できる。

Outline of Final Research Achievements

In this research, we developed a series of methods for mining commonsense knowledge about various concepts from multimedia data such as images, videos, and text, preparing it as a database with a graph structure (named Multimedia World Knowledge), and utilizing it for various artificial intelligence tasks. These results have been presented at a total of 27 international conferences and international journals (9 of which were accepted by top conferences), and have been highly acclaimed internationally.

Academic Significance and Societal Importance of the Research Achievements

画像やテキストの認識・生成を行う人工知能は近年驚異的な進歩を遂げ、社会的にも大きなインパクトを与えている。その原動力となったのは、大量のデータから学習した大規模言語モデルであるが、本研究では知識グラフという異なる形式の知識基盤の獲得方法とその有効性を示した点に独創性がある。知識グラフは、大規模言語モデルと併用することで人工知能の性能をさらに向上できる可能性があると同時に、解釈性や可制御性に優れた表現であるため、今後の社会でより信頼性の高い人工知能技術を実装・展開するための重要なアプローチになると考えられる。

Report

(5 results)
  • 2022 Annual Research Report   Final Research Report ( PDF )
  • 2021 Annual Research Report
  • 2020 Annual Research Report
  • 2019 Annual Research Report
  • Research Products

    (75 results)

All 2023 2022 2021 2020 2019 Other

All Int'l Joint Research (3 results) Journal Article (30 results) (of which Int'l Joint Research: 6 results,  Peer Reviewed: 30 results,  Open Access: 18 results) Presentation (40 results) (of which Int'l Joint Research: 31 results,  Invited: 6 results) Book (1 results) Remarks (1 results)

  • [Int'l Joint Research] University of California, Los Angeles/Amazon(米国)

    • Related Report
      2022 Annual Research Report
  • [Int'l Joint Research] National Yang Ming Chiao Tung University/Academia Sinica/National Taiwan University(その他の国・地域)

    • Related Report
      2022 Annual Research Report
  • [Int'l Joint Research] ニューヨーク大学(米国)

    • Related Report
      2019 Annual Research Report
  • [Journal Article] A-CAP: Anticipation Captioning with Commonsense Knowledge2023

    • Author(s)
      Duc Minh Vo, Quoc-An Luong, Akihiro Sugimoto, Hideki Nakayama
    • Journal Title

      Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

      Volume: -

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] LED: A Dataset for Life Event Extraction from Dialogs2023

    • Author(s)
      Yi-Pei Chen, An-Zi Yen, Hen-Hsen Huang, Hideki Nakayama, Hsin-Hsi Chen
    • Journal Title

      Findings of the Association for Computational Linguistics: EACL 2023

      Volume: - Pages: 384-398

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Indirect Adversarial Losses via an Intermediate Distribution for Training GANs2023

    • Author(s)
      Rui Yang, Duc Minh Vo, Hideki Nakayama
    • Journal Title

      Proceedings of the 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)

      Volume: - Pages: 4641-4650

    • DOI

      10.1109/wacv56688.2023.00463

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Improving Noised Gradient Penalty with Synchronized Activation Function for Generative Adversarial Networks2022

    • Author(s)
      Rui Yang, Raphael Shu, Hideki Nakayama
    • Journal Title

      IEICE Transactions on Information and Systems

      Volume: E105.D Issue: 9 Pages: 1537-1545

    • DOI

      10.1587/transinf.2022EDP7019

    • ISSN
      0916-8532, 1745-1361
    • Year and Date
      2022-09-01
    • Related Report
      2022 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Character-Centric Story Visualization via Visual Planning and Token Alignment2022

    • Author(s)
      Hong Chen, Rujun Han, Te-Lin Wu, Hideki Nakayama and Nanyun Peng
    • Journal Title

      Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP)

      Volume: - Pages: 8259-8272

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] StoryER: Automatic Story Evaluation via Ranking, Rating and Reasoning2022

    • Author(s)
      Hong Chen, Duc Minh Vo, Hiroya Takamura, Yusuke Miyao, Hideki Nakayama
    • Journal Title

      Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP)

      Volume: - Pages: 1739-1753

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Weakly Supervised Formula Learner for Solving Mathematical Problems2022

    • Author(s)
      Yuxuan Wu, Hideki Nakayama
    • Journal Title

      Proceedings of the 29th International Conference on Computational Linguistics (COLING)

      Volume: - Pages: 1743-1752

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Neural Networks in a Product of Hyperbolic Spaces2022

    • Author(s)
      Jun Takeuchi, Noriki Nishida, Hideki Nakayama
    • Journal Title

      Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Student Research Workshop

      Volume: - Pages: 211-221

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] DJMix: Unsupervised Task-agnostic Image Augmentation for Improving Robustness of Convolutional Neural Networks"2022

    • Author(s)
      Ryuichiro Hataya, Hideki Nakayama
    • Journal Title

      Proceedings of the 2022 International Joint Conference on Neural Networks (IJCNN)

      Volume: - Pages: 1-8

    • DOI

      10.1109/ijcnn55064.2022.9892068

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Pixel to Binary Embedding Towards Robustness for CNNs2022

    • Author(s)
      Ikki Kishida, Hideki Nakayama
    • Journal Title

      Proceedings of the 26th International Conference on Pattern Recognition (ICPR)

      Volume: - Pages: 2279-2285

    • DOI

      10.1109/icpr56361.2022.9956572

    • Related Report
      2022 Annual Research Report 2021 Annual Research Report
    • Peer Reviewed
  • [Journal Article] NOC-REK: Novel Object Captioning with Retrieved Vocabulary from External Knowledge2022

    • Author(s)
      Duc Minh Vo, Hong Chen, Akihiro Sugimoto, Hideki Nakayama
    • Journal Title

      2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

      Volume: - Pages: 17979-17987

    • DOI

      10.1109/cvpr52688.2022.01747

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] OSSGAN: Open-Set Semi-Supervised Image Generation2022

    • Author(s)
      Kai Katsumata, Duc Minh Vo, Hideki Nakayama
    • Journal Title

      Proceedings of the 2022 IEEE/CVF Computer Vision and Pattern Recognition Conference (CVPR)

      Volume: なし Pages: 11175-11183

    • DOI

      10.1109/cvpr52688.2022.01090

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] PPCD-GAN: Progressive Pruning and Class-Aware Distillation for Large-Scale Conditional GANs Compression2022

    • Author(s)
      Duc Minh Vo, Akihiro Sugimoto, Hideki Nakayama
    • Journal Title

      Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)

      Volume: - Pages: 1422-1430

    • DOI

      10.1109/wacv51458.2022.00149

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Meta Approach to Data Augmentation Optimization2022

    • Author(s)
      Ryuichiro Hataya, Jan Zdenek, Kazuki Yoshizoe, Hideki Nakayama
    • Journal Title

      Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)

      Volume: - Pages: 3535-3544

    • DOI

      10.1109/wacv51458.2022.00359

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] NOC-REK: Novel Object Captioning with Retrieved Vocabulary from External Knowledge2022

    • Author(s)
      Duc Minh Vo, Hong Chen, Akihiro Sugimoto, Hideki Nakayama
    • Journal Title

      Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

      Volume: -

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] OSSGAN: Open-Set Semi-Supervised Image Generation2022

    • Author(s)
      Kai Katsumata, Duc Minh Vo, Hideki Nakayama
    • Journal Title

      Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

      Volume: -

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] DCT-based Fast Spectral Convolution for Deep Convolutional Neural Networks2021

    • Author(s)
      Yuhao Xu and Hideki Nakayama
    • Journal Title

      Proceedings of the International Joint Conference on Neural Networks (IJCNN)

      Volume: - Pages: 1-8

    • DOI

      10.1109/ijcnn52387.2021.9534135

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed
  • [Journal Article] GraphPlan: Story Generation by Planning with Event Graph2021

    • Author(s)
      Hong Chen, Raphael Shu, Hiroya Takamura, Hideki Nakayama
    • Journal Title

      Proceedings of the 14th International Conference on Natural Language Generation (INLG)

      Volume: - Pages: 377-386

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] JokerGAN: Memory-Efficient Model for Handwritten Text Generation with Text Line Awareness2021

    • Author(s)
      Jan Zdenek and Hideki Nakayama
    • Journal Title

      Proceedings of the 29th ACM International Conference on Multimedia (ACMMM)

      Volume: - Pages: 5655-5663

    • DOI

      10.1145/3474085.3475713

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed
  • [Journal Article] SciXGen: A Scientific Paper Dataset for Context-Aware Text Generation2021

    • Author(s)
      Hong Chen, Hiroya Takamura, Hideki Nakayama
    • Journal Title

      Findings of the Association for Computational Linguistics: EMNLP 2021

      Volume: - Pages: 1483-1492

    • DOI

      10.18653/v1/2021.findings-emnlp.128

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Commonsense Knowledge Aware Concept Selection for Diverse and Informative Visual Storytelling2021

    • Author(s)
      Hong Chen, Yifei Huang, Hiroya Takamura, and Hideki Nakayama
    • Journal Title

      Proceedings of the 35th AAAI Conference on Artificial Intelligence (AAAI)

      Volume: -

    • Related Report
      2020 Annual Research Report
    • Peer Reviewed
  • [Journal Article] DCT-based Fast Spectral Convolution for Deep Convolutional Neural Networks2021

    • Author(s)
      Yuhao Xu, Hideki Nakayama
    • Journal Title

      Proceedings of the International Joint Conference on Neural Networks (IJCNN)

      Volume: -

    • Related Report
      2020 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Faster AutoAugment: Learning Augmentation Strategies using Backpropagation2020

    • Author(s)
      Ryuichiro Hataya, Zdenek Jan, Kazuki Yoshizoe, Hideki Nakayama
    • Journal Title

      Computer Vision - ECCV 2020

      Volume: LNCS 12370 Pages: 1-16

    • DOI

      10.1007/978-3-030-58595-2_1

    • NAID

      130007857024

    • ISBN
      9783030585945, 9783030585952
    • Related Report
      2020 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Unsupervised Visual Relationship Inference2020

    • Author(s)
      Taiga Kashima, Kento Masui, Hideki Nakayama
    • Journal Title

      Proceedings of the IEEE International Conference on Image Processing (ICIP)

      Volume: - Pages: 1476-1480

    • DOI

      10.1109/icip40778.2020.9190770

    • Related Report
      2020 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Latent-Variable Non-Autoregressive Neural Machine Translation with Deterministic Inference Using a Delta Posterior2020

    • Author(s)
      Raphael Shu, Jason Lee, Hideki Nakayama, Kyunghyun Cho
    • Journal Title

      Proceedings of the 34th AAAI Conference on Artificial Intelligence (AAAI)

      Volume: -

    • Related Report
      2019 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] LOL: Learning To Optimize Loss Switching Under Label Noise2019

    • Author(s)
      Ryuichiro Hataya, Hideki Nakayama
    • Journal Title

      Proceedings of the 26th IEEE International Conference on Image Processing (ICIP)

      Volume: - Pages: 3621-3625

    • DOI

      10.1109/icip.2019.8803512

    • Related Report
      2019 Annual Research Report
    • Peer Reviewed
  • [Journal Article] DCT Based Information-Preserving Pooling for Deep Neural Networks2019

    • Author(s)
      Yuhao Xu, Hideki Nakayama
    • Journal Title

      Proceedings of the 26th IEEE International Conference on Image Processing (ICIP)

      Volume: - Pages: 894-898

    • DOI

      10.1109/icip.2019.8802962

    • Related Report
      2019 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Bipolar Gan: Double Check the Solution Space and Lighten False Positive Errors in Generative Adversarial Nets2019

    • Author(s)
      Rui Yang, Hideki Nakayama
    • Journal Title

      Proceedings of the 26th IEEE International Conference on Image Processing (ICIP)

      Volume: - Pages: 4260-4264

    • DOI

      10.1109/icip.2019.8803613

    • Related Report
      2019 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Empirical Study of Easy and Hard Examples in CNN Training2019

    • Author(s)
      Ikki Kishida, Hideki Nakayama
    • Journal Title

      Proceedings of the 26th International Conference on Neural Information Processing (ICONIP 2019), Communications in Computer and Information Science

      Volume: 1142 Pages: 179-188

    • DOI

      10.1007/978-3-030-36808-1_20

    • ISBN
      9783030368074, 9783030368081
    • Related Report
      2019 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Shifted Spatial-Spectral Convolution for Deep Neural Networks2019

    • Author(s)
      Yuhao Xu, Hideki Nakayama
    • Journal Title

      Proceedings of the 1st ACM International Conference on Multimedia in Asia (MMAsia)

      Volume: - Pages: 1-6

    • DOI

      10.1145/3338533.3366575

    • Related Report
      2019 Annual Research Report
    • Peer Reviewed
  • [Presentation] A-CAP: Anticipation Captioning with Commonsense Knowledge2023

    • Author(s)
      Duc Minh Vo, Quoc-An Luong, Akihiro Sugimoto, Hideki Nakayama
    • Organizer
      The 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Indirect Adversarial Losses via an Intermediate Distribution for Training GANs2023

    • Author(s)
      Rui Yang, Duc Minh Vo, Hideki Nakayama
    • Organizer
      The 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Character-Centric Story Visualization via Visual Planning and Token Alignment2022

    • Author(s)
      Hong Chen, Rujun Han, Te-Lin Wu, Hideki Nakayama and Nanyun Peng
    • Organizer
      The 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP)
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] StoryER: Automatic Story Evaluation via Ranking, Rating and Reasoning2022

    • Author(s)
      Hong Chen, Duc Minh Vo, Hiroya Takamura, Yusuke Miyao, Hideki Nakayama
    • Organizer
      The 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP)
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Weakly Supervised Formula Learner for Solving Mathematical Problems2022

    • Author(s)
      Yuxuan Wu, Hideki Nakayama
    • Organizer
      The 29th International Conference on Computational Linguistics (COLING)
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Neural Networks in a Product of Hyperbolic Spaces2022

    • Author(s)
      Jun Takeuchi, Noriki Nishida, Hideki Nakayama
    • Organizer
      The 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Student Research Workshop
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] DJMix: Unsupervised Task-agnostic Image Augmentation for Improving Robustness of Convolutional Neural Networks"2022

    • Author(s)
      Ryuichiro Hataya, Hideki Nakayama
    • Organizer
      The 2022 International Joint Conference on Neural Networks (IJCNN)
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Pixel to Binary Embedding Towards Robustness for CNNs2022

    • Author(s)
      Ikki Kishida, Hideki Nakayama
    • Organizer
      The 26th International Conference on Pattern Recognition (ICPR)
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] NOC-REK: Novel Object Captioning with Retrieved Vocabulary from External Knowledge2022

    • Author(s)
      Duc Minh Vo, Hong Chen, Akihiro Sugimoto, Hideki Nakayama
    • Organizer
      The 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] OSSGAN: Open-Set Semi-Supervised Image Generation2022

    • Author(s)
      Kai Katsumata, Duc Minh Vo, Hideki Nakayama
    • Organizer
      The 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] ラベルノイズ付きオープンセット半教師あり画像生成2022

    • Author(s)
      勝又 海, Duc Minh Vo, 原田 達也, 中山 英樹
    • Organizer
      第25回 画像の認識・理解シンポジウム
    • Related Report
      2022 Annual Research Report
  • [Presentation] Incorporating External Knowledge for Vision and Language Systems2022

    • Author(s)
      Hideki Nakayama
    • Organizer
      2nd Workshop on Trends and Advances in Machine Learning and Automated Reasoning for Intelligent Robots and Systems (in conjunction with IROS 2022)
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research / Invited
  • [Presentation] PPCD-GAN: Progressive Pruning and Class-Aware Distillation for Large-Scale Conditional GANs Compression2022

    • Author(s)
      Duc Minh Vo, Akihiro Sugimoto, Hideki Nakayama
    • Organizer
      IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)
    • Related Report
      2021 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Meta Approach to Data Augmentation Optimization2022

    • Author(s)
      Ryuichiro Hataya, Jan Zdenek, Kazuki Yoshizoe, Hideki Nakayama
    • Organizer
      IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)
    • Related Report
      2021 Annual Research Report
    • Int'l Joint Research
  • [Presentation] NOC-REK: Novel Object Captioning with Retrieved Vocabulary from External Knowledge2022

    • Author(s)
      Duc Minh Vo, Hong Chen, Akihiro Sugimoto, Hideki Nakayama
    • Organizer
      IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
    • Related Report
      2021 Annual Research Report
    • Int'l Joint Research
  • [Presentation] OSSGAN: Open-Set Semi-Supervised Image Generation2022

    • Author(s)
      Kai Katsumata, Duc Minh Vo, Hideki Nakayama
    • Organizer
      Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
    • Related Report
      2021 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Pixel to Binary Embedding Towards Robustness for CNNs2022

    • Author(s)
      Ikki Kishida, Hideki Nakayama
    • Organizer
      International Conference on Pattern Recognition (ICPR)
    • Related Report
      2021 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Efficient Training of Neural Module Networks and Applications2021

    • Author(s)
      Hideki Nakayama
    • Organizer
      Fifth International Workshop on Symbolic-Neural Learning (SNL-2021)
    • Related Report
      2021 Annual Research Report
    • Int'l Joint Research / Invited
  • [Presentation] DCT-based Fast Spectral Convolution for Deep Convolutional Neural Networks2021

    • Author(s)
      Yuhao Xu and Hideki Nakayama
    • Organizer
      International Joint Conference on Neural Networks (IJCNN)
    • Related Report
      2021 Annual Research Report
    • Int'l Joint Research
  • [Presentation] GraphPlan: Story Generation by Planning with Event Graph2021

    • Author(s)
      Hong Chen, Raphael Shu, Hiroya Takamura, Hideki Nakayama
    • Organizer
      International Conference on Natural Language Generation (INLG)
    • Related Report
      2021 Annual Research Report
    • Int'l Joint Research
  • [Presentation] JokerGAN: Memory-Efficient Model for Handwritten Text Generation with Text Line Awareness2021

    • Author(s)
      Jan Zdenek and Hideki Nakayama
    • Organizer
      ACM International Conference on Multimedia (ACMMM)
    • Related Report
      2021 Annual Research Report
    • Int'l Joint Research
  • [Presentation] SciXGen: A Scientific Paper Dataset for Context-Aware Text Generation2021

    • Author(s)
      Hong Chen, Hiroya Takamura, Hideki Nakayama
    • Organizer
      Conference on Empirical Methods in Natural Language Processing (EMNLP)
    • Related Report
      2021 Annual Research Report
    • Int'l Joint Research
  • [Presentation] 深層学習におけるデータ拡張の原理と最新動向2021

    • Author(s)
      中山 英樹, 幡谷龍一郎
    • Organizer
      第27回画像センシングシンポジウム OS2:続・限られたデータからの深層学習
    • Related Report
      2021 Annual Research Report
    • Invited
  • [Presentation] Commonsense Knowledge Aware Concept Selection for Diverse and Informative Visual Storytelling2021

    • Author(s)
      Hong Chen, Yifei Huang, Hiroya Takamura, and Hideki Nakayama
    • Organizer
      AAAI Conference on Artificial Intelligence
    • Related Report
      2020 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Commonsense Knowledge Aware Concept Selection For Diverse and Informative Visual Storytelling2021

    • Author(s)
      Hong Chen, Yifei Huang, Hiroya Takamura, and Hideki Nakayama
    • Organizer
      言語処理学会 第27回年次大会
    • Related Report
      2020 Annual Research Report
  • [Presentation] Faster AutoAugment: Learning Augmentation Strategies using Backpropagation2020

    • Author(s)
      Ryuichiro Hataya, Zdenek Jan, Kazuki Yoshizoe, Hideki Nakayama
    • Organizer
      European Conference on Computer Vision (ECCV)
    • Related Report
      2020 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Unsupervised Visual Relationship Inference2020

    • Author(s)
      Taiga Kashima, Kento Masui, Hideki Nakayama
    • Organizer
      IEEE International Conference on Image Processing (ICIP)
    • Related Report
      2020 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Incorporating Horizontal Connections in Convolution by Spatial Shuffling2020

    • Author(s)
      Ikki Kishida, Hideki Nakayama
    • Organizer
      第23回 画像の認識・理解シンポジウム (MIRU)
    • Related Report
      2020 Annual Research Report
  • [Presentation] 潜在変数の精緻化による非自己回帰型ニューラル機械翻訳2020

    • Author(s)
      朱中元, Jason Lee, Kyunghyun Cho, 中山英樹
    • Organizer
      言語処理学会 第26回年次大会
    • Related Report
      2019 Annual Research Report
  • [Presentation] Latent-Variable Non-Autoregressive Neural Machine Translation with Deterministic Inference Using a Delta Posterior2020

    • Author(s)
      Raphael Shu, Jason Lee, Hideki Nakayama, Kyunghyun Cho
    • Organizer
      The 34th AAAI Conference on Artificial Intelligence (AAAI 2020)
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Unifying semi-supervised and robust learning by mixup2019

    • Author(s)
      Ryuichiro Hataya, Hideki Nakayama
    • Organizer
      第22回 画像の認識・理解シンポジウム (MIRU2019)
    • Related Report
      2019 Annual Research Report
  • [Presentation] 深層学習の発展と応用2019

    • Author(s)
      中山英樹
    • Organizer
      応用物理学会 第152回微小光学研究会
    • Related Report
      2019 Annual Research Report
    • Invited
  • [Presentation] 機械学習とマルチメディア認識理解2019

    • Author(s)
      中山英樹
    • Organizer
      第4回 統計・機械学習若手シンポジウム
    • Related Report
      2019 Annual Research Report
    • Invited
  • [Presentation] マルチモーダル情報と機械翻訳2019

    • Author(s)
      中山英樹
    • Organizer
      アジア太平洋機械翻訳協会年次セミナー AAMT 2019, Tokyo ~機械翻訳最前線~
    • Related Report
      2019 Annual Research Report
    • Invited
  • [Presentation] LOL: Learning To Optimize Loss Switching Under Label Noise2019

    • Author(s)
      Ryuichiro Hataya, Hideki Nakayama
    • Organizer
      The 26th IEEE International Conference on Image Processing (ICIP 2019)
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research
  • [Presentation] DCT Based Information-Preserving Pooling for Deep Neural Networks2019

    • Author(s)
      Yuhao Xu, Hideki Nakayama
    • Organizer
      The 26th IEEE International Conference on Image Processing (ICIP 2019)
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Bipolar Gan: Double Check the Solution Space and Lighten False Positive Errors in Generative Adversarial Nets2019

    • Author(s)
      Rui Yang, Hideki Nakayama
    • Organizer
      The 26th IEEE International Conference on Image Processing (ICIP 2019)
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Empirical Study of Easy and Hard Examples in CNN Training2019

    • Author(s)
      Ikki Kishida, Hideki Nakayama
    • Organizer
      The 26th International Conference on Neural Information Processing (ICONIP 2019)
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Shifted Spatial-Spectral Convolution for Deep Neural Networks2019

    • Author(s)
      Yuhao Xu, Hideki Nakayama
    • Organizer
      The 1st ACM International Conference on Multimedia in Asia (MMAsia 2019)
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Unifying Semi-Supervised and Robust Learning by Mixup2019

    • Author(s)
      Ryuichiro Hataya, Hideki Nakayama
    • Organizer
      The 2nd Learning from Limited Labeled Data (LLD) Workshop, ICLR 2019
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research
  • [Book] 深層学習からマルチモーダル情報処理へ2022

    • Author(s)
      中山 英樹、二反田 篤史、田村 晃裕、井上 中順、牛久 祥孝
    • Total Pages
      248
    • Publisher
      サイエンス社
    • ISBN
      9784781915548
    • Related Report
      2022 Annual Research Report
  • [Remarks] 画像認識・自然言語処理に関する研究内容と成果

    • URL

      http://www.nlab.ci.i.u-tokyo.ac.jp/projects.html

    • Related Report
      2019 Annual Research Report

URL: 

Published: 2019-04-18   Modified: 2024-01-30  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi