• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

A mechanism on cross-modal interactions of synesthesia-Functional neuroimaging of colored-hearing synesthesia and digital media

Research Project

Project/Area Number 21300086
Research Category

Grant-in-Aid for Scientific Research (B)

Allocation TypeSingle-year Grants
Section一般
Research Field Sensitivity informatics/Soft computing
Research InstitutionKwansei Gakuin University

Principal Investigator

NAGATA Noriko  関西学院大学, 理工学部, 教授 (50368453)

Co-Investigator(Kenkyū-buntansha) FUJISAWA Takashi  長崎大学, 医歯薬学総合研究科, 助教 (90434894)
INOKUCHI Seiji  宝塚大学, 造形芸術学部, 講師 (90029463)
KURODA Kagayaki  東海大学, 情報理工学部, 教授 (70205243)
AIBA Eriko  関西学院大学, 理工学部, 研究員 (40569761)
SHIMOTOMAI Takayuki  玉川大学, 脳科学研究所, 特別研究員 (50415361)
Project Period (FY) 2009 – 2011
Project Status Completed (Fiscal Year 2011)
Budget Amount *help
¥10,660,000 (Direct Cost: ¥8,200,000、Indirect Cost: ¥2,460,000)
Fiscal Year 2011: ¥3,120,000 (Direct Cost: ¥2,400,000、Indirect Cost: ¥720,000)
Fiscal Year 2010: ¥3,380,000 (Direct Cost: ¥2,600,000、Indirect Cost: ¥780,000)
Fiscal Year 2009: ¥4,160,000 (Direct Cost: ¥3,200,000、Indirect Cost: ¥960,000)
Keywords感性情報処理 / 計測工学 / 脳 / 神経 / マルチモーダルインタフェース / 共感覚 / 色聴 / fMRI / 感性情報学 / 脳・神経
Research Abstract

Synesthesia is an interesting phenomenon in which the stimulation of one sensory modality involuntarily elicits a perception in another modality. In this study, we focused on one typical synesthesia : seeing colors while listening to sounds, which is called "colored hearing." We examined the brain's mechanism with neuroimaging technology. Furthermore, we applied the synesthesia phenomenon to digital media creations, and we achieved scientific-based interactive arts between music and colors. The results of the research have been made public through mass media, including TV and press reports, and we succeeded in helping the public understand synesthesia better.

Report

(4 results)
  • 2011 Annual Research Report   Final Research Report ( PDF )
  • 2010 Annual Research Report
  • 2009 Annual Research Report
  • Research Products

    (59 results)

All 2012 2011 2010 2009 Other

All Journal Article (13 results) (of which Peer Reviewed: 10 results) Presentation (33 results) Book (1 results) Remarks (12 results)

  • [Journal Article] 和音進行による情動の知覚: fMRIおよび印象評価によるアプローチ2012

    • Author(s)
      饗庭絵里子・田中里弥・藤澤隆史・赤塚諭・下斗米貴之・長田典子
    • Journal Title

      音楽知覚認知研究

    • NAID

      40019636175

    • Related Report
      2011 Final Research Report
    • Peer Reviewed
  • [Journal Article] 和音進行による情動の知覚:fMRIおよび印象評価によるアプローチ2012

    • Author(s)
      長田典子, 饗庭絵里子
    • Journal Title

      音楽知覚認知研究

      Volume: (accepted)

    • NAID

      40019636175

    • Related Report
      2011 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Identification of Motion Features Affecting Perceived Rhythmic Sense of Virtual Characters through Comparison of Latin American and Japanese Dances2011

    • Author(s)
      Iwai, D.
    • Journal Title

      映像情報メディア学会誌

      Volume: 65(2) Pages: 203-210

    • Related Report
      2010 Annual Research Report
    • Peer Reviewed
  • [Journal Article] 音を聴くと色が見える:共感覚のクロスモダリティ2010

    • Author(s)
      長田典子
    • Journal Title

      日本色彩学会誌

      Volume: 34(4) Pages: 348-353

    • NAID

      110007989409

    • Related Report
      2011 Final Research Report 2010 Annual Research Report
  • [Journal Article] メディア分野における多次元センシング技術の現状と動向2010

    • Author(s)
      長田典子・大城英裕・加藤邦人・輿水大和・佐川立昌・藤原孝幸, 他
    • Journal Title

      電気学会論文誌D

      Volume: 131(4) Pages: 433-440

    • NAID

      10027980297

    • Related Report
      2011 Final Research Report 2010 Annual Research Report
    • Peer Reviewed
  • [Journal Article] バーチャル夜光虫:水の流れを媒介とした光と音のインタラクション2010

    • Author(s)
      相田恭兵
    • Journal Title

      インタラクション2010論文集

      Pages: 187-190

    • Related Report
      2009 Annual Research Report
    • Peer Reviewed
  • [Journal Article] 共感覚の脳機能イメージング2009

    • Author(s)
      長田典子・藤澤隆史
    • Journal Title

      システム制御情報学会誌

      Volume: 53(4) Pages: 149-154

    • Related Report
      2011 Final Research Report
  • [Journal Article] Virtual Noctiluca : Interaction between Light and Water using Real-time Fluid Simulation and 3D Motion Measurement2009

    • Author(s)
      Aida, K., & Nagata, N.
    • Journal Title

      ICEC2009, Lecture Notes in Computer Science

      Volume: 5709 Pages: 157-166

    • Related Report
      2011 Final Research Report
    • Peer Reviewed
  • [Journal Article] Dance Motion Control of a Humanoid Robot Based on Real-Time Tempo Tracking from Musical Audio Signals2009

    • Author(s)
      Nakahara, N., Miyazaki, K., Sakamoto, H., Fujisawa, T. X., Nagata, N., & Nakatsu, R.
    • Journal Title

      ICEC2009, Lecture Notes in Computer Science

      Volume: 5709 Pages: 36-47

    • Related Report
      2011 Final Research Report
    • Peer Reviewed
  • [Journal Article] Dance Motion Control of a Humanoid Robot Based on Real-Time Tempo Tracking from Musical Audio Signals.2009

    • Author(s)
      Nakahara, N.
    • Journal Title

      ICEC2009, Lecture Notes in Computer Science 5709

      Pages: 36-47

    • Related Report
      2009 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Virtual Noctiluca : Interaction between Light and Water using Real-time Fluid Simulation and 3D Motion Measurement.2009

    • Author(s)
      Aida, K.
    • Journal Title

      ICEC2009, Lecture Notes in Computer Science 5709

      Pages: 157-166

    • Related Report
      2009 Annual Research Report
    • Peer Reviewed
  • [Journal Article] 音楽情報を用いたロボットのダンスモーションのリアルタイム制御2009

    • Author(s)
      中原直人
    • Journal Title

      インタラクション2009論文集

      Pages: 137-138

    • Related Report
      2009 Annual Research Report
    • Peer Reviewed
  • [Journal Article] 共感覚の脳機能イメージング2009

    • Author(s)
      長田典子
    • Journal Title

      システム制御情報学会誌 53(4)

      Pages: 149-154

    • Related Report
      2009 Annual Research Report
  • [Presentation] 色字共感覚の脳内メカニズム解明:日本語文字呈示時におけるfMRI研究2012

    • Author(s)
      真崎大・矢山隆三・赤塚諭・下斗米貴之・饗庭絵里子・長田典子
    • Organizer
      日本認知心理学会第10回大会
    • Place of Presentation
      岡山大学
    • Year and Date
      2012-06-02
    • Related Report
      2011 Final Research Report
  • [Presentation] 色字共感覚の脳内メカニズム解明:日本語文字呈示時におけるfMRI研究2012

    • Author(s)
      真崎大・矢山隆三・赤塚諭・下斗米貴之・饗庭絵里子・長田典子
    • Organizer
      日本認知心理学会第10回大会
    • Place of Presentation
      岡山大学(発表確定)
    • Year and Date
      2012-06-02
    • Related Report
      2011 Annual Research Report
  • [Presentation] 和音進行聴取における脳機能計測および印象評価2012

    • Author(s)
      高瀬雅良・饗庭絵里子・田中里弥・藤澤隆史・赤塚諭・下斗米貴之・長田典子
    • Organizer
      日本認知心理学会第10回大会
    • Place of Presentation
      岡山大学(発表確定)
    • Year and Date
      2012-06-02
    • Related Report
      2011 Annual Research Report
  • [Presentation] 和音の色彩へのマッピングに関する分析2012

    • Author(s)
      下斗米貴之・饗庭絵里子・三軒谷友美・長田典子・大森隆司
    • Organizer
      第7回日本感性工学会春季大会
    • Place of Presentation
      サンポート・ホール(香川)
    • Year and Date
      2012-03-02
    • Related Report
      2011 Annual Research Report 2011 Final Research Report
  • [Presentation] 色聴共感覚の脳機能イメージング2011

    • Author(s)
      長田典子
    • Organizer
      日本心理学会ワークショップ「共感覚と共感覚的認知」
    • Place of Presentation
      日本大学
    • Year and Date
      2011-09-06
    • Related Report
      2011 Annual Research Report 2011 Final Research Report
  • [Presentation] Brain activity in grapheme-color synesthetes : when seeing japanese letters : an fMRI study2011

    • Author(s)
      Mazaki, H., Nishimoto, M., Akatsuka, S., Yayama, R., Shimotomai, T., Aiba, E., & Nagata, N.
    • Organizer
      16th Annual Meeting of the Organization for Human Brain Mapping(OHBM2011)
    • Place of Presentation
      Quebec City, Canada
    • Year and Date
      2011-06-27
    • Related Report
      2011 Final Research Report
  • [Presentation] Brain Activity in the Color Area When Listening to Music : An fMRI Study of Synesthesia2011

    • Author(s)
      Yayama, R., Shimotomai, T., Takahashi, R., Akatsuka, S., Aiba, E., Fujisawa, T. X., & Nagata, N.
    • Organizer
      16th Annual Meeting of the Organization for Human Brain Mapping(OHBM2011)
    • Place of Presentation
      Quebec City, Canada
    • Year and Date
      2011-06-27
    • Related Report
      2011 Final Research Report
  • [Presentation] Brain activity in grapheme-color synesthetes : when seeing japanese letters : an fMRI study2011

    • Author(s)
      Mazaki, H., Nishimoto, M., Akatsuka, S., Yayama, R., Shimotomai, T., Aiba, E., Nagata, N.
    • Organizer
      16th Annual Meeting of the Organization for Human Brain Mapping (OHBM2011)
    • Place of Presentation
      Quebec City, Canada
    • Year and Date
      2011-06-27
    • Related Report
      2011 Annual Research Report
  • [Presentation] Brain Activity in the Color Area When Listening to Music : An fMRI Study of Synesthesia2011

    • Author(s)
      Yayama, R., Shimotomai, T., Takahashi, R., Akatsuka, S., Aiba, E., Fujisawa, T.X., Nagata, N.
    • Organizer
      16th Annual Meeting of the Organization for Human Brain Mapping (OHBM2011)
    • Place of Presentation
      Quebec City, Canada
    • Year and Date
      2011-06-27
    • Related Report
      2011 Annual Research Report
  • [Presentation] Synesthetic cross-modal sound-color mapping2011

    • Author(s)
      Wang, J., Cai, D., Asai, N., Nagata, N., & Fukumoto, A.
    • Organizer
      The 15th annual meeting of the The Association for the Scientific Study of Consciousness
    • Place of Presentation
      Kyoto, Japan
    • Year and Date
      2011-06-09
    • Related Report
      2011 Annual Research Report 2011 Final Research Report
  • [Presentation] Synesthetic Sound-Color Cross-Modality in Animations2011

    • Author(s)
      Cai, D., Goto, S., Wang, J. P., Asai, N., Nagata, N., Fukumoto, A., & Kurumizawa, J.
    • Organizer
      Eurographics 2011 : posters
    • Place of Presentation
      Llandudno, UK
    • Year and Date
      2011-04-15
    • Related Report
      2011 Final Research Report
  • [Presentation] Synesthetic Sound-Color Cross-Modality in Animations2011

    • Author(s)
      Cai, D., Goto, S., Wang, J.P., Asai, N., Nagata, N., Fukumoto, A., Kurumizawa, J.
    • Organizer
      Eurographics 2011
    • Place of Presentation
      Llandudno, UK
    • Year and Date
      2011-04-15
    • Related Report
      2011 Annual Research Report
  • [Presentation] trace (cross-wiring)2011

    • Author(s)
      四宮優
    • Organizer
      The lounge (Dandans Exhibition No.7)
    • Place of Presentation
      ブルガリ銀座タワー(東京)
    • Year and Date
      2011-01-13
    • Related Report
      2010 Annual Research Report
  • [Presentation] 共感覚の脳機能イメージング2010

    • Author(s)
      長田典子
    • Organizer
      音楽と科学研究会
    • Place of Presentation
      相愛大学(大阪)
    • Year and Date
      2010-10-30
    • Related Report
      2011 Final Research Report
  • [Presentation] 共感覚の脳機能イメージング2010

    • Author(s)
      長田典子
    • Organizer
      音楽の科学研究会
    • Place of Presentation
      相愛大学(大阪)
    • Year and Date
      2010-10-30
    • Related Report
      2010 Annual Research Report
  • [Presentation] Synesthetic Color Scheme in Fantasia2010

    • Author(s)
      Cai, D. S., Goto, S., Shinohara, T., Nagata, N., Fukumoto, A., Kurumisawa, J., & Asai, N.
    • Organizer
      ACM SIGGRAPH 2010 : Talks, Visualization for Art & Design
    • Place of Presentation
      Los angeles. USA
    • Year and Date
      2010-07-27
    • Related Report
      2011 Final Research Report
  • [Presentation] Synesthetic Color Scheme in Fantasia2010

    • Author(s)
      Cai, D.S.
    • Organizer
      ACM SIGGRAPH 2010 : Talks, Visualization for Art & Design
    • Place of Presentation
      Los angeles
    • Year and Date
      2010-07-27
    • Related Report
      2010 Annual Research Report
  • [Presentation] Measurement of Brain Function When Listening to Chords : fMRI Study2010

    • Author(s)
      Akatsuka, S., Fujisawa, T. X., Aiba, E., & Nagata, N.
    • Organizer
      16th Annual Meeting of the Organization for Human Brain Mapping
    • Place of Presentation
      Barcelona. SPAIN
    • Year and Date
      2010-06-26
    • Related Report
      2011 Final Research Report
  • [Presentation] Measurement of Brain Function When Listening to Chords : fMRI Study2010

    • Author(s)
      Akatsuka, S.
    • Organizer
      16th Annual Meeting of the Organization for Human Brain Mapping
    • Place of Presentation
      Barcelona
    • Year and Date
      2010-06-26
    • Related Report
      2010 Annual Research Report
  • [Presentation] 調和度に基づく和音性評価モデルと楽曲ムードの色彩表現インタフェース2010

    • Author(s)
      谷光彬・饗庭絵里子・下斗米貴之・長田典子・藤澤隆史
    • Organizer
      日本音響学会音楽音響研究会
    • Place of Presentation
      筑波大学
    • Year and Date
      2010-03-11
    • Related Report
      2011 Final Research Report
  • [Presentation] 調和度に基づく和音性評価モデルと楽曲ムードの色彩表現インタフェース2010

    • Author(s)
      谷光彬
    • Organizer
      日本音響学会 音楽音響研究会
    • Place of Presentation
      筑波大学
    • Year and Date
      2010-03-11
    • Related Report
      2009 Annual Research Report
  • [Presentation] バーチャル夜光虫:水の流れを媒介とした光と音のインタラクション2010

    • Author(s)
      相田恭兵・長田典子
    • Organizer
      インタラクション2010
    • Place of Presentation
      一橋記念講堂,東京
    • Year and Date
      2010-03-01
    • Related Report
      2011 Final Research Report
  • [Presentation] 音と色のマルチモーダルインタラクション2010

    • Author(s)
      長田典子
    • Organizer
      画像応用技術専門委員会研究会、精密工学会
    • Place of Presentation
      東京電機大学
    • Year and Date
      2010-01-15
    • Related Report
      2011 Final Research Report
  • [Presentation] 音と色のマルチモーダルインタラクション2010

    • Author(s)
      長田典子
    • Organizer
      精密工学会 画像応用技術専門委員会研究会
    • Place of Presentation
      東京電機大学
    • Year and Date
      2010-01-15
    • Related Report
      2009 Annual Research Report
  • [Presentation] コミュニケーションと感性2009

    • Author(s)
      長田典子
    • Organizer
      ヒューマンインタフェース学会コミュニケーション支援専門研究会
    • Place of Presentation
      関西学院大学大阪梅田キャンパス
    • Year and Date
      2009-11-28
    • Related Report
      2011 Final Research Report 2009 Annual Research Report
  • [Presentation] ディジタルメディアの感性的リアリティを支援する技術2009

    • Author(s)
      長田典子
    • Organizer
      兵庫エレクトロニクス研究会
    • Place of Presentation
      兵庫県立工業技術センター
    • Year and Date
      2009-11-19
    • Related Report
      2009 Annual Research Report
  • [Presentation] ひと・メディア・インタラクション2009

    • Author(s)
      長田典子
    • Organizer
      KMDvs.KHMトークバトル
    • Place of Presentation
      関西学院大学上ヶ原キャンパス
    • Year and Date
      2009-11-14
    • Related Report
      2009 Annual Research Report
  • [Presentation] 人の感覚・感性を支援する技術2009

    • Author(s)
      長田典子
    • Organizer
      (社)日本鉄鋼協会第142回制御技術部会
    • Place of Presentation
      神戸製鋼所加古川製鉄所
    • Year and Date
      2009-11-13
    • Related Report
      2011 Final Research Report
  • [Presentation] 人の感覚・感性を支援する技術2009

    • Author(s)
      長田典子
    • Organizer
      (社)日本鉄鋼協会 第142回制御技術部会
    • Place of Presentation
      神戸製鋼所加古川製鉄所
    • Year and Date
      2009-11-13
    • Related Report
      2009 Annual Research Report
  • [Presentation] 色字共感覚に関する比較研究:非共感覚者における文字の出現頻度と連想色の関連性2009

    • Author(s)
      西本真由香
    • Organizer
      日本認知心理学会第7回大会
    • Place of Presentation
      立教大学
    • Year and Date
      2009-07-19
    • Related Report
      2009 Annual Research Report
  • [Presentation] 和音聴取時の脳機能計測:fMRI研究2009

    • Author(s)
      赤塚諭
    • Organizer
      日本認知心理学会第7回大会
    • Place of Presentation
      立教大学
    • Year and Date
      2009-07-19
    • Related Report
      2009 Annual Research Report
  • [Presentation] 共感覚の脳機能イメージング-音楽色聴の検討-2009

    • Author(s)
      藤澤隆史
    • Organizer
      日本認知心理学会第7回大会
    • Place of Presentation
      立教大学
    • Year and Date
      2009-07-19
    • Related Report
      2009 Annual Research Report
  • [Presentation] ヒューマンメディアにおける脳機能イメージング2009

    • Author(s)
      長田典子
    • Organizer
      第25回近赤外フォーラム
    • Place of Presentation
      名古屋大学
    • Year and Date
      2009-05-15
    • Related Report
      2011 Final Research Report 2009 Annual Research Report
  • [Book] 「なぜ音や文字に色が見えるのか」ケータイ版イミダス2012

    • Author(s)
      長田典子
    • Publisher
      集英社
    • Related Report
      2011 Annual Research Report 2011 Final Research Report
  • [Remarks] (1)長田典子(2012.5.19).音と色のハーモニー-科学・技術・アートを融合した感性価値の創出-,東京オトナ大学.

    • Related Report
      2011 Final Research Report
  • [Remarks] (2)楽しい学問「感性情報学」,読売新聞, 2012.4.22.

    • Related Report
      2011 Final Research Report
  • [Remarks] (3) NHK総合爆問学問(爆笑問題のニッポンの教養)「世界はもっとカラフルだ~共感覚のフシギ」, 2011.11.3

    • Related Report
      2011 Final Research Report
  • [Remarks] (4)共感覚現象,読売新聞科学欄, 2011.9.26

    • Related Report
      2011 Final Research Report
  • [Remarks] (5) BSフジガリレオX,「ガリレオチャンネル共感覚のミステリー」,再放送, 2011.4.1

    • Related Report
      2011 Final Research Report
  • [Remarks] (6)四宮優・長田典子・鈴木輝昭・鈴木あずさ(2011.1.13,銀座ブルガリタワー8階プライベート・ラウンジ(東京)). trace(cross-wiring), The Lounge(Dandans Exhibition No.7),オープニングパフォーマンス,鈴木あずさ氏による、鈴木輝昭ピアノ独奏曲初演.

    • URL

      http://dandans.jp/archiveDetail.php?exhibitionId=7.

    • Related Report
      2011 Final Research Report
  • [Remarks] (7) NHK教育テレビ(ETV),「ガリレオチャンネル共感覚のミステリー」第51回科学技術映像祭入選作品から, 2010.05.29.

    • Related Report
      2011 Final Research Report
  • [Remarks] (8)長田典子.フジテレビ「とくダネ!」で,共感覚に関して取材協力. 2010.04.26.

    • Related Report
      2011 Final Research Report
  • [Remarks] (9)東京MXテレビ「ガリレオチャンネル共感覚のミステリー」が第51回科学技術映像祭文部科学大臣賞受賞,入選作品上映会が全国13都市で開催, 2010.03.11.

    • Related Report
      2011 Final Research Report
  • [Remarks] (10)長田典子.共感覚のミステリー音や文字に色が見える!?-東京MXテレビ「ガリレオチャンネル」に出演, 2009.11.21.

    • Related Report
      2011 Final Research Report
  • [Remarks]

    • Related Report
      2011 Final Research Report
  • [Remarks]

    • URL

      http://ist.ksc.kwansei.ac.jp/~nagata/

    • Related Report
      2009 Annual Research Report

URL: 

Published: 2009-04-01   Modified: 2016-04-21  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi