Symmetric and Structural Cross-Modal Mapping between Musics and Colors and its Application to Media Designs

2013 Fiscal Year Final Research Report

• Project/Area Number: 23650040
• Research Category: Grant-in-Aid for Challenging Exploratory Research
• Allocation Type: Multi-year Fund
• Research Field: Media informatics/Database
• Research Institution: University of Tsukuba
• Principal Investigator: CAI DongSheng 筑波大学, システム情報系, 准教授 (70202075)
• Project Period (FY): 2011 – 2013
• Keywords: 共感覚 / 感覚統合現象 / 脳内クロスモダリティ / MRI計測 / MEG / 応用認知科学 / カラーグレーディング

Research Abstract

Synesthesia is a neurological condition in which stimulation of one sensory or cognitive pathway includes automatic and involuntary responses in a second sensory or cognitive pathway. People who report such experiences are known as "synesthetes". A subset of synesthetes, sound-color synesthetes "see" colors and simple animated shapes in association with music and other sounds. We can find a color-sound synesthetes out of two to three thousands. It has been suggested that "...every infant is synesthetic, although most gradually grow out of it" p 12 (Stein and Meredith 1993) and the synesthetes possess these cross-modal connections even they grown up. We show scientifically how these synesthetic cross-modal mapping are used in animations and movies.

Research Products

• [Journal Article] Memory color effect induced by familiarity of brand logos (2013)
  • Author(s): Atsushi Kimura, Yuji Wada, Tomohiro Masuda, Sho-ichi Goto, Daisuke Tsuzuki, Haruo Hibino, Dongsheng Cai, Ippeita Dan
  • Journal Title: PLoS ONE
  • DOI: 10.1371/journal.pone.0068474
  • Peer Reviewed
• [Journal Article] Model of Aggregation - A Topological Approach (2012)
  • Author(s): Masud Rana and DongSheng Cai
  • Journal Title: GRAPP/IVAPP Pages: 355-360
  • Peer Reviewed
• [Journal Article] Rendering Morphobutterflies based on high accuracy nanooptical simulation, Journal of Optics (2012)
  • Author(s): Okada, Naoki, Zhu, Dong, Cai, Dongsheng, Cole, JamesB., Kambe, Makoto, Kinoshita, Sh uichi
  • Journal Title: Journal of Optics Volume: 1 Pages: 25-36
  • DOI: 10.1007/s12596-012-0092-y
  • Peer Reviewed
• [Journal Article] Stable and convenient spatial registration of stand-alone NIRS data through anchor- based probabilistic registration (2012)
  • Author(s): DaisukeTsuzuki, DongShengCai, Haruka Dan, YasushiKyutoku, AkifumiFujita, EijuW atanabe, IppeitaDan
  • Journal Title: Neuroscience Research Volume: 72, 2 Pages: 1632013171
  • Peer Reviewed
• [Journal Article] Amplification of Weibel instability in the relativistic beam-plasma interactions due to ion streaming
  • Author(s): KazemArdane, DongShengCai, Ken-Ichi Nishikawa
  • Journal Title: NewAstronomy Volume: Vol.33 Pages: 1-6
  • DOI: 10.1016/j.newast.2014.04.004
  • Peer Reviewed
• [Journal Article] 色聴共感覚者の一貫性実験によるクロスモーダルマッピング
  • Author(s): 蔡東生, 丁策立
  • Journal Title: 情報処理学会研究報告, グラフィクスとCAD研究会報告 Pages: 1-8
  • Peer Reviewed
• [Presentation] Dynamics of the cusp within a selfconsistent global 3D PIC simulation approach (2014)
  • Author(s): Cai, D., B. Lembege, K.-I. Nishikawa, A. Esmaeili
  • Organizer: Union Rado Scientifique Internationale
  • Place of Presentation: Beijing, China
  • Year and Date: 20140816-23
  • Invited
• [Presentation] Emotion of Colors : Synesthetic CrossModal Key Modulation (2014)
  • Author(s): Cai, D., N.Asai, andN.Nagata
  • Organizer: International Confe rence on Computer Graphics and Interactive Techniques
  • Place of Presentation: Vancouver, Canada
  • Year and Date: 20140810-14
• [Presentation] 3D Global Simulation of Magnetosphere, International Conference on Storms (2013)
  • Author(s): Cai, D., B. Lembege, K.-I. Nishikawa, A. Esmaeili
  • Organizer: Substorms, and Space Weather (ICSSSW)
  • Place of Presentation: ShenZheng, China
  • Year and Date: 20131104-07
  • Invited
• [Presentation] Large Scale 3D Full particle simulations of the solarwind-terrestrial magnetosphere interaction : impact of the IMF rotation on the magnetospheric dynamics (2013)
  • Author(s): Cai, D., B.Lembege, K.-I.Nishikawa, A. Esmaeili
  • Organizer: The 23rd International Con ference on Numerical Simulation of Plasmas (23rd ICNSP)
  • Place of Presentation: Beijin China
  • Year and Date: 20130914-16
• [Presentation] 3D Global Simulation of Magneto sphere (2013)
  • Author(s): Cai, D., B.Lembege, K.-I.Nishikawa, A. Esmaeili
  • Organizer: The11th International School for Space Simulations (ISSS-11)
  • Place of Presentation: Taipei, Taiwan
  • Year and Date: 20130721-28
  • Invited
• [Presentation] Dynamicsofthecuspboundariesa ndparticlesentry : 3-DPIClargescalesimulati on (2012)
  • Author(s): B.Lembege, D.S.Cai, K.Nishikawaand A.Esmaeili
  • Organizer: the11th Annual International Astrophysics Conference
  • Place of Presentation: PalmSprings, USA
  • Year and Date: 20120319-23
  • Invited
• [Presentation] Model of Aggregation A Topological Approach (2012)
  • Author(s): M.Rana, D.Cai
  • Organizer: GRAPP2012 : International Conferenceon Computer Graphics Theory and Application
  • Place of Presentation: Rome, Italy
  • Year and Date: 20120224-26
• [Presentation] Lorenz bifurcation and its visualization in magnetospheric dynamics (2011)
  • Author(s): Cai, D., B.Lembege, andK.-I.Nishikawa
  • Organizer: The10th International School for Space Simulations (ISSS-10)
  • Place of Presentation: Banff Canada
  • Year and Date: 20110724-29
  • Invited
• [Presentation] SynestheticCr oss-ModalSound-ColorMapping (2011)
  • Author(s): J.Wang, D.Cai, NobuyoshiAsai, Noriko Nagata, andAsakoFukumoto
  • Organizer: The15th annual meeting of the Association for the Scientific Study of Consciousness : poster(DVD)
  • Place of Presentation: 京都
  • Year and Date: 20110609-12
• [Presentation] Synesthetic Sound-Color Cross-Modality in Animations (2011)
  • Author(s): Cai, D., Goto, S., Wang, J.P., Asai, N., Nagata, N., Fukumoto, A., Kurumizawa, J
  • Organizer: Proc. Eurographics 2011 : posters (DVD)
  • Place of Presentation: Llandudno UK
  • Year and Date: 20110422-25
• [Remarks]
  • URL: http://www.cavelab.cs.tsukuba.ac.jp/~gotoh/syneth/index.html