Spatiotemporal integration of multimodal sensory information

2011 Fiscal Year Final Research Report

• Project/Area Number: 19001004
• Research Category: Grant-in-Aid for Specially Promoted Research
• Allocation Type: Single-year Grants
• Review Section: Science and Engineering; Engineering
• Research Institution: Tohoku University
• Principal Investigator: SUZUKI Yoiti 東北大学, 電気通信研究所, 教授 (20143034)
• Co-Investigator(Kenkyū-buntansha): GYOBA Jiro 東北大学, 大学院・文学研究科, 教授 (50142899) ; KAWASE Tetsuaki 東北大学, 大学院・医工学研究科, 教授 (50169728) ; SAKURAI Kenzo 東北学院大学, 教養学部, 教授 (40183818) ; SUGITA Yoichi 産業技術総合研究所, ヒューマンライフテクノロジー研究部門, 主任研究員 (40221311) ; IWAYA Yukio 東北大学, 電気通信研究所, 准教授 (10250896) ; SAKAMOTO Shuichi 東北大学, 電気通信研究所, 准教授 (60332524) ; MIYAUCHI Ryota 北陸先端科学技術大学院大学, 情報科学研究科, 助教 (30455852) ; TANAKA Akihiro 早稲田大学, 高等研究所, 助教 (80396530) ; YAIRI Satoshi 仙台高等専門学校, 情報デザイン学科, 助教 (00447187)
• Co-Investigator(Renkei-kenkyūsha): OTANI Makoto 信州大学, 工学部, 准教授 (40433198) ; KAWACHI Yosuke 東北福祉大学, 感性福祉研究所, 講師 (20565775) ; KAWABATA Hideaki 慶應義塾大学, 文学部, 准教授 (70347079) ; KUBODERA Toshio 東京大学, 大学院・総合文化研究科, 教務補佐員 ; KOBAYASHI Maori 明治大学, 研究・知財研究機構, 研究員 (90451632) ; HIDAKA Souta 立教大学, 現代心理学部, 准教授 (40581161) ; TERAMOTO Wataru 室蘭工業大学, 工学研究科, 助教 (30509089) ; HONDA Akio 東北福祉大学, 総合福祉学部, 助教 (80433564)
• Project Period (FY): 2007 – 2011
• Keywords: マルチモーダル感覚情報 / マルチモーダル情報通信技術 / 迫真性・臨場感・自然性 / 3次元ディスプレイ

Research Abstract

This research project aims at precisely describing multimodal perception from the view point of spatiotemporal event marking to build up brain computation models to describe perception of high-level kansei information such as verisimilitude, presence and naturalness for multimedia information. Based on such knowledge, this project also aims at providing quantitative design guidelines for multimedia spatiotemporal information display systems. As results of this project, we have accumulated a lot of new knowledge on multimodal spatiotemporal perception and have described human perception models of high-level kansei information. Finally, we have drawn up the quantitative design guidelines for multimedia spatiotemporal information display systems.

Research Products

• [Journal Article] Indiscriminable sounds determine the direction of visual motion (2012)
  • Author(s): M. Kobayashi, W. Teramoto, S. Hidaka and Y. Sugita
  • Journal Title: Scientific Reports Volume: 2 Pages: 365
  • Peer Reviewed
• [Journal Article] Toward 3D spatial audio systems with high sense-of-presence (2011)
  • Author(s): Y. Suzuki, T. Okamoto, J. Trevino, T. Kimura, S. Sakamoto, Z. L. Cue, M. Katsumoto and Y. Iwaya
  • Journal Title: Proc. 5th IUCS, TS-3-1 (10 pages manuscript)
  • Peer Reviewed
• [Journal Article] 「迫真性」を規定する時空間情報 (2010)
  • Author(s): 寺本渉, 吉田和博, 日高聡太, 浅井暢子, 行場次朗, 坂本修一, 岩谷幸雄, 鈴木陽一
  • Journal Title: 日本バーチャルリアリティ学会論文誌 Volume: 15(3) Pages: 483-486
  • Peer Reviewed
• [Journal Article] Sounds move a static visual object (2010)
  • Author(s): W. Teramoto, S, Hidaka and Y. Sugita
  • Journal Title: PLoS ONE Volume: 5(8) Pages: e12255
  • Peer Reviewed
• [Journal Article] I feel your voice : Cultural differences in the multisensory perception of emotion (2010)
  • Author(s): A. Tanaka, A. Koizmi, H. Imai, S. Hiramatsu and B. de Gelder
  • Journal Title: Psychological Science Volume: 21(9) Pages: 1259-1262
  • Peer Reviewed
• [Journal Article] Alternation of sound location induces visual motion perception of a static object (2009)
  • Author(s): S. Hidaka, Y. Manaka, W. Teramoto, Y. Sugita, R. Miyauchi, J. Gyoba, Y. Suzuki and Y. Iwaya
  • Journal Title: PLoS ONE Volume: 4 Pages: e8188
  • Peer Reviewed
• [Journal Article] Insufficient visual information leads to spontaneous bipedal walking in monkeys (2009)
  • Author(s): Y. Manaka and Y. Sugita
  • Journal Title: Behavioral Processes Volume: 80 Pages: 104-106
  • Peer Reviewed
• [Journal Article] Selective effects of auditory stimuli on tactile roughness perception (2008)
  • Author(s): Y. Suzuki, J. Gyoba and S. Sakamoto
  • Journal Title: Brain Research Volume: 1242 Pages: 87-94
  • Peer Reviewed
• [Journal Article] Compression of auditory space during forward self-motion
  • Author(s): W. Teramoto, S. Sakamoto, F. Furune, J. Gyoba and Y. Suzuki
  • Journal Title: PLoS ONE Volume: (in press)
  • Peer Reviewed
• [Presentation] バーチャル・リアリティのリアリティ (2012)
  • Author(s): 寺本渉, 吉田和博, 浅井暢子, 日高聡太, 行場次朗, 坂本修一, 岩谷幸雄, 鈴木陽一
  • Organizer: 映像情報メディア学会立体映像研究会
  • Place of Presentation: 東京
  • Year and Date: 2012-03-05
• [Presentation] 異種感覚間相互作用の神経基盤について (2012)
  • Author(s): 杉田陽一
  • Organizer: 第3回多感覚研究会
  • Place of Presentation: 東京
  • Year and Date: 2012-01-18
• [Presentation] Auditory Displays and Microphone Arrays for Active Listening (2010)
  • Author(s): Y. Suzuki, Y. Iwaya, S. Sakamoto and T. Okamoto
  • Organizer: 40th Audio Engneering Society (AES)Conference-Spatial Audio
  • Place of Presentation: Tokyo
  • Year and Date: 2010-10-09
• [Presentation] Self-motion perception by multimodal stimulation based on virtual reality technology (2010)
  • Author(s): S. Sakamoto, T. Kubodera, P. M. Grove, K. Sakurai, J. Gyoba and Y. Suzuki
  • Organizer: 11th International Multisensory Research Forum (IMRF)
  • Place of Presentation: Liverpool, UK
  • Year and Date: 2010-06-18
• [Presentation] Perception of emotion in face and voice : Crosscultural comparison (2009)
  • Author(s): A. Tanaka, A. Koizumi, H. Imai, S. Hiramatsu, E. Hiramoto and B. de Gelder
  • Organizer: Proc. 10th International Multisensory Research Forum(IMRF)
  • Place of Presentation: NewYork, USA
  • Year and Date: 2009-06-30
• [Book] Principles and applications of spatial hearing (2011)
  • Author(s): Y. Suzuki, D. Brungart, Y. Iwaya, K. Iida, D. Cabrera and H. Kato (Edited)
  • Total Pages: 520
  • Publisher: World Scientific
• [Book] バーチャルリアリティ学 (2009)
  • Author(s): 鈴木陽一, 岩谷幸雄(分担執筆)
  • Total Pages: 110-114
  • Publisher: 工学調査会
• [Remarks] 受賞:寺本渉, 吉田和博, 浅井暢子, 日高聡太, 行場次朗, 鈴木陽一, 臨場感の素朴な理解, 日本バーチャルリアリティ学会論文賞, 2011
• [Remarks] 河地庸介,行場次朗,知覚的に消失した物体の削除が再起させる視覚的アウェアネス,電子情報通信学会ヒューマンコミュニケーション賞,2007
• [Remarks]
  • URL: http://www.ais.riec.tohoku.ac.jp/multimodal.html