• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to project page

2007 Fiscal Year Final Research Report Summary

The computation of multimodal representation through dynamic interaction

Research Project

Project/Area Number 16200020
Research Category

Grant-in-Aid for Scientific Research (A)

Allocation TypeSingle-year Grants
Section一般
Research Field Cognitive science
Research InstitutionKyoto University

Principal Investigator

INUI Toshio  Kyoto University, Graduate School of Informatics, Professor (30107015)

Co-Investigator(Kenkyū-buntansha) SAIKI Jun  Kyoto University, Graduate School of Human and Environmental Studies, Professor (60283470)
SUGIO Takeshi  Doshisya University, Faculty of Culture and information Science, Lecturer (60335205)
SASAOKA Takafumi  Kyoto University, Graduate School of Informatics, Assistant Professor (60367456)
Project Period (FY) 2004 – 2007
Keywordsmultimodal / information integration / reach-to-grasp / vision / motor command
Research Abstract

Interactions between hand movements and visual feedback are investigated. Results suggest that the central nervous system controls movements based on the prediction or the estimation of consequent motor accuracy taking into account both sensory and motor error. It is also suggested that the pre-supplementary motor area (pre-SMA), the right posterior parietal cortex (PPC), and the right temporo-parietal junction (TPJ) are involved in internal movement prediction, online evaluation of visuomotor error, and movement estimation to compensate for visual feedback delay, respectively. Experiments on cross-modal integration between vision and haptics with dynamic objects revealed that representation of dynamics as a weighted linear summation of sensory cues mediates cross-modal integration, and that temporal cues used in integration is independent of those used in simultaneity judgments. Experiments on cross-modal object recognition showed that recognition of 3-D objects within single modality … More utilized both viewpoint-dependent and viewpoint-invariant information, whereas object recognition by a modality not used in learning exclusively utilized viewpoint-invariant information.
It was found that the motor program for a familiar object is highly activated during the execution of the action in conventional manner, while the role of visual feedback is more dominant for an unfamiliar action. These results suggest that the motor representation for a familiar object is based on the previous interactions between the part of the object to be grasped and a particular hand shape. It was experimentally shown that active exploration of object views using a track ball facilitates object recognition. It suggested that a rule for view transformation of objects was learned by the interaction between information provided by hand movements from the motor system and visual information. Moreover, it was demonstrated that active exploration leads to a significant decrease of the amplitude of the magnetic field in the left hemisphere at the time period when the current dipole was estimated in the left parietal cortex. Less

  • Research Products

    (56 results)

All 2008 2007 2006 2005 Other

All Journal Article (41 results) (of which Peer Reviewed: 10 results) Presentation (15 results)

  • [Journal Article] 動的な変形に対する視触覚間同時性判断2008

    • Author(s)
      高橋康介, 齋木潤
    • Journal Title

      心理学研究 78(6)

      Pages: 599-606

    • Description
      「研究成果報告書概要(和文)」より
    • Peer Reviewed
  • [Journal Article] Realignment of temporal simultaneity between vision and touch2008

    • Author(s)
      Takahashi, K., Saiki, J., & Watanabe, K.
    • Journal Title

      Neuroreport 19(3)

      Pages: 319-322

    • Description
      「研究成果報告書概要(和文)」より
    • Peer Reviewed
  • [Journal Article] Visual-haptic simultaneityl iudgments for dynamic deformation2008

    • Author(s)
      Takahashi, K., Saiki, J
    • Journal Title

      Shinrigaku Kenkvu 78(6)

      Pages: 599-606

    • Description
      「研究成果報告書概要(欧文)」より
  • [Journal Article] Realignment of temporal simultaneity between vision and touch2008

    • Author(s)
      Takahashi, K., Saiki, J., Watanabe, K
    • Journal Title

      Neuroreport 19(3)

      Pages: 319-322

    • Description
      「研究成果報告書概要(欧文)」より
  • [Journal Article] イメージ生成とイメージ障害の認知脳理論2007

    • Author(s)
      乾 敏郎
    • Journal Title

      現代思想 5

      Pages: 233-245

    • Description
      「研究成果報告書概要(和文)」より
    • Peer Reviewed
  • [Journal Article] Neural correlates of state estimation in visually guided movements: anevent-related fMRI study2007

    • Author(s)
      Ogawa, K., Inui, T., & Sugio, T.
    • Journal Title

      Cortex 43(3)

      Pages: 289-300

    • Description
      「研究成果報告書概要(和文)」より
    • Peer Reviewed
  • [Journal Article] Lateralization of the posterior parietal cortex for internal monitoring of self-versus externally generated movements2007

    • Author(s)
      Ogawa, K., & Inui, T.
    • Journal Title

      Journal of Cognitive Neuroscience 19

      Pages: 1827-1835

    • Description
      「研究成果報告書概要(和文)」より
    • Peer Reviewed
  • [Journal Article] 視覚と触覚を用いた物体認識における視点独立性2007

    • Author(s)
      上田祥行, 齋木潤
    • Journal Title

      基礎心理学研究 26(1)

      Pages: 11-19

    • Description
      「研究成果報告書概要(和文)」より
    • Peer Reviewed
  • [Journal Article] 触覚刺激が視知覚に与える影響2007

    • Author(s)
      上田祥行, 齋木潤
    • Journal Title

      電子情報通信学会技術報告 HIP2007-105

      Pages: 33-37

    • Description
      「研究成果報告書概要(和文)」より
  • [Journal Article] 仮想物体の変形に対する視触覚間同時性 知覚の順応2007

    • Author(s)
      高橋康介, 齋木潤, 渡邊克巳
    • Journal Title

      電子情報通信学会技術報告 MVE2007-5

      Pages: 21-26

    • Description
      「研究成果報告書概要(和文)」より
  • [Journal Article] An integrative theory of mental image generation : evidence from cognitive neuroscience and neuronsvchologv2007

    • Author(s)
      Inui, T
    • Journal Title

      revue de la pemee d'aujourd'hui 5

      Pages: 233-245

    • Description
      「研究成果報告書概要(欧文)」より
  • [Journal Article] Neural correlates of state estimation in visually guided movements : an event-related fMRI study2007

    • Author(s)
      Ogawa, K., Inui, T., Sugio, T
    • Journal Title

      Cortex 43(3)

      Pages: 289-300

    • Description
      「研究成果報告書概要(欧文)」より
  • [Journal Article] Lateralization of the posterior parietal cortex for internal monitoring of self-versus externally generated movements2007

    • Author(s)
      Ogawa, K., Inui, T
    • Journal Title

      Journal of Cognitive Neuroscience 19

      Pages: 1827-1835

    • Description
      「研究成果報告書概要(欧文)」より
  • [Journal Article] Viewpoint independence in visual and haptic obiect recognition2007

    • Author(s)
      Ueda, Y., Saiki, J
    • Journal Title

      The Japanese Journal of Pscychonomic Science 26(1)

      Pages: 11-19

    • Description
      「研究成果報告書概要(欧文)」より
  • [Journal Article] Visual perception is varied by tactile input2007

    • Author(s)
      Ueda, Y., Saiki, J
    • Journal Title

      Technical Report of IEICE 2007-105

      Pages: 33-37

    • Description
      「研究成果報告書概要(欧文)」より
  • [Journal Article] The adaptation of vision-haptics simultaneity judgments for the deformation of virtual obiect2007

    • Author(s)
      Takahashi, K., Saiki, J., Watanabe, K
    • Journal Title

      Technical Report of IEICE 2007-5

      Pages: 21-26

    • Description
      「研究成果報告書概要(欧文)」より
  • [Journal Article] The effect of viewing the moving limb and target object during the early phase of movement on the online control of grasping2006

    • Author(s)
      Fukui, T., and Inui, T.
    • Journal Title

      Human Movement Science 25(3)

      Pages: 349-371

    • Description
      「研究成果報告書概要(和文)」より
    • Peer Reviewed
  • [Journal Article] 内省・直感過程に基づく身体化による認知2006

    • Author(s)
      乾 敏郎
    • Journal Title

      神経心理学 22(1)

      Pages: 62-69

    • Description
      「研究成果報告書概要(和文)」より
    • Peer Reviewed
  • [Journal Article] Separating the brain regions involved in intenally guided and visual feedback control of moving effectors: an event-related fMRI study2006

    • Author(s)
      Ogawa, K., Inui, T., & Sugio, T.
    • Journal Title

      Neuroimage 32(4)

      Pages: 1760-1770

    • Description
      「研究成果報告書概要(和文)」より
    • Peer Reviewed
  • [Journal Article] 自己 vs.他者運動の内的推定における後部頭頂皮質の側性:fMRI研究2006

    • Author(s)
      小川 健二, 乾 敏郎
    • Journal Title

      電子情報通信学会技術研究報告 106

      Pages: 13-18

    • Description
      「研究成果報告書概要(和文)」より
  • [Journal Article] 物体の変形に対する視触覚間同時性判断2006

    • Author(s)
      高橋康介, 齋木潤
    • Journal Title

      電子情報通信学会技術研究報告 HIP2006-49

      Pages: 25-50

    • Description
      「研究成果報告書概要(和文)」より
  • [Journal Article] 日常物体の構造が到達把持運動の運動学的特性に及ぼす影響2006

    • Author(s)
      杉尾 武志, 乾 敏郎
    • Journal Title

      電子情報通信学会技術研究報告 HIP2006-50

      Pages: 31-36

    • Description
      「研究成果報告書概要(和文)」より
  • [Journal Article] Visuomotor transformation process in goal-directed prehension: Utilization of online vision during preshaping phase of grasping2006

    • Author(s)
      Fukui, T., Takemura, N., & Inui, T.
    • Journal Title

      Japanese Psychological Research 48(3)

      Pages: 188-203

    • Description
      「研究成果報告書概要(和文)」より
    • Peer Reviewed
  • [Journal Article] The effect of viewing the moving limb and target object during the early phase of movement on the online control of grasping2006

    • Author(s)
      Fukui, T., Inui, T
    • Journal Title

      Human Movement Science 25(3)

      Pages: 349-371

    • Description
      「研究成果報告書概要(欧文)」より
  • [Journal Article] Embodied cognition by reflective and reflexive processes2006

    • Author(s)
      Inui, T
    • Journal Title

      Japanese Journal of Neuropsychology 22(1)

      Pages: 62-69

    • Description
      「研究成果報告書概要(欧文)」より
  • [Journal Article] Separating the brain regions involved in intenally guided and visual feedback control of moving effectors : an event-related fMRI study2006

    • Author(s)
      Ogawa, K., Inui, T., Sugio, T
    • Journal Title

      Neuroimage 32(4)

      Pages: 1760-1770

    • Description
      「研究成果報告書概要(欧文)」より
  • [Journal Article] Lateralization of posterior parietal cortex for internal movement estimation of self vs. others : an fMRI study2006

    • Author(s)
      Ogawa, K., Inui, T
    • Journal Title

      Technical Report of IEICE 106

      Pages: 13-18

    • Description
      「研究成果報告書概要(欧文)」より
  • [Journal Article] Visuo-haptic simultaneity judgment of the deformation of an object2006

    • Author(s)
      Takahashi, K., Saiki, J
    • Journal Title

      Technical Report of IEICE 2006-49

      Pages: 25-50

    • Description
      「研究成果報告書概要(欧文)」より
  • [Journal Article] How structural properties of a familiar object affect reach-to-grasp kinematics2006

    • Author(s)
      Sugio, T., Inui, T
    • Journal Title

      Technical Report of IEICE 2006-50

      Pages: 31-36

    • Description
      「研究成果報告書概要(欧文)」より
  • [Journal Article] Visuomotor transformation process in goal-directed prehension : utilization of online vision during preshaping phase of grasping2006

    • Author(s)
      Fukui, T., Takemura, N., Inui, T
    • Journal Title

      Japanese Psychological Research 48(3)

      Pages: 188-203

    • Description
      「研究成果報告書概要(欧文)」より
  • [Journal Article] 到達把持動作の運動時間がオンライン視覚情報の利用特性に及ぼす影響2005

    • Author(s)
      福井 隆雄, 乾 敏郎
    • Journal Title

      電子情報通信学会技術研究報告 HIP2005-72

      Pages: 119-124

    • Description
      「研究成果報告書概要(和文)」より
  • [Journal Article] 仮想物体の変形に対する視覚触覚統合過程の心理物理学的検討2005

    • Author(s)
      高橋 康介, 齋木 潤
    • Journal Title

      電子情報通信学会技術研究報告 MVE2005-65

      Pages: 45-50

    • Description
      「研究成果報告書概要(和文)」より
  • [Journal Article] 視覚触覚統合過程における感覚間時間遅延の効果に関する実験心理学的検討2005

    • Author(s)
      高橋 康介, 齋木 潤
    • Journal Title

      電子情報通信学会技術研究報告 HIP2005-78

      Pages: 153-158

    • Description
      「研究成果報告書概要(和文)」より
  • [Journal Article] 視覚触覚統合による物体変形量の知覚:感覚間時間遅延の効果2005

    • Author(s)
      高橋 康介, 齋木 潤
    • Journal Title

      電子情報通信学会技術研究報告 HIP2005-154

      Pages: 1-6

    • Description
      「研究成果報告書概要(和文)」より
  • [Journal Article] 日常物体に対する到達把持運動特性の検討2005

    • Author(s)
      杉尾 武志, 乾 敏郎
    • Journal Title

      電子情報通信学会技術研究報告 HIP2005-3

      Pages: 13-18

    • Description
      「研究成果報告書概要(和文)」より
  • [Journal Article] Influence of movement duration on the utilization of online vision for reach-to-grasp movements2005

    • Author(s)
      Fukui, T., Inui, T
    • Journal Title

      Technical Report of IEICE 2005-72

      Pages: 119-124

    • Description
      「研究成果報告書概要(欧文)」より
  • [Journal Article] Psychophysical investigation on visual-haptic integration of a virtual object2005

    • Author(s)
      Takahashi, K., Saiki, J
    • Journal Title

      Technical Report of IEICE 2005-65

      Pages: 45-50

    • Description
      「研究成果報告書概要(欧文)」より
  • [Journal Article] The effect of temporal delay between senses in visual and haptic integration2005

    • Author(s)
      Takahashi, K., Saiki, J
    • Journal Title

      Technical Report of IEICE 2005-78

      Pages: 153-158

    • Description
      「研究成果報告書概要(欧文)」より
  • [Journal Article] Perceiving deformation of an object through vision and haptics : the effect of temporal delay between modalities2005

    • Author(s)
      Takahashi, K., Saiki, J
    • Journal Title

      Technical Report of IEICE 2005-154

      Pages: 1-6

    • Description
      「研究成果報告書概要(欧文)」より
  • [Journal Article] Kinematic properties of a reach-to-grasp movement toward familiar objects2005

    • Author(s)
      Sugio, T., Inui, T
    • Journal Title

      Technical Report of IEICE 2005-3

      Pages: 13-18

    • Description
      「研究成果報告書概要(欧文)」より
  • [Journal Article] Effect of active exploration of 3-D object views on the view matching process in object recognition

    • Author(s)
      Sasaoka, T., Asakura, N., Kawahara, T
    • Journal Title

      (submitted)

    • Description
      「研究成果報告書概要(欧文)」より
  • [Presentation] An online control model for prehension based on stochastic prediction considering future sensory feedback2007

    • Author(s)
      Takemura, N., Inui, T., & Fukui, T.
    • Organizer
      3rd Annual Computational Cognitive Neuroscience Conference
    • Place of Presentation
      San Diego, USA
    • Year and Date
      20071101-02
    • Description
      「研究成果報告書概要(和文)」より
  • [Presentation] Recalibration of vision-haptics temporal simultaneity2007

    • Author(s)
      Takahashi, K., Saiki, J., & Watanabe, K.
    • Organizer
      8th International Multisensory Research Forum
    • Place of Presentation
      Sydney, Australia
    • Year and Date
      20070705-07
    • Description
      「研究成果報告書概要(和文)」より
  • [Presentation] Can human transfer an object's representation across modalities?2006

    • Author(s)
      Ueda, Y., & Saiki, J.
    • Organizer
      47th Annual Meeting of the Psychonomic Society
    • Place of Presentation
      Houston, USA.
    • Year and Date
      20061116-19
    • Description
      「研究成果報告書概要(和文)」より
  • [Presentation] Can human transfer an object's representation across modalities2006

    • Author(s)
      Ueda, Y., Saiki, J
    • Organizer
      47th Annual Meeting of the Psychonomic Society
    • Place of Presentation
      Houston, USA
    • Year and Date
      20061116-19
    • Description
      「研究成果報告書概要(欧文)」より
  • [Presentation] Lateralization of posterior parietal cortex for internal movement estimation of self versus others: an fMRI study2006

    • Author(s)
      Ogawa, K., & Inui, T.
    • Organizer
      36th annual meeting of the Society for Neuroscience
    • Place of Presentation
      Atlanta, USA.
    • Year and Date
      20061014-18
    • Description
      「研究成果報告書概要(和文)」より
  • [Presentation] Lateralization of posterior parietal cortex for internal movement estimation of self versus others : an fMRI study2006

    • Author(s)
      Ogawa, K., Inui, T
    • Organizer
      36th annual meeting of the Society for Neuroscience
    • Place of Presentation
      Atlanta, USA
    • Year and Date
      20061014-18
    • Description
      「研究成果報告書概要(欧文)」より
  • [Presentation] Non-random human performance under conditions of lacking information2006

    • Author(s)
      Tarasenko, S., Abdikeev. N., & Inui, T.
    • Organizer
      The 5th International Conference of the Cognitive Science Society
    • Place of Presentation
      Sheraton Vancouver Wall Centre, Vancouver British Colombia, Canada
    • Year and Date
      20060726-29
    • Description
      「研究成果報告書概要(和文)」より
  • [Presentation] Distinguishable Functional Modes in the Human Processing of New Information contained in Pattern-based Structures2006

    • Author(s)
      Tarasenko, S., Inui, T., & Abdikeev. N.
    • Organizer
      The Twenty Eighth Annual Conference of the Cognitive Science Society
    • Place of Presentation
      Sheraton Vancouver Wall Centre, Vancouver British Colombia, Canada
    • Year and Date
      20060726-29
    • Description
      「研究成果報告書概要(和文)」より
  • [Presentation] Non-random human performance under conditions of lacking information2006

    • Author(s)
      Tarasenko, S., Abdikeev, N., Inui, T
    • Organizer
      The 5th International Conference of the Cognitive Science Society(Sheraton Vancouver Wall Centre, Vancouver British Colombia)
    • Place of Presentation
      Canada
    • Year and Date
      20060726-29
    • Description
      「研究成果報告書概要(欧文)」より
  • [Presentation] Illusory spatial perception induced by the temporal discrepancy between modalities in dynamic vision-haptics integration2006

    • Author(s)
      Takahashi, K., & Saiki, J.
    • Organizer
      The 7th International Multisensory Research Forum
    • Place of Presentation
      Dublin, Ireland.
    • Year and Date
      20060718-21
    • Description
      「研究成果報告書概要(和文)」より
  • [Presentation] A stochastic feedback control model for grasping movement: Simulation of visual occlusion2006

    • Author(s)
      Takemura, N., Fukui, T., & Inui, T.
    • Organizer
      Workshop: Bridging the gap between sensation and motor control
    • Place of Presentation
      Giessen, Germany
    • Year and Date
      20060713-16
    • Description
      「研究成果報告書概要(和文)」より
  • [Presentation] A stochastic feedback control model for grasping movement : Simulation of visual occlusion2006

    • Author(s)
      Takemura, N., Fukui, T., Inui, T
    • Organizer
      Workshop : Bridging the gap between sensation and motor control
    • Place of Presentation
      Giessen, Germany
    • Year and Date
      20060713-16
    • Description
      「研究成果報告書概要(欧文)」より
  • [Presentation] Illusory spatial perception induced by the temporal discrepancy between modalities in dynamic vision-haptics integration2006

    • Author(s)
      Takahashi, K., Saiki, J
    • Organizer
      The 7th International Multisensory Research Forum
    • Place of Presentation
      Dublin, Ireland
    • Year and Date
      20060618-21
    • Description
      「研究成果報告書概要(欧文)」より
  • [Presentation] Brain-body-environment interactions in cog2005

    • Author(s)
      Inui, T
    • Organizer
      Proceedings of the 4th IEEE international conference : Development and learning-from interaction to Cognition
    • Place of Presentation
      INTEX Osaka, Japan
    • Year and Date
      20050719-21
    • Description
      「研究成果報告書概要(欧文)」より
  • [Presentation] Brain-body-environment interactions in cognition2005

    • Author(s)
      Inui, T
    • Organizer
      Proceedings of the 4th IEEE international conference: Development and learning-from interaction to cognition
    • Place of Presentation
      INTEX 大阪
    • Year and Date
      20050719-20
    • Description
      「研究成果報告書概要(和文)」より

URL: 

Published: 2010-02-04  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi