• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Sonic Visual Interface

Research Project

Project/Area Number 13680443
Research Category

Grant-in-Aid for Scientific Research (C)

Allocation TypeSingle-year Grants
Section一般
Research Field Intelligent informatics
Research InstitutionNagoya Institute of Technology

Principal Investigator

SATO Jun  Nagoya Institute of Technology, Department of Comp. Sci. Eng., professor, 工学研究科, 教授 (20303688)

Co-Investigator(Kenkyū-buntansha) SATO Yukio  Nagoya Institute of Technology, Department of Comp. Sci. Eng., professor, 工学研究科, 教授 (80134790)
Project Period (FY) 2001 – 2003
Project Status Completed (Fiscal Year 2003)
Budget Amount *help
¥4,100,000 (Direct Cost: ¥4,100,000)
Fiscal Year 2003: ¥800,000 (Direct Cost: ¥800,000)
Fiscal Year 2002: ¥1,600,000 (Direct Cost: ¥1,600,000)
Fiscal Year 2001: ¥1,700,000 (Direct Cost: ¥1,700,000)
KeywordsSonic Visual Interface / Computer Vision / Sound Control / Mixed Reality / Mutual Projection / 音響視覚インターフェース / 視覚障害者
Research Abstract

In this research, we investigated methods for taking consistency between 3D visual information and 3D sound information, and for representing visual information by using sound information. We first analyzed the relationship between 3D image space, 3D sound space and 3D auditory perception space. If cameras are uncalibrated, 3D space reconstructed from images has 3D projective ambiguity. IF 3D sound generators are uncalibrated, 3D sound space has 3D projective ambiguity. Also, We have 3D affine ambiguity in human auditory perception. Thus, the relationship between 3D image space, 3D sound space and 3D auditory space can be described by 3D projective transformations. By using these properties, we proposed a method for representing 3D visual shapes and positions by using 3D sound.
We next applied the method for taking geometric consistency of visual and auditory information in mixed reality applications. In particular, we showed that by using the mutual projection of multiple cameras, we can compute 3D visual information much more accurately than the existing method. We also applied the proposed method for mixed reality systems of multiple users, and showed that by recovering 3D information in each user's camera coordinates, we can generate 3D sound information for each person. We applied the proposed method for generating virtual images and virtual sounds in virtual fighting systems and in virtual music instruments

Report

(4 results)
  • 2003 Annual Research Report   Final Research Report Summary
  • 2002 Annual Research Report
  • 2001 Annual Research Report
  • Research Products

    (22 results)

All Other

All Publications (22 results)

  • [Publications] 坪井 秀樹, 小竹 論季, 古橋 万里, 佐藤 淳: "未校正ソニックビジュアルインターフェイス"画像の認識・理解シンポジウム講演論文集. 1. 489-494 (2002)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2003 Final Research Report Summary
  • [Publications] 伊藤 満, 佐藤 淳: "カメラの相互射影によるエピポーラ幾何計算の安定化"電子情報通信学会論文誌. J85-DII, 3. 416-424 (2002)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2003 Final Research Report Summary
  • [Publications] 杉村 健之, 佐藤 淳: "カメラの相互投影によるTrifocal Tensor計算の安定化"情報処理学会論文誌(コンピュータビジョンとイメージメディア). 43, SIG11. 113-120 (2002)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2003 Final Research Report Summary
  • [Publications] J.Sato: "Recovering Multiple View Relations from Mutual Projections of Multiple Cameras"Proc.Asian Conference on Computer Vision. 1. 7-12 (2004)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2003 Final Research Report Summary
  • [Publications] 河合 潤也, 佐藤 淳: "複数ユーザカメラの相互投影を用いた3次元復元と複合現実空間の共有"第10回画像センシングシンポジウム論文集. (2004)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2003 Final Research Report Summary
  • [Publications] 江川 宗一郎, 佐藤 淳: "複合現実感による仮想楽器の実現"第10回画像センシングシンポジウム論文集. (2004)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2003 Final Research Report Summary
  • [Publications] H.Tsuboi, T.Kotake, M.Furuhashi, J.Sato: "Uncalibrated Sonic Visual Interface"Proc. of Meeting on Image Recognition and Understanding. Vol.1. 489-494 (2002)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2003 Final Research Report Summary
  • [Publications] M.Ito, J.Sato: "Robust Computation of Epipolar Geometry from Mutual Projection of Cameras"Transaction of the Institute of Electronics, Information and Communication Engineers. J85-DII, No.3. 416-424 (2002)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2003 Final Research Report Summary
  • [Publications] T.Sugimura, J.Sato: "Robust Computation of Trifocal Tensor from Mutual Projection of Cameras"IPSJ Transactions on Computer Vision and Image Media. Vol.44,No.SIG17. 113-120 (2002)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2003 Final Research Report Summary
  • [Publications] J.Sato: "Recovering Multiple View Relations from Mutual Projections of Multiple Cameras"Proc. Asian Conference on Computer Vision. Vol.1. 7-12 (2004)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2003 Final Research Report Summary
  • [Publications] J.Kawai, J.Sato: "3D Reconstruction and Mixed Reality f from Mutual Projection of Multiple Cameras"Proc. Symposium on Sensing via Image Information. (to appear). (2004)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2003 Final Research Report Summary
  • [Publications] S.Egawa, J.Sato: "Virtual Music Instruments from Mixed Reality"Proc. Symposium on Sensing via Image Information. (to appear). (2004)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2003 Final Research Report Summary
  • [Publications] J.Sato: "Recovering Epipolar Geometry from Mutual Projections of Multiple Cameras"Proc.Australia-Japan Advanced Workship on Computer Vision. 111-117 (2003)

    • Related Report
      2003 Annual Research Report
  • [Publications] J.Sato: "Recovering Multiple View Relations from Mutual Projections of Multiple Cameras"Proc.Asian Conference on Computer Vision. Vol.1. 7-12 (2004)

    • Related Report
      2003 Annual Research Report
  • [Publications] 河合潤也, 佐藤 淳: "複数ユーザカメラの相互投影を用いた3次元復元と複合現実空間の共有"第10回画像センシングシンポジウム講演論文集. (2004)

    • Related Report
      2003 Annual Research Report
  • [Publications] 杉村良彦, 佐藤 淳: "未校正マルチカメラの相互投影を用いた弱校正と物体形状の復元"情報処理学会研究報告(2003-CVIM-140). Vol.2003, No.88. 17-24 (2003)

    • Related Report
      2003 Annual Research Report
  • [Publications] 坪井秀樹, 小竹論季, 古橋万里, 佐藤淳: "未校正ソニックビジュアルインターフェイス"画像の認識・理解シンポジウム講演論文集. 1. 489-494 (2002)

    • Related Report
      2002 Annual Research Report
  • [Publications] 杉村健之, 佐藤淳: "カメラの相互投影によるTrifocal Tensorの計算と形状復元の安定化"情報処理学会論文誌(コンピュータビジョンとイメージメディア). 43・SIG11. 113-120 (2002)

    • Related Report
      2002 Annual Research Report
  • [Publications] M.Ito, T.Sugimura, J.Sato: "Recovering Structures and Motions from Mutual Projection of Cameras"Proc. International Conference on Pattern Recognition. 3. 676-679 (2002)

    • Related Report
      2002 Annual Research Report
  • [Publications] 佐藤 淳: "マルチカメラの相互投影による3次元形状復元と複合現実感の実現"電気関係学会東海支部連合大会講演論文集. S19-S20 (2002)

    • Related Report
      2002 Annual Research Report
  • [Publications] 佐藤 淳: "カメラの相互投影を用いたマルチカメラの弱校正"計測自動制御学会システムインテグレーション部門講演会論文集. 3. 83-84 (2002)

    • Related Report
      2002 Annual Research Report
  • [Publications] 小竹論季, 古橋万里, 佐藤淳: "未校正ソニックビジュアルインターフェイス"画像の認識・理解シンポジウム講演論文集. (2002)

    • Related Report
      2001 Annual Research Report

URL: 

Published: 2001-04-01   Modified: 2016-04-21  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi