• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Feature extraction of eye and lip movements from video image of the face

Research Project

Project/Area Number 63460129
Research Category

Grant-in-Aid for General Scientific Research (B)

Allocation TypeSingle-year Grants
Research Field 電子通信系統工学
Research InstitutionKanazawa Institute of Technology

Principal Investigator

KAKITA Yuki  Kanazawa Institute of Technology, Faculty of Engineering, Professor, 工学部, 教授 (00098823)

Co-Investigator(Kenkyū-buntansha) HONDA Kiyoshi  Kanazawa Institute of Technology, Faculty of Engineering, Associate Professor, 工学部, 助教授 (10114638)
KAWARA Tetsuo  Kanazawa Institute of Technology, Faculty of Engineering, Professor, 工学部, 教授 (40112776)
SHIMADA Youichi  Kanazawa Institute of Technology, Faculty of Engineering, Professor, 工学部, 教授 (50113155)
Project Period (FY) 1988 – 1990
Project Status Completed (Fiscal Year 1990)
Budget Amount *help
¥5,900,000 (Direct Cost: ¥5,900,000)
Fiscal Year 1990: ¥1,400,000 (Direct Cost: ¥1,400,000)
Fiscal Year 1989: ¥1,800,000 (Direct Cost: ¥1,800,000)
Fiscal Year 1988: ¥2,700,000 (Direct Cost: ¥2,700,000)
KeywordsCommunication / Face posture / Eye movement / Lip movement / Image Processing / Speech / Language / Human Interface / コミュニケ-ション
Research Abstract

A. System development
A group of computers and peripheral devices related to the image, sound and bio-electric signals, were facilitated to our research group by the aid from Monbusho for special research facility in 1987 (Showa 62). We developed the additional system for this particular research project. Newly purchased facilities by the aid of this project were a workstation for image processing, image capture board, and RAMs. Completed systems were good for our research purpose.
B. Preliminary Results
Pilot experiments were done for the extraction of lip movements and eye movements.
(1) eye movements
Experiments of detecting the eye pointing object were conducted. The image system estimate the direction of eye sight by the image of iris of one eye when looking at the 5x5 objects configured in matrix. Under the head still condition, good correlation was obtained between the position of object and the image parameters of iris image. For head free condition, a preliminary method, by applying a special coordinate conversion, was examined. It was also promising.
(2) lip movements
Using 3 parameters, vertical opening of lips and upper-lower incisors, and horizontal opening, differentiation of 11 consonants in the context of /aCa/. Almost all the consonants, except /k,g/ and /r/, were differentiated successfully. For the error of recognition was suspected to be caused by the low frame rate. A new image interpolation method were developed. The method is promising because it can enhance the time resolution substantially.

Report

(4 results)
  • 1990 Annual Research Report   Final Research Report Summary
  • 1989 Annual Research Report
  • 1988 Annual Research Report
  • Research Products

    (20 results)

All Other

All Publications (20 results)

  • [Publications] 栗田、本多、垣田: "口唇画像情報を併用する音声の分析"電子情報通信学会技術研究報告音声研究会資料. SP88-94. 41-48 (1988)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      1990 Final Research Report Summary
  • [Publications] Honda,Kusakawa,and Kakita: "Sequential control model of speech articulation on producing word utterances"Proc.Int.Conf.on Spoken Language Processing. 20. 445-448 (1990)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      1990 Final Research Report Summary
  • [Publications] 倉部、垣田: "通常ビデオ画像からの注視方向の推定-コミュニケーションの補足手段への応用-"日本ME学会大会論文集. 1-B-3 (1992)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      1990 Final Research Report Summary
  • [Publications] 津守、垣田: "口唇運動のビデオ画像に基づく/aCa/環境下での子音の分類"日本音響学会講演論文集. 2-2-5 (1992)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      1990 Final Research Report Summary
  • [Publications] 栗田、本多、垣田: "口唇形状を生成する生理学的モデル"日本音響学会講演論文集. 3-5-5 (1992)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      1990 Final Research Report Summary
  • [Publications] 栗田、本多、垣田: "唇音の生成に関する筋電図学的研究-閉鎖,円唇化と調音結合-"日本音響学会講演論文集. 2-2-4 (1992)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      1990 Final Research Report Summary
  • [Publications] 倉部勇一: "眼球運動を利用したポインタ・インターフェースの基礎研究"平成5年度 金沢工業大学修士学位論文(未公表学位論文). 50 (1993)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      1990 Final Research Report Summary
  • [Publications] 津守正樹: "口唇運動による音韻情報の抽出"平成5年度 金沢工業大学修士学位論文(未公表学位論文). 79 (1993)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      1990 Final Research Report Summary
  • [Publications] Kurita, Honda, and Kakita: "Speech analysis referring lip gestures"Trans. Speech Res. IECIE. SP88-94(In Japanese). 41-48 (1988)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1990 Final Research Report Summary
  • [Publications] Honda, Kusakawa and Kakita: "Sequential control model of speech articulation on producing word utterances"1990, Proc. Int. Conf. on Spoken Language Processing, Kobe. 445-448

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1990 Final Research Report Summary
  • [Publications] Kusakawa, Honda and Kakita: "Sequential control model of speech articulation on producing word utterances"Proc. Acoust. Soc. Jpn.. (In Japanese). 3-6-10 (1991)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1990 Final Research Report Summary
  • [Publications] Kusakawa, Honda and Kakita: "2-D model of the tongue derived by EMG"Proc. Acoust. Soc. Jpn.. (In Japanese). 2-2-11 (1992)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1990 Final Research Report Summary
  • [Publications] Kurabe and Kakita: "Estimation of eye point -Application to aid for communicative handicap-"Proc. BME Soc. Jpn.. (In Japanese). 1-B-3 (1992)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1990 Final Research Report Summary
  • [Publications] Tsumori and Kakita: "Differentiation of consonants in the context of /aCa/ based on the video images of the lip movements"Proc. Acoust. Soc. Jpn.. (In Japanese). 2-2-5 (1992)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1990 Final Research Report Summary
  • [Publications] Kurita, Honda, and Kakita: "Physiological model generating lip gestures"Proc. Acoust. Soc. Jpn.. (In Japanese). 3-5-5 (1992)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1990 Final Research Report Summary
  • [Publications] Kurita, Honda, and Kakita: "EMG study of generating labial articulation -closure, rounding and coarticulation-"Proc. Acoust. Soc. Jpn.. (In Japanese). 2-2-4 (1992)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1990 Final Research Report Summary
  • [Publications] Kurabe: "Basic study on pointer interface using eye movement"(Unpublished MS Thesis) Kanazawa Institute of Technology (In Japanese). (1993)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1990 Final Research Report Summary
  • [Publications] Tsumori: "Extraction of phonetic information from lip movements"(Unpublished MS Thesis) Kanazawa Institute of Technology (In Japanese). (1993)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1990 Final Research Report Summary
  • [Publications] Kurita: "Study on generation and perception of lip articulation combining the speech production and audiovisual processing mechanism"(Unpublished Ph.D. Thesis) Kanazawa Institute of Technology (In Japanese). (1994)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1990 Final Research Report Summary
  • [Publications] 栗田知好・本多清志・垣田有紀: 電子情報通信学会技術研究報告. SP88-42. 43-46 (1988)

    • Related Report
      1988 Annual Research Report

URL: 

Published: 1988-04-01   Modified: 2017-10-19  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi