• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Vision-based Sign Language Recognition in Complicated Background and Occlusion

Research Project

Project/Area Number 15300058
Research Category

Grant-in-Aid for Scientific Research (B)

Allocation TypeSingle-year Grants
Section一般
Research Field Perception information processing/Intelligent robotics
Research InstitutionOsaka University

Principal Investigator

SHIRAI Yoshiaki  Osaka University, School of Engineering, Professor, 大学院・工学研究科, 教授 (50206273)

Co-Investigator(Kenkyū-buntansha) MIURA Jun  Osaka University, School of Engineering, Associate Professor, 大学院・工学研究科, 助教授 (90219585)
SAKIYAMA Takuro  Osaka University, School of Engineering, Research Associate, 大学院・工学研究科, 助手 (70335371)
SHIMADA Nobutaka  Ritsumeikan University, Associate Professor, 情報理工学部, 助教授 (10295371)
島田 伸敬  立命館大学, 情報理工学部, 助教授 (10294034)
Project Period (FY) 2003 – 2004
Project Status Completed (Fiscal Year 2004)
Budget Amount *help
¥8,500,000 (Direct Cost: ¥8,500,000)
Fiscal Year 2004: ¥4,600,000 (Direct Cost: ¥4,600,000)
Fiscal Year 2003: ¥3,900,000 (Direct Cost: ¥3,900,000)
KeywordsHand posure estimation / Sign Language Recognition / Complex background / 3-D Shape Model / Apperance variation modeling / Tracking racking in image sequence / Transitin Network / Hidden Markov Model / 3次元形状モデル / 見え変動の学習 / 動画像追跡
Research Abstract

This research achieved techniques which enable to recognize sign language words from image sequences in background-free conditions. It includes tree sub-themes below :
1.hand region detection and tracking in complex backgrounds using sign language word dictionary
Hand region and its shape change was stably detected and tracked even when the hand moves quickly in cluttered backgrounds using a Transition network built from examples of sign language scenes. The network registers hand state models using motion feature significantly when hand moves quickly or using shape feature when the motion is slow. A novel model-matching criterion based on the probabilities of image contrast occurrance in both of hand and background was proposed.
2.Sign language word recognition based on temporal and spacio image features
2-D image features including the number of finger-like parts, principal axes of hand region, position of hand relative to the face and 2-D motion parameters were extracted from about 50 s … More ign language words and trained Hidden Markov Model (HMM) for word recognition. Because mutual occlusion between both hands and the face, when occlusion is detected, each region is tracked using textures stored in the preceding frames. Recognition success rate was improved by using two stage matching : first rough matching by the HMM using only position and motion features is done, and then precise matching by the HMM using all of features is done.
3.Hand shape estimation based on 3-D shape model and learning of appearance deformation
3-D shape parameters was estimated from monocular image sequence based on silouette matching. A number of possible hand shape silhouette images are generated in advance and the pairs of the silhouette and the joint angles and viewing parameters are registered in a model database. Because the degree of freedom of human hand is quite large, the models have to been sparsely sampled. In compensation for it, locally-compressed manifold architechture, which extracts and stored the possible appearance changes around each model in a parametric way, is novelly proposed. A PC-clustered estimation system was built and established 10 fps estimation even far the complicatedly self-occluded hand shape. Less

Report

(3 results)
  • 2004 Annual Research Report   Final Research Report Summary
  • 2003 Annual Research Report

Research Products

(25 results)

All 2005 2004 2003 Other

All Journal Article (21 results) Publications (4 results)

  • [Journal Article] 輪郭の変形の学習による3-D手指姿勢の認識2005

    • Author(s)
      今井, 島田, 白井
    • Journal Title

      電子情報通信学会論文誌D-II (採録決定)

    • NAID

      110003225303

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] 3-D Hand Posture Recognition by Learning Contour Variation2005

    • Author(s)
      A.Imai, N.Shimada, Y.Shirai
    • Journal Title

      IEICE Trans.Division D-II (in Japanese) (in printing)

    • NAID

      110003225303

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] HMMを用いた手指単語の認識2004

    • Author(s)
      金山, 白井, 島田
    • Journal Title

      電子情報通信学会PRMU/WIT/MI研究会技術報告 PRMU2004-16

      Pages: 21-28

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] 手話認識のための動画像を用いた複雑背景における手指形状推定2004

    • Author(s)
      浜田, 島田, 白井
    • Journal Title

      第43回日本エム・イー学会予稿集

      Pages: 203-203

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] Hand Shape Estimation under Complex Backgrounds for Sign Language Recogntion2004

    • Author(s)
      Y.Hamada, N.Shimada, Y.Shirai
    • Journal Title

      Proc.Of 6^<th> Int.Conf.on Automatic Face and Gesture

      Pages: 589-594

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] 3-D Hand Posture Recogntion by Training Contour Variation2004

    • Author(s)
      A.Imai, N.Shimada, Y.Shirai
    • Journal Title

      Proc.Of 6^<th> Int.Conf.on Automatic Face and Gesture

      Pages: 895-900

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] 輪郭の変形の学習による3-D手指姿勢の認識2004

    • Author(s)
      今井, 島田, 白井
    • Journal Title

      画像の認識・理解シンポジウムMIRU2004予稿集

      Pages: 672-677

    • NAID

      110003225303

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] Sign Language Word Recognition using HMM2004

    • Author(s)
      K.Kanayama, Y.Shirai, N.Shimada
    • Journal Title

      IEICE PRMU/WIT/MI2004-16

      Pages: 21-28

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] Hand Shape Estimation from Image Sequence under Complex Background for Sign Language Recognition2004

    • Author(s)
      Y.Hamada, N.Shimada, Y.Shirai
    • Journal Title

      Proc.of 43th Conf.of Japan Society of Medical Electronics and Biological Engineering

      Pages: 203-203

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] Hand Shape Estimation under Complex Backgrounds for Sign Language Recogntion2004

    • Author(s)
      Y.Hamada, N.Shimada, Y.Shirai
    • Journal Title

      Proc.of IEEE Int.Conf Automatic Face and Gesture Recognition

      Pages: 589-594

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] 3-D Hand Posture Recognition by Training Contour Variation2004

    • Author(s)
      A.Imai, N.Shimada, Y.Shirai
    • Journal Title

      Proc.of IEEE Int.Conf Automatic Face and Gesture Recognition

      Pages: 895-900

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] 3-D Hand Posture Recognition by Learning Contour Variation2004

    • Author(s)
      A.Imai, N.Shimada, Y.Shirai
    • Journal Title

      Proc.of Meeting on Image Recognition and Understanding 2004

      Pages: 672-677

    • NAID

      110003225303

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] HMMを用いた手話単語の認識2004

    • Author(s)
      金山和功, 白井良明, 島田伸敬
    • Journal Title

      電子情報通信学会技報WIT/PRMU/MI 2003-104-89

      Pages: 21-28

    • NAID

      110003171998

    • Related Report
      2004 Annual Research Report
  • [Journal Article] 手話認識のための動画像を用いた複雑背景における手指形状推定2004

    • Author(s)
      浜田康志, 島田伸敬, 白井良明
    • Journal Title

      第43回日本エム・イー学会予稿集 CDROM

    • Related Report
      2004 Annual Research Report
  • [Journal Article] Hand Shape Estimation under Complex Backgrounds for Sign Language Recognition2004

    • Author(s)
      Y.Hamada, N.Shimada, Y.Shirai
    • Journal Title

      Proc.of 6th Int.Conf.on Automatic Face and Gesture Recognition

      Pages: 589-594

    • Related Report
      2004 Annual Research Report
  • [Journal Article] 3-D Hand Posture Recognition by Training Contour Variation2004

    • Author(s)
      A.Imai, N.Shiamda, Y.Shirai
    • Journal Title

      Proc.of 6th Int.Conf. on Automatic Face and Gesture Recognition

      Pages: 895-900

    • Related Report
      2004 Annual Research Report
  • [Journal Article] Recognition of Two-Hand Gestures Using Coupled Linear Model2003

    • Author(s)
      M.H.Jeong, Y.Kuno, N.Shimada, Y.Shirai
    • Journal Title

      IEICE Trans.Division D E86-D, No.8

      Pages: 1416-1425

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] 手話認識のための複雑背景における手指形状推定2003

    • Author(s)
      浜田, 島田, 白井
    • Journal Title

      電子情報通信学会PRMU研究会技術報告 PRMU2003-152

      Pages: 17-19

    • NAID

      110003272013

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] 手話認識のための画像に基づく非接触手指形状計測2003

    • Author(s)
      島田, 白井
    • Journal Title

      第17回日本エム・イー学会秋季大会 OS06-02(CDROM)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] Hand Shepe Estimation for Sign Language Recognition under Complex Background2003

    • Author(s)
      Y.Hamada, N.Shimada, Y.Shirai
    • Journal Title

      IEICE PRMU Tech.Report PRMU2003-152

      Pages: 17-19

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] Image-based Contactless Hand Shape Estimation under Complex Background for Sign Laguage Recognition2003

    • Author(s)
      N.Shimada, Y.Shirai
    • Journal Title

      Proc.of 17^<th> Automn Conf.of Japan Society of Medical Electronics and Biological Engineering OS06-02(CDROM)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Publications] M.H.Jeong, Y.Kuno, N.Shimada, Y.Shirai: "Recognition of Two-Hand Gestures Using Coupled Switching Linear Model"IEICE Transactions Division D. Vol.E86-D, No.8. 1416-1425 (2003)

    • Related Report
      2003 Annual Research Report
  • [Publications] 浜田康志,, 島田伸敬, 白井良明: "手話認識のための複雑背景における手指形状認識"電子情報通信学会PRMU研究会技術報告. 2003-152. 7-12 (2003)

    • Related Report
      2003 Annual Research Report
  • [Publications] A.Imai, N.Shimada, Y.Shira: "3-D Hand Posture Recognition by Training Contour Variation"Proc.of 6th Int.Conf.on Automatic Face and Gesture Recognition. (発表予定). (2004)

    • Related Report
      2003 Annual Research Report
  • [Publications] Y.Hamada, N.Shimada, Y.Shirai: "Hand Shape Estimation under Complex Backgrounds for Sign Language Recognition"Proc.of 6th Int.Conf.on Automatic Face and Gesture Recognition. (発表予定). (2004)

    • Related Report
      2003 Annual Research Report

URL: 

Published: 2003-03-31   Modified: 2016-04-21  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi