• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Automatic Measuring of English Language Proficiency for Global Communications

Research Project

Project/Area Number 16300048
Research Category

Grant-in-Aid for Scientific Research (B)

Allocation TypeSingle-year Grants
Section一般
Research Field Intelligent informatics
Research InstitutionDoshisha University (2005-2007)
Advanced Telecommunications Research Institute International (2004)

Principal Investigator

YAMAMOTO Seiichi  Doshisha University, Faculty of Engineering, Professor (20374100)

Co-Investigator(Kenkyū-buntansha) SUMITA Eiichiro  Doshisha University, ATR, SLP Laboratories, Department head (90395020)
YASUDA Keiji  Doshisha University, ATR, SLP Laboratories, Researcher (50395018)
YANAGIDA Masuzo  Doshisha University, Faculty of Engineering, Professor (00116120)
YOSHIDA Kensaku  Sophia, University, Faculty of Foreign studies, Professor (80053718)
NISHINO Haruo  Doshisha University, Institute for Language and Culture, Associate Professor (50172680)
松本 一則  株式会社 KDDI研究所, 主任研究員
Project Period (FY) 2004 – 2007
Project Status Completed (Fiscal Year 2007)
Budget Amount *help
¥14,650,000 (Direct Cost: ¥13,900,000、Indirect Cost: ¥750,000)
Fiscal Year 2007: ¥3,250,000 (Direct Cost: ¥2,500,000、Indirect Cost: ¥750,000)
Fiscal Year 2006: ¥2,100,000 (Direct Cost: ¥2,100,000)
Fiscal Year 2005: ¥7,000,000 (Direct Cost: ¥7,000,000)
Fiscal Year 2004: ¥2,300,000 (Direct Cost: ¥2,300,000)
Keywordse-Learning / TOEIC / translatability / English corpus / learner corpus / fill-in-the-blank question / measuring of translation quality / measuring of proficiency / 学習者コーパス
Research Abstract

Purposes of this re arch are (1) creation of a learner corpus which contains English sentences translated by various Japanese subjects of different English proficiency and some additional items available for subjective and objective evaluation of translation quality, and (2) development of an automatic measuring method of English proficiency. Main research results are as follows ;
1. As tools for automatic measuring English proficiency, we created a learner corpus of 150,000 English sentences translated by 500 Japanese subjects, of which Japanese source sentences were randomly selected from BTEC (Basic Travel Expression Corpus), English-Japanese parallel corpus of 500,000 paired sentences, and textbooks on English for junior and senior high schools. Ten reference sentences per each source sentence were made to measure some features such as edit-distance, similarity of n-grams between the reference sentences and translated sentences by the subject. Qualities of some of translated sentenc … More es in the learner corpus were subjectively evaluated by native English speakers who are frequent speaker of Japanese.
2. Correlation were calculated between subjective evaluation by the English native speakers and objective evaluation such as BLEU for some English sentences translated by the subjects, and high correlation was assured if dozens of English sentences were utilized. This experimental result assured that the objective evaluation method of BLEU, which was proposed to objectively evaluate quality of sentences translated with MT technologies, can be used for measuring English proficiency of the subjects.
3. Different method for selecting target sentences for translation were checked necessary number of target sentences in order to obtain a high correlation score, and one selection method for obtaining high correlation with smaller number sentences was proposed.
4. The learner corpus is expected to be used for various applications besides automatic measuring of English proficiency. One of the other applications is to create a probabilistic language model which represents lexical and syntactical difference between utterances by native English speakers and Japanese. In order to verify the idea, we created a probabilistic language model such bi-gram and tri-gram and linearly interpolated them with a language models which were trained with BTEC corpus, a large text corpus of canonical English sentences. The experimental results using the linearly interpolated language model shown that the language model provided better word accuracy than a language model trained with BTEC alone. This shows that the language model adaptation between a model trained with the learner corpus and a model trained with large corpus of native English is effective for compensating for the mismatch between the lexical and syntactical characteristics of native speakers and second language speakers. Less

Report

(5 results)
  • 2007 Annual Research Report   Final Research Report Summary
  • 2006 Annual Research Report
  • 2005 Annual Research Report
  • 2004 Annual Research Report
  • Research Products

    (40 results)

All 2008 2007 2006 2005 2004

All Journal Article (27 results) (of which Peer Reviewed: 4 results) Presentation (11 results) Book (2 results)

  • [Journal Article] 機械翻訳の研究・開発における翻訳自動評価技術とその応用2008

    • Author(s)
      安田 圭志
    • Journal Title

      人工知能学会誌 23

      Pages: 2-9

    • Related Report
      2007 Annual Research Report
    • Peer Reviewed
  • [Journal Article] 日本人英語学習者による日英翻訳コーパスの開発2007

    • Author(s)
      喜多村 圭祐
    • Journal Title

      電子情報通信学会総合全国大会 D-15-25

      Pages: 1785-1785

    • Related Report
      2006 Annual Research Report
  • [Journal Article] Comparative study on corpora for speech translation2006

    • Author(s)
      Kikui, G.
    • Journal Title

      IEEE Trans.ASLP 14

      Pages: 1674-1682

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2007 Final Research Report Summary
    • Peer Reviewed
  • [Journal Article] The ATR Multilingual Speech-to-Speech Translation System2006

    • Author(s)
      S. Nakamura, K. Markov, H. Nakaiwa, G. Kikui, H. Kawai, T. Jitsuhiro, J. Zhang, H. Yamamoto, E. Sumita, S. Yamamoto
    • Journal Title

      IEEE Trans. ASLP Vol.14, No.2

      Pages: 365-376

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2007 Final Research Report Summary
  • [Journal Article] Using Multiple Edit Distances to Automatically Grade Outputs from Machine Translation Systems2006

    • Author(s)
      Y. Akiba, K. Imamura, E. Sumita, H. Nakaiwa, S. Yamamoto, H. Okuno
    • Journal Title

      IEEE Trans. ASLP Vol.14, No.2

      Pages: 393-402

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2007 Final Research Report Summary
  • [Journal Article] Comparative Study on Corpora for Speech Translation2006

    • Author(s)
      G. Kikui, S. Yamamoto, T. Takezawa, E. Sumita
    • Journal Title

      IEEE Trans. ASLP Vol.14, No.5

      Pages: 1674-1682

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2007 Final Research Report Summary
  • [Journal Article] The ATR multilingual speech-to-speech translation system2006

    • Author(s)
      中村 哲
    • Journal Title

      IEEE Trans. ASLP 14,2

      Pages: 365-376

    • Related Report
      2006 Annual Research Report
  • [Journal Article] Comparative study on corpora for speech translation2006

    • Author(s)
      菊井 玄一郎
    • Journal Title

      IEEE Trans. ASLP 14,5

      Pages: 1674-1682

    • Related Report
      2006 Annual Research Report
  • [Journal Article] Web上のテキスト情報と翻訳モデルを利用した翻訳品質評価法の検討2006

    • Author(s)
      宮下 広平
    • Journal Title

      情報処理学会 自然言語処理研究会 報告 NL-177

      Pages: 17-23

    • Related Report
      2006 Annual Research Report
  • [Journal Article] Using the Web to disambiguate acronyms2006

    • Author(s)
      隅田 英一郎
    • Journal Title

      Proc. of HLT-NAACL2006

      Pages: 161-164

    • Related Report
      2006 Annual Research Report
  • [Journal Article] Word pronunciation disambiguation using the Web2006

    • Author(s)
      隅田 英一郎
    • Journal Title

      Proc. of HLT-NAACL2006

      Pages: 165-168

    • Related Report
      2006 Annual Research Report
  • [Journal Article] WWW上のテキスト情報を利用した翻訳品質評価法の検討2006

    • Author(s)
      宮下広平
    • Journal Title

      情報処理学会 自然言語処理研究会 報告 NL-171

      Pages: 89-94

    • Related Report
      2005 Annual Research Report
  • [Journal Article] Using Multiple Edit Distances to Automatically Grade Outputs from Machine Translation Systems2006

    • Author(s)
      秋葉泰弘
    • Journal Title

      IEEE Trans.ASLP Vol.14 No.2

      Pages: 393-402

    • Related Report
      2005 Annual Research Report
  • [Journal Article] An objective method for evaluating speech translation systems:using a second language learner's corpus2005

    • Author(s)
      Yasuda, K.
    • Journal Title

      Trans.IEICE INF.& SYST. 88-D

      Pages: 569-577

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2007 Final Research Report Summary
    • Peer Reviewed
  • [Journal Article] Multiple translation-engine-based hypotheses and edit-distance-based rescoring for a greedy decoder for statistical machine translation2005

    • Author(s)
      Paul, M.
    • Journal Title

      IPSJ Journal 46

      Pages: 2753-2767

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2007 Final Research Report Summary
    • Peer Reviewed
  • [Journal Article] An Objective Method for Evaluating Speech Translation System : Using a Second Language Learner's Corpus2005

    • Author(s)
      K. Yasuda, F. Sugaya, T. Takezawa, G. Kikui, S. Yamamoto, M. Yanagida
    • Journal Title

      Trans. IEICE INF. & SYST. Vol.E88-D, No.3

      Pages: 569-577

    • NAID

      110003214220

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2007 Final Research Report Summary
  • [Journal Article] A proposal for an evaluation set selection method for corpus-based speech translation2005

    • Author(s)
      F. Sugaya, K. Yasuda, S. Yamamoto
    • Journal Title

      Systems and Computers in Japan Vol.36, No.1

      Pages: 1-11

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2007 Final Research Report Summary
  • [Journal Article] Automatic Grader of MT Outputs in Colloquial Style by Using Multiple Edit Distances2005

    • Author(s)
      Y. Akiba, K. Imamura, E. Sumita, H. Nakaiwa, S. Yamamoto, H. Okuno
    • Journal Title

      IPSJ Journal Vol.20, No.3-A

      Pages: 139-148

    • NAID

      10022005063

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2007 Final Research Report Summary
  • [Journal Article] Multiple Translation-Engine-Based Hypotheses and Edit-Distance-based Rescoring for a Greedy Decoder for Statistical Machine Translation2005

    • Author(s)
      M. Paul, E. Sumita, S. Yamamoto
    • Journal Title

      IPSJ Journal Vol.46, No.11

      Pages: 2753-2767

    • NAID

      130000058423

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2007 Final Research Report Summary
  • [Journal Article] Measuring Non-native Speakers' Profiency of English by Using a Test with Automatically-Generated Fill-in-the-Blank Questions2005

    • Author(s)
      隅田英一郎
    • Journal Title

      Proc.2nd Workshop on Building Educational Applications Using NLP

      Pages: 61-68

    • Related Report
      2005 Annual Research Report
  • [Journal Article] Multiple Translation-Engine-based Hypotheses and Edit-Distance-based Rescoring for a Greedy Decoder for Statistical machine Translation2005

    • Author(s)
      Michael Paul
    • Journal Title

      情報処理学会 論文誌 Vol.46 No.11

      Pages: 2753-2767

    • Related Report
      2005 Annual Research Report
  • [Journal Article] Generating Language Profiency Tests for "Anyone, Anytime, and Anyplace"2005

    • Author(s)
      隅田英一郎
    • Journal Title

      Proc. of EuroCALL

      Pages: 200-200

    • Related Report
      2005 Annual Research Report
  • [Journal Article] 機械翻訳システム評価法の最前線2005

    • Author(s)
      隅田英一郎
    • Journal Title

      情報処理 46巻5号

      Pages: 552-557

    • NAID

      110002768407

    • Related Report
      2005 Annual Research Report
  • [Journal Article] An Objective Method for Evaluating Speech Translation System : Using a Second Language Learner's Corpus2005

    • Author(s)
      安田圭志
    • Journal Title

      IEICE TRANSACTIONS on Information and Systems Vol.E88-D No.3

      Pages: 569-577

    • NAID

      110003214220

    • Related Report
      2004 Annual Research Report
  • [Journal Article] A Proposal for an Evaluation Set Selection Method for Corpus-Based Speech Translation2005

    • Author(s)
      菅谷史昭
    • Journal Title

      Systems and Computers in Japan Vol.36 No.1

      Pages: 1-11

    • Related Report
      2004 Annual Research Report
  • [Journal Article] 機械翻訳システム評価法の最前線2005

    • Author(s)
      隅田英一郎
    • Journal Title

      情報処理 46巻5号(掲載予定)

    • NAID

      110002768407

    • Related Report
      2004 Annual Research Report
  • [Journal Article] 英語能力測定のための空所補充問題の自動作成手法2004

    • Author(s)
      隅田英一郎
    • Journal Title

      電子情報通信学会,思考と言語研究会,信学技報 TL2004-22 WIT2004-56

      Pages: 17-22

    • Related Report
      2004 Annual Research Report
  • [Presentation] 英語学習者コーパスの開発と英語表現能力評価尺度の検討2008

    • Author(s)
      喜多村 圭祐
    • Organizer
      電子情報通信学会 教育工学研究会
    • Place of Presentation
      徳島大学
    • Year and Date
      2008-03-08
    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2007 Final Research Report Summary
  • [Presentation] 英語学習者コーパスの開発と英語表現能力評価尺度の検討2008

    • Author(s)
      喜多村 圭祐
    • Organizer
      電子情報通信学会教育工学研究会
    • Place of Presentation
      徳島大学
    • Year and Date
      2008-03-08
    • Related Report
      2007 Annual Research Report
  • [Presentation] Using the Web to Disambiguate Acronyms2006

    • Author(s)
      E. Sumita, F. Sugaya
    • Organizer
      HLT/NAACL2006
    • Place of Presentation
      New York
    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2007 Final Research Report Summary
  • [Presentation] Word Pronunciation Disambiguation using the Web2006

    • Author(s)
      E. Sumita, F. Sugaya
    • Organizer
      HLT/NAACL2006
    • Place of Presentation
      New York
    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2007 Final Research Report Summary
  • [Presentation] Generating Language Proficiency Tests for Anyone, Anytime, and Anyplace2005

    • Author(s)
      E. Sumita, F. Sugaya, S. Yamamoto
    • Organizer
      EuroCALL2005
    • Place of Presentation
      Cracow(Poland)
    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2007 Final Research Report Summary
  • [Presentation] Relieving Teachers and Students : Generating a Compact Language Proficiency Test Set2005

    • Author(s)
      F. Suagaya, E. Sumita, S. Yamamoto
    • Organizer
      EuroCALL2005
    • Place of Presentation
      Cracow(Poland)
    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2007 Final Research Report Summary
  • [Presentation] Incremental Methods to Select Test Sentences for Evaluating Translation Ability2004

    • Author(s)
      Y. Akiba, E. Sumita, H. Nakaiwa, S. Yamamoto, H. Okuno
    • Organizer
      LREC2004
    • Place of Presentation
      Lisbon(Portugal)
    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2007 Final Research Report Summary
  • [Presentation] Example-based Rescoring of Statistical Machine Translation Output2004

    • Author(s)
      M. Paul, E. Sumita, S. Yamamoto
    • Organizer
      HLT/NAACL2004
    • Place of Presentation
      Boston
    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2007 Final Research Report Summary
  • [Presentation] Using a Mixture of N-Best Lists from Multiple MT Systems in Rank-Su-Based Confidence Measure for MT Outputs2004

    • Author(s)
      Y Akiba, E. Sumita, H. Nakaiwa, S. Yamamoto, H. Okuno
    • Organizer
      COLING2004
    • Place of Presentation
      Geneva
    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2007 Final Research Report Summary
  • [Presentation] Automatic Measuring of English Language Proficiency Using MT Evaluation Technology2004

    • Author(s)
      K. Yasuda, F Sugaya, E. Sumita, T. Takezawa, G. Kikui, S. Yamamoto
    • Organizer
      COLING2004
    • Place of Presentation
      Geneva
    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2007 Final Research Report Summary
  • [Presentation] Measuring Non-native Speakers'Proficiency of English by Using a Test with Automatically Generated Fill-in-the-Blank Questions2004

    • Author(s)
      E. Sumita, F. Sugaya, S. Yamamoto
    • Organizer
      2nd Workshop on Building Education Applications Using Natural Language Processing at ACL
    • Place of Presentation
      Geneva
    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2007 Final Research Report Summary
  • [Book] Multilingual Speech Processing2006

    • Author(s)
      Shultz, T., et. al.
    • Total Pages
      508
    • Publisher
      Academic Press
    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2007 Final Research Report Summary
  • [Book] Multilingual Speech Processing2006

    • Author(s)
      Tanja Schultz
    • Total Pages
      508
    • Publisher
      Academic Press
    • Related Report
      2006 Annual Research Report

URL: 

Published: 2004-04-01   Modified: 2016-04-21  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi