Learner Corpus-based Research on Automatic Evaluation Methods forReading Ability and Writing Ability

2012 Fiscal Year Final Research Report

• Project/Area Number: 22300299
• Research Category: Grant-in-Aid for Scientific Research (B)
• Allocation Type: Single-year Grants
• Section: 一般
• Research Field: Educational technology
• Research Institution: Kansai Gaidai University
• Principal Investigator: KOTANI Katsunori 関西外国語大学, 外国語学部, 准教授 (30440994)
• Co-Investigator(Kenkyū-buntansha): YOSHIMI Takehiko 龍谷大学, 理工学部, 准教授 (50368031) ; ISAHARA Hitoshi 豊橋技術科学大学, 情報メディア基盤センター, 教授 (20358881) ; NANJO Hiroaki 龍谷大学, 理工学部, 助教 (50388162)
• Project Period (FY): 2010 – 2012
• Keywords: 学習者コーパス / 自動評価法 / 作文 / 発音 / 読解 / 聴解

Research Abstract

The present study constructed a learner corpus by collecting linguistic data generated in learners' writing, pronunciation, reading comprehension, and listening comprehension. This learner corpus was subsequently used for developing automatic evaluation methods for learners' linguistic skills of writing, pronunciation, reading comprehension and listening comprehension. In addition, automatic evaluationmethods were developed to measure the difficulty of sentences for each linguistic skill. Experimental results for the validation of these automatic evaluation methods showed the efficacy of our learner corpus and automatic evaluation methods for multiple linguistic skills.

Research Products

• [Journal Article] Corpus Materials for Constructing Learner Corpus Compiling Speaking, Writing, Listening, and Reading Data (2013)
  • Author(s): Kotani, K., Yoshimi, T. Nanjo, H. and Isahara, H.
  • Journal Title: International Journal of Computational Linguistics and Applications
  • Peer Reviewed
• [Journal Article] Application of Reading Data in an Integrated Learner Corpus (2013)
  • Author(s): Kotani, K., Yoshimi, T. and Isahara, H.
  • Journal Title: Procedia Social and Behavioral Sciences
  • Peer Reviewed
• [Journal Article] Automatic Classification of Texts Written by Learners of English as a Foreign Language based on Linguistic Features and Learner Features, Proceedings of International Technology (2013)
  • Author(s): Kotani, K., Yoshimi. T. and Uchida, M
  • Journal Title: Education and Development Conference (INTED) Pages: 6305--6314
  • Peer Reviewed
• [Journal Article] 統合的日本手話学習者コーパスの開発構想 (2012)
  • Author(s): 小谷克則,吉見毅彦,小谷志穂,井佐原均
  • Journal Title: 手話学研究 Volume: Vol.21 Pages: 25--43
  • Peer Reviewed
• [Journal Article] Use of a New Set of Linguistic Features to Improve Automatic Assessment of Text Readability (2012)
  • Author(s): Yoshimi, T., Kotani, K. and Isahara, H.
  • Journal Title: US-China Education Review B Volume: Vol.2, No.1 Pages: 55--62
  • Peer Reviewed
• [Journal Article] Applicability of Readability Formulae to the Measurement of Sentence-level Readability (2012)
  • Author(s): Kotani, K., Yoshimi, T., Nanjo, H. and Isahara, H.
  • Journal Title: Proceedings of International Conference of Education, Research and Innovation Pages: 6023--6031
  • Peer Reviewed
• [Journal Article] Statistical Analysis of a Learner Corpus Integrating Linguistic Output and Language Processing Data in Speaking, Listening, Writing, andReading, G. R. S. Weir and S. Ishikawa (Eds.) (2011)
  • Author(s): Kotani, K., Yoshimi, T., Nanjo, H. and Isahara, H.
  • Journal Title: Corpora and Language Technologies in Teaching, Learning and Research, University of Strathclyde Press Pages: 21--28
  • Peer Reviewed
• [Journal Article] 読解効率に基づく英文読解能力測定テストの開発とその信頼性・妥当性の検証 (2011)
  • Author(s): 小谷克則,吉見毅彦,井佐原均
  • Journal Title: 情報処理学会論文誌 Volume: Vol.52,No.4 Pages: 1843--1851
  • Peer Reviewed
• [Journal Article] A Scoring Method for Second Language Writing Based on Word Alignment (2011)
  • Author(s): Kotani, K. and Yoshimi, T.
  • Journal Title: Procedia-Social and Behavioral Sciences Volume: No.27 Pages: 42--49
  • Peer Reviewed
• [Journal Article] A Machine Learning Approach to Measurement of Text Readability for EFL Learners Using Various Linguistic Features (2011)
  • Author(s): Kotani, K., Yoshimi, T. and Isahara, H.
  • Journal Title: US-China Education Review B Volume: Vol.1, No.6 Pages: 767--777
  • Peer Reviewed
• [Journal Article] Compiling Learner Corpus Data of Linguistic Output and LanguageProcessing in Speaking, Listening, Writing, and Reading (2011)
  • Author(s): Kotani, K., Yoshimi, T., Nanjo, H. and Isahara, H.
  • Journal Title: Proceedings of The 5th International Joint Conference on Natural Language Processing Pages: 1418--1422
  • Peer Reviewed
• [Journal Article] Second Language Writing Classification System Based on Word Alignment Distribution, Themes in Science and Technology Education (2010)
  • Author(s): Kotani, K. and Yoshimi, T.
  • Journal Title: Special Issue on ICT in Language Learning Volume: Vol.3, No.1&2 Pages: 223--238
  • Peer Reviewed
• [Journal Article] A Machine Learning-Based Evaluation Method for Machine Translation, Lecture Notes inComputer Science, Vol.6040, Springer (2010)
  • Author(s): Kotani, K., and Yoshimi, T.
  • Journal Title: Berlin Pages: 351--356
  • Peer Reviewed
• [Journal Article] Validation of a Reading Speed Test for Japanese Learners of English as a Foreign Language, G. R. S. Weir and S. Ishikawa (Eds.), Corpus, ICT, and Language Education (2010)
  • Author(s): Kotani, K., Yoshimi, T. and Isahara, H.
  • Journal Title: University of Strathclyde Press Pages: 215--224
  • Peer Reviewed
• [Journal Article] A Prediction Model of Foreign Language Reading Proficiency based on Reading Time and Text Complexity (2010)
  • Author(s): Kotani, K., Yoshimi, T. and Isahara, H.
  • Journal Title: US-China Education Review Volume: Vol.17, No.10 Pages: 1--9
  • Peer Reviewed
• [Journal Article] Classification of Language Learners' Sentences into Native Speaker-like or Non-native speaker-like Sentences Using Learner Sentences and Machine Translation Sentences as Training Data, Proceedings of International Conference ofEducation (2010)
  • Author(s): Kotani, K., Yoshimi, T. and Isahara, H.
  • Journal Title: Research and Innovation Pages: 2672--2677
  • Peer Reviewed
• [Presentation] An Application of Reading Part of the Integrated Learner Corpus (2013)
  • Author(s): Kotani, K., Yoshimi, T., Nanjo, H., and Isahara, H.
  • Organizer: 5th International Conference on Corpus Linguistics
  • Place of Presentation: Alicante, Spain.
  • Year and Date: 20130314-16
• [Presentation] 英語学習者の習熟度を考慮した文単位の聴解難易度予測式の構築 (2013)
  • Author(s): 上田翔太,南條浩輝,吉見毅彦,小谷克則
  • Organizer: 言語処理学会第19回年次大会
  • Place of Presentation: 名古屋大学
  • Year and Date: 20130312-15
• [Presentation] Automatic Classification of Texts Written by Learners of English as aForeign Language based on Linguistic Features and Learner Features, International Technology (2013)
  • Author(s): Kotani, K., Yoshimi, T. and Uchida, M.
  • Organizer: Education and Development Conference (INTED)
  • Place of Presentation: Valencia, Spain.
  • Year and Date: 20130304-06
• [Presentation] 英文読解能力測定モデルへの熟語難易度の導入 (2012)
  • Author(s): 奥村将成,吉見毅彦,南條浩輝,小谷克則
  • Organizer: FIT2012第11回情報科学技術フォーラム
  • Place of Presentation: 法政大学
  • Year and Date: 20120904-06
• [Presentation] 英語学習支援のための聴解コーパスの有効性の検証 (2012)
  • Author(s): 上田翔太,南條浩輝,吉見毅彦,小谷克則
  • Organizer: FIT2012第11回情報科学技術フォーラム
  • Place of Presentation: 法政大学
  • Year and Date: 20120904-06
• [Presentation] 英文読解速度テストに用いたテキストの難易度を考慮したテスト有効性の検証 (2012)
  • Author(s): 岡本隆志,南條浩輝,吉見毅彦,小谷克則
  • Organizer: FIT2012第11回情報科学技術フォーラム
  • Place of Presentation: 法政大学
  • Year and Date: 20120904-06
• [Presentation] Applicability of Readability Formulae to the Measurement of Sentence-level Readability, International Conference of Education (2012)
  • Author(s): Kotani, K., Yoshimi, T., Nanjo, H. and Isahara, H.
  • Organizer: Research and Innovation (ICERI)
  • Place of Presentation: Madrid, Spain.
  • Year and Date: 2012-11-21
• [Presentation] Corpus Materials for Constructing Learner Corpus Compiling Speaking, Writing, Listening, andReading (2012)
  • Author(s): Kotani, K., Yoshimi, T., Nanjo, H. and Isahara, H.
  • Organizer: 13th International Conference on Intelligent Text Processing and Computational Linguistics (CICLING2012)
  • Place of Presentation: New Delhi, India.
  • Year and Date: 2012-03-12
• [Presentation] Compiling Learner Corpus Data of Linguistic Output and language Processing in Speaking, Listening (2011)
  • Author(s): Kotani, K., Yoshimi, T., Nanjo, H. and Isahara, H.
  • Organizer: The5th International Joint Conference on Natural Language Processing (IJCNLP2011)
  • Place of Presentation: Kuala Lumpur, Malaysia.
  • Year and Date: 2011-11-11
• [Presentation] Statistical Analysis of a Learner Corpus Integrating Linguistic Output and Language Processing Data (2011)
  • Author(s): Kotani, K., Yoshimi, T., Nanjo, H. and Isahara, H.
  • Organizer: 7th International Conference Corpora and Language Technologies in Teaching, Learning and Research
  • Place of Presentation: Khon Kaen, Thailand.
  • Year and Date: 2011-09-11
• [Presentation] A Scoring Method for Second Language Writing Based on Word Alignment (2011)
  • Author(s): Kotani, K. and Yoshimi, T.
  • Organizer: 12th Pacific Association for Computational Linguistic Conference (PACLING 2011)
  • Place of Presentation: Kuala Lumpur, Malaysia.
  • Year and Date: 2011-07-19
• [Presentation] Classification of language learners' sentences into native speaker-like or non-native speaker-like sentences using learner sentences and machine (2010)
  • Author(s): Kotani, K. and Yoshimi, T.
  • Organizer: translation sentences as training data, International Conference of Education, Research and Innovation(ICERI2010)
  • Place of Presentation: Madrid, Spain.
  • Year and Date: 20101115-17
• [Presentation] Validation of a reading speed test for Japanese Learners of English as a Foreign Language (2010)
  • Author(s): Kotani, K., Yoshimi, T. and Isahara, H.
  • Organizer: The Teaching and Learning of Language International Conference 2010 Kyoto Conference
  • Place of Presentation: Kyoto, Japan.
  • Year and Date: 20100921-23
• [Presentation] A Machine Learning-Based Evaluation Method for Machine Translation (2010)
  • Author(s): Kotani, K. and Yoshimi, T.
  • Organizer: The 6th Hellenic Conference on Artificial Intelligence (SETN 2010)
  • Place of Presentation: Athens, Greece.
  • Year and Date: 20100504-07
• [Remarks]
  • URL: http://nlp.i.ryukoku.ac.jp/