• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

An investigation of rater variability in the context of teacher employment examinations using a mixed approach

Research Project

Project/Area Number 25370726
Research Category

Grant-in-Aid for Scientific Research (C)

Allocation TypeMulti-year Fund
Section一般
Research Field Foreign language education
Research InstitutionBunkyo University

Principal Investigator

Akiyama Tomoyasu  文教大学, 文学部, 教授 (20383218)

Project Period (FY) 2013-04-01 – 2017-03-31
Project Status Completed (Fiscal Year 2016)
Budget Amount *help
¥2,730,000 (Direct Cost: ¥2,100,000、Indirect Cost: ¥630,000)
Fiscal Year 2015: ¥910,000 (Direct Cost: ¥700,000、Indirect Cost: ¥210,000)
Fiscal Year 2014: ¥1,300,000 (Direct Cost: ¥1,000,000、Indirect Cost: ¥300,000)
Fiscal Year 2013: ¥520,000 (Direct Cost: ¥400,000、Indirect Cost: ¥120,000)
Keywords教員採用試験 / パフォーマンステスト / 模擬授業 / 評価者 / 言語テスト / 評価過程 / 信頼性 / 応用言語学 / 言語 / 評価者認知
Outline of Final Research Achievements

This study aims to examine possible reasons of rater variability in employing prospective English teachers, both quantitatively (Many-Facet Rasch Analysis, MFRA) and qualitatively (think-aloud method). Thirty candidates undertook microteaching tests. Seventeen raters rated all candidates using assessment criteria. Results indicate that most of the raters show consistent rating patterns but with various levels of severity. Results of the think-aloud method show that raters rated candidates not only with different interpretations of and different attention to subcategories of assessment criteria but also according to different core teaching values and ideal images of English teachers or English classes possessed by the rater themselves, leading to divergence between lenient and harsh ratings of the same candidates by different raters.

Report

(5 results)
  • 2016 Annual Research Report   Final Research Report ( PDF )
  • 2015 Research-status Report
  • 2014 Research-status Report
  • 2013 Research-status Report
  • Research Products

    (23 results)

All 2017 2016 2015 2014 2013 Other

All Journal Article (4 results) (of which Peer Reviewed: 3 results,  Acknowledgement Compliant: 2 results,  Open Access: 1 results) Presentation (15 results) (of which Int'l Joint Research: 5 results,  Invited: 1 results) Book (3 results) Remarks (1 results)

  • [Journal Article] Assessing teacher, 20th Anniversary2016

    • Author(s)
      Tomoyasu Akiyama
    • Journal Title

      Issue of JLTA Journal.

      Volume: special issue Pages: 168-172

    • Related Report
      2016 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Assessing teacher2016

    • Author(s)
      Tomoyasu Akiyama
    • Journal Title

      JLTA 20周年記念特別号

      Volume: 12 Pages: 1-5

    • Related Report
      2015 Research-status Report
    • Peer Reviewed / Acknowledgement Compliant
  • [Journal Article] 採用試験の評価者は模擬授業をどのように評価しているのか-think-aloud 法を用いて-2014

    • Author(s)
      秋山朝康
    • Journal Title

      英語英文学41号

      Volume: 41 Pages: 9-22

    • Related Report
      2014 Research-status Report
    • Open Access / Acknowledgement Compliant
  • [Journal Article] 採用試験の評価者は模擬授業をどのように評価しているのか2014

    • Author(s)
      秋山朝康
    • Journal Title

      英語英文学

      Volume: 41 Pages: 9-22

    • Related Report
      2013 Research-status Report
    • Peer Reviewed
  • [Presentation] Searching into rater variability in employing English teachers using multiple analytic approaches2017

    • Author(s)
      Tomoyasu
    • Organizer
      AAAL
    • Place of Presentation
      Oregon, USA
    • Year and Date
      2017-03-18
    • Related Report
      2016 Annual Research Report
    • Int'l Joint Research
  • [Presentation] A Closer Take a Look at Rater Bias in English Teacher Employment Examinations2016

    • Author(s)
      Tomoyasu Akiyama
    • Organizer
      The Pan-Pacific Association of Applied Linguistics
    • Place of Presentation
      Tamkang University, Taiwan
    • Year and Date
      2016-08-22
    • Related Report
      2016 Annual Research Report
    • Int'l Joint Research
  • [Presentation] An Investigation of Rater Variability in the Context of Teacher Employment Examinations Using a Mixed Approach2016

    • Author(s)
      Tomoyasu Akiyama
    • Organizer
      International Symposium on Language, Linguistics, Literature and Education
    • Place of Presentation
      Loisir Hotel, Okinawa, Japan
    • Year and Date
      2016-08-02
    • Related Report
      2016 Annual Research Report
  • [Presentation] Investigations of teacher employment examinations: How do raters assess prospective English teachers?2015

    • Author(s)
      Tomoyasu Akiyama
    • Organizer
      ALAA/ALANZ/ALTAANZ 2015
    • Place of Presentation
      University of South Australia, AUS
    • Year and Date
      2015-11-30
    • Related Report
      2015 Research-status Report
    • Int'l Joint Research
  • [Presentation] An exploration of rater cognition: The impact of rater differences on teacher employment examinations2015

    • Author(s)
      Tomoyasu Akiyama
    • Organizer
      The East Coast Organization of Language Testers (ECOLT)
    • Place of Presentation
      Georgetown University, USA
    • Year and Date
      2015-10-09
    • Related Report
      2015 Research-status Report
    • Int'l Joint Research
  • [Presentation] An Examination of Rater Orientations in English Teacher Employment Examinations,2015

    • Author(s)
      Tomoyau Akiyama
    • Organizer
      CASE International Conference,
    • Place of Presentation
      in University of Hong Kong, HK.
    • Year and Date
      2015-06-11
    • Related Report
      2015 Research-status Report
    • Int'l Joint Research
  • [Presentation] The possibility of introducing speaking tests  nto senior high school entrance examinations2014

    • Author(s)
      Tomoyasu Akiyama
    • Organizer
      JLTA(日本言語テスト学会)
    • Place of Presentation
      立命館大学
    • Year and Date
      2014-10-20
    • Related Report
      2014 Research-status Report
    • Invited
  • [Presentation] Who Assesses Tomorrow’s English Teachers?” An impact of Rater Differences on Teacher Employment Decisions2014

    • Author(s)
      Tomoyasu Akiyama
    • Organizer
      AILA
    • Place of Presentation
      ブリスベン、豪州
    • Year and Date
      2014-08-10 – 2014-08-15
    • Related Report
      2014 Research-status Report
  • [Presentation] The Establishment of Modelsfor Integrated-skills Assessment. From the Standpoint of Assessment Literacy –2014

    • Author(s)
      Yuji Nakamura, Tomoyasu Akiyama, Yasuko Ito, Kahoko Matsumoto, Kei Miyazaki*, Adam
    • Organizer
      AILA (Applied Linguistics Association Conference)
    • Place of Presentation
      ブリスベン、豪州
    • Year and Date
      2014-08-10 – 2014-08-15
    • Related Report
      2014 Research-status Report
  • [Presentation] Examining test fairness: Focusing on Rater Behavior in the Case of Employing Prospective English Teachers2014

    • Author(s)
      Tomoyasu Akiyama
    • Organizer
      ALTE (The Association of Language Testers in Europe)
    • Place of Presentation
      フランス、パリ
    • Year and Date
      2014-04-10 – 2014-04-12
    • Related Report
      2014 Research-status Report
  • [Presentation] 評価者はどのように評価しているのか~教員採用試験の模擬授業の場合~2013

    • Author(s)
      秋山朝康
    • Organizer
      全国英語教育学会
    • Place of Presentation
      北星学園
    • Related Report
      2013 Research-status Report
  • [Presentation] Investigations of Teacher Employment Examinations: How do Raters Assess Prospective English teachers?2013

    • Author(s)
      秋山朝康
    • Organizer
      International Conference on Language Learning and Teaching
    • Place of Presentation
      Park Royal Hotel Batu Feringghi, Penang, Malaysia
    • Related Report
      2013 Research-status Report
  • [Presentation] Issues with integrative teaching & assessment in Japan: From assessment (exit) to teaching (entrance)2013

    • Author(s)
      秋山朝康
    • Organizer
      International Conference on Language Learning and Teaching
    • Place of Presentation
      Park Royal Hotel Batu Feringghi, Penang, Malaysia
    • Related Report
      2013 Research-status Report
  • [Presentation] Issues Surrounding Integrated-skills Assessment in Japan - An Attempt toward the Improvement of Integrated-skills Speaking Tests2013

    • Author(s)
      中村優治・秋山朝康・松本佳穂子・伊藤泰子・宮崎啓・土平泰子
    • Organizer
      Asia TEFL
    • Place of Presentation
      Ateneo de Manila University,Philippines
    • Related Report
      2013 Research-status Report
  • [Presentation] スキル統合型評価モデルの確立2013

    • Author(s)
      中村優治・秋山朝康・松本佳穂子・伊藤泰子・宮崎啓・土平泰子
    • Organizer
      関東甲信越英語教育学会
    • Place of Presentation
      松本歯科大学
    • Related Report
      2013 Research-status Report
  • [Book] Studies in Language Testing vol. 44, Language Assessment for Multilingualism2016

    • Author(s)
      Tomoyasu Akiyama
    • Total Pages
      24
    • Publisher
      Cambridge University Press
    • Related Report
      2016 Annual Research Report
  • [Book] Language assessment for multingualism2016

    • Author(s)
      Tomoyasu Akiyama
    • Total Pages
      25
    • Publisher
      Cambridge University Press
    • Related Report
      2015 Research-status Report
  • [Book] Language Assessment for Multilingualism2016

    • Author(s)
      Tomoyasu Akiyama
    • Total Pages
      25
    • Publisher
      Cambridge English Language Assessment and Cambridge University Press.
    • Related Report
      2014 Research-status Report
  • [Remarks] NVivo sucess story

    • Related Report
      2013 Research-status Report

URL: 

Published: 2014-07-25   Modified: 2019-07-29  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi