• LTRG Home
• Membership
• Programme
• Projects
• PhD Students' Research
• Language Testing Update
• Resources in Language Testing
• LAEL Home

TOEFL Impact Study: The impact of changes in the TOEFL on teaching in Central and Eastern Europe

Researchers: Dianne Wall (Principal Investigator) and Tania Horák (Research Associate)

This project is funded by Educational Testing Services (ETS), the producers of the Test of English as a Foreign Language (TOEFL). The TOEFL is an examination of English language proficiency taken by non-native speakers who wish to study in the medium of English at institutions of higher education in the United States and many other countries. The examination is generally described as 'high stakes', as the results contribute to the institutions consider when deciding whether to accept students unconditionally, to require them to take additional language classes before enrolling, or to reject them.

A new version of this exam (the internet-Based TOEFL, or 'iBT') was launched in several countries in September 2005 and introduced in much of the rest of the world in 2006. ETS commissioned two studies to look at the 'washback' that the iBT might produce on teaching and learning in various locations around the world. Our study looks at six countries in Central and Eastern Europe .

In Phase 1 (January 2003 to June 2004) we produced a baseline study documenting TOEFL exam preparation practices before it was generally known that the exam would be undergoing changes. It was necessary to determine what effects the new examination was intended to have, not only so that we could measure whether it was present in classes after the introduction of the exam but so that we could see whether these features were indeed absent from the classes that were currently taking place at that time. We combined the results of our survey of intention with a description we had built up of what the new examination would look like and our understanding of previous washback studies, and compiled interview and observation schedules for use in the institutions in our sample. We visited 10 institutions; interviewed 10 teachers, 21 students and 10 directors of study; and we observed both TOEFL preparation and non-TOEFL preparation lessons. We analysed our data using Henrichsen's (1989) Hybrid Model of the Diffusion Diffusion/Implementation of Innovations. The results have been published in TOEFL Monograph MS-34 (details below).

Phase 2 of the project lasted from October 2004 to March 2006. The title of this phase was Coping with Change'. We investigated teachers' reactions during the transition period between the first release of detailed information about the iBT and the time it went 'live'. We collected data on what teachers understood of the new examination and the thinking that lay behind it and how this affected their plans for preparation classes as the introduction date grew nearer. We used computer-mediated communication to maintain regular contact with the teachers, to probe their awareness and attitudes and to find out what materials and methods they deemed most appropriate for preparing for the new exam. The results have been published in the new TOEFL iBT research series - TOEFLiBT-05 (details below).

Phase 3 of the project lasted from April 2006 to March 2007. We continued to track teachers and developments in their institutions during the early days of iBT, concentrating on their choice and use of TOEFL preparation coursebooks.

Phase 4 (the final phase) lasted from April 2007 to March 2008. In this phase we re-visited three of the original research sites, conducting interviews and observations that were parallel to the ones we conducted in Phase 1. Our goal was to find out whether any changes had occurred in teaching and to determine whether there was an 'evidential link' (Messick 1996) which tied any changes in the classroom practices to the changes in the TOEFL examination. The report for this phase will be available at the end of Summer 2008.

References

Henrichsen, L E (1989) Diffusion of innovations in English language teaching: The ELEC effort in Japan , 1956-1968. New York : Greenwood Press.

Messick, S (1996) Validity and Washback in Language Testing. Language Testing 13(3). pp 241-256.

Wall, D., & Horák, T. (2006). The impact of changes in the TOEFL examination on teaching and learning in central and eastern Europe . Phase I: The baseline study (TOEFL Monograph No. MS-34). Princeton , NJ : ETS.

Wall, D., & Horák, T. (2008). The impact of changes in the TOEFL examination on teaching and learning in central and eastern Europe . Phase 2: Coping with change. (TOEFLiBT-05). Princeton , NJ : ETS.

For further information about the TOEFL Impact Study please contact Dianne Wall, Principal Investigator.