• LTRG Home
• Membership
• Programme
• Projects
• PhD Students' Research
• Language Testing Update
• Resources in Language Testing
• LAEL Home

IELTS Impact Study

This research project, funded by the University of Cambridge Local Examinations Syndicate, developed research instruments suitable for the investigation of the impact of the IELTS test. It comprised four sub-projects: instruments to enable the analysis of teaching materials, both published and unpublished; observation schedules for classroom observation and questionnaires for teachers and students post observation; questionnaires to investigate the attitudes of various stakeholders in the test; and instruments designed to capture in-depth information about candidates on a semi-routine basis.

A total of 11 instruments were developed and piloted:

Impact on the classroom

Classroom observation schedule

Teacher's post-observation questionnaire/interview schedule

Students' post-observation questionnaire

Impact on materials design

Textbook analysis

Impact on attitudes towards IELTS

Administrators' questionnaire

IELTS Teachers' questionnaire

EAP Teachers' post-IELTS questionnaire

Questionnaire for subject teachers (in accepting institution)

Questionnaire for students currently studying for IELTS

Questionnaire for students who have taken IELTS

Test-takers' language learning behaviour

Test-takers' background questionnaire (including sections on: (i) attitudes to learning English and taking tests; (ii) how you learn English; (iii) what you do when you learn English; (iv) what you do in language tests)

Once these instruments had been designed, the question arose of how their validity and reliability might be established. This led to further funding in which procedures for validating data collection instruments (such as questionnaires and interview schedules) were investigated. A report was submitted to the University of Cambridge Local Examinations Syndicate (UCLES) in December 1996.

This work has also been presented as a paper (Alderson, J.C. and Banerjee, J. (1997) Validation and Impact Studies) at the annual conference of the American Association of Applied Linguistics (1997) and has been published as Alderson, J.C. and Banerjee J., (2001) Impact and washback research in language testing, in C. Elder, A. Brown, E. Grove, K. Hill, N. Iwashita, T. Lumley, T. McNamara and K. O’Loughlin (eds.), Experimenting with uncertainty: Essays in honour of Alan Davies, Cambridge: CUP, pp. 150 – 161.

A major finding of this work was that it is common to pilot data collection instruments but relatively rare to validate them. Drawing largely on validation literature in testing and evaluation, the report emphasised the importance of establishing the reliability and validity of such instruments i.e. whether they are capturing the same information each time they are administered or, indeed, whether they are capturing the information that the researchers intend them to capture.

Further information is available from Charles Alderson or Jayanti Banerjee