English for Academic Purposes Testing Workshop
On Friday 8th January, seven of us who are involved in leading and teaching on the English Pre-sessional Programmes participated in a day-long Testing Workshop.
The workshop was delivered by two lecturers from the University of Reading’s International Study and Language Institute (ISLI): Professor John Slaght, Director of Assessment and Test Development, and Bruce Howell, a Surrey alumnus and one of the writers and markers for Reading University’s Test of English for Educational Purposes (TEEP), which is used to assess students’ levels in all four skills at Reading and in a number of other British universities. The purpose of the workshop was to provide us with some training in designing reliable and valid reading and listening tests. This is important in ensuring that only students who have achieved the required levels in these skills are able to progress from the year-round Pre-sessionals (PS1-3) onto the Summer Pre-sessionals and then onto their degree courses.
The day was divided into four sessions, two of which focussed on consideration of some of the more general points related to test development. These included the creation of test specifications, the use of integrated skills tests, specific issues related to test reliability and validity, and an introduction to the use of Excel to conduct simple statistical analyses of students’ answers to individual questions in order to improve reliability.
The other two sessions centred around the design of reading and listening tests. These included discussion of the advantages and disadvantages of different question types, the importance of including question types that test different sub skills rather than just comprehension, text authenticity, advice on searching for suitable texts, and using lexical frequency software to check the level of difficulty of the texts. We also analysed some poorly designed examples of tests to raise our awareness of the many potential pitfalls. The listening test session included some interesting discussion as to whether note-taking can be assessed and how many times students should be permitted to listen. Traditionally it has been thought that in the interests of replicating an authentic lecture they should only be allowed to listen once, but in this age of students recording lectures and lecture capture, perhaps listening twice is becoming more authentic.
By the end of the day we all felt much better equipped to design our own tests and less daunted by the prospect of statistical analysis. We are very grateful to our colleagues at Reading for sharing their wealth of knowledge and experience with us.