Test Specifications, Assessors Guidelines, and Assessment Criteria - Literature review Example

Evaluative report on speaking test The testing instrument that this paper is all about is intended to be a placement test that is meant to gauge the English language competency of two (2) Saudi nationals, who have recently moved to the UK and planned to enroll in secondary education. Both of them are assessed to be in level B1 of CEFR. The testing instrument specifically dwells on the most significant competencies that are needed for speaking or oral communication. A survey of literature The concerned testing instrument zeroes in on the three competencies of task achievement and communication skills, clarity and naturalness of speech and the range of accuracy of vocabulary and familiarity with grammatical rules. Facilitating interpersonal relationships, communication skills relate to competence to relay one’s message to other people clearly and unambiguously and to receive information that others are conveying with as little distortion as possible. Ellis (2003) points out that that skills to communicate are not only learned formally – as in schools – but also through consistent practice and reflection on one’s experiences in communication (see pp. 3). These skills are varied; but, in this paper, it is the students’ oral communication skills – including the sub-skills related to sentence structure, pronunciation, word endings, tenses – among others (see Condelli, Wrigley & Kwang 2009, pp. 133) that is singled out. Clarity and naturalness (or obviousness) of speech is the ability of a speaker to produce clear and natural speech by using standard pronunciation and stress and by producing fluent utterances (Mewald, Gassner & Sigott 2009, pp. 1). Contributing to speech intelligibility (Nusbaum, Francis, & Henly 1995; see also Ratcliff, Coughlin & Lehman 2002), they are considered among the most important points for conversation test scenarios (see Moeller 2000, pp. 76). One’s range and accuracy of vocabulary and familiarity with grammatical rules may be gauged by how well one communicates particularly in unplanned speech (Mewald, Gassner & Sigott 2009, pp. 1). Conventionally known, a grammar-less language would leave us handicapped. It involves syntax, or about how words can (and cannot) be combined to form a sentence (Batstone 2003, pp. 3). And, as such, grammar is an important aspect of second language acquisition. The speaking test – specifically, its adopted format – is patterned after the three-part International English Language Testing System (IELTS). Every part of the IELTS speaking test is designed to ascertain whether the test taker is able to express himself or herself in English on personal and general situations. And, IELTS is marked in consideration of the four features of language – i.e., fluency and coherence, vocabulary, grammar and pronunciation (see Jakeman & McDowell 2008, pp. 78-79). Too, it resembles the oral/speaking examination for the Cambridge First Certificate in English (FCE) exam (see The Oral (Speaking) Test of the FCE Exam [n.d.]). Further, Mewald, Gassner & Sigott (2009) is very close to the format and order of this paper’s testing instrument; in fact, the scale that was used to gauge the English competence of the test takers and the allocation of time for every section of the test are likewise based on theirs. What differentiates this paper’s testing instrument is how the topics were pursued through promptings – e.g., in the form of questions – by the interlocutor. As it will be said in the later section, the test takers were observably pausing and taking their time to answer. To make the testing adhere to allocated times, and to induce the test takers to continually speak, questions were raised by the interlocutor following the answers of the test takers. Thus, all throughout the testing, the individuality of the test takers was taken into account as the testing was ensured to be student-centered. Test specifications, assessors’ guidelines and assessment criteria As earlier stated, the test takers are two Saudi Arabian nationals – Mohammed, 15, and (state the name, please), 13). Their English competence is assessed to be at the lower level of independent users, or the B1 (Threshold), following the Common European Framework of Reference: Learning, Teaching Assessment. As such, they possess the capacity to understand the main points of clear standard input on familiar matters that they regularly encounter in school, leisure, community, etc. They, too, are able to deal with most situations likely to arise when they move about in UK. Further, they possess the capability to produce simple connected texts on topics that are familiar or of personal interest to them. And, finally, they are believed to have the ability to describe their experiences and events, dreams, hopes and ambitions and succinctly give reasons and explanations for their personal opinions and plans (see Council of Europe [n.d.]). And, since these two Saudi Arabians are set to enroll in high school in UK, they took the placement test. The test follows the usual format of an interview between the individual test taker and one especially trained examiner. The latter’s role is to lead the test takers through the three parts of the test. The three parts of the test are assigned their particular topics, regulated by time limits, and are designed to check on specific elements of oral communication competence. The first part of the test has as its topic “Who are you?” In this part, the test taker is led to provide fundamental information about his/her personal identities. Allotting two (2) minutes to each test taker for this part, the examiner fields three (3) to five (5) questions– e.g., what is your name? Can you spell your first name, please? How are you today? Where do you live? What time does school finish today? What do you do in your free time? What do you normally do at weekends? This part is designed to diagnose the test taker’s achievement and communication skills. The second part of the test prompts the test taker to do a monologue based on provided textual stimulus. The test taker is given the instruction written on a card, asking him/her to describe the village or city he/she came from. An alternative topic for this part is about the test taker’s dream holiday. The test taker is given one (1) minute to read and understand the instruction and prepare for his/her two-to-three (2-3) minute monologue. This part of the test is intended to diagnose the test taker’s clarity and naturalness of speech. In case the test taker manifests hesitations or loss of ideas to share, verbal prompts may be used – e.g., what is village/city like? What can one do there? Why do you like (or do not like) your village/ city? Would you want to change anything about your village/ city?; or, for the alternative topic, where would you like to go and where do you stay for your dream holiday? What is this place like? What would you do on your dream holiday? Why do you like this holiday? The third part is designed to mark the test taker’s range and accuracy of vocabulary and familiarity with grammatical rules. In this part, the test taker is made to engage the examiner in a dialogue on the topic “Money for Teenagers”. Prompting question is “should teenagers be given a big amount of pocket money by their parents?” Repair questions may be – e.g., how much money should teenagers get? What kind of jobs can teenagers do? What about finding a job during holiday? Should teenagers be allowed to buy their own clothes? Why do you think so? Should teenagers get money to get out? What do you do with your money? What do teenagers do with their money? A one (1) minute time is allotted for the test taker’s reading and understanding the instruction, and the speaking time for the dialogue is two (2) to three (3) minutes. Over-all, the testing time – inclusive of the welcoming the test taker and wrapping up the testing session – for every test taker is fifteen (15) to seventeen (17) minutes. The entire proceeding is assessed in situ and vide recorded for double marking. The marking criteria for this testing consist of four (4) grades. And, each criteria grade has distinctive expectations for the test taker to fulfill. Grade 1 expects the test taker to adeptly use basic linear connectors such as “and” and “then” and to carry out communication in simplest everyday situation. It also anticipates that the test taker pronounces words with strong accent and is only able to make very short and isolated utterances. The primary grade settles for the test taker’s extremely limited range of simple sentence structure, frequent breakdown of communication, extremely limited – and, thus, inadequate – range of vocabulary, communication of just few clear ideas, and lifting of several chunks from the prompts. The secondary grade is for the test take who can communicate limited information on familiar and routine matters, narrate simple stories and make plain descriptions, and use simple connectors. The second grade, too, is the level of the test taker who has noticeable foreign accent – although the speech is mostly clear enough to be understood – and can make short exchanges on familiar topics albeit with hesitations and false starts. This grade level is also for the test taker with limited range of simple sentence structures, which are inaccurate but does not cause breakdown, and with a limited range of vocabulary and the tendency to lift phrases from prompts. The third grade is for the test taker who communicates simple, straightforward information, makes straightforward descriptions or narratives, and is adept in linear sequence of simple elements. It is also for one who shows effectiveness in initiating, maintaining and closing simple conversations, pronounces clearly and intelligibly, and is capable of comprehensible stretches of language – although with pauses. The test taker’s grammar is marked by generally sufficient range of structures for familiar contexts, occasional inaccuracies, and clear message; the vocabulary is sufficient to clearly communicate an idea, though occasionally inaccurate. The fourth, and the highest, grade is meant for the test taker who can communicate detailed information, does clear and detailed description or narrative, and makes coherent discourse – though, with some jumpiness. The test taker, too, manifests clear and natural pronunciation and intonation, carries out fluent and spontaneous conversations and his/her long stretches of language are marked by natural pauses. This grade is also reserved for the test taker who has good range of structures and few inaccuracies in sentence structures, and has a good stock of vocabulary that is generally accurate. Qualitative analysis of tasks types and the candidates’ responses Over-all, the actual testing proceeded as planned. The test takers individually went through the three parts of the test. The tasks were found to be just within their capacity – i.e., all the designed tasks were successfully accomplished by them. The individual test taker was able to proficiently supply personal information. The first test taker chose to speak about her dream holiday; the second test taker spoke about Riyadh, his city of origin. For both of them, the question in the last part of the test was applied. The prompts and the repair questions proved to be very helpful, specifically when the test takers appeared to need some prodding when they were pausing too long – hence, not natural pauses – during the test. The testing instrument and, particularly, the marking criteria, served their purpose. At the end of it, the examiner is able to come up with specific marks for each of the test taker’s elements of language competence. The first test taker is found to be in the grade 4 for all the elements of communication competence, while the second test taker is marked 4 for communication skills, 3 for clarity and naturalness of speech, 4 for grammar and 3 for vocabulary. These marks would prove to be very handy for the purpose of ascertaining the level of these Saudi Arabian nationals’ placement relative to their command of English language. And, given the rather detailed criteria for each of the levels or grades, the result can likewise stand as a basis for succeeding ESL lessons for the two test takers. Noticeably, though, the actual testing lasted less than its anticipated time of fifteen (15) to seventeen (17) minutes. For the first taker, the testing session lasted only for ten (10) minutes. She was timed beginning from the first second that she spoke. The second test taker was time eleven (11) minutes to finish the test. It needs to be indicated here that, for both of the test takers, no time was allotted for preparation for the last part of the testing. Otherwise, it would have taken them eleven (11) (for the first test taker) and twelve (12) (for the second test taker) minutes. And, this could have meant a couple of things – that is, it may be that the test is really doable in less than amount of time, and that the estimated time of the test is mistaken. It also proved to be helpful to have standing-by repair questions. At least, when the test takers lost their bearing – for whatever reason – the examiner can have ready prompts to induce them to resume speaking. However, as this particular testing session showed, it also paid for the examiner to improvise repair questions. When the first test taker who spoke about her dream holiday paused for unusually long period of time, she was prompted by impromptu questions like “Why do you choose Dubai?” and “Why do you choose Jumairah Hotel?” The second test taker was similarly prompted by the instruction, “Tell me more about the deserts in Riyadh,” when he paused a lot as he tried to describe his city. References: Batstone, R. (2003). Grammar. Oxford: Oxford University Press. Condelli, L., Wrigley, H.S. & Kwang, S.K. (2009). “What works” for adult literacy students of English as a second language. In S. Reder & J. Bynner, eds. Tracking adult literacy and numeracy skills. New York: Routledge, pp. 132-159. Council of Europe [n.d.]. Levels. Retrieved 7 April 2010, from http://www.coe.int/T/DG4/Portfolio/?M=/main_pages/levels.html Ellis, R. (2003). Communication skills: stepladders to success for the professional. Bristol: Intellect Books. Jakeman, V. and McDowell, C. (2008). Action plan for IELTS self-study pack academic module. Cambridge: Cambridge University Press. Mewald, C., Gassner, O. & Sigott, G. (2009). E8 speaking test specifications version 01. Retrieved 7 April 2010, from http://www.uni-klu.ac.at/ltc/downloads/Speaking_Specs.pdf Moeller, S. (2000). Assessment and prediction of speech quality in telecommunications. Norwell (MA): Kluwer Academic Publishers. Nusbaum, H., Francis, A. & Henly, A. (1995). Measuring the naturalness of synthetic speech [ Abstract]. Retrieved 7 April 2010, from http://www.springerlink.com/content/f261w8h321633629/ Ratcliff, A., Coughlin, S. & Lehman, M. (2002). Factors influencing ratings of speech naturalness in augmentative and alternative communication [Abstract]. Retrieved 7 April 2010, from http://www.informaworld.com/smpp/content~content=a714043393&db=all The oral (speaking) test of the FCE exam, [n.d.]. Retrieved 7 April 2010, from http://www.ukstudentlife.com/English/Exams/Oral.htm Read More