Evaluating Assessment and Testing at YCT - Report Example

Evaluating Assessment and Testing at YCT A serious and major problem encountered by EFL students in Saudi Arabia is their inability to communicate effectively in English. The Kingdom of Saudi Arabia has tried to address this issue by establishing educational programs in different institutions in the country which are expected to improve the quality of English language teaching. Yanbu College of Technology (YCT) is one of the colleges where programs aimed at enhancing and improving the quality of English language teaching has been established. Like many other colleges, YCT offers students credit points of General English and 6-9 credit points of English for Specific Purposes (ESP), so that students can learn English for their specific field of study. YCT is currently facing some problems related to modes of assessment of students learning English language. For example, the tests are generally poorly written as they are very much text-book oriented. This is only one issue among many other issues that can be examined further. This essay is written as an evaluation of the current testing system in YCT. It addresses issues such as test administration, validity, reliability, practicality, and language skills on the basis of the relevant literature. By the end of the essay, suggestions will be provided on how to overcome the problems related to assessment being faced at YCT. In terms of test administration, YCT runs the assessment through the following criteria: 1) Participation = 10 %; 2) First Mid-Term Exam = 20 %; 3) Second MidTerm Exam = 20 %; 4) Final Exam = 50 %. This assessment system is entirely conducted by the teachers of YCT. Although every teacher is responsible for designing the Mid-Term and the Final Exam, each exam is administered differently. Whereas the Mid-Term exam does not need any approvals from the administration, such approvals are necessary from the Head of the English Language Department for the Final Exam. As a result, this sometimes creates some sort of confusion and conflict of interest among the faculty members as to which test is to be approved and which is to be redesigned. In addition, designing the test itself is another issue. Most teachers who have to design the test generally have insufficient training and knowledge related to language testing and assessment. Teachers copy exercises directly from the textbook and administer them to the students as exams Since, students are already conversant with these exercises after completing such exercises in class work most of them end up performing very well. However, when a completely new exam containing questions that students are not conversant with is administered most of them end up failing. What this means is that the college administration as well as the teachers find easy ways of ensuring that their students pass without exposing students to challenging exams. After examining the test design in YCT, I found that the test design in YCT does not clearly address some of the technical steps of test design as suggested by Harmer (2008). A major rationale for that argument is that it does not involve people who have relevant background related to the nature and context of the test. This issue is of paramount importance because background of the test designers determines the quality of the test (Harmer, 2008). Additionally test administration in YCT needs to be re-evaluated because there is no piloting session that can ensure the appropriateness of the test for the level of the students. If piloting sessions are conducted, then it may be expected that the test would not be too easy or too difficult for the students. When it comes to test validity, some experts have come up with interesting insights. Jones (1997) noted that if one can make secure decisions on the basis of a test results, then a test is likely to be valid. Moreover, Popham (1995) divided test validity into three categories, including: 1) Content-related validity (the extent to which an assessment procedure adequately represents the content of the assessment domain being sampled); 2) Criterion-related validity (The degree to which performance on an assessment procedure accurately predicts the examinees performance on an external criterion); 3) Construct-related validity (The extent to 2 which empirical evidence confirms that a given assessment procedure is measuring the inferred construct accurately). In examining the validity of testing at YCT, several issues were discovered. First, tests produced at YCT lacked content validity. The tests are produced without even considering the objectives of the unit. As a result, the tests do not match the objectives of the unit. In addition, the teachers also do not use test methods which are prescribed in the Unit Outline. Clearly, this is a violation of the unit objectives. Second, the tests lacked criterion validity. The reason is that when setting the tests the teachers do not accurately follow the laid down standards in order to ensure that the tests match the unit objectives. Third, since placement tests are not provided at the college it is hard to say that there is predictive validity which is useful in measuring or predicting the most appropriate class for any particular students (Hughes 1989, p. 30). As a consequence, students may be placed in the wrong class full of students with heterogeneous levels. Fourth, in measuring the construct-related validity, it was clear that the tests were not valid because they were too simple. The students can easily answer the questions as the tests ask general knowledge rather than subject-specific field knowledge. 02. What are the short forms for these terms? * Amplifier :_( AMP) _ _ _ * Video recorder: _ (VCR) __ _ * Television :_(TV) _ _ Some experts have shared insights related to test reliability. According to Brown (2004, p. 20), "a reliable test is consistent and dependable". This means that the test should give similar results if it is given to the same students or matched students on two different occasions. In other words, reliability suggests an accurate reflection of a candidate's actual proficiency. According to Jones (2002), there are a number of factors that can contribute to the unreliability of a test. These include unfavourable administrative conditions, like unclear instructions, too little time, too much time, distractions and cheating. In YCT, the teachers are concerned about administrative conditions that may affect the reliability of the test. That is why they supervise the test carefully so that the students cannot cheat during the test. They also provide clear, easily understandable, and unambiguous instructions for the test, so that students can do the test as expected. However, the content of the test makes it an unreliable test because the students have to do the same questions as the ones that they do as exercises in the classroom. Also, the time provided for completing the test is long because the test can actually be completed within one hour instead of two hours as indicated in the test booklet. Another thing that affects the reliability of the test is poor guidelines on how to deal with the moderation of the test. It has been widely reported that disparities exists among the markers of the tests. One marker may give too high marks whereas another marker may give very low marks when they assess the same work. This depends on the training, knowledge and skills of the markers in a particular unit. Because of this inconsistency of marking develop negative attitudes students towards the assessment system (Light & Cox, 2001) and this case is obvious in YCT. According to Hughes (1989) “a practical test is one that is easy and cheap to construct, administer, score and interpret”. In YCT, the test is administered in such a way that is affordably constructed and easily scored and interpreted. In addition, not only is the administration of the tests easy but also the marking of the tests. After the papers are marked, the unit teacher reviews every test paper in order to ensure that the marker did not miss any of the students' answers. Then a second reviewer is assigned to review the grades of each student and make sure they are the same as the grades awarded. After receiving the final approval from the Head of English Language Department, the teachers begin entering the final scores into the college's system. The students can collect their results through the internet. Considering all of the factors above, I believe that the test in YCT is practical. Many experts such as Brown (2004, 2001) and Harmer (2008) have shown agreement that a good test should be able to assess the macro skills (reading, speaking, listening, and writing) and micro skills (grammar, vocabulary, and pronunciation). In YCT, however, it is generally evident that the test only assesses the macro and micro skills. Reading and writing are the only macro skills which are tested while listening and speaking macro skills have been ignored by the teachers. There are several reasons why speaking and listening are not tested. For example, poor technological support (e.g. insufficient cassette recordings) or because of the teachers' lack of motivation to work hard. In addition, grammar and vocabulary especially the ESP terminology receive an extensive emphasis. Questions about word definitions appear every now and then in the test and are definitely the major components of the test. To sum up, it is interesting to reveal the major findings in regard to testing at YCT. First, it has been shown that test administration at YCT fails to fulfil important technical steps in designing test such as mastery of background knowledge of the test among the test designers. Second, the test has been proven to be invalid because it does not match the content, criteria, and predictive validities as suggested by Popham (1995).Third, the test is also unreliable, because students are provided with the same questions encountered in classroom exercises. At the same time, lack of consistency while moderating the tests makes the attitudes of the students towards the assessment system in YCT quite low. My suggestion regarding the type of assessment undertaken at YCT is that YCT administration and teachers should make some changes and improvements in the testing and assessment system. One serious action is that there should be a placement test for students' entry level. This is important as a predictive validity of the test. Second, the test should be in accordance with the unit objectives in order to meet the content validity. Third, in order to make the test more reliable, the test designers should not integrate a wide range of questions in the tests. Some questions may be picked from textbooks and integrated with questions drawn from other textbooks and past papers. Reference List Brown, H. D. (2001). Assessing language skills: Basic concepts in test development. In H.DBrown, Teaching by principles an interactive approach to language pedagogy (pp. 386-387). New York: Addison Wesley Longman. Brown, H.D. (2004). Language assessment: Principles and classroom practices. New York: Longman. Harmer, J. (2008). Testing and evaluation. In J. Harmer, The practice of English language teaching (pp. 379-389). Essex: Pears0!l' Hughes, A. (2003). Testing for language Teachers. Cambridge: Cambridge University Press. Jones, 1. F. (2002). Classroom language testing and assessment at Osaka University. Journal of Language and Culture (OU) , Volume(N1 A)( pp.29-35). Light, G. & Cox, R. (2001). Learning & teaching in Higher Education: The reflective professional. London: Paul Chapman Publishing. Popham, W. (1995). Classroom Assessment. Needham Heights, MA: Allyn & Bacon Technica and Vocational Training Corporation . (2003). Education. Retrieved 8 1, 2009, from Technica and Vocational Training Corporation (TVTC): Http://wwv.·.tvtc.gov.salEnglishieducation.htm Read More