Benefits and Drawbacks of Computer-based Testing - Assignment Example

 What are the benefits and drawbacks of computer-based testing? Discuss with specific reference to the TOEFL paper-based test and internet-based test. Following the increasing use of technology in school, powerful microcomputers have had a great influence on the design and content of school curricula and the whole teaching and learning process (Dunkel, 1999). Technology plays a significantly important role in testing used for assessment purposes as computerized tests, administered on personal computers or at computer terminals are considered a practical alternative to paper-based testing. Proponents of computer-based testing (CBT) and its subcategory, computer-adaptive testing (CAT) mention several benefits of this form of assessment in a foreign language classroom. To start with, a CAT test is adapted to the individual needs and abilities of each student (Dunkel, 1999). When working on a personalized activity, learners avoid boredom that would accompany a task far above their activity level, or anxiety that could accompany a difficult task. Furthermore, the test can be scored immediately, thus providing a learner with immediate feedback. Multimedia tests that can include graphics, photographs, and even video clips are attractive and stimulating for language learners (Dunkel, 1999). New technologies such as CD-ROM or interactive video make the students’ learning experience meaningful and authentic as they provide test takers with challenging tasks that require simultaneous use of different language skills (Chaloub-Deville, 2001; Sawaki, Stricker, & Oranje, 2008). Apart from benefits for students, computer-based tests also facilitate test delivery and administration. Technology also enables test developers to adopt integrated test approach, which produces a more accurate image of test-takers’ abilities (Chaloub-Deville, 2001). Web-based tests (WBT) have further specific advantages, including flexibility in time and space, easy test design, and affordable price (Roever, 2001). However, there are several drawbacks concerning the validationof computer-based tests. First, there is a concern that such factors as computer unfamiliarity or poor technological conditions can significantly affect test-takers’ performance. Furthermore, CBTs and CATs are not considered valid instruments for assessing particular language skills. They are also characterized by lack of fidelity and comprehensiveness (Dunkel, 1999). In case of web-based tests (WBT), their major disadvantage is high exposure to cheating and item confidentiality. Other issues that need to be addressed involve problems with self-scoring tests, data storage, server failure and browser incompatibility (Roever, 2001). Nevertheless, computer-based tests are increasingly used by academic and testing organizations. In 1998, a computer-based test TOEFL was introduced as an alternative to a traditional paper-based test. Since 2005, an Internet-based test has been used in test centers in the US and all over the world (Sawaki et al., 2008). TOEFL is the most widely accepted English test in the world, recognized by more than 7.000 colleges in over 130 countries. It measures learners’ use of four basic language skills: reading, listening, speaking, and writing in the university classroom. The test enables the students to apply to colleges and universities, in which English is the language of instruction. Apart from different form, the Internet-based test differs from the traditional paper-based one in terms of design principles on which it is based: the main objective of the new test is the adaptation of the test design to different language use tasks that test takers are likely to encounter in everyday academic life (Sawaki et al., 2008). To achieve this aim, the Internet-based test (IBT) contains a speaking section and integrated tasks, which involve simultaneous use of different language skills. Given the growing use of Internet-based tests in TOEFL test centers worldwide, it is important to discuss the benefits and drawbacks of this form of testing as well as to address challenging issues concerning computer-based and Internet-based testing. Individualized approach A CAT test as well as an IBT test adjusts the item difficulty to the test taker’s performance, thus providing precise information across a wide range of ability level (Dunkel, 1999; Chaloub-Deville, 2001). Due to the test’s individualized approach, low-ability take testers avoid anxiety that often accompanies excessively difficult tasks, while high-ability learners are not bored with tasks that are far beyond their ability level (Dunkel 1999). Furthermore, the TOEFL Internet-based test provides learners with feedback on their performance and advice for improvement, adjusted to their abilities level. As an example, while students with a low listening level are advised to listen to recordings to English lessons and recordings on familiar topic, those with high listening skills should focus on new topics, listen to programs with academic content, and take live and audio-recorded tours in museums (Educational Testing Service, 2007). Thus, the test does not only assess their language skills, but also encourages them to further practice Authenticity of tasks The Internet-based TOEFL test is based on the principle of authenticity: the test design is adapted to a variety of different language situations test takers are likely to encounter in the university classroom (Sawaki et al., 2008). An IBT TOEFL test involves a variety of interacting components and processes, which make testing experience more authentic and meaningful. To start with, an obligatory speaking section, consisting of six different tasks, has been added to the test. Two of them are independent tasks, in which learners have to express their opinions on familiar topics, concerning their personal experience or background (Swain, Huang, Barkaoui, Brooks, & Lapkin, 2009). The other tasks involve simultaneous use of speaking, reading and listening skills. In two exercises test takers need to listen and respond to a short conversation or lecture. In the other two, their task is to read and listen to the text on the same topic and then combine the information from both sources in their responses. The reading and listening material concerns campus-life situations and academic course content, thus preparing a student for situations they are likely to experience in English-speaking university classrooms (Swain, 2009). Furthermore, the integrated tasks that require students to use different language skills simultaneously prepare students for real life situations, in which they need to give responses to what they read and hear. Thus, IBT tests provide test takers with more authentic learning experience than traditional paper-based TOEFL tests, in which the four language skills are measured separately. The principle of authenticity is a principal advantage of computer based testing. Attractiveness for users Multimedia tests that contain graphics, photographs and video clips are stimulating and attractive tools for test takers (Dunkel, 1999). The research report concerning the impact of changes in TOEFL on language teaching in Central and Eastern Europe reveals that nearly all students of institutions offering TOEFL preparation courses that participated in the research felt comfortable when using computers (Wall & Horak, 2006). As a result of lowering costs of technology and the increased use of computers in schools, the level of computer competence among students has considerably increased. Most students had access Internet at home and, thus, could search the Web for information related to the TOEFL test and language exercises. Many of them also used various CD-ROMs for additional practice. The majority of the students interviewed found Internet-based tests more attractive and easier than paper-based tests. Flexibility in time and space The crucial logistical advantage of an Internet-based test is its flexibility in time and space (Roever, 2001). Given that test takers have access to a computer with a Web browser and Internet connection, they can take a Web-based test whenever and wherever it is convenient to them. Thus, existing facilities such as computer laboratories can be used as IBT tests centers. Furthermore, due to the use of scoring scripts for dichotomously scored items, which comprise the majority of items in the Reading and Listening part, the test is independent from the tester, which makes it even more flexible and, thus, convenient for the user (Sawaki et al, 2008). The ETS website offers a variety of preparation and practice materials for the ITB test takers. Students can answer free sample questions, work with interactive skill-building programs, take practice tests online and obtain their results immediately (ETS, 2009). While learners who have taken the traditional, paper-based TOEFL test need to wait five weeks for the results, the scores of IBT tests are mailed to students within two weeks after the test. Flexibility and convenience make computer-based tests and Internet-based tests useful tools for both learners and test designers. Integrated test approach Technology enables test designers to adopt a more integrated and systematic test approach, aimed to create interrelations among task characteristics, test takers’ behavior, and the abilities and processes used to solve the tasks (Chaloub-Deville, 2001). This approach provides testers with a more meaningful picture of the test takers’ language skills and abilities, which can be assessed from a more holistic point of view. Furthermore, when working on computer-based tasks and Internet-based tasks, test developers can establish close links between task characteristics and language skills involved. Thus, analysis, classification and systematization of prototype tasks with identified characteristics permits the creation of new tasks with the desired linguistic, situational, cognitive, and measurement characteristics (Chaloub-Deville, 2001). In the context of the TOEFL IBT test, the test specification classification can be used to identify different purposes of Reading and Listening tasks (Sawaki et al., 2008). In this classification, items are grouped according to the definitions of the abilities assessed in these sections, based on the test specifications. This grouping has enabled researchers to identify the three main objectives of Reading and Listening tasks: Basic Understanding, Pragmatic Information, and Connecting Information. The knowledge of different items used to achieve each of these purposes allows test developers to design new exercises with the desired characteristics. Easy design In comparison with traditional computer-based tests, which require a high level of programming expertise and the use of specially-designed, sophisticated delivery platforms, Internet-based tests are relatively easy to design and use. The HTML language, used to create Web-based tests is an easy tool, which does not require advanced programming skills (Roever, 2001). It contains several elements that support the creation of common types, which include input boxes for short response items, radio buttons, or text areas for extended response items, such as dictations or essays. Everybody with access to a computer and a free, standard browser can take an Internet-based test. These factors influence the widespread use of Internet-based tests in TOEFL centers worldwide. Affordable price Another advantage of Internet-based tests is affordable price for both test designers and test takers. As Internet-based tests do not require sophisticated software, they do not incur any additional production costs. These tests can be written by hand or with a free edition program and uploaded freely to the servers provided by the testers’ institutions or to one of several commercial megaservers that offer free web space (Roever, 2001). Due to a small size of an average IBT file, there is enough space on a free server to upload a few tests. As test takers are provided with free access to Internet in many test centers as well as in school and university libraries, they can practice for a test without any extra cost. Thus, even though the cost of IBT TOEFL is slightly higher than the price of PBT TOEFL, the wide variety of practice exercises available on the website makes the test affordable for students. In contrasts, when preparing for a paper based test, students have to buy books containing practice exercises. Furthermore, especially in smaller towns students do not have access to printed TOEFL sources (Wall & Horak, 2006). Thus, due to its accessibility and relatively inexpensive overall costs, more and more students decide to take an Internet-based TOEFL test. Validation of CBTs and IBTs Many experts in the field are concerned about several issues concerning the validation of computer-based test and Internet-based tests. To start with, the test takers’ performance may be considerably affected by their familiarity with computers, thus causing construct-irrelevant variance (Roever, 2001). However, this problem can be solved with providing the learners with tutorials, aimed to improve their computer literacy skills. Furthermore, the study published by Roever reveals that the use of standard web-browsers in IBTs increases the likelihood that the testing environment is familiar to test takers. The study conducted with intermediate-level English learners at the University of Hawaii did not find any significant correlation between self-assessments of Web browser familiarity and scores on a Web-based test of second language pragmatics. Another factor affecting the test performance is typing speed, which is a more serious source of error variance and, unlike browser familiarity, cannot be improved through quick training. While IBT TOEFL test-takers from Center and Eastern Europe, interviewed by Wall and Horak (2006) claimed to be familiar with computers, they displayed poor typing skills. Thus, test takers often complained about not having enough time for the discourse completion section of the test, which allowed only 90 seconds per item. Furthermore, the Internet-based test may “skip” items due to technical problems, such as errors in the algorithm in an adaptive or randomized test (Roever, 2001). Such factors as server traffic, complexity of the page, or client computer speed may negatively affect download times of a test delivered via the Web. Limitations of Internet based tests A common concern about IBTs is their lack of security, related to cheating and high confidentiality (Roever, 2001). When taking tests at home, students can cheat or download items into the web browser’s cache on their computers. However, this problem does not affect TOEFL IBTs, which are taken in testing centers under supervision. The presence of trustworthy supervisors also prevents other common problem, which accompanies the use of self-scoring tests and scripts. Java Script contains all the answers and, thus, a test-taker may easily view them if he or she looks at the source’s code. Finally, there are problems concerning data storage; because of security reasons, the only file to which the test taker’s work is a cookie file, which works only in Microsoft Internet Explorer. Thus, a system crash in the middle of the session erases the cookie, thus deleting all test information (Roevel, 2001). As test data in TOEFL IBT are sent back to the tester’s server, even in case of a browser crash it is possible to recuperate the test taker’s responses. Furthermore, a Java Script function can “harvest” the responses and send them to a different server, thus providing a test taker with an additional security form. Thus, an Internet based testing deals successively with common limitations associated with computer-based and web-based testing. Conclusion Computer-based testing and, more recently, Internet-based testing are more and more increasingly used in TOEFL testing centers, being recognized as a practical and attractive alternative to traditional, paper-based tests. Such features as individualized and integrated approach, authenticity, attractive design, integrated test approach, accessibility and affordable price make computer-based tests a useful tool for both test designers, test administrators, and, finally, language learners themselves. As the test adjusts the item difficulty to the test taker’s performance, the learners are neither discouraged by difficult tasks, nor bored by activities far beyond their activity level. Authentic, integrated tasks prepare learners for the communication situations they are likely to encounter in the English-speaking university classroom. Furthermore, the Internet-based TOEFL test provides all users with individualized feedback, which helps them to realize their weak points and search for strategies to improve their performance. In today’s computerized society, learners who are familiarized with computers with IBTs easier and more attractive than traditional paper-based tests. Finally, a wide range of free practice tests available on the Web enables test-takers to prepare themselves for the test and, consequently, perform satisfactorily on the test. A common problem concerning he influence of computer literacy skills on testing performance may be solved given that test takers are provided with a tutorial, aimed to improve their computer familiarity. Furthermore, such issues associated with Internet-based testing as cheating and item exposure, server failure, or the risk of losing data do not concern TOEFL IBTs, which are administered by trustworthy administers and protected with several security forms. With the continuous research aimed to develop and improve the IBTs’ design, approach, and testing procedures, computer-based tests and Internet-based TOEFL tests are bound to be successfully used by test takers all over the world. References: Chaloub-Deville, M. (2001). Language testing and technology: past and future. Language Learning and Technology, 5(2), 95-98. Retrieved from http://llt.msu.edu/vol5num2/deville/default.html on 11 January 2010 Dunkel, P. (1999). Considerations in developing or using second/foreign language proficiency computer-adaptive tests. Language Learning and Technology, 2(2), 77-93. Retrieved from http://llt.msu.edu/vol2num2/article4/ on 11 January 2010 Educational Testing Service (ETS). (2007). TOEFL IBT Performance. Feedback For Test Takers. Retrieved from www.toefl.org on 12 January 2010 Roever, C. (2001). Web-based language testing. Language Learning and Technology, 5(2), 84-94. Retrieved from http://llt.msu.edu/vol5num2/roever/default.html on 10 January 2010 Sawaki, Y., Stricker, L., & Oranje, A. (2008). Factor Structure of the TOEFL Internet- based test (IBT): Exploration in a Field Trial Sample. Princeton, NJ: ETS. Retrieved from http://www.ets.org/Media/Research/pdf/RR-08-09.pdf Swain, M., Huang, L., Barkaoui, K., Brooks, L., & Lapkin, S. (2009). The Speaking Section of the TOEFL IBT (SSTIBT): Test Takers’ Reported Strategic Behaviors. TOEFL IBT Research Report, ETS. Retrieved from http://www.ets.org/Media/Research/pdf/RR-09-30.pdf on 12 January 2010 Wall, D. & Horak, T. (2006) The Impact of Changes in the TOEFL Examination on Teaching and Learning in Central and Eastern Europe: Phase 1, The Baseline Study. TOEFL Monograph Series, ETS. Retrieved from http://www.ets.org/Media/Research/pdf/RR-06-18.pdf on 12 January 2010 Read More