Functional Writing Skills and Grammar - Case Study Example

RUNNING HEAD: Test Design and Evaluation Test Design and Evaluation Client Inserts His/her Name Client Inserts Name of Institution Introduction The test assignment described in this text is designed for junior high school English students. It seeks to evaluate their understanding of grammar and functional writing skills. Benson (1979) emphasizes that such tests should reflect what students have learned and is able to apply. The designs of the questions in the tests reflect the standard High school requirement for such tests. Question 1 of the test demands that a student compose an 800 word essay discussing the importance of girl child education. Question 2 examines the student’s mastery of grammar. The mark guide for each question has been generated. The test was developed after the possible requirements were ascertained. The general instructions were written. Purpose of the test The purpose of this test is to evaluate the attainment of functional writing skills and grammar by grade 2 students. The language used is at the level of this group. Test Design Test Type This test is composed of a functional writing and a grammar test. The first question will be answered based on candidates’ skills on composition and understanding of the topic of discussion (girl child education). In the 2nd test, students are expected to complete sentences by inserting missing words to make meaningful statements. Rationale Statement Language students at 2nd stage of secondary education are expected to be able to create own composition which must be standard and specific to certain guidelines. Students at this level should be able to complete sentences by using appropriate words to ensure coherence (peci.org, 2012). This will reflect their creativity and innovation on grammar work. They have been taught these skills and the test; therefore, seeks to evaluate their understanding of the skills. This test is useful since it holds the six aspects of test quality measures i.e reliability, construct validity, authenticity, interactive, impact and practicality. The following are a link between the test and the quality features. i) Reliability. The test results can be reproduced. Results from the pilot study are consistent with ability of the subjects used in the study. This means results of the actual test will also be reflective. ii) Validity. The tests are valid since it measures ability of the test takers to apply grammar and literary skills. The 1st test demands that candidates give their opinions on girl child education. It also measures their level of writing and application of functional skills. The 2nd test measures candidates’ ability to communicate effectively in an interactive environment. iii) Authenticity. The design of this test is within the knowledge scope of the test taker. The issues addressed in the questions are relevant to what is taught either at a lower level or a level of this class. The terminologies used and the general language in which the essay in test 1 is framed are understandable by the examinee. iv) Instructiveness. The language of the test is very friendly to the candidates. The source from which the comprehension text was derived is a novel the students interact with frequently in class. The functional writing questions talks about girl child education, a prerequisite information the students have interacted with at home, in school (this being a mixed school), or they have heard from the media. The candidates, therefore, understand the background from which the tests were derived. v) Impact. This test has an impact on both the candidates and teacher. The candidates will improve their knowledge on the issues raised in the questions. Through feedback given by candidates, the examiner will be informed of the candidates’ level of awareness and the attainment of the test objectives enabling the language teacher to identify the “missing links” for future improvements. The perspectives presented by candidates in their arguments will inform the examiner of the available knowledge. vi) Practicality. Questions in the test are relevant to contemporary social issues. For instance, the candidates understand what girl child education entails and have a capacity to suggest importance of it. Test Duration This test will be taken for duration of 60 minutes. This includes 5 minutes to be used in reading the instructions and all questions before the student embark on the test. Condition for the Test The questions must only be taken by English language students at the 2nd stage of secondary education. The examinee must be prepared having been informed of an impending test through a timetable. Start time need to be observed and the test be done in a confined environment away from any distraction. Test Instruction and Formatting Candidates are required to attempt all the questions in the test and write answers in the order in which the questions appear. Secondly, the composition to be written in question 1 must be discursive, on the specified topic and not exceed 800 words. The question 1 will take an essay approach. Scoring Method Scores will be given based on student’s attainment of the test objectives. The specific requirements of the tests give the basis of scoring. The mark distribution is as contained in the scheme in the appendix. Clarke (2006) maintains that if scores are what is used to gauge the extent of learning, then they should be awarded objectively. Pilot Study of the Test Introduction The pilot study used 3 students selected through stratification. The examiner classified the candidates in 3 groups based on their abilities: The high achievers, the average, and the low achievers. He then picked randomly a student from each group to be used in the study. The rationale of this sampling is to the view that the test results should not be skewed to any direction and as such moderation of the mark scheme and/or test design is necessary to achieve the objectives of the subject/ unit in evaluating students understanding and application of the learned skills (Lancaster, 2004). Procedure The piloting began by sampling the 3 students. The students were then notified to prepare for the test within a 24vhours. They then undertook the examinations in a room typical of the intended examination class. At the end of this test, the mark guide was collected and their papers examined against the mark guide. Evaluation was based on the attainment of the test’s specifications i.e style of writing, language of communication, correctness of the response given. Grammar checks were also done and students were denied marks if wrong usage of grammar was detected or non-adherence to test instructions was observed. Before marking begun, the teacher went through all the three papers to ascertain the variations in response. This was necessary to enable him moderate the mark scheme and cater for individual variations in the three examinees. The mark scheme was the modified in cases where it was necessary and the three scripts were marked according to the revised guidelines. Results Variations occurred in the area of students’ performance in the test. Even though the mark scheme was moderated, the three students scored marks that reflected their former academic abilities of high achiever, average and low achiever. The results however displayed some strengths and weaknesses of the paper. Areas of Strength The strength of the assessment was seen in the conformity of all examinee to the test guidelines and the consistency of their responses with the test objectives. This indicates that students grasped the concept of functional writing and grammar at the 2nd level of secondary education. They possess the skills and can apply them to real life situations. The instructions of the tests also appear to be simple and easily understood. This was reflected in the relevance of examinees response. Reading.ac.uk (2012) suggests that if the instructions for a test are clear, it will be less likely that students’ response to the tests will be incoherent with what the examiner expects. This therefore calls for clear indication of the test instructions. Secondly, the test appeared to be balanced or fair to all categories of the learners. This was reflected in its result which replicated the three student’s academic abilities just as was predicted. The sampling technique was beneficial to the final outcome since it considered diversity thereby ensuring a fair balance. Weaknesses of the paper were also observed. The very first was time limitation. It was realized that students struggled to adjust to the 60 minute test period to cover the entire work. The results showed that only one student out of the three (33%) managed to respond to all the questions. And when asked why they didn’t finish, the other two alluded that the test gave them little time to respond to all the questions. This might have affected the final outcome of the test. In addition, it also might have affected the accuracy of their responses prompting incorrect answers. In addition, it was identified that the 1st test needed a higher word count. The test allowed students to use just 800 words for the creative composition. However, it appeared from the writings that this number of words limited the candidates’ exploration of the topic. From the results, it was noted that all the three candidates went above the threshold of 800 words. The average number of words used was 950. The limitation of words to 800 could impair performance in the 1st question since the candidates did not apply skills for writing summary to this work. Despite the mentioned weaknesses of the test questions, the test’s objectivity and the identified relevant response to it reflects its applicability to the class. Recommendations Analysis of test displayed areas of weaknesses and strengths. While the strengths observed justify the relevance of this assignment, there is need for modifications to it so as to lessen the identified shortcomings which could impair good performance in it. The actual test should therefore consider the following: Time adjustment: instead of taking the test in 60 minutes, this pilot study recommends an additional 20 minutes. The added time will necessitate candidates to complete the entire task and have time to verify the correctness of information written. The basis of adding time is to enable all candidates to complete the paper regardless of individual speed. This will create a uniform evaluation ground limiting bias caused by incomplete assignments. Thabane (2010) suggests that by providing a leverage to each candidate, the individual variations can then be displayed making the tests more rational. Secondly, the 1st question of the test needed an additional allocation of space. Instead of the proposed 800 words, students could be allowed a word range of between 800 – 1000 words. This will solve the bias created by fewer words that did not favour students without good mastery of summary skills (Usq.edu.,2012). The mark scheme could also be moderated. This can be done using a larger sample of the actual candidates. For instance 15% of the class will give a better pointer of the whole class ability other than using 3 students. The sampling technique should replicate the design employed during this piloting. This will provide leverage for all the candidates. The procedure for marking should ensure impartiality so that human error of empathy is limited. Reference list Peci.org (2012). Functional test Procedure. Retrieved at: www.peci.org/ftguide/ftg/Test.../Tests/TG16-Writing_a_test.do Benson J (1979). A Comparison of Three Types of Test Development Procedures Using Classical and Latent Trait Methods. Retrived from: www.eric.ed.gov/ERICWebPortal/recordDetail?accno Clarke, J,P et al. (2006). Development, design, and flight test evaluation of a continuous descent approach. Retrieved from: web.mit.edu/aeroastro/partner/reports/cda_rpt.pdf Georgian Printers (2012). Kericho District Mock Examination- 101/2 & 101/3. Retrieved from : www.georgianprinters.edu.ke Lancaster, G,A . (2004). Design and analysis of pilot studies. Retrieved from: www.ncbi.nlm.nih.gov/pubmed/15189396 Reading.ac.uk (2012) Marking and Moderation of Assessments. Retrieved from: http://www.reading.ac.uk/web/FILES/exams/20-eaph-marking.pdf Thabane L. (2010). A tutorial on pilot studies: the what, why and how . Retrieved from: www.biomedcentral.com/content/pdf/1471-2288-10-1.pdf Usq.edu. (2012) Moderation of Assessments. Retrieved from: www.usq.edu.au/~/media/.../moderationofassessmentspdf.ashx APPENDIX 1. Original Test and Mark Scheme Test assignment Instructions: Attempt all the questions according to the instructions given. Time Allocated: 60 Minutes 1. Discuss the importance of educating a girl child in the society. (Use 800 words) (20mks) 2. Fill in the gap using the correct word from the bracket. i) Let us ________________ (play/pray) that we win the _________ (race/lace) this_______ (time/dime). ii) The pilot was told that it was against the __________ (raw/law) to _______ (fly/fry) ________ (row/low). iii) Walking _______ (town/down) River _________(load/road)with a _______ (road/load) of goods can be dangerous. iv) Winnie was _________ (shocked/chocked) when she __________ (heart/heard) that the thieves had not only stolen her _________(shoes/choose) but also her ______ (backs/bags). v) They _______(shut/sat) near the _________ (river/liver) and _______ (aid/ate) their ________ (river/liver). Answers 1. - Must be explanatory / discursive, if not AD 4 marks - The number of points does not matter, but it is the linguistic competence that counts. - Accept candidate’s choice of girl child education in discussion or exposition 2. i. Pray; race; time ( 1 mark each . Total = 3marks) ii. Law; fly (1 mark each. Total = 2marks) iii. Down; road; load (1 mark each. Total = 3marks) iv. shocked; eared; shoes; bags (1mk each. Total = 4marks) v. Sat; river; ate; liver (1 mk each. Total = 4marks) 2. Final test and Mark Scheme Test assignment Instructions: Attempt all the questions according to the instructions given. Time allocated: 80 Minutes 1. Discuss the importance of educating a girl child in the society. (Use 800 - 1000 words) (20mks) 2. Fill in the gap using the correct word from the bracket. i) Let us ________________ (play/pray) that we win the _________ (race/lace) this_______ (time/dime). ii) The pilot was told that it was against the __________ (raw/law) to _______ (fly/fry) ________ (row/low). iii) Walking _______ (town/down) River _________(load/road)with a _______ (road/load) of goods can be dangerous. iv) Winnie was _________ (shocked/chocked) when she __________ (heart/heard) that the thieves had not only stolen her _________(shoes/choose) but also her ______ (backs/bags). v) They _______(shut/sat) near the _________ (river/liver) and _______ (aid/ate) their ________ (river/liver). Answers 1. - Must be explanatory / discursive, if not AD 4 marks - The number of points does not matter, but it is the linguistic competence that counts. - Accept candidate’s choice of girl child education in discussion or exposition 2. i. pray; race; time ( 1 mark each . Total = 3marks) ii. law; fly (1 mark each. Total = 2marks) iii. down; road; load (1 mark each. Total = 3marks) iv. shocked;heared; shoes; bags (1mk each. Total = 4marks) v. sat; river; ate; liver (1 mk each. Total = 4marks) Read More