Rubrics In Evaluating Student Works - Case Study Example

Rubrics In Evaluating Student Works Introduction This paper seeks to discuss and analyze how to create a scoring rubric for an essay exam. This will therefore include into the discussion available theories of rubric creation and a sample rubric will be produced as a result. 2. Analysis and Discussion 2.1. What are rubrics? Mertler, (2001) in discussing “Designing scoring rubrics for your classroom” gives the following definitions: Rubrics are rating scales-as opposed to checklists-that are used with performance assessments. They are formally defined as scoring guides, consisting of specific pre-established performance criteria, used in evaluating student work on performance assessments. Rubrics are typically the specific form of scoring instrument used when evaluating student performances or products resulting from a performance task.” 2.2 Types of Rubric Mertler, (2001) further discussed two types of rubrics: holistic and analytic. The author citing (Nitko, 2001) (Moskal, 2000) said, “ A holistic rubric requires the teacher to score the overall process or product as a whole, without judging the component parts separately. In contrast, with an analytic rubric, the teacher scores separate, individual parts of the product or performance first, then sums the individual scores to obtain a total score.” 2.3 When to use each type of rubric? Mertler, (2001) citing Chase (1999) suggested that holistic rubrics are customarily utilized when errors in some part of the process can be tolerated provided the overall quality is high. He cited Nitko (2001) further saying that use of holistic rubrics is probably more appropriate when performance tasks require students to create some sort of response and where there is no definitive correct answer Hence when talking of overall quality holistic is preferable aside from somewhat quicker scoring process than use of analytic rubrics (Mertler, 2001, citing Nitko,(2001). This is basically due to the fact that the teacher is required to read through or otherwise examine the student product or performance only once, in order to get an "overall" sense of what the student was able to accomplish (Mertler, 2001). Mertler, 2001 citing Nitco ( 2001) prefers the use of analytic rubrics when a fairly focused type of response is required, that is, for performance tasks in which there may be one or two acceptable responses and creativity is not an essential feature of the students' responses. He also suggested the use of the same in making assessment on a multidimensional level. 2. 4 Decision to choose type of Rubric A rubric measures performance and measuring performance is just one of the functions of a teacher hence prior to design a specific rubric, a choice scoring holistically or analytically must be made. Mertler (2001) citing Airasian, (2000 & 2001). Specific performance criteria and observable indicators need to be identified at the start. It is the use of the results that could determine the choice of approach. Mertler (2001) cited Montgomery (2001) that if an overall, summative score is desired, a holistic scoring approach would be more desirable but if formative feedback is the goal, an analytic scoring rubric should be used. To measure performance, using either quantitative (i.e., numerical) or qualitative (i.e., descriptive) labels is also a choice and both may also be used sometimes. Mertler (2001) suggested that if a rubric contains four levels of proficiency or understanding on a continuum, quantitative labels would typically range from "1" to "4." but if teachers want more flexibility and creativity, using qualitative labels, might be more useful. 2.5 Converting Rubric to grades Although rubrics are rating scales that measures performance there is nothing that could prevent the teachers by making a more detailed equivalent of the scores. Mertler (2001) suggested that the process of converting rubric scores to grades or categories is more a process of logic than it is a mathematical one. Mertler (2001) cited Trice (2000) as suggesting that in a rubric scoring system, there are typically more scores at the average and above average categories (i.e., equating to grades of "C" or better) than there are below average categories. As an example rubric consisted of several score categories, the equivalent grades and categories might include the grade of A and A+ for Excellent, B and B+ for Good, C+ and C for Fair, and U for Unsatisfactory. As will be seen same categories are used in rubric created in Table I found at the latter part of this paper (Mertler, 2001) (Paraphrasing made). What is important in grade conversion, that teachers must find a system of conversion that works for them and fits comfortably into their individual system of reporting student performance (Mertler, 2001). 2.6 Steps in the Design of Scoring Rubrics Mertler (2001) citing various works from Airasian, 2000 & 2001; Mertler, 2001; Montgomery, 2001; Nitko, 2001; Tombari & Borich, 1999, gives us below a step-by-step process for designing scoring rubrics for classroom use. Step 1: Re-examine the learning objectives to be addressed by the task. This allows you to match your scoring guide with your objectives and actual instruction. Step 2: Identify specific observable attributes that you want to see (as well as those you don’t want to see) your students demonstrate in their product, process, or performance. Specify the characteristics, skills, or behaviors that you will be looking for, as well as common mistakes you do not want to see. Step 3: Brainstorm characteristics that describe each attribute. Identify ways to describe above average, average, and below average performance for each observable attribute identified in Step 2. Step 4a: For holistic rubrics, write thorough narrative descriptions for excellent work and poor work incorporating each attribute into the description. Describe the highest and lowest levels of performance combining the descriptors for all attributes. Step 4b: For analytic rubrics, write thorough narrative descriptions for excellent work and poor work for each individual attribute. Describe the highest and lowest levels of performance using the descriptors for each attribute separately. Step 5a: For holistic rubrics, complete the rubric by describing other levels on the continuum that ranges from excellent to poor work for the collective attributes. Write descriptions for all intermediate levels of performance. Step 5b: For analytic rubrics, complete the rubric by describing other levels on the continuum that ranges from excellent to poor work for each attribute. Write descriptions for all intermediate levels of performance for each attribute separately. Step 6: Collect samples of student work that exemplify each level. These will help you score in the future by serving as benchmarks. Step 7: Revise the rubric, as necessary. Be prepared to reflect on the effectiveness of the rubric and revise it prior to its next implementation. 2.7 An application of the rules of on creating rubric for an essay exam. In applying the rules and steps as discussed above discussed above, an example or rubric for an essay example is shown below. Aspects Criteria (Descriptions of Scoring Levels) Score Editorial Aspect 1: Clarity 1. Paper is well written; no grammatical errors; easy and interesting reading; clear evidence of regular revision & proofing; key ideas are fully elaborated and illustrate what is meant; examples are provided as appropriate 2. Paper is reasonably well written; a few minor grammatical errors; easy and interesting reading; evidence of regular revision and proof reading; key points are made, but not always elaborated. 3. Paper is tolerably well written; a fair number of minor grammatical errors; a few major errors; confusing to reader; no evidence of regular review and revision; key points are made, but not often elaborated. 4. Paper is poorly written; a number of major and minor grammatical errors; essentially unreadable; paragraphs are a jumble of sentences and sentences are a jumble of words; gibberish; key points missing and/or not elaborated. . Paper is poorly written; a number of major and minor grammatical errors; essentially unreadable; paragraphs are a jumble of sentences and sentences are a jumble of words; gibberish; key points missing and/or not elaborated. Editorial Aspect 2: Accuracy 1. Essay shows no discernable errors in fact; well-grounded conclusions. 2. Essay shows minor errors in fact; reasonably well drawn conclusions 3. Essay shows multiple minor errors in fact; poorly-drawn conclusions. 4. Essay shows multiple and gross errors in fact; grossly inaccurate conclusions. Editorial Aspect 3: Precision 1. Essay shows concise statements rather than broad generalities; provides details; provides substantial evidence. 2. Essay shows mostly concise statements but a few broad generalities; moderate use of supporting evidence. 3. Essay shows mostly broad generalities, a few concise statements; very limited use of supporting evidence. 4. Essays shows broad statements of generalities; provides no details and no supporting evidence for claims. Editorial Aspect 4: Compliance with requirements as number of words. 1. Work perfectly matches with requirements 2. Work exceeds or lacks by more or less than 1% 3. Work violates between 2 % tot 5% 4. Work violates by more than 5% Content Aspect 1: Professionalism 1. Work is very professional; material of substantive value; research based; no needed improvement; strong evidence of “deep learning.” 2. Work is more professional; material mostly of substance, but includes some opinions; shows more “deep learning” than “surface learning.” 3. Work is less professional; some material of substance, but lots of personal commentary; shows more “surface learning” than “deep learning.” 4. Work is unprofessional; provides mostly general commentary and personal opinions; strong evidence of “surface learning.” 5. 6. Content Aspect 2: Relevance 1. All statements and assertions are relevant to the topic or bear on the question at hand; assists in clarifying topic or resolving issue 2. Most arguments are cogent, concise, and relevant; a small number of arguments provided and all are well reasoned. 3. Arguments are not always cogent, concise, and relevant; many arguments are given but they are poorly reasoned. 4. Arguments are not cogent, concise, and relevant; few arguments are given and they are poorly reasoned, and insufficient to the task. Content Aspect 3: Logic 1. Arguments supporting propositions as provided are all well reasoned, “win the day” and make sense; conclusions flow from evidence; order of presentation suggests use of a mental outline. 2. Arguments show fairly good use of logic; provides good data, but perhaps draws improper conclusions on the basis of that data; orderly presentation of information and arguments. 3. Arguments show weak logic, perhaps flawed, but attempts to draw appropriate conclusions from the limited amount of data provided; somewhat disorganized presentation of information. 4. There is evidence of flawed logic; draws inappropriate conclusions from data or draws conclusions without supporting data; garbled presentation; lacks logical flow of presentation Content Aspect 4: Depth and Breadth 1. Work fully applies main factors that make the topic important; deals with complexities; identifies difficulties; shows evidence of thorough knowledge of major critical resources; shows full range of subject matter very thoroughly; includes multiple important perspectives if pertinent to topic. 2. Work applies most of the main factors that make this topic important; shows considerable evidence of knowledge of several resources; shows full range of subject matter adequately; includes other important perspectives if pertinent to topic. 3. Work applies some of the main factors that make this topic important; shows some evidence of knowledge of resources; shows full range of subject matter irregularly; provides no or incorrect alternative perspectives. 4. Work applies few if any of the main factors that make this topic important; clearly lacks evidence of appropriate knowledge of resources, shows full range of subject matter poorly; provides biased alternative perspectives Note: Some parts of the above rubric include paraphrased portions of rubric from Illinois State University Physics Teacher Education Program (2005) Conclusion and Recommendation As found out, making rubrics for an essay exam should be guided by the objectives of the teacher. As defined, rubrics are rating scales that are used with performance assessments or scoring guides, consisting of specific pre-established performance criteria, used in evaluating student’s work or performance of a task, so they must accomplished their purpose. But the teacher may have specific purpose of measuring like developing creativity among students of which holistic approach should be preferred, other the teacher is an advised to use analytic approach as was illustrated in this paper. Collecting samples of student work to exemplify each level of performance is also helpful for future benchmarks or allow revision of rubric as necessary to attain effectiveness as desired. Bibliography 1. Airasian, P. W. (2000). Assessment in the classroom: A concise approach (2nd ed.). Boston: McGraw-Hill. 2. Airasian, P. W. (2001). Classroom assessment: Concepts and applications (4th ed.). Boston: McGraw-Hill. 3. Chase, C. I. (1999). Contemporary assessment for educators. New York: Longman. 4. Illinois State University Physics Teacher Education Program (2005), PTE Essay Exam Scoring Rubric, {www document} URL http://www.phy.ilstu.edu/pte/Essay_Exam_Scoring_Rubric.pdf, Accessed September 15,2006 5. Mertler, C. A. (2001). Using performance assessment in your classroom. Unpublished manuscript, Bowling Green State University. 6. Mertler, Craig A. (2001). Designing scoring rubrics for your classroom. Practical Assessment, Research & Evaluation, 7(25), {www document} URL http://PAREonline.net/getvn.asp?v=7&n=25 , Accessed September 15,2006. 7. Montgomery, K. (2001). Authentic assessment: A guide for elementary teachers. New York: Longman. 8. Moskal, B. M. (2000). Scoring rubrics: what, when, and how?. Practical Assessment, Research, & Evaluation, 7(3). Available online: http://pareonline.net/getvn.asp?v=7&n=3 9. Nitko, A. J. (2001). Educational assessment of students (3rd ed.). Upper Saddle River, NJ: Merrill. 10. Tombari, M. & Borich, G. (1999). Authentic assessment in the classroom: Applications and practice. Upper Saddle River, NJ: Merrill. 11. Trice, A. D. (2000). A handbook of classroom assessment. New York: Longman. Read More