Assessing Learning from Instruction - Essay Example

Assessing Learning from Instruction Objective of the Lesson: To provide an overview of the methodology in assessing learning from Instructions Purposes of Evaluation The primary aim of assessment is to foster learning of worthwhile academic content for all students (Wolf, Bixby, Glenn, & Gardner, 1991). School communities use assessment results in a formative way to determine how well they are meeting instructional goals and how to alter curriculum and instruction so that goals can be better met. But unless the content of assessment (what schools assess) and the format of assessment (how schools assess) match what is taught and how it is taught, the results are meaningless, if not potentially harmful. The same is true if assessment tools are not of high quality. Theres also potential for harm when decisions affecting students futures are being made based on results of assessments made with tools that are not appropriate for the purpose. Three Basic questions which help guide the planner: Where are we going? How will we get there? How do we know when we are there? Assessment Models Sequence Assessment Design Activity A good designer generally begins to think about assessment instruments as she develops the learning objectives. This is done to help ensure that not only does the assessment match the objectives but appropriately amplifies and clarifies the objectives themselves. Norm-referenced: Norm-referenced test designers define the scope of the material for the assessment instrument more broadly and with less precision. It is designed to rank or compare learners. Criterion-referenced: Tests developed to determine competence. Designers use the precise objectives that they developed in the learning task description phase. Characteristics of Assessment Instruments Validity: If a test does not measure what it purports to measure the test is worthless or perhaps worse, misleading. Reliability: An Assessment Instrument is reliable if it consistently measures what it claims to measure and if we have a high degree of confidence in the scores that it produces. Practicality: Optimal assessment might be a many-itemed constructed response or performance exam that is as close as possible to the real life situation in which the learners will apply their knowledge for all objectives with a carefully developed set of criteria for evaluating the responses. Design courses often have learners complete designs for one situation for some degree of validity and use printed scenarios in written assessments in an attempt to obtain some degree of reliability. Trade-offs in Assessment Design: It is indeed important to consider carefully decisions as to which factors are most important for the particular assessment situation and then male trade-offs among the factors of validity, reliability and practicality. All design work involves consideration of trade-offs (Petroski, 2003). Formats for Assessment There are two major formats of tests: i. Performance assessment, and ii. Pencil and paper tests. Performance Assessments Swanson, Norman, and Linn (1995) proposed that “authentic assessment” is synonymous with performance assessment. They defined performance assessment as “testing complex, ‘higher order’ knowledge and skills in the real world context in which they are actually used, generally with open-ended tasks that require substantial examinee time to complete. There are four most common performance assessment forms: 1 On the job observation 2 Simulations 3 Essays, and 4 Portfolios. 1 Observation of On-the-job performance: The best way to see if students have learned what we want them to learn at the necessary level is to take into the real world and have them perform what they have been instructed to do. The test instruments can be a checklist or rating scale. Checklists may be appropriate for on-the-job assessments if all critical cognitions in the newly acquired learning are manifested in observable behavior. Rating scale is used when a more refined evaluation of a process or product than just whether it is correct or incorrect, i.e., that indicates degree or levels of quality for a performance or quality. 2 Simulations: Simulations are excellent assessment forms, particularly for assessment of higher-order rule learning and attitude change. Simulations can be delivered with print, video, group interactions, computers, or interactive multi-media. 3 Essays: A correct response to an essay question may vary considerably in terms of sequence, language, and specificity, not to mention correctness of style and grammar. 4 Portfolio: A purposeful collection of student work that tells the story of the student’s efforts, progress, or achievements. This collection must include student participation in selection of portfolio content, the guidelines for selection, the criteria for judging merit and evidence of student self-reflection. Stringent criteria is required to evaluate the adequacy of performance assessment – standards of validity, consequences of assessments on instructional practice, directness, transparency, farness, transfer and generalizability, adequate cognitive complexity of the task content quality, content coverage, meaningfulness, cost, and efficiency. Pencil and paper tests All of the test item form can have a finite and relatively small number of correct responses - with imagination and ingenuity, multiple-choice, matching, completion, and short answer items may be used to assess declarative knowledge, intellectual skills, attitudes, cognitive strategies, and portions of motor skills. There are three different forms: 1 Recall, 2 Recognition, and 3 Constructed answer 1. Recall: Recall items ask learners to simply reproduce the knowledge that they have presented in instruction, in verbatim, paraphrased, or summarized form. 2. Recognition items: Recognition items require the learner to recognize or identify the correct answer from a group of alternatives. 3. Constructed Answer Items: Constructed answer items require that learners actually produce or construct a response. Assessment Item Specification In writing item specifications, the designer describes the characteristics of the items and the forms that responses will take. Item specifications include the following components: 1. Objective: the Objective should be composed of the components – conditions, performance, and perhaps a standard or criterion. 2. Description of test form: The statement may be as simple as multiple-choice form or as complex as a computer-based simulation. The more clarity and detail that is provided in this step, the easier the following steps will be. 3. Sample item: should assess the ability to perform the behaviour described in the objective under the conditions described in the objective’s condition. 4. Question characteristics: defines the qualities of the questions, pencil-and-paper assessments, scenarios in simulations, or situations that will be observed during on-the-job assessments. 5. Response characteristics: the designer must outline the characteristics of the correct answer to be selected, which is relatively easy, and the incorrect answers, which is much more difficult. 6. Number of items and mastery criterion: Objective should have a minimum of eight items with one each from the eight possible directions. A maximum number of items is 24 with three items from each direction. Mastery is 75% of the items correct. Assessment Instrument Blueprint An assessment instrument typically evaluates learners’ performance on more than one objective. Therefore the assessment designer must consider all the item specifications together and define the entire instrument. This plan is sometimes called an instrument blueprint. A blueprint includes the following components: 1. Objectives 2. Forms of items 3. Total number of items in instruments: You may need to determine the number of written items, the number of simulation experiences, the number of on-the-job-observations or the total of items combined. 4. Proportionality of items: the percentage of items that come from each objective is closely related to the total number of items. 5. Direction for administration: The designer will need to write directions for the entire instrument for the students and directions to administrations of the instrument. 6. Scoring methods: This key may be a simple template for items such as multiple-choice and true-false items, or it may need to be as complete and detailed as possible to alleviate errors in scoring other forms of items such as on-the-job-observation, simulations, and essay items. 7. Weighting of items: Since all objectives that are being assessed may not be of equal importance, you may give weight to those important objectives by including more items for those objectives. 8. Passing or cut-off level: The designer may wish to assign or suggest a passing or mastery criterion for the entire test. References: 1 http://www.lessonplans4teachers.com/templates.php 2 Cotter, K. (2001), Summary of instructional observation, Retrieved May 14, 2003 from Http://www.pres-sport.com.obs.pdf 3 http://www.ncrel.org/sdrs/areas/issues/methods/assment/as500.htm 4 Read More