The Use of Reports in University Assessment - Example

RUNNING HEAD: THE USE OF REPORTS IN UNIVERSITY ASSESSMENT The Use of Reports in University Assessment [Name of the Writer] [Name of the Institution] The Use of Reports in University Assessment Introduction There are many possible ways of assessing and evaluating a university or student’s knowledge, skill acquisition, or total learning after a course. Students could sit traditional written exams, which may be essay-type or multiple choice exams. They could be examined in one-to-one, or examiner-panel, oral (viva voce) examinations. Equally, they could be evaluated on a series of coursework assignments or one major project or dissertation. Alternative assessments have enjoyed enormous popularity over the last decade for any number of reasons. Proponents believe that alternative assessments, such as that shown in the first item above, better represent the kinds of disciplinary knowledge students should be acquiring, the complex thinking and communication skills that are essential to students' fixture success, and the rigorous standards for student accomplishment that are being formulated nationally, in most states, and in many localities. More than traditional multiple choice, exemplified by the second item, alternative assessments reflect the kinds of learning activities that current cognitive and curriculum theories advocate. Such assessments provide teachers with rich data on how well students are able to understand a topic, use the tools of the discipline, and support a point of view. The Role of Assessment in Instruction Why is there such belief in the power of assessment to motivate changes in teaching and learning? The sources are many: Advocates of measurement-driven instruction (Popham, Cruse, Rankin, Sandifer, & Williams, 2000) have long seen benefits in establishing and measuring learning objectives; creators of total quality management systems similarly advocate the power of establishing quality goals and measuring and reflecting on their accomplishment (Deming, 2002); and the Governors' Education Summit, National Goals 2000, and other federal programs champion similar conceptions. The logic of using assessment to motivate improvement appears relatively simple: 1. Assessments can communicate meaningful standards to which university systems, universities, teachers, and students can aspire. 2. These standards can provide focus and direction for teaching and learning. 3. Results from the assessment support important insights on the nature, strengths, and weaknesses of student progress relative to the standards. 4. Educators and students will use this feedback to understand and direct their attention to improving relevant aspects of student learning. 5. Coupled with appropriate incentives and/or sanctions—external or self-motivated and directed —assessment will motivate students to learn better, teachers to teach better, and universities to be more educationally effective. The same general stream of logic applies whether the issue is the classroom, with students raising familiar questions—“What's going to be on the test, teacher?” “How will this be graded?” “What do I need to do to get an ‘A’?”—or the context is teachers and administrators responding to local or state testing mandates. Assessment thus can provide valuable focus to the system and has the potential to be a powerful and beneficial engine of change. Alignment with Standards If an assessment is to serve a communication function, focus people's attention on what is important, and provide good information on students' progress, then it must represent the knowledge and skills we expect students to learn: the standards or goals we hold for student accomplishment. While obvious to say, this often is not easy to accomplish. Standards and assessments often exist in quite different forms, formulated at different levels of specificity. In addition a logical sequence of development—first standards, then assessments—infrequently occurs in practice. What features or criteria are essential to establishing such a match? Topic coverage and the level and complexity of cognitive processes are among the important elements of concordance (Webb, 2006). For example, do the assessments and standards evoke the enduring themes, central concepts and principles, and important topics of the discipline? Do the assessment reports call for the kinds of complex thinking and problem solving capabilities called for by the standards or goals? Are the types of problem situations similar and authentic to real life or to the work of the discipline? Based on such guiding questions, the match between valued goals and large scale assessments currently is imperfect. While almost all states and many districts are in the process of developing or have developed high standards for student reports, many testing programs still emphasize standardized multiple choice testing (Bond, Braskamp, & Roeber, 2006). The result is mixed, with confusing messages being delivered to universities and teachers. When test data are not aligned with student learning standards, they cannot tell universities about student progress relative to standards. At the same time, some observers have noted problems in the report and substance of available alternative assessments that limit their coverage of fundamental disciplinary report. Too often, more attention has been given to devising assessments that appear to engage students in complex thinking and problem solving than to assuring that the assessment also deals with essential academic report or that scoring credits higher levels of disciplinary thinking (Baxter & Glaser, 2006). Utility for Improving Instruction and Learning While accuracy of information is a clear precursor to usefulness, the clarity of the feedback provided and its implications for action are very important for instructional utility. Is it clear to teachers, students, and parents what is being assessed and what good reports looks like? To what extent do results provide feedback that clarifies sources of students' strengths and weaknesses and gives guidance on what to do next to improve student reports? Such questions argue for public specifications of reports assessments that make clear what is being assessed and what is expected in student responses. Also of value in this context are analytic scoring schemes. Rather than providing a single, holistic score, analytic scoring can furnish diagnostic feedback on important dimensions of student reports and thus focus instruction and learning. Do existing large scale reports assessments provide such clarity for instructional decision making? For a variety of reasons, including professional disagreements about what good reports assessments should look like and how they should be specified and scored, the answer is frequently “no.” Reports assessments often are ill-defined, drawing on multiple abilities in unique combinations and making it difficult to identify precisely what is being assessed. Technical and practical cost concerns, furthermore, often discourage analytic scoring, because using a single, holistic score is cheaper and faster. Nonetheless, despite some uncertainty about what is being assessed, there is a growing body of research to suggest that new forms of assessment do influence classroom instruction. Studies looking at assessment systems in Vermont, Maryland, Arizona, North Carolina, Kentucky, and various districts consistently show that teachers take the signal from new forms of assessment seriously and modify their classroom practices in line with the goals of the assessment and the standards that underlie them (Koretz, Barron, Mitchell, & Stecher, 2006; Koretz, Mitchell Barro, & Keith, 2006; Koretz, Stecher, Klein, McCaffrey, & Deibert, 1993; McDonnell & Choisser, in press; Smith, 2006). In line with state and constructivist reform goals, teachers in Maryland, for example, reported increases in their students' involvement in analysis of text, literary comprehension, mathematical communication, data analysis, use of graphs and tables, meaningful problem solving, and writing for a variety of purposes. Similarly, district and university level projects that have involved teachers in the development and implementation of alternative assessments document a variety of positive effects on classroom teaching and learning. These include raising teachers' expectations, encouraging teachers to rethink their roles and change their teaching strategies, supporting more active learning, and nurturing reflective practices (Aschbacher, 2004; Flexor, 2005; Gearhart, Wolf, Burkey, & Whittaker, 2004). What Next? The Road Ahead How do new forms of assessment measure up? Where are we with regard to having high quality measures of student reports that can support university reform and the improvement of student learning? The answer is mixed based on the last decade's explosion of interest and experimentation by states, local universities, and teachers, showing clear patterns of strength and weakness. The consequences of using reports assessment in large scale assessment systems appear to be a clear strength: Teachers and principals take the new assessments and the goals they represent seriously, they move to incorporate new pedagogical practices into their teaching, and they engage their students in the kinds of activities they see embodied in the assessment. The assessments serve as a strong signal to focus instruction, and with appropriate professional development and ongoing local support, new assessment can support meaningful change and improvement of practice. While available methods do make possible university-level estimates that can be used for a variety of purposes, educators and the public alike clamor for individual student results from their assessment systems. Parents want to know—and demand—formal, comparable evidence of what their children are learning, and students likewise want to know their progress. Similarly, teachers seek information they can use to understand individual students and how best to support their learning. The public controversies in a number of states and communities, furthermore, underscore a real diversity of opinion about what children ought to know and be able to do and the types of assessments that should be used to measure student achievement. These factors, combined with the current fiscal environment of public austerity and the costs of new forms of assessment, give pause to those considering changes in policy. As we move forward to design better assessment systems, we need to be realistic about the role of assessment in student learning and how much the results from a large scale assessment system can contribute to improvement. References .Baxter, G., & Glaser, R. (2006). An approach to analyzing the cognitive complexity of science performance assessments. Los Angeles: University of California, Center for the Study of Evaluation. Aschbacher, P.R. (2004, June). Helping educators to develop and use alternative assessments: Barriers and facilitators. Educational Policy, 8, 202–223 Bond, L.A., Braskamp, D., & Roeber, E. (2006). The status report of the assessment programs in the United States. Oakbrook, IL: North Central Regional Educational Laboratory (NCREL)/Council of Chief State School Officers. Deming, W.E. (2002). Quality, productivity, and competitive position. Cambridge, MA: Center for Advanced Engineering Study. Flexor, R. (2005). How “messing about” with performance assessment in mathematics affects what happens in classrooms (CSE Technical Report No. 396). Los Angeles: University of California, Center for Research on Evaluation, Standards, and Student Testing (CRESST). Gearhart, M., Wolf, S., Burkey, B., & Whittaker, A. (2004). Engaging teachers in students' narrative writing (CSE Technical Report No. 377). Los Angeles: University of California, Center for Research on Evaluation, Standards, and Student Testing (CRESST). Koretz, D.M., Barron, S., Mitchell, K.J., & Stecher, B.M. (2006). Perceived effects of the Kentucky Instructional Results Information System (KIRIS). Santa Monica, CA: RAND. Popham, W.J., Cruse, K.L., Rankin, S.C., Sandifer, P.D., & Williams, R.L. (2000). Measurement-driven instruction: It's on the road. Phi Delta Kappan, 66, 628–634. Webb, N. (2006). Alignment of content standards and assessment measures in mathematics and science. Invited session at the annual CRESST Conference. Los Angeles: University of California, Center for Research on Evaluation, Standards, and Student Testing (CRESST). Read More