Evaluation and Performance of School Principals - Assignment Example

?Evaluation and Performance of School Principals I. Overview and Implication on Research Although evaluation of principals has been around for at least a decade, the practice of evaluation of principals remains relatively new. This is especially because research has found that they have been inconsistently implemented or that school districts have been lax in implementing evaluation systems for principals. Research projects that were reviewed in this work indicate that the practice of evaluation of school principals has been non-uniform in the United States. There is even no evidence that the impact of evaluation systems for principals has been studied. Further, studies on evaluation systems for principals are very view and scant. Nevertheless, we review what we have gathered for their validity and reliability and identify from the research the direction for future research. Validity refers to the ability of a particular research instrument to measure what it purports to measure (Cohen et al., 2005, p. 105). Cohen et al. (2005, p. 105) clarified, however, that there are several types of validity. According to Cohen et al. (2005, p. 105), some of the concepts of validity relevant for research are as follows: content validity, criterion-related validity; construct validity; internal validity; external validity; concurrent validity; face validity; jury validity; predictive validity; systematic validity; catalytic validity; descriptive validity; interpretative validity; theoretical validity; and evaluative validity. Many of the concepts of validity mentioned are discussed in Cohen et al. (2005, p. 105-117). Meanwhile reliability refers to consistency and replicabilty over time (Cohen et al., 2005, p. 117). According to Cohen et al. (2005, p. 117), reliability is also concerned with precision and accuracy. They emphasized that “for research to be reliable it must demonstrate that if it were to be carried out on a similar group of respondents in a similar context, similar results would happen (Cohen et al., 2005, p. 117). Some of the other concepts related with reliability are stability, equivalence, and internal consistency (Cohen et al., 2005, p. 117-120). According to Cohen et al. (2005, p.120-133), validity and reliability should be simultaneously considered in education research. My review of research done on evaluation systems for principals indicate that further research on evaluation systems for principals should be along these lines: 1. Identifying the elements of the principal evaluation systems adopted by schools making great progress in instruction work and the role played by principal evaluation. 2. Identifying whether there is a correlation between academic performance of schools and the evaluation systems they have adopted for principals. 3. The role played by VAL-Ed or the Vanderbilt Assessment of Leadership in Education or VAL-ED in improving school performance, if any. The VAL-Ed supposedly met the high standards for content validity and reliability among the principal evaluation systems investigated by the New Leaders for New Schools in 2010. 4. The constraints encountered by schools in implementing a good evaluation system for principals. II. Sun and Youngs (2009) Sun and Youngs (2009, p. 2) described their work to have used hierarchical multivariate linear models “to investigation the relationships between principal evaluation purpose, focus and assessed leadership activities in 13 school district in Michigan”. Sun and Youngs (2009, p. 2) argued that their study found that “principals were more likely to engage in learning-centered leadership behaviors when the purposes of evaluation included principal professional development, school restructuring and accountability”. However, Sun and Youngs (2009, p. 2) also clarified that when the focus of evaluation was related to instructional leadership in the school setting, the thrust of the school principal activities have been in curriculum design, teacher professional development and evaluation, and monitoring student learning. In short, the Sun and Youngs study found that evaluation purpose influenced the emphasis of the principals’ activities. It is noteworthy that the Sun and Young study employed a sophisticated statistical measure called the hierarchical multivariate linear model. However, it seems also viable to do the same study using relatively simple statistical techniques like the good chi-square. How the researcher classified evaluation purpose between principal professional development and instructional leadership had probably influenced the statistical results and, thus, it can be argued that it is viable to criticize both the validity and reliability of the research design. Other investigators can use the categorization system used by Sun and Youngs but implementation of the categorization system can vary from investigator to investigator and, thus, one can validly criticize the reliability of Sun and Young research design. Nevertheless, it appears consistent with intuition that principals would be modifying their behavior or style of academic leadership based on the principal criteria or criterion that will be used to evaluate their performance. Despite the lack of reliability, it can be argued that the Sun and Youngs study is valid. However, the same study results can be obtained by qualitative research methods as well. III. Goldring et al. (2008) The Goldring et a. (2008) study merely identified how evaluation of principals vary across states and districts. Goldring et al. (2008) found that states and districts vary on how schools principals are evaluated. Futher, Goldring et al. (2008, p. 34) found that “districts focus on a variety of performance areas when evaluating their principals” and the states and districts use “different formats at various levels of specificity”. Unfortunately, they also found that the evaluation on principals does not promote enough rigorous curriculum development and quality instructions (Goldring et al., 2008, p. 29). For example, Goldring et al. (2008, p. 29) pointed out that most of the assessment lack justification and documentation in terms of “utility, psychometric properties, and accuracy of instruments”. The Goldring et al. (2008) study used document review or content analysis as the main method in their research. In the content review and content analysis, the Goldring et al. (2008, p. 25) team developed a set of codes and glossary to “capture the content of each of the times on the principal leadership assessment instruments”. It is highly likely that another set of investigators would be using another set of codes and glossary if the same set of data or documents were reviewed. Nevertheless, it is likely however that if there are different groups or sets of investigators for the same documents or instruments subjected to content review and content analysis, it is likely that all investigators would agree that the state and district evaluations used “different formats at various levels of specificity”. As to whether, different set of investigator would have the same conclusion that the assessments do not promote rigor, the finding would probably depend on the professional exposure and competence of the investigators involved. However, it is likely though that all investigators would be able to note that the assessment used by states and district lack justification and documentation in terms of the“utility, psychometric properties, and accuracy of instruments”. IV. Kimball (2006) Kimball (2006) documented the “standards-based principal performance evaluation” when it was initiated or pilot-tested in the Washoe Country School District in the fall of 2005. As per Kimball’s description (2006, p. 3), “standards-based leadership evaluation systems” have three features. The first feature is that leadership qualities that were used for the performance evaluation were the leadership qualities found by research to enhance improvements in teaching and student achievement. The second feature is that the rubrics in the standards-based evaluation systems clarify good performance. Finally, the third feature of a standards-based evaluation systems is that evaluation results can be used as the foundation for a “coordinated human resource management system” related to selection, induction, development, assessment, and compensation. Kimball’s case study (2006) covered a description of how the Washoe Country School District standards-based principal performance evaluation system was designed, including its context, impetus for the principal evaluation reform, design process, structure of the new principal performance evaluation system, and an overview of the initial pilot phase used to test the system prior to full implementation (p. 4). As of 2006, the research was still in progress to assess the impact of the pilot phase (Kimball 2006, p. 4). The Kimball (2006) research is a documentation research assessed from his perspective or point of view. Given that the process were documented by Kimball from his point of view Unfortunately, follow-up research on the evaluation are not immediately available but the Washoe Country School District, in its report in October 2010 prepared by its Public Policy, Accountability, and Assessment Group, students three show improvements in reading by 6%, grade five by 8%, grade eight by 9% over the last three years. Although earlier educational targets set the Washoe Country School Districts were not met, the gains were considered substantial (Washoe Country School District 2010, p. 3). Although we are unable to attribute the significant achievements to the evaluation system established and conducted among the principals, we can say that the evaluation system did not worsened the principal’s performance. V. De Grauwe (2008) Evaluation of principals is typically part of a monitoring and evaluation (also known as M&E) systems for schools. Based on the work of Kimball (2006), the evaluation of principals was just being pilot-tested or initiated in the fall of 2005. Thus, unfortunately, as the evaluation of principals appear to be a relative new thing based on the work of Kimball (2006), the impact of evaluation on principals may not be immediately possible to discover. Meanwhile, a United Nations study conducted by Anton de Grauwe (2008) attempted to find out the impact of school monitoring system on disparities. The De Grauwe (2008) methodology is unclear but reading the entire article would indicate that document review and content analysis of a truckload of documents and studies were the main techniques used for the research. In addition, the De Grauwe (2008) also study covered how monitoring personnel have allocated their working hours (p. 4-5). The study covered no less than ten countries of Asia and Africa. Unfortunately, however, although the subject matter of the work of De Grauwe (2005) was supposedly the impact of monitoring system on schools, his conclusions are not those pertaining to monitoring systems. Instead, his conclusions had touched instead on how the school monitoring systems can be improved and not on the impact of school monitoring systems. De Grauwe (2008) made four conclusions (p. 29-30): 1. There must be a balance between the mandate assigned to supervision and the resources available at their disposable. 2. Once a clear mandate has been assigned, the mandate must should be the basis for the organization and structure of the service. 3. There must be a balance between support and control in the supervision of principals and teachers. 4. All personnel must be mobilized for the monitoring system and not only the supervisors. In other words, De Grauwe’s conclusions are very far from what one should expect on an evaluation of what an impact a school monitoring system should be able to produce or unable to produce. Viewed from the standpoint of the study’s own research objectives, it is not even appropriate to say that the De Grauwe study is invalid and unreliable. To say the least, the De Grauwe (2008) study is a mismatch among study objectives, methodology, and conclusions. Unfortunately, the sponsor for the study is a United Nations body. To emphasize and to say the least, there is not one iota or shred of evidence the De Grauwe addressed both the objective and topic of his study. An interpretation that I can offer on the work of De Grauwe (2008) is that the study has been poorly designed or that the author attempted to cover up that the monitoring system had no impact yet at the time of the study. Of course, to his credit, De Grauwe may just have attempted to be diplomatic by providing the wrong answers to logical questions that can be posed on the study. VI. US National Comprehensive Center for Teacher Quality (July 2010) The United States National Comprehensive Center for Teacher Quality recommended that in conducting evaluation systems for school principals, evaluators must see to that clear expectations and goals for assessment are established (2010, p. 8-14). In particular, the US National Comprehensive Center for Teacher Quality recommended that evaluations must establish what will be assessed, who will be helping provide the feedback on the school principals, how the findings of the assessment will be used, and the frequency of the evaluation of the school principals. In addition, the national center also recommended that multiple forms assessment be used and that one must vary the data collected to obtain a holistic of principal performance (US National Comprehensive Center for Teacher Quality, 2010, p. 18-19). In the opinion of this scholar, more than anything else the document indicate that evaluation of principals is a relatively a new thing in the United States and perhaps in the entire educational community. As the sources of data and insights for the assessment appear to be not established in the document, a viable interpretation of the document is that systems for evaluation of principals are not yet fully established in the United States. Further, the document indicates that sources of data and frequency of the evaluation for principals is not yet established in the United States. The national document also suggests that evaluation of principals is not yet uniformly practice in the United States that is also suggestive that systems for the evaluation of school principals are unevenly implemented in the United States. The National Comprehensive Center for Teacher Quality (2010, p. 20) identified the evaluation system for schools principals in Pittsburg as an example of an evaluation system for principals with a positive impact although it can be argued that this is an effect rather than an impact. In particular, according to National Comprehensive Center for Teacher Quality (2010, p. 20), the evaluation system for principals enabled school administrators to classify principals into four categories (rudimentary, emerging, proficient, and accomplished) versus the old classification system of only two categories, satisfactory and unsatisfactory. In addition, the new evaluation system for principals in Pittsburg enabled principals to have a bonus in US$10,000 annually if they can demonstrate growth in student achievement. The issues of validity and reliability do not appear to be applicable in how data in the Pittsburg evaluation system for principals were used by the National Comprehensive Center for Teacher Quality. This is because the National Comprehensive Center for Teacher Quality as well as the agencies, which actually conducted the study of the evaluation system for principals at Pittsburgh, merely described the Pittsburg classification system and the associated bonuses or rewards if the principals are able to display qualities that would qualify them as accomplished. However, the criteria used by Pittsburg for classifying principals as “accomplished” can be questioned for validity and reliability. Unfortunately, no other details on the Pittsburg evaluation system for principals are available and, thus, we are unable to assess the reliability and discuss the details of the validity of the Pittsburg system for categorizing principals. VII. New Leaders for New Schools (2010) Based on 300 principal evaluation instruments covering 21 states of the United States, the New Leaders for New Schools (2010, p. 11-12) identified four key findings from field research on evaluation of school principals. First, according to the New Leaders for New Schools (2010, p. 11-12), principal evaluation systems tend not to focus enough on the right things. In particular, evaluation systems for principals tend to assess “general management”. The New Leaders for New Schools believe however that evaluation of principals should “narrow and deepen the focus on a small number of leadership practices, with particular attention to leadership in the domains of instruction, school culture, and human capital management” (New Leaders for New Schools, 2010, p. 11). Second, according to the New Leaders for New Schools (2010, p. 12), another shortcomings of current principal evaluation systems are that they are not to be based on clear performance standards. Thus, the New Leaders for New Schools (2010, p. 12) echoed the view of Doug Reeves that good evaluations systems for principals must have a “clear definitions of performance levels and precise rubrics allowing evaluators to effectively measure aspects of performance.” The New Leaders for New Schools (2010, p. 12) articulated a third point: principal evaluation systems are limited in the rigor of their design. In particular, they do not have clear psychometric properties and have been at least a decade old and were not constructed with the latest research in mind. However, related to this, the New Leaders for New Schools (2010, p. 12) mentioned the Vanderbilt Assessment of Leadership in Education or VAL-ED met the high standards for content validity and reliability. Finally or fourth, the evaluation systems for principals have been limited by the rigor of their implementation (New Leaders for New Schools, 2010, p. 12). In other words, the implementation of the evaluation systems for principals has been weak in the first place to contribute significantly to the improvement in instructions. In developing the findings, the New Leaders for New Schools reviewed the literature and undertook content analysis of the document. Of course, questions can be raised on the reliability and validity of the findings but the four points identified by the New Leaders and New Schools can be argued as reliable and valid findings as they seem to provide a reasonable narration on the status of evaluation systems for school principals. VIII. Catano and Stronge (2007) The study of Catano and Stronge used quantitative and qualitative methods of content analysis (2007, p. 379). As articulated by Catano and Stronge (2007, p. 379) themselves, the purposes of their study on the evaluation systems for principals in the school districts of Virginia have been the following: 1. Determine the emphasis place upon leadership and management behaviors expected from school principals; 2. Explore the congruence the instruments used for evaluation and the emphasis placed on leadership and management expected from the principals; and 3. Explore the congruence between the instrument used for evaluation and state and professional standards. The study has found, however, that instructional leadership invested considerable time on written language to the area of instructional quality and delivery (Catano and Stronge (2007, p. 394). Further, Catano and Stronge (2007, p. 394) found that although the school districts expectations that were in common with both state and professional standards, the instruments for principal evaluation have been inconsistent and produced role conflicts and role strains as “principals strive to comprehend which expectations they should focus their attention upon”. The quantitative methods used by Catano and Stronger (2007) simply made use of frequency counts and percentage computations of select themes as they were found in the instruments of the Virginia schools. Examples of these are those related to vision, program, organizational management, community relations, and the like. However, the authors made the interpretation as to whether the themes are consistent with the behavior expected from them and whether the themes found in the instrument were in congruence with professional standards and expectations of state authorities. Again, this point can raise questions of validity and reliability because other researchers can have a different set of view on the mater. Nevertheless, as the discussion of Catano and Stronge are plausible, they can be considered as valid and reliable unless proven otherwise. References Catano, N. and Stronge, J. (2007). What do we expect of school principals? Congruence between principal evaluation and performance standards. International Journal for Leadership in Education, 10 (4), 379-399. Cohen, L., Manion, L., and Morrison, K. (2005). Research methods in education. 5th Edition. London and New York: Routledge/Falmer Taylor & Francis Group. De Grauwe, A. (2008). School monitoring systems and their impact on disparities. Background paper prepared for the Education for All Global Monitoring Report 2009. United Nations: United Nations Educational, Scientific and Cultural Organization. Goldring, E., Cravens, X., Murhy, J., Elliot, S., and Carson, B. (2008). The evaluation of principals: What and how do states and districts asses leadership. The Elementary School Journal, 110 (1), 19-39. Kimball, S. (2006). Case study of the initiation of standards-based principal performance evaluation in Washoe County School District. US Department of Education: Institute for Education Sciences, Education Finance, Leadership, and Management Research Program Grant R305EO5135. Available from: http://cpre.wceruw.org/principal/washoe_prin_case.pdf [Accessed 9 April 2011]. New Leaders for New Schools, 2010. Evaluating principals: Balancing accountability with professional growth. Washington: New Leaders for New Schools. Sun, M. and Youngs, P. (2009). How does district principal evaluation affect learning-centered principal leadership? Evidence from Michigan School Districts. Leadership and Policy in Schools, 8(4), 411-445. As available from: http://www.allacademic.com/meta/p_mla_apa_research_citation/3/7/8/0/8/p378087_index.html [Accessed 9 April 2011]. US National Comprehensive Center for Teacher Quality. (July 2010). Evaluating School Principals. Washington: US Department of Education in cooperation with Vanderbilt University. Washoe County School District, 2010. Response to instruction/intervention year two implementation evaluation report Washoe Country School District, 2009-2010. Washoe Country School District: Public Policy, Accountability and Assessment. Available from: http://www.washoecountyschools.org/rti/RTI%20Eval%20Executive%20Summary_2009-2010.pdf [Accessed 9 April 2011]. Read More