Efficacy of 360 Feedback in Management Development - Literature review Example

0. Research Questions Objectives The 360 feedback is a measure of how well a manager performs in the behavioural areas (Shaver 2000). It is defined as a “systematic collection and feedback of performance data on an individual or group derived from a number of the stakeholders in their performance” (Ward, 1997 p. 4). In this day and age where top level management do not generally always see every aspect of their managers performance, a multi-tier appraisal in the form of a 360-degree feedback usually does a wonderful job. The theme of this study revolves around the efficacy of this instrument in management development. This study will venture to assess the qualifications of three candidates for branch manager in a new office being set up in UK Company using the 360-degree feedback. Specifically, the following research questions will be addressed: 1. What is the profile of the three candidates in terms of the variables: gender, age, civil status, educational attainment, and experience as manager? 2. What is the profile of the raters in terms of the variables: gender, age, civil status, educational attainment, position, and years of service with the company? 3. What is the level of qualification of each candidate in each of the following behavioural areas of management: leading change, leading people, results-driven characteristics, business acumen, and building coalitions / communications? 4. Are there significant differences in the assessment of the manager qualifications in each area of the three candidates when the respondents are grouped according to each profile variable? 5. Is there a significant difference in the assessment of the manager qualifications among the three candidates? 6. Did the personal rating of each candidate significantly differ with the overall assessment of the raters? 7. What are the greatest strengths and weaknesses of each candidate? 2.0. Justification for Research Topic and Background (300) The primary justification for the research topic are the emerging issues in the UK company where management believes that they do not have sufficient evidence to come up with a sound judgment regarding the choice for the best candidate. To further confound their woes, a meeting among top level management and the middle level managers ended in a deadlock as to which candidate is best qualified to take the helm as manager of the new branch office being set up within a year’s time. A look at each candidate’s profile and track record in the company showed that the candidates’ qualifications are comparable. Top management believes that candidates 1 or 3 would be an ideal choice, whereas the middle managers thought that candidate 2 will be the best choice. Top level management is, however, honest enough to admit that they based their nod for candidate 1 or 3 on gut feel. Middle managers confessed that they do not really know candidate 2 very well, but have experienced several altercations with both candidates 1 and 3 on separate instances. Hence, their choice for candidate 2 might have been somehow affected by bias against the other two. Being known to be a business student, one top level manager requested this researcher for some suggestions regarding the aforementioned issues. Without hesitation, and driven by the fact that the new branch office entail management development, 360-degree feedback was the first thing that crossed this researcher’s mind. The suggestion was well-taken and this researcher was asked to help in the conduct of the procedure. The researcher obliged to accept the challenge as an opportunity to put to use a concept learned in business school and to assist a friend who is immersed in the industry. This researcher lost no time in gathering preliminary information. It was learned that the three candidates currently hold middle manager positions in three departments with a comparable number of subordinates, but candidate three handles more employees in the supervisory positions. All three candidates usually deal with many of the company’s clients. This researcher realized that the 360-degree feedback might as well be developed into a full blown research study so that a thoroughly informed decision may be rendered. This served as the essential motivation in the development of this research proposal. 3.0. Critical Literature Review (1000) Brett and Atwater (2001) investigated the relationship between 360? feedback ratings and several factors such as reactions to feedback, feedback accuracy, usefulness of the feedback, and recipients’ openness to development. The results reveal a number of relationships. First, less favourable ratings is linked to less accurate feedback and negative reactions. Second, negative reactions and less accurate feedback are related to perceptions of less useful feedback. Third, less useful feedback is linked to less development-focused respondents. Fourth, a relationship exists between goal orientation and perceptions on the usefulness of feedback. Fifth, no relationship exists between goal orientation, feedback accuracy, and reactions to feedback. Overall, the result of the study challenges the current belief regarding positive change being brought about by negative and discrepant feedback. Facteau and Bartholomew (2001) evaluated a multisource performance appraisal instrument in relation to measurement invariance across varied groups of raters. To test for invariance of the rating instrument across peer, supervisor, and subordinate respondents, techniques such as multiple-groups confirmatory factor analysis and item response theory were utilized. The results reveal that the rating instrument was consistent across all groups. In addition, IRT results show some trivial effects on differential and test functioning. Overall, it has been suggested that the rating instrument is consistent across all groups which support the practice of direct comparison of ratings. Fletcher (2001) described the broadening of performance appraisal as a concept, set of practices, and as a form of performance management as a strategic element of integrating company policies and HR activities. Performance appraisal activities are perceived to have evolved from a measurement technique to a social and motivational driver of performance. These changes may be attributed to the multicultural landscape and impact of new technologies. DeNisi and Kluger (2000) described performance feedback as an invaluable tool in various company initiatives. However, despite the popular notion that performance feedback can influence job performance, feedback mechanisms are not as effective as it may seem. It is argued that feedback mechanisms might be less effective or harmful in certain situations. Thus, it is suggested that feedback mechanisms should undergo systemic evaluations. Van Wart (2009) described continual learning as the process of assimilating new information, finding new uses for old information, and creating opportunities to utilize old and new information in creative ways. In a leadership standpoint, London (2002) recognizes the importance of a leader keeping abreast with the developments around him. An effective leader is able to recognize, understand, respond, and create change. Leaders are expected to be able to identify development needs within the organization and are responsible in addressing such needs. Mumford (2008) considers leadership as a potent tool for social influence and how it affects creativity and innovation has sparked attention for the past 5 to 10 years. Most of the literature deals with how leadership affects the creative process and its effects on the outputs of subordinates. However, the link between leadership and creativity should be viewed in a multi-level perspective. The interaction which exists between leaders and the people behind the creative process is complex and may pose a number of obstacles. To alleviate the problem, leaders should have a receptive attitude towards change and maintain support towards his subordinates. In addition, the leader should act as a champion of the creative vision wherein they assume the responsibility of maintaining the culture of creativity and to influence others to play an active role in the creative process. Furthermore, an effective leader should consider the following facets of creativity: problem solving, vision, planning, implementation, garnering support, and engagement (Mumford 2008). Prather (2010) observed that an effective leader acknowledges that the adoption of leadership style depends on the situation and the required results. A leader may assume an authoritarian, participatory, mentor, or coach role based on the circumstances. An authoritarian role is assumed in situations wherein time is of the essence and results are expected to be delivered in a limited amount of time. Meanwhile, a leader may take a participatory role during decision-making wherein inputs from all the members are taken into consideration with the goal of coming up with a decision which will ultimately make an impact on the group. On the other hand, a mentor leader shares his knowledge, expertise, and experience to the group with the aim of assisting members in improving their performance. A mentor may also help new members learn important information in a limited amount of time or build up future leaders. Lastly, a coach leader engages members of the group in activities designed to maximize their personal and professional potential. Coach leaders usually produce more creativity and innovation from members compared to other leadership styles (Prather 2010). 4.0. Methods The procedure proposed in the conduct of this proposed study is the descriptive – quantitative research design. The descriptive method of research determines prevailing conditions. It delves on the “what is” in terms of the variables being examined in the study (Adanza & Martinez 2002). The descriptive research is deemed most suitable for conduct of this study since its general objective is to appraise the current qualifications of three managerial candidates for purposes of management development. Additionally, a descriptive research methodology has been proven in its efficacy in the collection of data for hypothesis testing and evaluating relationships (Monsen & Van Horn 2008). The survey method will be used in the collection of data. However, purposive sampling will be used in the sampling design. In purposive sampling, the researcher identifies the ideal characteristics of the population of interest and finds the individuals with such characteristics as sample of the study (Johnson & Christensen 2011). In this regard, the raters will be purposively sampled to be the best people who can provide an objective feedback to each of the three candidates. Accordingly four top level managers, five middle level managers, three subordinates and three supervisors in each department handled by the candidates, and five clients who have communicated at least once to all three managers. Each candidate will also be requested to make a self-report of their managerial skills using the same instrument. Hence, the total respondents in the proposed study will be a total of 35. A canned 360-degree feedback questionnaire retrieved from Exhibit Dynamics (2006) will serve as the main research instrument. Permission will be sought from the organization to use the 360-degree feedback questionnaire in this study. The questionnaire was divided into 5 sections and consists of 27 items. Section 1 appraises manager qualifications for leading change and consists of 8 items (1-8). Meanwhile, section 2 deals with manager qualifications for leading people and is comprised of 4 items (9-12). Section 3 evaluates results-driven characteristics of managers and is made up of 6 items (13-18). Section 4 examines the business acumen of the candidates based on 3 items (19-21). The last section scrutinizes the manager qualifications for building coalitions / communications and consists of 6 items (22-27). Responses in the questionnaire correspond to a five-point Likert scale. Following is a researcher-devised interpretation scale which will be useful in analyzing the findings of the study, including the statistical limits for each level: Scale Questionnaire Response Statistical Limits Qualitative Interpretation 1 Strongly disagree (SD) 1.00 – 1.35 Not qualified (NQ) 2 Disagree (DA) 1.36 – 2.61 Barely qualified (BQ) 3 Neither agree nor disagree (NA) 2.62 – 3.38 Can not evaluate in this regard (CN) 4 Agree (AG) 3.39 - 4.64 Moderately qualified (MQ) 5 Strongly agree (SA) 4.65 – 5.00 Highly qualified (HQ) A profile questionnaire was appended at the end of the canned 360 degree questionnaire. Reliability of the 360-feedback questionnaire was examined using the pre-test-post-test method. As explained in Cottrell and McKenzie (2011), reliability is the “extent to which a measurement instrument will produce the same or nearly the same result (measure or score) each time it is used” (p. 151). The test-retest reliability or pretest-posttest reliability, otherwise known as stability reliability, will be used to assess stability over a period of time (Cottrell & McKenzie 2011). In test-retest reliability, the research instrument will be administered to a quota sample of 7 employees in the same UK company twice over a period of one week. The same questionnaire will be administered under approximately the same conditions. Employees who will agree to participate in the pilot testing will not anymore be requested to be included in the purposive sampling during the research proper. The responses in each item of the assessment during the pre-test will be correlated with each item of the assessment during the post-test. The resulting Pearson coefficient of correlation will be computed using a 0.05 level of significance in a two-tailed analysis. Only the main questionnaire will included in the pilot testing, equivalent to 27 items. The table from De Vaus (2001, p. 259) will be used to describe the relationship among the responses between the pre-test and post-test. Data analysis will be performed using SPSS Statistics (2008). Both descriptive and inferential measures will be used in the statistical treatment of data. In hypothesis testing, analysis will be undertaken using a 0.05 level of significance (? = 0.05), and a non-directional or two-tailed approach. Profile variables will be described using frequency and percentage distributions in the form of a pie chart. Meanwhile, the level of qualification of each candidate in each of the five behavioural areas of management will be described using the simple mean, the standard deviation and simple ranking. Significant differences in the assessment of the manager qualifications in each area for the three candidates when the raters are grouped according to each profile variable will be statistically analyzed in the following manner: (1) gender and qualification ratings – by using independent samples t-test; (2) the rest of the profile variables and qualification ratings – using one-way analysis of variance (ANOVA), multiple comparisons test, and Levene’s test. Levene’s test of homogeneity of variance will be utilized to check whether the ratings in the 360-degree feedback are normally distributed since normality is a pre-condition for the use of the t-test and ANOVA. The results of Levene’s test is crucial in the t-test since in SPSS, different values of the t-statistic, the degrees of freedom and the p-value are provided when equal variances are not assumed and when equal variances are not assumed. Results of Levene’s test is also important in ANOVA in cases when significant differences among the group means are detected. If the F-statistic in Levene’s test is not significant, several post hoc multiple comparisons tests can be chosen, such as Least Significant Difference or Bonferroni. On the other hand, if the F-statistic in Levene’s test is significant, Tamhane’s T2 or Dunnett’s T3 may be used. The means among the groups are significantly different in ANOVA, when the F-statistic assumes a large value and the p-value or Sig. (meaning significance) is equal to or less than the hypothesized level of significance of 0.05. One-way ANOVA will also be used to examine whether significance difference exists in the assessment of the manager qualifications. One sample t-test will be used to determine whether the overall assessment of the raters significantly differed with the personal rating reported by each manager. Simple ranking will be used to outline the greatest strengths and weaknesses of each candidates. The 10 highest means in each item of the 360-degree feedback, without being lower than 3.39, will be considered as the strengths. On the other hand, the lowest 10 means in each item of the 360-degree feedback, lower than or equal to 2.61 will be considered as the weaknesses, if any of the three candidate managers. 5.00. Time Scale and Resources The time scale is shown next page. Resources needed for the project which would probably be shouldered by the UK company involved would include a ream of bond paper for printing the documentation of the study, at least 35 sets of the 6-page questionnaire, SPSS CD-ROM for the data analysis, and a personal computer or laptap for the processing of the 360-degree feedback responses. Activities Weeks 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 Request for use of instrument and approval Instrument Design if request is disapproved Pilot Testing Questionnaire Administration and Retrieval Processing of responses / encodng Data Analysis Writing-up of research 6.0. References Adanza, EG & Martinez, FN (2002), Methods of research for the health professions, Rex Book Store, Quezon City, Philippines. Brett, JF & Atwater, LE 2001, ‘360 feedback: Accuracy reactions and perceptions of usefulness’, Journal of Applied Psychology, 86(5), 930-942. Cottrell, RR & McKenzie, JF 2011, Health promotion & education research methods: Using the five-chapter thesis / dissertation model, 2nd edn, Jones and Bartlett Publishers, London, GBR. DeNisi, AS & Kluger, AN 2000, ‘Feedback effectiveness: Can 360-degree appraisals be improved?’, The Academy of Management Executive, 14(1), 129-139. Exhibit Dynamics 2006, 360° Feedback, viewed 31 March 2011, http://www.hr-survey.com/ sd360nq.htm Facteau, JD & Craig, SB 2001, ‘Are performance appraisal ratings from different rating sources comparable?’, Journal of Applied Psychology, 86(2), 215-227. Fletcher, C 2001, ‘Performance appraisal and management: The developing research agenda’, Journal of Occupational & Organizational Psychology, 74(4), 473-487. Johnson, B and Christensen, L. 2012, Educational research: Quantitative, qualitative, and mixed approach, Sage Publications, Thousand Oaks, CA. London, M (2002), Leadership development: paths to self-insight and professional growth, Lawrence Earlbaum Associates, Manwah, NJ. Monsen, ER & Van Horn, L 2008, Research: Successful approaches, third edn, American Dietetic Association, Chicago, IL. Mumford, MD (2008), Multi-level issues in creativity and innovation, Elsevier, Oxford. Shaver, WJ 2000, ‘What is a 360-degree feedback?’, In W Shaver (ed), How to build and use a 360-degree feedback system: Management development, American Society for Training and Development, Alexandria, VA. Toegel, g & Conger, JA 2003, ’60-degree assessment: Time for reinvention’, Academy of Management Learning & Education, 2(3), 297-311. Van Wart, M (2008), Leadership in public organizations: an introduction, M.E. Sharpe, Armonk, NY. Ward. P 1997, 360-degree feedback, Chartered Institute of Personnel and Development, London, GBR. Appendix 1 Research Instrument Instructions. As part of a 360 degree feedback process, you have been requested to assess yourself in terms of the items below. Your manager/s, peers, and subordinates will also evaluate you in order to generate a multi-perspective or 360 degree evaluation of your performance. The overall objective of this activity is to provide you with feedback which can be used as a basis for performance improvement. Improvement of performance will lead to improved team performance and overall organizational effectiveness as well. Leading Change Strongly Disagree Disagree Neither Agree nor Disagree Agree Strongly Agree 1. Continual Learning - Grasps the essence of new information; masters new technical and business knowledge; recognizes own strengths and weaknesses; pursues self-development; seeks feedback from others and opportunities to master new knowledge. ? ? ? ? ? 2. Creativity and Innovation - Develops new insights into situations and applies innovative solutions to make organizational improvements; creates a work environment that encourages creative thinking and innovation; designs and implements new or cutting-edge programs/processes. ? ? ? ? ? 3. External Awareness - Identifies and keeps up to date on key national and international policies and economic, political, and social trends that affect the organization. Understands near-term and long-range plans and determines how best to be positioned to achieve a competitive business advantage in a global economy. ? ? ? ? ? 4. Flexibility - Is open to change and new information; adapts behavior and work methods in response to new information, changing conditions, or unexpected obstacles. Adjusts rapidly to new situations warranting attention and resolution. ? ? ? ? ? 5. Resilience - Deals effectively with pressure; maintains focus and intensity and remains optimistic and persistent, even under adversity. Recovers quickly from setbacks. Effectively balances personal life and work. ? ? ? ? ? 6. Service Motivation - Creates and sustains an organizational culture which encourages others to provide the quality of service essential to high performance. Enables others to acquire the tools and support they need to perform well. Shows a commitment to public service. Influences others toward a spirit of service and meaningful contributions to mission accomplishment. ? ? ? ? ? 7. Strategic Thinking - Formulates effective strategies consistent with the business and competitive strategy of the organization in a global economy. Examines policy issues and strategic planning with a long-term perspective. Determines objectives and sets priorities; anticipates potential threats or opportunities. ? ? ? ? ? 8. Vision - Takes a long-term view and acts as a catalyst for organizational change; builds a shared vision with others. Influences others to translate vision into action. ? ? ? ? ? Leading People Strongly Disagree Disagree Neither Agree nor Disagree Agree Strongly Agree 9. Conflict Management - Identifies and takes steps to prevent potential situations that could result in unpleasant confrontations. Manages and resolves conflicts and disagreements in a positive and constructive manner to minimize negative impact. ? ? ? ? ? 10. Leveraging Diversity - Recruits, develops, and retains a diverse high quality workforce in an equitable manner. Leads and manages an inclusive workplace that maximizes the talents of each person to achieve sound business results. Respects, understands, values and seeks out individual differences to achieve the vision and mission of the organization. Develops and uses measures and rewards to hold self and others accountable for achieving results that embody the principles of diversity. ? ? ? ? ? 11. Integrity/Honesty - Instills mutual trust and confidence; creates a culture that fosters high standards of ethics; behaves in a fair and ethical manner toward others, and demonstrates a sense of corporate responsibility and commitment to public service ? ? ? ? ? 12. Team Building - Inspires, motivates, and guides others toward goal accomplishments. Consistently develops and sustains cooperative working relationships. Encourages and facilitates cooperation within the organization and with customer groups; fosters commitment, team spirit, pride, trust. Develops leadership in others through coaching, mentoring, rewarding, and guiding employees. ? ? ? ? ? Results Driven Strongly Disagree Disagree Neither Agree nor Disagree Agree Strongly Agree 13. Accountability - Assures that effective controls are developed and maintained to ensure the integrity of the organization. Holds self and others accountable for rules and responsibilities. Can be relied upon to ensure that projects within areas of specific responsibility are completed in a timely manner and within budget. Monitors and evaluates plans; focuses on results and measuring attainment of outcomes. ? ? ? ? ? 14. Customer Service - Balancing interests of a variety of clients; readily readjusts priorities to respond to pressing and changing client demands. Anticipates and meets the need of clients; achieves quality end-products; is committed to continuous improvement of services. ? ? ? ? ? 15. Decisiveness - Exercises good judgment by making sound and well-informed decisions; perceives the impact and implications of decisions; makes effective and timely decisions, even when data is limited or solutions produce unpleasant consequences; is proactive and achievement oriented. ? ? ? ? ? 16. Entrepreneurship - Identifies opportunities to develop and market new products and services within or outside of the organization. Is willing to take risks; initiates actions that involve a deliberate risk to achieve a recognized benefit or advantage. ? ? ? ? ? 17. Problem Solving - Identifies and analyzes problems; distinguishes between relevant and irrelevant information to make logical decisions; provides solutions to individual and organizational problems. ? ? ? ? ? 18. Technical Credibility - Understands and appropriately applies procedures, requirements, regulations, and policies related to specialized expertise. Is able to make sound hiring and capital resource decisions and to address training and development needs. Understands linkages between administrative competencies and mission needs. ? ? ? ? ? Business Acumen Strongly Disagree Disagree Neither Agree nor Disagree Agree Strongly Agree 19. Financial Management - Demonstrates broad understanding of principles of financial management and marketing expertise necessary to ensure appropriate funding levels. Prepares, justifies, and/or administers the budget for the program area; uses cost-benefit thinking to set priorities; monitors expenditures in support of programs and policies. Identifies cost-effective approaches. Manages procurement and contracting. ? ? ? ? ? 20. Human Resources Management - Assesses current and future staffing needs based on organizational goals and budget realities. Using merit principles, ensures staff are appropriately selected, developed, utilized, appraised, and rewarded; takes corrective action. ? ? ? ? ? 21. Technology Management - Uses efficient and cost-effective approaches to integrate technology into the workplace and improve program effectiveness. Develops strategies using new technology to enhance decision making. Understands the impact of technological changes on the organization. ? ? ? ? ? Building Coalitions / Communications Strongly Disagree Disagree Neither Agree nor Disagree Agree Strongly Agree 22. Influencing / Negotiating - Persuades others; builds consensus through give and take; gains cooperation from others to obtain information and accomplish goals; facilitates "win-win" situations. ? ? ? ? ? 23. Interpersonal Skills - Considers and responds appropriately to the needs, feelings, and capabilities of different people in different situations; is tactful, compassionate and sensitive, and treats others with respect. ? ? ? ? ? 24. Oral Communication - Makes clear and convincing oral presentations to individuals or groups; listens effectively and clarifies information as needed; facilitates an open exchange of ideas and fosters an atmosphere of open communication ? ? ? ? ? 25. Partnering - Develops networks and builds alliances, engages in cross-functional activities; collaborates across boundaries, and finds common ground with a widening range of stakeholders. Utilizes contacts to build and strengthen internal support bases. ? ? ? ? ? 26. Political Savvy - Identifies the internal and external politics that impact the work of the organization. Approaches each problem situation with a clear perception of organizational and political reality; recognizes the impact of alternative courses of action. ? ? ? ? ? 27. Written Communication - Expresses facts and ideas in writing in a clear, convincing and organized manner. ? ? ? ? ? Kindly provide the demographic information being requested by choosing the appropriate response and ticking off the category or writing the data on the space provided for. Profile of Candidates (Note to Raters: Please do not answer this section.) Gender ? Female ? Male Age Group (in years) ? 18 - 24 ? 25 - 35 ? 36 - 46 ? 47 – 60 ? 61 & older Civil status ? Single ? Marries ? Separated ? Divorced ? Widow / Widower Highest educational attainment ? High School ? Some College ? Vocational Course ? College ? Other: _______ Length of Experience as manager ? Less than 6 months ? 6 months to 11 months ? 1 to 2 years ? 3 to 4 years ? 5 or more years Profile of Raters (Note to Candidates: Please do not answer this section) Gender ? Female ? Male Age Group (in years) ? 18 - 24 ? 25 - 35 ? 36 - 46 ? 47 – 60 ? 61 & older Civil status ? Single ? Marries ? Separated ? Divorced ? Widow / Widower Highest educational attainment ? High School ? Some College ? Vocational Course ? College ? Other: _______ Position ? ? ? ? ? Years of service with the company ? Less than 6 months ? 6 months to 11 months ? 1 to 2 years ? 3 to 4 years ? 5 or more years Read More