The Assessment of an Employees Intelligence - Coursework Example

Organizations are increasingly coming to rely on Psychology and more specifically, psychometric testing, to dictate employee hiring, appraisal, and overall organizational effectiveness. For the purpose of this discussion, the assessment of an employee’s intelligence will be used as an example to demonstrate the inherent sources of bias throughout the psychological testing process. Intellectual assessment, in particular, is a subject of great debate amongst psychologists, educators, government officials, and test-takers. In general, problems of accurately defining ‘intelligence’ (or the inclusion of differing abilities in theoretical definitions) are directly reflected in the tests developed. In other words, new tests can be developed to describe the elements included in any particular theory. Anderson, Herriot, & Hodgkinson (2001) devoted an entire article to the necessity of “Pragmatic” Psychology being applied at work, wherein both methodological rigour and practical relevance are high. Unfortunately, they also claim that this is not where organizations are heading in their use of Psychology. It is therefore essential that the practitioner or test administrator understand and appreciate the limitations of the tests they administer, the need for stringent processes on part of tester, and the necessity of aiming for objective and unbiased results. There exist many sources of random error in psychometric testing and its subsequent interpretation. In order to minimize such error, researchers apply various technical criteria to ensure the psychological soundness of tests. However, some degree of systematic bias remains inherent in psychometrics, preventing accurate measurement. Although psychometric testing serves to indicate an aspect of a participant’s behavior, psychologists (and any test users) should understand that test limitations do exist. In fact, error is a necessary component of measurement, and therefore must be considered in all analysis of results. According to Cohen & Swerdlik (2005), error variance is “the component of a test score attributable to sources other than the trait or ability measured” (p.96). Therefore, it is assumed that a test taker has a ‘true’ score that would exist if not for error. Sources of error variance include test construction, administration, scoring and interpretation. Examples of these sources during testing range from the test-room temperature to the presence or absence of a test giver. A test is reliable when it consistently measures the same thing in the same manner. However, reliability and accuracy do not necessarily go hand in hand. For example, a scale measuring a two-pound bag of sugar at 1.8 pounds on three separate trials is reliable and inaccurate. The greater the proportion of variance that is ‘true’, the more reliable the test becomes. The differences in scores caused by error (error variance) are random, whereas true differences are stable. Therefore, the same scale (that weighed the two-pound sugar bag at 1.8 pounds) would be expected to weigh a five-pound bag of sugar at 4.5 pounds. Reliability, however, is not sufficient in determining a good test. Validity of a test ensures that the test measures what it was designed to. Yet, this is dependent on the definition of what is being measured. The scale example above is a valid form of measurement because it measures weight; however, it did not measure the weight correctly. Therefore, this scale is not a valid measurement tool. In terms of measuring intelligence, for example, the numerous definitions of the term ‘intelligence’ make the validity of any test attempting to measure it controversial. This becomes even more complicated when tests are applied across cultures (we will touch on this again later in the discussion). Validity can be sub-categorized into three areas: content, criterion-related, and construct. Content validity describes how well a test includes the various behaviors that make up the general behavior to be tested. For example, a test measuring creativity should look at creative examples in visual art, writing, singing, acting, every day tasks, appearance, etc. Across populations, however, content validity makes assumptions about the validity of a test and its items. Current tests of intelligence may also minimize the importance of certain abilities, like creativity. Since test-items require correct or incorrect responses, they leave no room for the processes common to creativity, namely fluency, originality, flexibility, and elaboration. Cohen & Swerdlik (2005) explain that intelligence tests focus on convergent thinking (deductive reasoning) and not divergent thinking (non-linear, imaginative thinking). Yet, tests developed to tap creativity (e.g. Mednick’s Remote Associates Test- RAT) fared low on validity and reliability. Ironically, although intelligence tests do not measure creativity, creativity tests may measure aspects of intelligence. In other words, the identification of what abilities measure intelligence is a continual matter of debate. Criterion-related validity is the extent to which a test can provide results that can be inferred to others’ measure of this particular behavior or trait. Concurrent validity indicates whether scores estimate current standing on criterion. Predictive validity indicates how well the test can predict the criterion in the future. Expectancy data is used to evaluate the criterion-validity of a test. Face validity, on the other hand, relates to what a test appears to measure from the perspective of the test taker. Therefore, a lack of face validity can eliminate all trust in test results since no connection is seen between the test and what it is supposed to measure. Test results can also provide other information about a test-taker, other than scores. When tests are administered individually, the test-giver has the opportunity to observe the test-taker. Cohen & Swerdlik (2005) make reference to ‘extra-test behavior’- the reactions, frustrations, and processes that the examiner sees when observing an examinee- claiming it should be used to supplement formal scores. Similarly, Alan Kaufman (In Benson, 2003) defends the need to integrate new information in intelligence testing: “The movement thats trying to get rid of IQ tests is failing to understand that these tests are valid in the hands of a competent practitioner who can go beyond the numbers--or at least use the numbers to understand what makes the person tick, to integrate those test scores with the kind of child youre looking at, and to blend those behaviors with the scores to make useful recommendations". (Questioning the test) Moreover, a test reflects the environment (or society) where the test is employed and different cultures have varying definitions of ‘intelligence’, test-takers from different cultures may produce invalid results. Moreover, interpretations of responses will vary depending on culture. For example, Serpell (1979, In Cohen & Swerdlik) tested Zambian and English children on a task of reconstructing models. The English children did best using paper-and-pencil reconstructions because they were most familiar with these materials. On the other hand, the use of wire by Zambians reflected their familiarity with this material. Both groups created reconstructions made out of clay equally well. Since all tests inherently reflect the culture within which they were devised and will be applied, it becomes impossible to create an intelligence test devoid of culture. This exacerbates the need for Guidelines, like those set forth by the International Test Commission (ITC) and the Psychological Testing Centre. The environmental influence on intelligence tests further includes the interaction between the test administrator and the examinee. The usefulness of results and the ability to correctly interpret test results depends on the test administrator’s reliance on the instructions provided by the test developer. Standardized instructions help ensure that all results are reliable. Standardized scoring procedures ensure accurate scoring and reporting of results. The best way to improve test standardization is to familiarize practitioners with the Standards set out by The British Psychological Society (2008) (or your local Psychological Society), and to ensure practitioners are properly trained to administer each test. The Society further publishes general guidelines for the test-takers, yet these are not always made available to them. Moreover, most Standards list the rights and responsibilities of the test providers rather than the test takers, and so the imbalance of power between the researcher and participant remains. Tindall (1994) purports that “In the final analysis it is the researcher’s version of reality that is given public visibility. It is not possible to achieve complete mutuality and equality” (p.155). The title of Anderson et al.’s (2001) article says it all: there exists a growing divide between practitioners and researchers in Industrial, Work, and Organizational Psychology; between the practical relevance of psychological experience regardless of theory, and entirely methodological theory-based Psychology. Therefore, organizations must be responsible for moving content into context: to move psychometric testing from a theoretical framework to a practical application. An educator, Rinaldo (2005), wrote that: Research provides us with a variety of perspectives from which to see things and a way of empirically analyzing our findings. It enables us to see from where we have come, provides us with a description of where we are now, and gives us some semblance of the direction in which we seem headed. (p.74) Yet, Rinaldo goes on to explain that real-life objectivity is itself theoretical, since it relies on individuals’ subjective judgments and decisions. For example, employee well-being and organizational effectiveness are socially constructed ideas that differ across organizations. To exemplify, she mentions the often-heard dictum “numbers don’t lie”. Yet, she says that “people do. And it’s the people that decide what numbers are used, and what methods of calculation are used to arrive at desired results” (p.75). In sum, employers need self-awareness and knowledge to be best able to integrate theory into practice. Today’s organizations are learning environments: People need to be able to solve problems, and understand the methods they use to do so (Clark, 2004). For example, meta-cognition is an important step in applying a theory, allowing learners to reflect on the learning process (theoretical proponents) from a personal point of view (one’s own learning capabilities). Therefore, effective leaders must realize their organizations consist of learners of past theories (including themselves), whom must see the organization as a place that provides the opportunity to learn further, and to lead. REFERENCES Anderson, N., Herriot, P., & Hodgkinson, G.P. (2001, November). The practitioner-researcher divide in Industrial, Work & Organizational (IWO) psychology: Where are we now, and where do we go from here? Journal of Occupational and Organizational Psychology, (74)4, 391-411. The British Psychological Society (2008, Feb 13). Code of Good Practice in Psychological Testing. [Electronic Version]. Retrieved November 4, 2008, from http://www.psychtesting.org.uk/downloadfile.cfm?file_uuid=129F8AF2-1143-DFD0-7EB7-C9C02C6C811E&ext=pdf Benson, E. (2003, Feb.). Intelligent intelligence testing: Psychologists are broadening the concept of intelligence and how to test it. [Electronic version]. APAOnline, 34, 2. Retrieved November 1st, 2008, from http://www.apa.org/monitor/feb03/intelligent.html Clark, D. R. (2004). Developing Instruction or Instructional Design. Retrieved November 4, 2008, from http://www.nwlink.com/~donclark/hrd/learning/development.html Cohen, J.C. & Swerdlik, M.E. (2005). Psychological Testing and Assessment: An introduction to Tests and Measurement (6th Ed.). New York: McGraw Hill. Rinaldo, V. (2005). Today’s practitioner is both qualitative and quantitative researcher. The High School Journal, 89 (1), 72-79. Read More