Problems on Validity and Reliability: Measuring Intelligence in terms of Grade Point Average - Assignment Example

Problems on Validity and Reliability: Measuring Intelligence in terms of Grade Point Average (GPA) The use of grade point average (GPA) has been widely used as a predictor and a measure of intelligence. Over the years, a number of reviews and literatures have relied on the qualitative analysis to determine the effectiveness and usefulness of grade point average in predicting human intelligence, however, much of what it accounts has been debated (Bretz: questia.com). To which extent does the grade point average measure the intelligence and to which extent it is reliable and valid? All tests that results to computation of grade point average have one characteristic in common that makes them both fascinating and remarkably practical: they promise to make it possible to find out a great deal about a person in a very short time (Gormly, 1996). Over the last several decades many researchers have tested the relationship between college grade point average and individual intelligence that qualitative reviews of the published literature more often leads to confusion in stating the true effect of GPA as a predictor of later success (Bretz: questia.com). It has been stipulated that learning in general would be measured by college grade point average (Hunter and Schmidt, 2004) such that grade point average or rank in class tends to be a more accurate predictor than test scores (Manning and Willingham, pp 125). The most common criterion of grade point average includes job success in business, engineering, teaching and medicine and less common includes social activity, and cultural participation. But then again, to what extent does grade point average measure individual intelligence, specifically in terms of its reliability and validity? This paper will then examine the reliability and validity of grade point average as a measure of intelligence as well as find out some problems that these factors may contain in generating the data needed. Problems of Reliability In order to get the grade point average, one has to undergo a series of tests that will establish the final grade evaluation. The reliability of the grade point average is dependent on the reliability of the test being issued to measure the claim. If the test is reliable, then does it also contend the reliability of the grade point average as well? The term reliability refers to the tests’ consistency- its ability to yield the same result under a variety of different circumstances and the extent to which scores on a test are relatively free from errors of measurement (Gormly, 1996: 292). There are three basic ways that determines a certain test’s reliability. First is the test’s stability or the test-re test reliability—if a student retakes a test within a short period of time the first testing, does he or she receive approximately the same score as the first one? For example, a certain student takes an aptitude test three times in the span of six months and scored 65, 90, and 70, in the first, second, and third times respectively, then the test in unreliable since it does not generate a measurement that is stable over time (Aiken, 1996). The second measure of the test’s reliability is when the test yields the same results when scored by different people. For example, when two test administrators give different scores on the same test taken by a student, then there is a good chance that the test is unreliable because in this case, the score received by the student depended more on the grader than the test-taker. In a reliable test, the score would still be the same no matter who graded the paper, given it is the same test and the student provided the same answer (Gormly, 1996: 292). And finally, one way to determine a test’s reliability is when the test is divided in half and a student scores each half separately and the two scores are approximately the same. If the test is supposed to measure one quality in a student (for example, mechanical and non verbal intelligence) then it should not have sections on which the student scores high and other sections which he will score low. But then again, in checking for test’s reliability, one must prevent chance factors from influencing the student’s score. There are a number of irrelevant matters that can interfere with the test that would generate the grade point average-- and these things question the reliability of the test issued. Problems of reliability in terms of test taking that produces the grade point average includes the psychological state of both test administrators (teachers, for example) and the students themselves the moment the test is administered and taken. If a student is having emotional difficulties at the time of the test or the entire duration of the term, then most often than not, he or she cannot do good in examinations that would result to a failing mark and a low grade point average- but this does not define his or her intelligence in any way (Bligh, Mcnay, Thomas: 1999). The teachers, on the other hand, are complex people and like everyone else have their own feelings, attitudes, and ideas of their own and their reactions to different students (e.g. short and tall students, students who speak with an accent, or students who remind them of someone they particularly like or dislike) may distort the data on their grading system that they have gathered. They may develop personal biases towards a particular student that may add or deduct certain points from the general point average, considering that part of the grade is the teacher’s impressions on the student’s behaviors (Gormly: 1996). Another challenge to the reliability of grade point average includes the standards of different schools and universities. For example, the grade point average of a student may be due to the fact that the schools that he or she went through have strict an up to the most grading standard that every graduate for that matter is expected to excel in all academic factors. Some of the institutions however, may devise measurements that are just appropriate for the skills taught that are too narrow in the first place. Considering the different educational standards of different universities and colleges, for example, it is highly possible that a student from one university may have higher grade point average compared to the student in another university even when the curriculum of both students are the same. For this reason, the reliability of the grade point average in the measure of individual intelligence may be compromised (Bligh, Mcnay, Thomas: 1999). The challenge on how the scores of the tests are interpreted as well as the method and considerations of the teachers in translating the students’ scores also prevails in getting the respective grade point average. Once the test results and all other factors that involves in the curriculum is obtained, the teachers must translate the scores and points into the final grade. Suppose a student accomplishes thirty two out of the fifty questions on a final exam, what would this mean? If the exam given is reliable and valid it could mean that the student understood a certain percentage of the contents and the subject matter being tested. In other words, the scores of the student in a certain examination predict how he or she understood and comprehends the subject matter (Gormly: 1996). In addition, when considering intelligence, the percentile on an aptitude test (such as Scholastic Achievement Tests for example) reflects the student’s standing among people of his or her age and year level who have taken the exam. Summarizing the above mentioned problems of reliability of grade point average in defining individual intelligence, these factors must be considered in evaluating the results; (a) If a similar test is taken within a short period of time, would a student receive similar score? (b) Is the performance of the student on the test given reflects his or her normal performance in the subject area that the test itself desires to measure? (c) If the scores on the tests administered will be compared with the scores of other students belonging in the same year level, class, or subject, would it reflect his or her general standing within that group of students? If one answers yes to the above questions, then most probably, the test administered in getting the student’s grade point average is reliable but considering the factors that may affect the performance, it does not directly follow that the scores define individual intelligence. Problems of Validity A test that measures individual intelligence may be reliable but still not be valid. Validity is the ability of a test to measure what it is supposed to measure--for example, a test that consists primarily of verbal ability will not measure aptitude for mechanical engineering and a mathematical test will not measure general learning ability for that matter (Babbie, 2007). Verifying the validity of a test as an academic requirement is more complex than assessing its reliability. One of the primary methods for measuring is to determine how well a particular test or appraisal predicts the student’s performance. For example, a group of researchers that would design a test that will measure the student’s non verbal intelligence will ask and formulate questions pertaining to logical, abstract, and numerical ability- same goes with the attempt to measure one’s verbal ability such that the test designed will ask questions about semantics, language coherence, and syntax (Lewis, 1996). So how would we know if the test is valid? Let’s cite a company that devises its own test for management ability for example. The test they formulated (along with their third party consultants or psychologists perhaps) will ask questions pertaining to management systems, attitudes towards employees and so on. What would be the assurance though that people who scored high on these tests will really make good managers? Suppose the test makers will check the validity of the test that they have formulated and find out how much improvement does an employee have in terms of their productivity in one year and they found out that those who scored high points produced more compared to those who scored low points, then the test may be considered valid. Then they can decide to use the test they have designed in the future if they have to hire additional employees (Gormly: 1996). But what if the employees who are good at raising productivity are poor at either leadership or decision making? It may be that this test measures talent for improving productivity, and not the leadership or management ability. This is one of the challenges in determining the validity of a test. As the above example shows, there is nothing that can be said about a test’s validity until the purpose of the test is clear cut. Along with these challenges are the problems that are also encountered in determining validity of a test especially if this test will attempt to define the intelligence of an individual. One of the main problems in validity is the difficulties in test designs. True to the premise that an assessment is valid only if it tests the abilities it intended to test but it only takes a few moments thought to realize that it is extremely difficult to design an assessment that will test the aims of higher education (Bligh, Mcnay, Thomas-A: 1999). How would the test makers devise the assessment such that critical thinking is defined within the boundary from the test of knowledge? In this connection, the examiners or test makers themselves are next in line for the problems associated with validity. The difficulties in devising an assessment (i.e. formulating assessment that will recognize problem solving skills from critical thinking) is said to be one of the major problems in validity of a test (Bligh, Mcnay, Thomas-B: 1999). Therefore it can be concluded that it is indeed difficult to design assessment that test the high level intellectual skills of the individual- the test maker has to work even higher level in order to measure the claim. Second, most examiners have no training in making these ruling, for example, research has shown that more marks for factual recall by administrators are evident like when the same examiner may give different marks to the same piece of work on different occasions when he or she doesn’t remember the mark he or she gave the first time. And lastly, there are no generally accepted examining techniques such that examiners will happily add marks together or calculate averages as if they were quantities not considering the fact that examination marks are not numbers, they are qualities that were based upon judgment of the quality of the students work (Bligh, Mcnay, Thomas-C: 1999). For example, a study conducted by the research in higher education supports the view that the predictive validity of pre-admissions measures such as standardized tests and high school grades may be understated because of the correctable defect in both the freshmen years and cumulative grade point average (Young: 1990). In conclusion, although the undergraduate grade point average has been the measure of choice in research on student achievement, the validity of the grade point average and self reported learning gains has been critically contended such that the validity of the grades and the grade point average typically do not take into account the students performance level at the beginning of the course; they are normative rather than criterion based indicators of academic performance much more than individual intelligence (Castellanos, and Jones: 2003). In other words, critics of these measures maintained that grades do not actually measure academic progress or intelligence, but these are mainly tools that determine the present performance of a particular student in a given period of time. References: 1. Gormly, Ann V. 1996. Understanding psychology. New York. Random House. 2. Hunter, John and Frank L. Schmidt. Meta analysis: correcting error and bias in research finding. 2004. SAGE Publications Ltd. Thousand Oaks California. 3. Boyle, Gregory John, Gerald Matthews, and Donald Saklofske. 2008. The SAGE handbook of personality theory and assessment: personality theories. Cromwell Press Ltd. Trowbridge Wiltshire. 4. Manning, Winton Howard and Warren W. Willingham. Selective admissions in higher education. 5. Castellanos, Jeannete and Lee Jones. 2003. The majority in the minority: expanding the representation of Latina faculty, administrators and students in higher educatio. Stylus Publishing LLC, Sterling Virginia. 6. Aiken, Lewis R. 1996. Assessment of intellectual functioning 2nd edition. Plenum Press NY: Spring St. NY. 7. Babbie, Earl R. 2007. The practice of social research 12th edition. Wadsworth Cengage Learning CA. 8. Bligh, Donald A., Ian Mcnay Harold R. Thomas. 1999. Understanding higher education. Intellect Books. Portland Oregon. 9. John W. Young. Are validity coefficients understated due to the correctable defects in the GPA? Research in Higher Education, Vol. 31, No. 4, 1990. 10. College grade point average as a predictor of adult success: a meta-analytic review and some additional evidence. [retrieved from questia.com]. March 28, 2010. Read More