The Association Between Quizzes and Psychological Variables - Lab Report Example

? Item Analysis: Estimating the association between two quizzes and two psychological variables Results The researcher wished to test the hypotheses that student performance on the two quizzes administered during the semester was associated. Additionally, the researcher was also interested in testing the association between two psychological attributes, emotional intelligence and happiness. Finally, the researcher wished to determine whether emotional intelligence and happiness might be correlated with quiz II performance. Before testing these hypotheses on the data gathered, basic psychometric analyses were performed on the quiz items for both quizzes as well as for the items on the happiness and emotional intelligence scales. Quiz I: Psychometric Analysis The 12 item quiz described as Quiz I was associated with a mean of 6.4 and an SD of 2.657 (min = 0, max = 12). The distribution of the total scores was not very normal with skew observed to be 0.111 and kurtosis to be -0.888. The Pearson correlations between the 12 items in the quiz are reported in table 1. This shows that while the test has otherwise acceptable psychometric properties, some items may need to be revised or removed. An item discrimination analysis was conducted to investigate the contribution of each item to the test. Specifically, D index values were estimated for each of the 12 items in the quiz based on splitting the sample into high (73rd percentile and above) and low (27th percentile and below) quiz performers in accordance with Kelly (1939). Based on a frequency table analysis of the ‘quiz I performance’ variable, the split corresponded to scores of 4 or lower being associated with the lower set while scores of 9 or higher were associated with the higher set. Table 2 shows discriminatory index for each of the items, and it is evident that items 11 and 12 did not achieve a D index of more than 29%. This could mean that these items are poorly constructed or that they do not contribute to the test as well as the other items (Crocker & Algnia, 1986). The item total correlations also verify that these items contribute poorly to the quiz and have correlation coefficient s of less than 0.20. The internal consistency analysis conducted using the Cronbach’s ? was also marginally below the requisite 0.70 level (Nunnally & Brenstien, 1994) with a statistic value of 0.683. Given this data, it was believed that the quiz needed to be revised. Thus, items 12 and 11 were sequentially removed from the quiz and the Cronbach’s ? was re-estimated. The quiz was now composed of 10 items and was associated with an acceptable ? level of 0.734. The revised scale had a mean score of 5.128 with SD = 2.56 (min = 0, max = 10).the distribution of scores for the new quiz was also slightly more normalized with skew = 0.20 and kurtosis = -0.780. Quiz II: Psychometric analysis The 11 item quiz described as Quiz II was associated with a mean of 5.48 and an SD of 2.67 (min = 0, max = 11). The distribution of the total scores was approximately normal with skew observed to be 0.043 and kurtosis to be -0.8. The Pearson correlations between the 11 items in the quiz are reported in table 3. This shows that the test has acceptable psychometric properties. An item discrimination analysis was conducted to investigate the contribution of each item to the test. Specifically, D index values were estimated for each of the 11 items in the quiz based on splitting the sample into high (73rd percentile and above) and low (27th percentile and below) quiz performers in accordance with Kelly (1939). Based on a frequency table analysis of the quiz performance variable, the split corresponded to scores of 4 or lower being associated with the lower set while scores of 7 or higher were associated with the higher set. Table 4 shows discriminatory index for each of the items, all of which managed to achieve an acceptable D index (Crocker & Algnia, 1986). The item total correlations also verify that all items did contribute adequately to the quiz and have correlation coefficients of more than 0.20. The internal consistency analysis conducted using the Cronbach’s ? was also above the requisite 0.70 level (Nunnally & Brenstien, 1994) at 0.717. As there was no indication that deletion of modification of items would be required, the test was left intact. Emotional Intelligence scale: Psychometric analysis The 10 item scale described as the Emotional intelligence scale was constructed such that each item was assigned a score between 1 and 5, with lower scores depicting strong disagreement and higher scores associated with strong agreement on the items. This scale was associated with a mean of 36.65 and an SD of 5.18 (min = 21, max = 49). The distribution of the total scores was approximately normal with skew observed to be -0.1 and kurtosis to be -0.134. The Pearson correlations between the 12 items in the quiz are reported in table 5. This shows that the scale has otherwise acceptable psychometric properties. Since the scale was not binary in nature, discriminatory analysis was not considered appropriate for item analysis, and Item total correlations were use to verify that all items correlated adequately with the total. Each correlation was above 0.20, and thus was considered acceptable (shown in table 6). The internal consistency analysis conducted using the Cronbach’s ? was also adequate, yielding a statistic value of above 0.70 level (Nunnally & Brenstien, 1994). The ? value found was 0.78, which was considered as very acceptable. As there was no indication that deletion of modification of items would be required, the scale was left intact. Happiness scale: Psychometric analysis The 8 item scale described as the Happiness scale was constructed such that each item was assigned a score between 1 and 6, with lower scores depicting strong disagreement and higher scores associated with strong agreement on the items. This scale was associated with a mean of 34.48 and an SD of 6.73 (min = 19, max = 47). The distribution of the total scores was acceptably close to normal with skew observed to be -0.306 and kurtosis to be -0.655. The Pearson correlations between the 12 items in the quiz are reported in table 7. This shows that the scale has otherwise acceptable psychometric properties. Since the scale was not binary in nature, discriminatory analysis was not considered appropriate for item analysis, and Item total correlations were use to verify that all items correlated adequately with the total. Each correlation was above 0.20, and thus was considered acceptable (reported in Table 8). The internal consistency analysis conducted using the Cronbach’s ? was also adequate, yielding a statistic value of above 0.70 level (Nunnally & Brenstien, 1994). The ? value found was 0.783. As there was no indication that deletion of modification of items would be required, the scale was left intact. Correlation analysis In order to estimate the association between the different variables, Pearson product moment correlations were computed among the variables. The observed correlation between the two quizzes was r = 0.717, p < 0.01. This is a large effect when compared with Cohen’s (1992) guidelines. However, since scores on both quizzes were not associated with perfect reliability, the observed correlation was disattenuated based on disattenuation formula described by Furr and Bacharach (2008). The reliability estimates in question were ? = 0.734 for Quiz I and 0.717 for Quiz II. The corrected correlation was estimated at r = 0.988, demonstrating that the two quizzes shared about 98% of score variance. The observed correlation between the two scales was r = 0.421, p < 0.01. This is a relatively large effect when compared with Cohen’s (1992) guidelines. However, since scores on both quizzes were not associated with perfect reliability, the observed correlation was disattenuated based on disattenuation formula (Furr & Bacharach, 2008). The reliability estimates in question were ? = 0.78 for Emotional intelligence scale and 0.783 for Happiness scale. the corrected correlation was estimated at r = 0.538, demonstrating that the two quizzes shared about 28.95% of score variance. The observed correlation between Quiz II and the Happiness scale was r = 0.6.3, p < 0.01. this is a large effect when compared with Cohen’s (1992) guidelines. However, since scores on both quizzes were not associated with perfect reliability, the observed correlation was disattenuated based on disattenuation formula described by Furr and Bacharach (2008). The reliability estimates in question were ? = 0.717 for Quiz II and 0.783 for Happiness Scale. The corrected correlation was estimated at r = 0.805, demonstrating that the two quizzes shared about 64.8% of score variance. The observed correlation between Quiz II and the Emotional intelligence scale was r = 0.422, p < 0.01. This is a relatively large effect when compared with Cohen’s (1992) guidelines. However, since scores on both quizzes were not associated with perfect reliability, the observed correlation was disattenuated based on disattenuation formula described by Furr and Bacharach (2008). The reliability estimates in question were ? = 0.717 for Quiz II and 0.78 for the Emotional Intelligence scale. The corrected correlation was estimated at r = 0.564, demonstrating that the two quizzes shared about 31.8% of score variance. Thus, it may be said that all the associations expected by the researcher were found to be verified by the data. References Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155-159. Furr, R.M. & Bacharach, V.R. (2008). Psychometrics: An introduction. Thousand Oaks, CA: Sage. Kelly, T.L. (1939). Selection of upper and lower groups for the validation of test items. Journal of educational psychology, 30, 17-24. Nunnally, J.C., & Bernstein, I.H. (1994). Psychometric theory (3rd ed.). New York, NY: McGraw-Hill. Table 1: Correlations for Quiz I quiz1.1 quiz1.2 quiz1.3 quiz1.4 quiz1.5 quiz1.6 quiz1.7 quiz1.8 quiz1.9 quiz1.10 quiz1.11 quiz1.12 quiz1.1 1 quiz1.2 .135 1 quiz1.3 .267** .239** 1 quiz1.4 .161* .216** .235** 1 quiz1.5 .304** .163* .299** .277** 1 quiz1.6 .194** .116 .153* .176* .298** 1 quiz1.7 .238** .142 .333** .289** .326** .445** 1 quiz1.8 .278** .151* .249** .182* .285** .270** .307** 1 quiz1.9 .177* .256** .155* .155* .272** .270** .374** .123 1 quiz1.10 .242** .022 .104 .127 .156* .164* .096 .182* .146* 1 quiz1.11 .016 -.009 -.091 .039 -.043 .053 -.033 .075 -.068 -.012 1 quiz1.12 .122 -.016 .016 .016 -.075 .065 .040 .057 -.023 -.002 .070 1 Table 2: Descriptive statistics for Quiz I Descriptives Item Discrimination Item-total correlation Mean SD Variance High Low D quiz1.1 .21 .411 .169 50.8 0 50.8 0.537 quiz1.2 .68 .468 .219 90.8 46.2 44.6 0.427 quiz1.3 .57 .496 .246 87.7 25 62.7 0.533 quiz1.4 .57 .496 .246 86.2 23.1 63.1 0.521 quiz1.5 .53 .500 .250 86.2 19.2 67 0.588 quiz1.6 .65 .479 .230 93.8 26.9 66.9 0.571 quiz1.7 .40 .491 .242 72.3 0 72.3 0.636 quiz1.8 .39 .489 .239 72.3 7.7 64.6 0.567 quiz1.9 .79 .411 .169 98.5 50 48.5 0.486 quiz1.10 .33 .472 .223 53.8 9.6 44.2 0.392 quiz1.11 .46 .500 .250 63.1 42.3 20.8 0.189 quiz1.12 .81 .395 .156 87.7 69.2 18.5 0.196 Table 3: correlations for Quiz II quiz2.1 quiz2.2 quiz2.3 quiz2.4 quiz2.5 quiz2.6 quiz2.7 quiz2.8 quiz2.9 quiz2.10 quiz2.11 quiz2.1 1 quiz2.2 .400** 1 quiz2.3 .256** .271** 1 quiz2.4 .189** .212** .100 1 quiz2.5 .262** .262** .298** .019 1 quiz2.6 .103 .190** .197** .314** .025 1 quiz2.7 .252** .275** .278** .245** .143 .303** 1 quiz2.8 .012 .054 .098 .358** .037 .283** .199** 1 quiz2.9 .207** .413** .307** .078 .237** .046 .211** .133 1 quiz2.10 .158* .210** .073 .257** .154* .206** .131 .238** .239** 1 quiz2.11 .175* .144* .055 .172* .178* .204** .110 .027 .192** .138 1 Table 4: Descriptive statistics for Quiz II Descriptives Item Discrimination Item-total correlation Mean SD Variance High Low D quiz2.1 .62 .487 .237 87.1 21.3 65.8 0.538 quiz2.2 .39 .489 .239 78.6 6.4 72.2 0.612 quiz2.3 .76 .429 .184 97.1 44.7 52.4 0.510 quiz2.4 .44 .498 .248 74.7 12.8 61.9 0.533 quiz2.5 .65 .477 .228 85.7 29.8 55.9 0.460 quiz2.6 .42 .494 .244 72.9 14.9 58 0.518 quiz2.7 .51 .501 .251 80 12.8 67.2 0.569 quiz2.8 .49 .501 .251 70 19.1 50.9 0.442 quiz2.9 .32 .468 .219 58.6 2.1 56.5 0.537 quiz2.10 .21 .407 .166 42.9 2.1 40.8 0.480 quiz2.11 .67 .472 .223 85.7 38.3 47.4 0.425 Table 5: Correlations for Emotional intelligence scale ei1 ei2 ei3 ei4 ei5 ei6 ei7 ei8 ei9 ei10 ei1 1 ei2 .080 1 ei3 .117 .363** 1 ei4 .386** .134 .144* 1 ei5 .142 .137 .049 .330** 1 ei6 .252** .103 .263** .365** .320** 1 ei7 .436** -.047 -.014 .449** .158* .190** 1 ei8 .178* .088 .103 .323** .368** .407** .308** 1 ei9 .538** .168* .224** .384** .265** .224** .385** .243** 1 ei10 .499** .110 .100 .483** .319** .306** .504** .346** .478** 1 Table 6: Descriptive statistics for Emotional Intelligence scale Descriptives Item-total correlation Mean SD Variance ei1 3.60 .970 .940 0.654 ei2 3.71 .923 .852 0.368 ei3 3.82 .822 .676 0.395 ei4 3.59 .827 .683 0.683 ei5 4.11 .642 .412 0.492 ei6 4.19 .759 .576 0.563 ei7 3.18 .919 .845 0.600 ei8 3.92 .903 .816 0.596 ei9 3.20 1.102 1.213 0.708 ei10 3.33 .999 .997 0.734 Table 7: Correlations for Happiness scale happy1 happy2 happy3 happy4 happy5 happy6 happy7 happy8 happy1 1 happy2 .430** 1 happy3 .579** .688** 1 happy4 .571** .253** .380** 1 happy5 .120 .094 .062 .086 1 happy6 .384** .336** .422** .299** .059 1 happy7 .382** .369** .426** .149* .157* .425** 1 happy8 .352** .307** .275** .299** .015 .172* .317** 1 Table 8: Descriptive statistics for Happiness scale Descriptives Item-total correlation Mean SD Variance happy1 4.18 1.421 2.021 0.776 happy2 4.76 1.146 1.313 0.685 happy3 4.15 1.383 1.913 0.773 happy4 3.78 1.340 1.796 0.618 happy5 5.21 .819 .671 0.241 happy6 3.64 1.547 2.392 0.645 happy7 4.28 1.332 1.774 0.648 happy8 4.48 1.536 2.358 0.575 Individual Questions: 1: What does the disattenuated correlation between the two quizzes mean? (2 sentences maximum) The disattenuated correlation between the quizzes demonstrates the percentage of shared variance (in this case about 98%) in the scores on these two quizzes without the effect of error. Measurement error can affect the observed correlation between the variables, and this is a method that reduces the effect that this error has on the measurement of correlation. 2: Using SPSS, calculate T-score variables for each of the following variables: revised quiz one, quiz two, emotional intelligence composite scores, happiness composite scores. 2.1: What T-score does a raw score of 5 correspond to on the revised quiz 1 composite variable? A T score of 49.5 corresponds to a raw score of 5 on revised Quiz I. 2.2: What T-score does a raw score of 4 correspond to on the quiz 2 composite variable? A T score of 44.46 corresponds to a raw score of 4 on Quiz II. 2.4: What T-score does a raw score of 37 correspond to on the emotional intelligence composite variable? A T score of 50.67 corresponds to a raw score of 37 on the Emotional Intelligence scale. 2.5: What T-score does a raw score of 39 correspond to on the happiness composite variable? A T score of 56.72 corresponds to a raw score of 39 on the Happiness scale. 3: Calculate the 95% true score confidence interval associated with a quiz two point estimate of 5.48 (include the calculations associated with your work). Based on the data available, the confidence interval would be calculated using the formula X ± ( Z95%) (sem) Thus, 5.48 +/- 1.96 * 0.195 5.48 +/- 0.382 Thus, the confidence interval ranges from 5.862 to 5.098. 4: Suppose the honours selection panel in a school of psychology decided to select students into fourth year based on their performance across all four composite variables: revised quiz 1, quiz 2, emotional intelligence, happiness. One of the panel members suggests that they should calculate an average T-score across the four variables for the purposes of selection. Would the creation of a composite score out of the four T-scores be defensible psychometrically? Why or why not? Use a maximum of 60 words in your answer. Creating a composite T score is useful when all the items represent variation associated with a single global variable. But in this case, while it could be acceptable to combine the scores for the two quizzes, it would not be appropriate to combine scores for all variables. The scales and the quizzes do seem to measure somewhat different variables. 5: The researcher administered the two questionnaires designed to measure emotional intelligence and happiness using a website. Thus, each student completed the two questionnaires individually on their own time. It is very conceivable that some of the students completed the questionnaires in a totally inattentive manner, i.e., provided responses to questions they have not even read. Based on the emotional intelligence and happiness data that are available, how might the researcher detect those students who likely responded to items without reading them? Use a maximum of 50 words to describe your proposed strategy. It would be possible to detect such cases by testing the correlations between scores on Happiness and Emotional Intelligence for individuals and by testing individual scores on one variable with mean score on the other vaariable. Those who have randomly marked responses are likely to show a non significant, perfect or negative correlation between the two scales. Read More