Statistical Methods In Psychology - Essay Example

Article Review Synopsis Article: Statistical Methods in Psychology Journals: Guidelines and Explanations. Published in American Psychologist. Vol. 54, No. 8, 594–604 Written by Leland Wilkinson. The article takes the ongoing dispute over the applications of significance testing in psychology journals into consideration. The article was motivated greatly by the publication of Cohens (1994) article, the Board of Scientific Affairs (BSA) of the American Psychological Association (APA) convened a committee called the Task Force on Statistical Inference (TFSI) whose charge was "to elucidate some of the controversial issues surrounding applications of statistics including significance testing and its alternatives; alternative fundamental models and data transformation; and newer methods made possible by powerful computers" (BSA, personal communication, February 28, 1996). Robert Rosenthal, Robert Abelson, and Jacob Cohen (co chairs) met initially and agreed on the desirability of having several types of specialists on the task force: statisticians, teachers of statistics, journal editors, authors of statistics books, computer experts, and wise elders. Nine individuals were subsequently invited to join and all agreed. These were Leona Aiken, Mark Appelbaum, Gwyneth Boodoo, David A. Kenny, Helena Kraemer, Donald Rubin, Bruce Thompson, Howard Wainer, and Leland Wilkinson. In addition, Lee Cronbach, Paul Meehl, Frederick Mosteller and John Tukey served as Senior Advisors to the Task Analytical Report After the-publication of this articles the response to, the task force recommended several possibilities for further action, most importantly, of which would be to revise the statistical sections of the American Psychological Association Publication Manual (APA, 1994). After extensive discussion, the BSA recommended that "before the TFSI undertook a revision of the APA Publication Manual, it might want to consider publishing an article in American Psychologist, as a way to initiate discussion in the field about changes in current practices of data analysis and reporting" (BSA, personal communication, November 17, 1997). This report follows that request. The sections in italics are proposed guidelines that the TFSI recommends could be used for revising the APA publication manual or for developing other BSA supporting materials. Following each guideline are comments, explanations, or elaborations assembled by Leland Wilkinson for the task force and under its review. This report is concerned with the use of statistical methods only and is not meant as an assessment of research methods in general. Psychology is a broad science. Methods appropriate in one area may be inappropriate in another The title and format of this report are adapted from a similar article by Bailar and Mosteller (1988). That article should be consulted, because it overlaps somewhat with this one and discusses some issues relevant to research in psychology. Further detail can also be found in the publications on this topic by several committee members (Abelson, 1995, 1997; Rosenthal, 1994; Thompson, 1996; Wainer, in press; see also articles in Harlow, Mulaik, & Steiger, 1997). An analysis of the methods was conducted and revealed the following: There are many forms of empirical studies in psychology, including case reports, controlled experiments, quasi-experiments, statistical simulations, surveys, observational studies, and studies of studies (meta-analyses). Some are hypothesis generating: They explore data to form or sharpen hypotheses about a population for assessing future hypotheses. Some are hypothesis testing: They assess specific a priori hypotheses or estimate parameters by random sampling from that population. Some are meta-analytic: They assess specific a priori hypotheses or estimate parameters (or both) by synthesizing the results of available studies. Some researchers have the impression or have been taught to believe that some of these forms yield information that is more valuable or credible than others (see Cronbach, 1975, for a discussion). Occasionally proponents of some research methods disparage others. In fact, each form of research has its own strengths, weaknesses, and standards of practice. Population The interpretation of the results of any study depends on the characteristics of the population intended for analysis. Define the population (participants, stimuli, or studies) clearly. If control or comparison groups are part of the design, present how they are defined. Psychology students sometimes think that a statistical population is the human race or, at least, college sophomores. They also have some difficulty distinguishing a class of objects versus a statistical population—that sometimes we make inferences about a population through statistical methods, and other times we make inferences about a class through logical or other no statistical methods. Populations may be sets of potential observations on people, adjectives, or even research articles. How a population is defined in an article affects almost every conclusion in that article. Sample Interval estimates for clustered and stratified random samples differ from those for simple random samples. Statistical software is now becoming available for these purposes. If you are using a convenience sample (whose members are not selected at random), be sure to make that procedure clear to your readers. Using a convenience sample does not automatically disqualify a study from publication, but it harms your objectivity to try to conceal this by implying that you used a random sample. Sometimes the case for the representative ness of a convenience sample can be strengthened by explicit comparison of sample characteristics Random assignment is included in the article and must also be analyzed. For research involving causal inferences, the assignment of units to levels of the causal variable is critical. Random assignment (not to be confused with random selection) allows for the strongest possible causal inferences free of extraneous assumptions. If random assignment is planned, provide enough information to show that the process for making the actual assignments is random. There is a strong research tradition and many exemplars for random assignment in various fields of psychology. Even those who have elucidated quasi-experimental designs in psychological research (e.g., Cook & Campbell, 1979) have repeatedly emphasized the superiority of random assignment as a method for controlling bias and lurking variables. "Random" does not mean "haphazard." Randomization is a fragile condition, easily corrupted deliberately, as we see when a skilled magician flips a fair coin repeatedly to heads, or innocently, as we saw when the drum was not turned sufficiently to randomize the picks in the Vietnam draft lottery. As psychologists, we also know that human participants are incapable of producing a random process (digits, spatial arrangements, etc.) or of recognizing one. It is best not to trust the random behavior of a physical device unless you are an expert in these matters. It is safer to use the pseudorandom sequence from a well-designed computer generator or from published tables of random numbers. The added benefit of such a procedure is that you can supply a random number seed or starting number in a table that other researchers can use to check your methods later. Speculation may be appropriate, but use it sparingly and explicitly. Note the shortcomings of your study. Remember, however, that acknowledging limitations is for the purpose of qualifying results and avoiding pitfalls in future research. Confession should not have the goal of disarming criticism. Recommendations for future research should be thoughtful and grounded in present and previous findings. Gratuitous suggestions ("further research needs to be done …") waste space. Do not interpret a single studys results as having importance independent of the effects reported elsewhere in the relevant literature. The thinking presented in a single study may turn the movement of the literature, but the results in a single study are important primarily as one contribution to a mosaic of study effects. Some had hoped that this task force would vote to recommend an outright ban on the use of significance tests in psychology journals. Although this might eliminate some abuses, the committee thought that there were enough counterexamples (e.g., Abelson, 1997) to justify forbearance. Furthermore, the committee believed that the problems raised in its charge went beyond the simple question of whether to ban significance tests. The task force hopes instead that this report will induce editors, reviewers, and authors to recognize practices that institutionalize the thoughtless application of statistical methods. Distinguishing statistical significance from theoretical significance (Kirk, 1996) will help the entire research community publish more substantial results. Encouraging good design and logic will help improve the quality of conclusions. And promoting modern statistical graphics will improve the assessment of assumptions and the display of results. References Wilkinson, Leland. August 1999. Statistical Methods in Psychology Journals: Guidelines and Explanations. American Psychologist. Vol. 54, No. 8, 594–604 Read More