Threats to Internal and External Validity in Experimental Research - Report Example

Origins of Internal and External Validity: Threats and How They Can Be Controlled Introduction Experimental is a situation whereby a researcher manipulates one variable and then randomizes the remaining variables. It consists of a control group, in which the subjects under study are randomly given between the sample groups. In such a case, only experiments on a single effect are conducted at a given time (Shadish,Cook and Campbell, 2002). The distinction existing between external validity and internal validity is famously known as fundamental. Internal validity is a simple requirement necessary for the interpretation of any experiment of any nature. On the other hand, external validity seeks to answer the question of generalization: to what treatments, populations, or setting variables can this outcome be regarded on the general basis. Ideally, the external and internal validity are crucial with respect to experiments. Their main undoing in most cases is that they are always opposing each other when it comes to research design and planning (Taylor et al., 2003). If a research is to obtain internal validity, control the entire variables is needed for one to do away with all threats that can interfere with the results of the observed experiment. When researchers try to do this, they put the external validity of the outcomes at risks. In research studies that have very strong internal validity, response to queries such as to whom, where, or what the outcome are usually generalized and in most cases are not always very clear. In cases of settings that are valid ecologically, not every aspect of the experiment is operated and controlled or rather checked just as it is done in the laboratory set up. It will be unrealistic and unreasonable for one to expect a single experiment to fulfill all the considerations of the entire research design. The most prudent way in such a study will be for the researcher to identify the specific targets and limitations with respect to the study. Choice of either internal or external validity should be ascertained in the early stages of the research study. After the decision is made, the researcher is free to plan the study with only one kind of validity which will be considered as the main target. Threats to Internal Validity External validity refers to the degree of which the results obtained in a study are considered to be a function of a given variable which can be systematically measured, observed, and manipulated by the researcher. In regard to the work of Campbell and Stanley (1963), a total of eight threats were established with respect to internal validity. Later on in his studies, Rosenthal (2002) documented another threat to internal validity bringing the number to nine. The threats posed to internal validity are testing, statistical regression, history, maturation, instrumentation, selection-maturation interaction, selection bias, experimental mortality, and expectancy. History History as a threat to internal validity implies that certain events that are not wanted during the period of treatments occur. For instance, in a research aimed at ascertaining the result of physical education in a semester with respect to physical fitness of fifth graders, the presence of 60% of the kids who took part in the exercise but did not participate in soccer activities will form part of the history threat in relation to internal validity. This is because the football event acted as a booster to physical fitness and in most cases it is not easy to distinguish from the rewards with regard to an education program that is physical in nature. Testing The testing threat is a state where the outcome of conducting an experiment once impacts the results recorded in subsequent trials of the same experiment. For example if a group of students is given multiple tests to evaluate their knowledge on a particular subject today and later on after three or four days, the students will perform better on the second test. Doing the test or experiment once aids in carrying it out once more on another occasion with higher chances of better performance. Statistical Regression Statistical regression poses a threat to the internal validity of the groups chosen on grounds of extreme scorers. In most cases it is not severe in the subsequent experiments. For example, if you consider a group of students on their activity basis (those who are more active to the ones who are less active) and then you establish two sets of students on the basis of less active and very active students, statistical regression is bound to happen when the next experiment is done. The ones who were less active will be more positive, and the ones who were more active will appear to be less active. To be specific, both sets of students will shift from extreme scores to an overall mean. It thus implies that scorers that are extreme may reflect an observation of a given performance as a high performance on extreme scores. But when the average scores of various samples under study are compared after some time, the group that looked like the leading performer does not maintain their best performance. In the same spirit, the group that seemed to record weak performance is the one that will post the best results or good performance (Onghen, and Edginten, 2005). Maturation In most cases, maturation is linked to aging. Maturation often happens in a fashion whereby one set of participants’ undergo numerous tests for an extended period. It is the biological and psychological conditional changes that happen as time elapses. For example testing a students’ retention ability of what they were taught after some time say six months and after one year. What is discovered is that after some time, the retention disappears. Instrumentation Instrumentation is a problem often experienced when conducting scientific research. For example, if one uses spring equipment that is loaded to certain strength, the springs tension eventually decreases unless it is calibrated on a regular basis. The amount of force applied will give an increased reading in terms of strength when compared to the initial results. The threat does also occur in research that employs observations techniques. It calls for regular training of the observers and proper checking to avoid the possibility of rating as seen by the observer to vary across the participants. The Selection-Maturation Interaction This happens precisely on types of styles or designs. In these styles, one sample is chosen due to certain traits and on the ground that the remaining sample does not meet the desired features to be selected as a sample for the study. The selection of the sample group for different tests or experiments is usually done on the basis of varying criteria such as the one quoted above. In such cases serious threats can be posed. Selection Bias This threat occurs in cases where groups or sets of the samples are established on certain grounds instead of using the random strategy. Therefore, when experiments are conducted, rival hypothesis ensue as a result of any differences existing which is as a consequence of the first bias in the selection process and not the experiment. Experimental Mortality This threat implies that there is the loss of participants. Regardless of a random selection of groups of participants, this threat is always seen as normality to happen. The reason is because in some cases the members may leave the group or walk out of the group to be tested. Some of the reason as to why this happens can be attributed to the perceived time consumption and also the technical nature of the experiment. Expectancy The expectancy threat was not identified by Campbell and Stanley (1963) but was established by Rosenthal (2002). It implies that tests or experiments are carried out with the expectation that the participants will post better performance. This is unconscious and in most cases occurs where members are vividly indicated or labeled. The expectancy impact also has influence on the members at times. For instance, in a study involving sports among the youths, trainers or coaches, poorer performance may be as a result of the coachs poor substitution. In some cases where the coach is not liked by the substitutes due to adverse treatment, the substitutes may decide to perform poorly to discipline the coach whom they perceive to be mistreating them. If the managers of the football clubs treat all his players equally, then each player will give their best. Therefore, a researcher needs to address every sample group in a fair way if the desired results are to be obtained. Threats to External Validity The external validity implies generalization across setting, time and individuals on the extent of the outcome of a sample under investigation. The external validity is usually dived into two; population validity and ecological validity. According to the works of Campbell and Stanley (1963), four threats to external validity were established with respect to physical research methods. These are reactive or the effects of interactive setting, multiple interference treatments, the reactive effect of experimental arrangements, and the interaction of the selection bias and the experimental treatment. Interaction of the Bias and the Experimental Bias The threat can limit the outcome of the external validity of the participants who do not have given traits with respect to the sample under study. For example, an education program about drugs can be relevant in changing the attitudes of a first-year college student with respect to the use of drugs. However, this very program may not be of interest to a third-year student who is studying a course in medicine since s/he is aware of drugs usage and their impacts. A researcher thus needs to know that one test may be satisfactory to one group of participants and yet fail to impress another group of participants. Therefore, the researcher has to know the best design to employ to each participant to obtain the best-desired outcome (Lerner, 2003). Reactive or Interactive Effects of Testing The interactive or reactive outcome of results can be severe to any strategy with an experiment. For instance in the event that the physical fitness analysis is done to the participants first, the sample used in the group to be tested may be aware that their fitness levels are relatively small and in specific have the desire to follow the program that is prescribed jointly. In the population that is not experimented, nevertheless, the activity may fail to be significant since the sample may be unaware of their low levels of physical fitness (Brewer, 2000). Multiple-Treatment Interference This threat to external validity is most often seen as a problem in situations which the sample is subjected to numerous experiments or tests. In this risk with respect to external validity, the ability of the researcher to generalise outcomes may be affected by the utilisation of the multiple treatments. Reactive Effects of Experimental Arrangements This threat implies that empirical tests may fail to meet the generalisation standards in a reality check. These outcomes are problems that are persistent for laboratory-based research. In such a study the researcher examines a process, an effect, or a result that is in particular meant for laboratory and thus cannot be generalized (Brown, 1954). In a case of this nature then external validity may fail to live up to its expectation because it fails to meet the generalization standards. Controlling Threats to Internal and External Validity External and internal validity threats can be monitored in varying ways by employing given strategies. In this area, important strategies are used in solving threats to internal validity in experimental design. Numerous threats with respect to internal validity are checked by making the samples in the tests be alike. This goal is in most cases achieved by employing a random selection of the groups to participate in the research study. Controlling Internal Validity Threats Some of the ways in which researchers should employ in controlling the internal validity threat include ensuring that all conditions are standardized. The particular ways in which every internal validity threat can be controlled are discussed. History The researcher can control the history threat by the use of control groups derived from the same population as the experimental or the sample group. The researcher should also encourage short durations of time when conducting the experiment. The researcher can also opt for more participants to facilitate the gathering of more information. If the researcher considers the aforementioned, then the history threat will be controlled. Testing The application of research strategy that does not incorporate pre-testing by the researcher can also play a significant role in the elimination of a testing threat. Researchers should be encouraged to apply research designs that do not always involve pretest requirements. The aforementioned factors will play a great role in ensuring that the researcher entirely controls the testing threat. Instrumentation Instrumentation threats can be controlled by the use of control groups that are obtained or selected from the same population under study. The use of shorter timespans in experiments should also be encouraged by the researcher as it is effective in the reduction of instrumentation threat. Careful specifications and the control of procedures that are used in the measurements can also eliminate the majority of instrumentation threat. Statistical Regression Statistical regression threat can be controlled by avoiding the use of the extreme scorers. The exclusion of the average scores will reduce the risk of statistical regression. Researchers need to avoid the extreme scorers at all costs if they are to control this threat in a better way. If this is not conducted with utmost care, then the risk cannot be reduced since the extreme scores are bound to change with time if a second test is done. Mortality The death threat can be reduced by selecting a big group and making sure that all of them are representing the population in which they were picked from. The large group of participants will help the researcher to obtain more information needed for the study. The researcher, therefore, has to ensure that all the participants under study are tested to avoid the differential loss. A loss of such nature occurs because some participants were not tested. When testing is done the mortality threat is reduced. Selection The selection threat can be controlled by the use of random assignments and random choice of the subjects of study. The researcher should also avoid using the same sample group for more than one test. For instance, one group may be tested first, and then the other group is tested later. What is more, the researcher should at all times avoid the selection of the sample group for different tests. This will indeed reduce the selection threat to internal validity. Controlling the External Validity Threats Effects of Testing Researchers can apply a research strategy or design that does not incorporate pretest. The use of Solomon four groups’ strategy is critical in ascertaining to what level pretesting may influence the outcome of the study. Thus, pre-test must be avoided for this threat to be eliminated or for it to be brought under check. This will be of great benefit to the external validity since the threat is reduced. Multiple Treatments The researcher should select a design whereby only one approach is tasked to a single subject. This will ensure that external validity is put under check. The researcher can also help in reducing this threat by avoiding the use of the same sample group being tested in various experiments more than once. The researcher can use other sample groups instead of testing one sample group on numerous occasions. Researchers need to decide the type of strategies to be used to help them in controlling this threat. Selection Treatment Interaction A researcher can apply random assignments and selection of the sample groups. In situations where the above is not of a practical nature, then one needs to use statistical strategies like the covariance analysis commonly used together with the quasi-experiment. In the event that the researcher uses this, then the selection treatment interaction threat will be significantly reduced and controlled in the best way possible. References Brewer, M (2000). Research Design and Issues of Validity: In Reis, H. and Judd, C (Eds) Handbook of Research Methods in social and personality psychology. Cambridge: Cambridge University Press. Campbell, D. And Stanley, J (1963). Experimental and Quasi-experiment Design for Research. Chicago. IL: Rand-McNally Lerner, P (2003). Hysterical Men: war, psychiatry, and the politics of trauma in Germany, 1890-1930. New York: Cornell University Press Onghen, P., And Edginten, E .S (2005). Customization of pain treatment: Single-case design and analysis. Clinical Journal of Pain, 21, 56-68 Rosenthall, R (2002). Experimenter effects in behavioral research. New York: Appleton- Century Crofts Shadish, W., Campbell, D.T and Cook, T.D (2002). Experimental Design for Generalized Casual Inference. Boston: Houghton Mifflin. Taylor, S., Maxfield, L., Thadarson, D., Federoff, I.C., Lovel, K., and Ognedniczik (2003). Comparative efficacy, speed & adverse effect of three treatments for PTSD: Exposure therapy EMDR, and relaxation training. Journal of Consulting and Clinical Psychology, 71, 330-338 Tylor, S (2000). Understanding and Treating Panic Disorder. New York: Wiley. Read More