Government Policies in a Population Survey about Good Government - Essay Example

Government Policies in a Population Survey about “Good Government” Much research in political science involves issues surrounding perceptions of good government and parameters of social well-being (Adsera, Boix & Payne 2003). For example, research studies have suggested that individuals who score higher on tests that measure personal satisfaction and well-being are more likely to inhabit countries whose governing standards are judged favourably based on established criteria (Baron 2005; Besley, Burgess & Prat 2002; Besley & Prat 2004). The purpose of this experiment is to test whether one’s perception of standards of good government may be easily manipulated by exposing the test subject to highly favourable (or unfavourable) commentary immediately prior to being given a survey on personal attitudes towards government policies and their effectiveness (Brunetti & Weder 2003). This is an experimental study involving a control group who will be given the survey without reading anything prior to testing and an experimental group who will read an essay on government policies prior to completing the research survey. This will be a randomised study of college seniors who are of similar age and educational background who will be randomly assigned to the control or experimental group. The experimental condition is a dramatic and colourful expose on the accomplishments (or, alternatively, abuses) of government that will be used to determine whether exposure to opinionated statements affects attitutinal responses on a government survey questionnaire. The effect will be measured by a comparison of survey responses between the two groups. The setting is controlled in that it comprises a peaceful library setting at the university attended by the study subjects. The randomisation will affect the potential generalisability of the study results and decrease the possible occurrence of random variables that could affect test responses. Randomisation of the study also provides a basis for the statistical assessment of study results. This is an example of a classic controlled experiment design (Campbell & Stanley 1963). As such, it will involve pre-tests and post-tests. Maximum internal and external validity may be obtained by the use research study designs that involve group studies rather than case studies or same subject design, subject randomisation, and statistical analyses that are appropriate for the type of research study to which they are applied (Bulpitt 1996). Single subject designs have a high internal validity but lower external validity as the responses depend on an individual subject under different experimental conditions (Shapiro 2002). This type of study may be of greater external validity when the experiment is repeated with large numbers of subjects. External validity relates to the generalisability of the research findings produced by an experimental study (Cook & Campbell 1976). External validity is a measure of the degree to which the results from an individual study may be applicable on a broader scale to different groups or different conditions. Group studies that utilise random selection criteria of study subjects generally have greater external validity than case studies or single subject design experiments (Peterson 2001). One factor affecting external validity refers to selection criteria that may affect the study responses. If the selection criteria are biased, this may affect study results and the generalisability of the study conclusions (Salanti et al 2008). . Pre-testing can also affect subject performance and the external validity of the data (Nuesch & Juni 2008). The experimental setting may also affect subject responses to the experimental conditions that affect broader applications of study results. Lastly, studies that involve multi-step processes, interventions or treatments may have lower external validity as the subject responses may be very much affected by the uniqueness and sequence of the experimental conditions (Kramer & Shapiro 1984). The threats to external validity in this study design may include population representativeness, in that university students represent a select and highly educated group (Salanti et al 2008). .To counteract this, a similar study could be carried out with other segments of society to see if a similar relationship is observed. Treatment representativeness should be considered, but most individuals in society read newspapers or listen to TV or radio discussions of government issues and policies that may be highly opinionated and thus might be expected to have at least temporary effects on attitudes towards government (assuming that the experimental study suggests this outcome) (Djankov et al 2003). This factor could affect the generalisability to some extent. The setting is also a factor that could affect the generalisability in that it is a university environment (Salanti et al 2008). The study would require replication in other settings (factories, hospitals, etc.) to afford external validity to the results. A difference between the responses of the control and experimental groups will provide evidence of a correlation between the reading material and study responses that may indicate causation. These results must be evaluated with regard to their internal validity. Internal validity refers to the factors that may affect the integrity or accuracy of the experimental results. Internal validity involves the design of the experiment, operational definitions, the choice of variables and the degree to which changes in an independent variable are the direct cause of changes in the dependent variable (Fitz-Gibbon & Morris 1978). Some factors that may affect internal validity include history variables (Little & Rubin 1987). These are defined as the unique experiences that may individuals and their testing responses during the course of an experiment. This type of variable is more important when the length of a study is extensive or involves repeated testing of study subjects. Maturation variables refer to the internal mood or physical state of the study during the course of an experiment (such as fatigue, boredom, hunger) that may affect subject responses (Wells & Littell 2009). Pre-testing can also contaminate study results by affecting subjects’ responses during the actual experimental testing procedure. The type of instrumentation or observation methods may produce differences in results based on the use of different tools or due to judgment factors (Garg, Hackam & Tonelli 2008). Statistical regression can affect study responses if subjects were selected for a research study based on unusual testing responses. Selection variables result when the experimental and the control groups are not functionally equivalent (Salanti et al 2008). Experimental mortality refers to subjects who do not complete research studies. In addition, interactions among the various variables can further contribute to a lack of internal validity in an experimental study (Rossi & Freeman 1993). The design of this study is randomised in order to minimize these threats to internal validity. Moreover, the subjects are all of similar age and educational background. This will help to prevent selection variables from affecting the internal validity of the data. Study designs that maximize internal validity include double pretest (.Holland 1986). This type of study involves two temporally separated pre-tests on subject candidates. This approach can be used to rule out maturation effects and with selection regression. It is also useful in preventing historical effects on subject responses. Switching replication design involves the use of two groups interchangeably to measure study responses (Salanti et al 2008). This design helps to prevent social threats of compensatory rivalry, compensatory equalization and resentful demoralisation. Because the differences between the groups are responsible for these social effects, the experimental design generates an equivalency that may largely prevent these social effects from occurring. The Solomon Four Group Design is also useful in promoting internal experimental validity (Mark & Cook 1984). This experimental design involves the random assignment of study subjects into four groups, two of which receive the first study application and two that do not (the controls). The first two groups receive pre-and post test, whereas the second set of two groups do not. This is useful in ruling out pre and post test effects on experimental results that may affect internal validity. The experimental design for this study could easily be manipulated in these ways to afford high level internal validity to the experimental results.   References Adsera, A.; Boix, C.; Payne, M. (2003). Are you being served? Political accountability and quality of government, Journal of Law, Economics and Organization, vol. 19 (2), 445 – 490. Baron DP (2005) Competing for the Public Through the News Media. Journal of Economics & Management Strategy 14, 339–376. Besley, T.; Burgess, R.; Prat, A. (2002). Mass media and political accountability, Working paper, Department of Economics, London School of Economics. Besley, T. and Prat, A. (2004). Handcuffs for the grabbing hand? The role of the media in political accountability. Working paper, Department of Economics, London School of Economics. Brunetti, A. and Weder, B. (2003). A free press is bad news for corruption. Journal of Public Economics, vol.87, no. 7-8,1801-1824. Bulpitt C (1996). Randomized Controlled Clinical Trials, 2nd ed. Boston, Kluwer Academics. Campbell, D. T., & Stanley, J. C. (1963). Experimental and quasi-experimental designs for research. Chicago: Rand McNally. Cook, T. D., & Campbell, D. T. (1976). The design and conduct of quasi-experiments and true experiments in field settings. In M. D. Dunnette (Ed.), Handbook of industrial and organizational psychology (pp. 223-326). New York: John Wiley and Sons. Djankov, S.; McLiesh, C.; Nenova, T.; Shleifer, A. (2003). Who owns the media? Journal of Law and Economics, 46, 341-381 Fitz-Gibbon, C. T., & Morris, L. L. (1978). How to design a program evaluation. Beverly Hills, CA: Sage Publications. Garg, A. X., Hackam, D., Tonelli, M. (2008). Systematic Review and Meta-analysis: When One Study Is Just not Enough. CJASN 3: 253-260.   Holland P. (1986). Statistics and causal inference. J Am Statistical Assoc. 81:945–960. Kramer M. S., & Shapiro S. (1984). Scientific challenges in the application of randomized trials. JAMA, 252:2739–2745. Little R. J.A., & Rubin, D.B. (1987). Statistical Analysis With Missing Data. New York, John Wiley & Sons. Mark, M. M., & Cook, T. D. (1984). Design of randomized experiments and quasi-experiments. In L. Rutman (Ed.), Evaluation research methods: A basic guide (2nd ed., pp. 65-120). Beverly Hills, CA: Sage Publications. Nuesch, E., Juni, P. (2008). Commentary: Which meta-analyses are conclusive?. Int J Epidemiol 0: dyn265v1-dyn265. Peterson, R. A. (2001) On the Use of College Students in Social Science Research: Insights from a Second-Order Meta-analysis. Journal of Consumer Research 28:3, 450-461 Rossi, P. H., & Freeman, H. E. (1993). Evaluation: A systematic approach (5th ed.). Newbury Park, CA: Sage Publications. Salanti, G., Higgins, J. P., Ades, A., Ioannidis, J. P. (2008). Evaluation of networks of randomized trials. Stat Methods Med Res 17: 279-301 Shapiro, M. (2002) Generalizability in Communication Research. Human Communication Research 29:4, 491-500. Wells, K., Littell, J. H. (2009). Study Quality Assessment in Systematic Reviews of Research on Intervention Effects. Research on Social Work Practice 19: 52-62.   Read More