Health Program Evaluation - Essay Example

and No Program Evaluation Paper #3 28 September Data Collection The proposed health program designed forprevention of childhood obesity must meet certain requirements for it to be viable. One of these criteria is how data will be collected. In this regard, it is necessary for data collection processes to be easy to do but at the same time assure data integrity. This means the collection processes can be done by the programs aides with only minimal instructions and can be followed easily by them. For example, instructions for getting the weight measurements of the kindergarten kids should be rounded off to nearest tenths of a pound. If a kid weighs 40.55 pounds, this weight figure will be rounded off to the nearest tenth of a pound, which will make the childs weight as 40.6 pounds. If the child had weighed 40.54 pounds, then the weight measurement will be recorded as 40.5 pounds. This is an example of how data collection can influence the program evaluation plan by making the collection of kids measurements easier to do but without compromising its accuracy. The program evaluation itself is a mixed method, meaning it will use both quantitative and qualitative ways of gathering the data. Getting the exact weight measurements of the kids is part of the qualitative process of data gathering while using open-ended surveys in the form of a questionnaire will form the qualitative part of the data collection process. For this part of the data collection which is the qualitative part, it is important that the observations by parents and teachers are made honestly and describes in full detail what they had observed in the kids behavior. If they found out kids are more active after eating their fruits, for example, then it must be described in a meaningful way such as using an increasing scale, with 1 as the lowest and 10 as the highest so this will give the researchers a good idea of how active they were. The programs overall integrity will be evaluated based on the way data is collected and how reliable the data will be. This will greatly determine whether the program is a good one or needs some revisions based on the difficulty or ease of getting all the necessary data. It is at this point when the programs researchers can have a good idea on the probable outcome of their research project because data collection is the very heart of the research study itself. It will determine whether the entire project will be successful or not because on implementation aspects of the program such as the aforementioned data collection procedures. Strengths and Weaknesses of Data Collection A strength of the data collection process is its reliance on concerned stakeholders: the parents primarily and the school teachers, secondarily. This simply means the parents have a vital stake in the success of the program and would therefore exert all efforts to make the data as accurate as possible. This means they will not make attempts to fudge the data collected; it becomes counterproductive and possibly even harmful to their kids if the data is not accurate. What is at stake and involved here is the health of their kids who can be prevented from being obese by using the appropriate prevention intervention programs. It would do the parents no good if the data collected is not accurate or even doctored or just made up. On the part of the teachers, eventual success of the program will also be to their own benefit, namely the number of school enrollees will be maintained for each school opening. It means there livelihood as professionals charged with teaching children is assured because of the high levels of enrollment. Furthermore, the programs success will also be the source of a great pride for the teachers for having successfully participated in a vital health program that involves the kids long-term health. The teachers feel they acted in a professional manner that contributed to the overall well-being of the entire community of which they are a big part of. Finally, teachers will feel they had successfully integrated a health module in the curriculum. However, there are a few probable weaknesses in the program not because of teachers and parents commitment to its successful implementation but more on time constraints. In the case of the teachers, imposing an additional burden of recording the kids physical activities may interfere with their academic duties of preparing the lesson plans and supervising all the kids in their various activities. The responsibilities of data gathering may cause some teachers to get distracted from their primary duties as educators; they are not researchers themselves. Another possible weakness, and this is related to the first probable weakness which is time constraint, is the chance that teachers will be tempted to just fill in needed data just for a sake of compliance with the programs requirements to produce data by a given deadline. This will skew the data profile, whether intentionally or not, because it will not be reflective of the true situation regarding the progress of the intervention program. Even assuming the teachers have no intentions of making up some data, the demands on their work time may make it near to impossible to record certain data immediately. This will force the teachers to fill out forms after the events they are supposed to observe had happened, and may therefore be relying on their memories and recollection on how the events they will record had really transpired. The data will not be as accurate as when these data were recorded immediately during the event. In a program like childhood obesity prevention programs where kids are the primary research subjects, it is important to use a variety of methods during the data collection process which are the interviews, questionnaires, observations and organizational records. A weakness of any program even like this obesity prevention is to rely on only one or two data collection methods (Swanson, 2007, p. 108). An example might be relying more on using interviews as the data source as this method is quick and easy to do but will not give a full picture. On the other hand, it is important to let the research questions determine the best appropriate method to use and not let the data collection methodology determine the formulation of the questions. Subsequent threats during the data collection process include personal and procedural reactivity. The first threat happens when the study subjects behave differently than they would normally behave. A reason for the different reaction from the subjects could be due to the very presence of the observer. An example would be the subjects behaving differently because the observer happened to be a male when in previous observations, the observers would usually be females. The second threat which is procedural is when the subject behave differently as they know they are being observed and they become self-conscious (Sapsford & Jupp, 2006, p. 87). This type of reaction is famously known as the Hawthorne Effect in which subjects are modifying their behaviors intentionally which adversely affects the results of data gathered. In either of the two instances cited above, both types of reactivity invalidates the data. It affects the validity of the data and the research study may have to be repeated again. A very common but sometimes unconscious mistake of researchers is collecting too much data (Wilson, 2009, p. 81). However, this is a very subjective decision and the only type of limit will the researchers’ own common sense with regards to data requirements. There are no hard and fast rules in this regard; everything will depend on the nature of the research. After all is said and done, it will ultimately be the researchers who will be swamped with data to be analyzed and the tendency will be to overlook the important details in those mountains of data. Data collection should be planned very well right from the very start. The only effective way to counter these threats to data collection is to train properly all the data collections, including the parents and teachers involved (Blankenship, 2009, p. 133). The whole idea is to keep the research environment consistent all throughout during the data collection sessions. As a final note, the threats to qualitative data collection are much different from the type of threats encountered when gathering quantitative data. In the first instance, qualitative data consists mostly of words, descriptions and observations. In qualitative studies, it is the researcher himself who is the primary data collection instrument (ibid. p. 137). Appropriate Data Analysis Procedure A new trend in data analysis today is to use geographic information systems or GIS. This is a GPS-enabled analytical procedure which makes data interpretation much easier as it translates the data gathered into maps. The visualization process using maps if often better than using tables or graphs; besides the researchers, the intended audience of study results might be laymen or ordinary people and using GIS makes them understand the data faster. Data presentation is made more compact and precise with GIS; large data sets can be reduced to a few points on a map which is easier to understand. In one fairly recent childhood obesity study, Zhang et al. (2006, p. 14) were able to project their tabular data into nice sets of maps showing the prevalence of childhood obesity based on some school neighborhood-level physical characteristics which are thought to be contributory to the development of obesity in children. Factors like land use, proximity to other commercial centers, census tracts, crime rates, levels of vehicular and human traffic are significant factors that predispose kids to becoming obese during their formative years. It is generally conceded that inactive or sedentary lifestyles and poor eating habits are contributors to childhood obesity but this study looked at obesity as additionally caused by the social and physical environments where children spend most of their time which is inside their schools. The concept of GIS only came about due to advances in technology of the global positioning system (GPS) and this new technology was used in the interpretation of many research data and carried forward or combined with data from geo-spatial locations to make data easier to interpret. Result is a neat presentation of complex data and make it easier to understand for everybody, even for laymen and non-experts alike. This is made possible by the installation of GIS into researchers’ computer data systems; statistical and graphical analytical procedures then provide an atmosphere where various data sets can be modeled and translated into more visual presentations which render it easier to understand. Data of any practical significance, in the above example, might include the number of school playground equipment located in each school. It may also include the walking distance of the school from the main highway, which by inference, would require the young students some distance to walk in order to reach their school. This would then be correlated with the obesity rates of children in that school; walking daily to and from school from the highway is an indicator of the physical activities children perform and therefore is a factor that reduces the obesity rates in the children of that particular school. The adoption of these methods is a result of the limitations observed in data sets of their statistical significance when using null hypothesis. There have now been several calls for the adoption of newer approaches when interpreting the practical significance of these data. A new perspective of seeing massive mountains of data and to look for practical significance is actually just an attempt to inject practicality into all that data collection efforts. Some of the data being collected by some researchers in their studies have no practical bearing and may constitute nothing of practical value even if obtained. A stubborn insistence of getting some of the data is a waste of time and effort especially when viewed in an era of limited budgets. The ultimate test of the practicality of any data is whether it is useful in the real world. Statistical significance pertains to the probability of some error being committed due to the effects of randomness. This variation in the data could significantly alter the results or the conclusions by skewing the data (Fletcher & Fletcher, 2005, p. 171). One must be aware of any statistical significance of the data gathered and account for these minor differences in the interpretation process. Randomness will always be present in any observation and one cannot completely eliminate chance in any experimental study. Statistical significance will help to account for how random chance can account for some findings in the research study. An understanding of this detail can lessen the probability of errors in the study. In this case of studying obesity in kids, it means whether the data is meaningful or not (Urdan, 2005, p. 57). Some of the essential elements in the program evaluation report will include standards to use when judging program performance, what items will be evaluated, what levels of the standards will have to be reached before the program can be considered as successful, what type of evidence will be used and how the conclusions are reached so that it can be said that the program was successful. For this particular research study, an example of such essential element will be the total number of school kids who were prevented from becoming obese in the two-year period of the study. This will serve as the standard by which to measure whether the program achieved its objectives or not in terms of obesity prevention in kids. The ultimate test of any research study, especially studies involving public health, is whether the program standards adhered to the generally-accepted procedures that ensures a conclusion that is credible, feasible, useful, ethical and most importantly, accurate. In this case of childhood obesity prevention, all these elements must be present and achieved in order for a conclusion about the effectiveness of the educational module implemented in the schools under study can be stated without any lingering doubts. The elements and parameters that were discussed at this point will constitute the framework that will guide public health care professionals when evaluating their programs. This framework is practical in the sense it can help in the processing of organizing and summarizing all the essential elements of evaluation. Utilization Strategies in Program Evaluation Program evaluation can be used either retroactively or proactively. In the former, it is used to revise existing programs and introduce improvements so that program effectiveness is enhanced and the results more useful and credible. This serves as a valuable input to decision making processes for program proponents. By utilizing the insights gained from the program evaluation, present programs are improved significantly that may result in cost savings, the shortening of the program period or greater accuracy in the data gathering. The result is that present programs are enhanced by using the knowledge gained (Howell et al. 1996, p. 38). Program evaluation used proactively means designing future programs more effective in order to achieve a program’s aims faster, cheaper and easier. The insights gained so far will help in pointing out to program proponents the potentials of a proposed program but the real challenge is how to translate these potentials into practical use in reality. If the results of this childhood obesity prevention program showed a significant number of the kids did not gain any weight at the end of the two-year duration of the study, then parents and teachers alike can be convinced to continue the educational program in subsequent batches of kids. These two stakeholders can then join together to refine the program so more so it can be used in even the higher grades, such as in the elementary grades 1-6. Better than that, public health policy makers can be induced to adopt this program on a wider scale and make it a standard feature in their advocacy about childhood obesity prevention. The idea can be adopted to make it a permanent feature in school programs affecting children health. Even the primary stakeholders who are the kids themselves will be convinced of its benefits to them. Conclusion Obesity is a serious enough problem that it can no longer be ignored. The need to act is now but intervention programs have to be properly evaluated whether they are useful or not in the fight against childhood obesity. To ensure various programs achieve their purpose, it is necessary to evaluate processes during the data collection phase of the program. This is to make sure the data gathered is valid, taking into account certain statistical significance factors which may distort the data and also to account for variability due to random chance happening during the data collection process. But more critically, any obesity prevention program will also have to be evaluated for its practical significance since so many programs fail due to their being of little practical value during implementation. This means their findings cannot be really useful in a practical sense and will remain in the esoteric realm of theory making only. An honest and serious program evaluation can help in designing future obesity programs. Reference List Blankenship, D. (2009). Applied Research and Evaluation Methods in Recreation. Champaign, IL: Human Kinetics. Fletcher, R. H. & Fletcher, S. W. (2005). Clinical Epidemiology: The Essentials. Baltimore, MD: Lippincott, Williams & Wilkins. Howell, S. L., Foster, R. L., Hester, N. O., Vojir, C. P. & Miller, K. L. (1996 Summer). Evaluating a Pediatric Pain Management Research Utilization Program. The Canadian Journal of Nursing Research, 28 (2), 37-57. Sapsford, R. & Jupp, V. (2006). Data Collection and Analysis. London, UK: Sage Publications, Ltd. Swanson, R. A. (2007). Analysis for Improving Performance. San Francisco, CA: Berrett-Koehler Publishers, Inc. Urdan, T. C. (2005). Statistics in Plain English. Mahwah, NJ: Routledge. Wilson, E. (2009). School-based Research: A Guide for Education Students. London, UK: Sage Publications, Ltd. Zhang, X., Christoffel, K. K., Mason, M. & Liu, L. (2006, March 30). Identification of Contrastive and Comparable School Neighborhoods for Childhood Obesity and Physical Activity Research. International Journal of Health Geographics, 5,14. Retrieved from http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1526711/ Read More