The Adequacy of Theory, Validity of Method and Reliability of Data - Assignment Example

Assessment 1 “Research Philosophy, Methodology and Quantitative Methods” Abstract This study points out the quantitative research philosophy, methodology and quantitative methods. This study investigates the philosophy that was grounded for building the quantitative research methods. It also discusses the issues associated with sampling, validity and reliability. I. Background and purpose 1.1 Relevant theoretical frameworks According to Harris (1979) science is 'a superior way for human beings to attain knowledge about the world in which we live'. Within the social sciences, there are two major theoretical perspectives for the scientific attainment of knowledge: positivism (the natural science paradigm) and phenomenology (Bogdan & Taylor 1975). The natural sciences, modelling the positivist philosophy, are concerned solely with observations of phenomena. The social sciences are additionally challenged with understanding the meaning of the phenomena, not a readily observable process (Schwartz & Jacobs 1979). Positivism, a deductive process of knowledge attainment, seeks to verify facts and causal relationships stated in existing theories. The true experiment is the classical example of positivism. Phenomenology, and inductive processes, generate theory from facts obtained within the natural setting of the phenomenon. The distinct contrast in the philosophy of this methodology from positivism is evident in grounded theory. The true experiment is a descendant of positivism and later logical positivism. The logical positivists devised two classes of statements: formal propositions based on logic or mathematics and factual propositions which are to be empirically verified (Harris 1979). In keeping with this philosophy, the true experiment seeks to verify facts or causes of social phenomena with little regard for subjective states of individuals (Bogdan & Taylor 1975). The goal is the establishment of general laws common to the phenomenon regardless of the setting. Mischler (1979) is particularly critical of the positivist methodology of context stripping. He believes the environment of the phenomenon is meaningful to the causal relationship and the understanding of human behaviour. Mischler accuses the true experimental design researcher of sacrificing meaning only to meet the assumptions of positivism. He criticizes: 1. the use of one method for a diverse setting of subjective matters; 2. the development of general laws which are expected to be applicable to all cases; and 3. the use of the statistical model as the ideal. Researchers of the phenomenological schools, i.e. grounded theorists, reject the assumptions of positivism, and attempt to humanize the research process. 'The project of phenomenological sociology thus concerns itself with the development of methodologies and modes of description which dispense with the distinction between (person) as subject and (person) as object' (Phillipson 1973). Grounded theory seeks to generate theory through an understanding of human behaviour from the actor's perspective in the natural setting (context) (Glaser & Strauss 1967). The major assumption of phenomenology is that knowledge of social facts is best attained when the researcher gets inside the natural setting and attempts to see (understand) the phenomenon as the subjects do. Data are thus collected and analysed in the natural language of the subjects rather than the statistical notations of positivism (Schwartz & Jacobs 1979). Mischler (1979) succinctly distinguishes the salient points of each philosophy. Phenomenology 1 Intertwines observer and phenomena 2 Many different but equal truths dependent upon the purpose and focus of the investigator 3 Seeks understanding of the meaning of the phenomena 4 Qualitative, holistic analysis 5 Increased validity Positivism 1. Outside observer; separate from phenomena 2. Seeks one truth to explain a phenomenon 3. Causal relationship 4. Context stripping assumptions and methodologies, quantitative 5. Increased reliability Positivism and phenomenology have grown to become polarities. Harris (1979) reminds us that both philosophies shared commonalities at the onset. The exaggerated reaction of positivists in support of their position was in response to narrow inductionism. The strengths and weaknesses of each position must be continually considered as nurses engage in research in a wide variety of settings and empirical foci. 1.2 Aims and objectives of the investigation The objectives of this essay are: 1. to study the philosophy of quantitative research method 2. to examine the quantitative methodology, and 3. to investigate the quantitative methods. II. Philosophy 2.1 The philosophical continuum and stances Deductive theory represents the commonest view of the nature of the relationship between theory and social research. The researcher, on the basis of what is known about in a particular domain and of theoretical considerations in relation to that domain, deduces a hypothesis (or hypotheses) that must then be subjected to empirical scrutiny. Embedded within the hypothesis will be concepts that will need to be translated into researchable entities. The social scientist must both skilfully deduce a hypothesis and then translate it into operational terms. This means that the social scientist needs to specify how data can be collected in relation to the concepts that make up the hypothesis (Bryan, 2004). 1. Theory ↓ 2. Hypothesis ↓ 3. Data collection ↓ 4. Findings ↓ 5. Hypotheses confirmed or rejected ↓ 6. Revision of theory The doctrine of positivism is extremely difficult to pin down and therefore to outline in a precise manner, because it is used in a number of different ways by authors. For some writers, it is a descriptive category— one that describes a philosophical position that can be discerned in research—though there are still disagreements about what it comprises; for others, it is a pejorative term used to describe crude and often superficial data collection (Bryan, 2004). Positivism is an epistemological position that advocates the application of the methods of the natural sciences to the study of social reality and beyond. But the term stretches beyond this principle, though the constituent elements vary between authors. However, positivism is also taken to entail the following: 1. Only phenomena and hence knowledge confirmed by the senses can genuinely be warranted as knowledge (the principle of phenomenalism). 2. The purpose of theory is to generate hypotheses that can be tested and that will thereby allow explanations of laws to be assessed (the principle of deductivism). 3. Knowledge is arrived at through the gathering of facts that provide the basis for laws (the principle of inductivism). 4. Science must (and presumably can) be conducted in a way that is value free (that is, objective). 5. There is a clear distinction between scientific statements and normative statements and a belief that the former are the true domain of the scientist. This last principle is implied by the first because the truth or otherwise of normative statements cannot be confirmed by the senses. III. Methodology 3.1 Critically Analyse Methodology Validity is 'the extent to which a measure measures what it is supposed to measure (internal) or the extent to which its use provides data comparable with other relevant evidence (external)' (Diers 1979). Vahdity assures that the causal relationship verified (or falsified) or the theory generated are not negated by rival hypotheses. External validity indicates the researcher's ability to generalize the findings beyond the study sample. Validity is a particular concern for quantitative research, i.e. the true experiment. In order to meet the requirements necessary to infer a causal relationship (concomitant variation, time order, non-spurious relationships), the internal structure of the study must control for or eliminate sources of rival hypotheses (extraneous variables). Internal validity is the basic minimum without which any experiment is uninterpretable: did in fact the experimental treatments make a difference in this specific experimental instance?' (Campbell & Stanley 1963). Striving to attain consistent internal validity, Campbell & Stanley (1963) developed eight classes of extraneous variables which have become standard criteria for measuring the validity of the true experiment. These criteria are: (1) history, other effects occurring during the study; (2) maturation, changes within the respondents; (3) testing, effects of taking one test on the second; (4) instrumentation, changes in the instrument or measurer; (5) statistical regression, sample selection on the basis of extreme scores; (6) selection biases; (7) experimental mortality, loss of sample members; and (8) interaction of extraneous variables (selectionmaturation, etc.). The presence of these extraneous variables threatens the researcher's ability to state with confidence that the outcome produced (dependent variable) resulted from the effects of the experimental variable. The question of external validity is never completely answerable. 'External validity asks the question of generalizability: to what population, settings, treatment variables, and measurement variables can this effect be generalized? (Campbell & Stanley 1963). The researcher desires to know that the results of the study are applicable to a larger population than simply the study sample. This concern is particularly acute when one recalls that the goal of positivism is to establish 'context free universal laws (Mischler 1979). The interactive effects of extraneous variables with experimental variables threaten generalization. These contrived interactions cause the study sample to be unique and therefore their outcome is not representative of the expected results of the independent variable on the population. The problems are painful because of a recurrent reluctance to accept Hume's truism that induction or generalization is never fully justified logically' (Campbell & Stanley 1963). Campbell & Stanley (1963) delineate the factors jeopardizing external validity. These factors are: (1) reactive or interactive effects of testing in which pretesting may sensitize the sample to the experimental variable, (2) interaction effects of selection biases and the experimental variable, (3) reactive effects of experimental arrangements which may alter the effects of the experimental variable in the non-experimental setting; and (4) multipletreatment interference since the effects of prior treatments cannot be eliminated. IV. Quantitative Methods 4.1 The characteristics and concepts of quantitative methods Paradigm Positivistic, acknowledging only a fixed reality that is measurable Relation to Theory/ Knowledge Theory testing using deductive approach Tests predetermined hypotheses Suitable when knowledge is available about variable(s) of interest and these variables can be measured Purpose Discovery of relationships, cause and effect Sample Larger sample size that is determined in advance and is representative of the population as a whole Setting Clinical, laboratory setting that can be controlled and manipulated Data Numbers Standardized biological and psychological measures Data analysis Numerical comparisons and statistical inferences Research product Acceptance or rejection of hypothesis based on the data gathered Source: Adopted from Vishnevsky and Beanlands (2004) 4.2 Sampling The effect of sampling limitations on a study can be devastating. A true experiment which does not have access to the required population (and subsequent sample) may be halted. Statistical sampling aims for random selection of the sample from the study population and then random assignment of the sample to the various study groups. Randomization is important in controlling for non-spurious relationships, a necessary condition of the true experiment. The sampling expertise of the researcher is critical during the development of the study. Keeping philosophically aligned with positivism, the true experiment is a logical and sequential research process outlined at the onset. Adherence to the criteria of probability sampling appears to be the only hope for generalizing the research findings. Random selection of participants is the ideal; but, this procedure is frequently sacrificed for a convenience sample. The convenience sample in the strictest sense negates the researcher's ability to generalize. Secondly, attention should be paid to the research setting. Compromises may have to be made in internal validity if one hopes to generalize the findings to the natural setting. Positivism purists would cringe and resist this suggestion. What they must realize is that the subjects (objects) and conditions of the social and behavioural sciences are born and must live in the natural setting. This environment cannot be controlled or manipulated as the environment of objects in the natural sciences. The artificiality that surrounds the current conduct of nursing research may tend to blind researchers to the empirical given if it not be boxed' (Dickoff et al., 1975). 4.3 The adequacy of theory, validity of method and reliability of data The true experiment is by design more reliable. Positivism is knowledge 'gained by interplay of reason and controlled observation' (Harris 1979). The controlled observation reflects the logical, systemized and delineated design of the true experiment. These details lend this design to being repeated in other settings. Unlike the controlled environment of the laboratory the field setting is complicated by extraneous variables which must be acknowledged to allow for repeatability. This systemized research methodology is an antithesis to the unstructured and sometimes spontaneous strategies of grounded theory. The subjective nature of grounded theory requires the researcher in each setting to follow the leads of the actors (sample). The reliability of the true experiment can be rather easily assessed by standardized testing. A measurement instrument such as a questionnaire can be pilot tested for computer analysis of rehability (coefficient alpha). The reliability of two human observers or raters can be tested by checking their inter-rater/inter-observer agreement levels. Physical data collection instruments such as recorders, EKGs, and so forth must also be tested for accuracy and congruency; changes being made in humans and physical objects to correct for low reliability. Concern for the sensitivity and precision of measurements increases reliability. Sensitivity allows the entire range of variation of the phenomenon to be measured, whereas precision addresses the instrument or measurements exactness in differentiating subtle variations within the sample. V. Practical example Hawaii is one of the most preferred destinations for Japanese along with the U.S. mainland, South Korea, and China. Hawaii Department of Business, Economic Development, and Tourism (2003) reported that more than 1 million Japanese visited Hawaii in 2002. Also, the number of Japanese tourists, students, and temporary workers visiting Hawaii has remained high since the mid–1990s, approximately 2 million a year (Hawaii Department of Business, and Tourism, 2003). In addition, more than half of Japanese visitors to Hawaii had visited previously. Hawaii is also popular for Japanese students. Several universities and colleges (e.g., University of Hawaii, Hawaii Pacific University, and Honolulu Community College) have a large number of Japanese international students. International Student Services at the University of Hawaii at Manoa (2003) reported that almost one third of international students on campus came from Japan. In addition, because a large number of Japanese Americans in Hawaii (17% of the total population) (U.S. Census Bureau, 2000), Hawaii keeps attracting Japanese business persons and temporary workers, as well (Benotsch, 2006). There are no available data identifying foreign countries where Japanese nationals became infected with HIV. As many recent Japanese heterosexual men were infected with HIV–1 subtype E, which is common in Southeast Asian countries, these cases may include heterosexual Japanese men who were infected with HIV in Asian countries through unprotected sex with HIV-positive women in these countries. When we consider that a huge volume of Japanese (annually more than 1 million) travel to Hawaii and the majority of these tourists are young adults, it is quite possible that they may engage in HIV–related risk behaviors while staying in Hawaii (Benotsch, 2006). However, little is known about their HIV–related drug use and sexual behaviors. It is beneficial to describe their risk taking behaviors in Hawaii in comparison with those in Japan in order to develop future HIV/STI prevention intervention programs targeting high–risk groups of Japanese. Because a large number of people travel to Hawaii and elsewhere, it is also important to describe HIV–related risk behaviors among tourists and business travelers, particularly for those transient Japanese who engage in unprotected sex with commercial and casual sex partners. They may be infected with HIV/STDs or may transmit diseases to local people while traveling, as well as transmit diseases to people when they return to Japan (Benotsch, 2006). Procedures: Based on ethnographic mapping, the target areas for sampling were identified by two survey workers and three local collaborators who were enrolled in a graduate school of the University of Hawaii. The survey workers and local collaborators who are bilingual in Japanese and English were trained for the study protocol, sampling and recruitment procedures, human subjects protection, and data collection and maintenance, which were approved by the Committee on Human Research, University of California, San Francisco. A total of 249 Japanese nationals were recruited from the identified target areas in Honolulu, Hawaii in 2000. We utilized stratified sampling methods by gender (approximately 50% each gender) and types of participants (targeted 100 each for tourists, students, and temporary workers) (Benotsch, 2006). Measurements and data analysis: The structured questionnaire included anchored questions and Likert–type scales asking about participants’ demographics, HIV–related sexual behaviors (e.g., condom use and sexual behaviors under the influence of substances), alcohol and drug use, AIDS knowledge, attitudes toward condom use in general and with casual partners, and adaptation to the American mainstream culture (adaptation to American life). The measurements were modified from those used for our previous study in HIV–related risk behaviors among Japanese drug users who were temporarily staying in the United States The measurements are written in Japanese and designed to be culturally appropriate and sensitive to Japanese who live in the U.S. as well as Japanese visiting the United States. VI. Conclusion In short, the study has shown the fundamental philosophy grounded in the quantitative research methods. There are issues associated with the quantitative research methodology and cautions must be taken when applying the research methods to the designated study. Read More