Quantitative and Qualitative Data - Essay Example

COLLECTING THE DATA Quantitative and Qualitative Data quantitative data is favoured by the positivist approach Qualitative data collected by the interpretist approach Ways of Collecting Data There are six major ways 1. Interviews 2. questionnaires 3. observation techniques 4. logs and journals 5. scales 6. secondary data -Scales –quantitative -logs and journals are in general qualitative data all the rest can be either qualitative or quantitative - Each method can be checked for neutrality and applicability -With neutrality the researchers does not lookout to influence the data in order to arrive at a preconceived truth - Applicability relates to whether a measuring instrument is suitable for a measuring particular problem Interviews they can be structured or semi-structured or unstructured a) structured the questions are known by the interviewer in advance b) semi-structured the interview is free to improvise with questions general directory are given but not the questions also known as open-ended questions c) unstructured are similar to a conversation except the researcher and subject are aware that they are in an interview Neutrality suffers limited neutrality: people react differently to different people depending also on the approach interview factors can play a role e.g. age, gender, ethnicity, dress, context, manner of speaking of the interviewer etc also each interviewer will react differently to these factors (i.e. the reactive effect) this can jeopardize objectively in the positivist approach it is less of a problem with interpretist approach but is still valuable one solution is to train interviewers so they can follow an explicit protocol. format advantage: -ambiguous questions can be clarified -response rates are high with this method esp. for face-t-face interviews and less for telephone interviews Applicability It is good if a. want high response rate b. want to gather in-depth information (esp. interpretive studied) c. in cases where anonymity is not important i.e. if the subject is not sensitive d. have enough time and money e. the respondent is illiterate Questionnaires There are many types a. Mailed questionnaires b. Face-to-face questionnaires c. Group questionnaires d. Bilingual questionnaires Mailed popular and used by agencies to survey clients in program evaluation or needs assessments Face-to-face -given in some was as structured interviews Group questionnaires -given to groups e.g. looking for feedback on foster parent training session then questionnaires are given to whole group at one time Bilingual questionnaires -depending on the community and population may be needed Neutrality relatively neutral: no interview bias and responses are clear if the questionnaire is well constructed , the responses will be neutral i.e. no ambiguities Questionnaires should be clear as possible. e.g. no double negative questions no double-barrelled questions Advantages: mailed questionnaires have low response rates, Factors: a. Limit length b. Structure of the questions c. Content of the questionnaire d. Format e. Cover letter f. Follow-up Solutions: they should be as short as possible questions should be close-ended or open-ended Close ended- questions give choices to the respondent Open-ended leaves room for the subject to create a response .they can be intimidating content limit sensitive questions or place them at the end of the questionnaire format – no typographical errors / clear uncluttered layout Cover letter –describes the purposes of the study and encourages a response. show how confidentiality will be ensured/ offer incentives coupons or a pen Follow-up a second mailing may improve response rate. Applicability They are good for when a. A high response rate is not a priority b. Anonymity is important c. Limited budgets d. Respondents are literate Observation techniques can be structured or unstructured some phenomena like illegal behaviour or children’s behaviour cannot be measured through interviews and questionnaires – so observation is better as a social worker you should be a good observer Structured observation good when behaviour are known and categorized prior to observation and quantitative data is needed behaviours are categorized as characteristics, frequency, direction and magnitude the categories can then be quantified Unstructured observation Good when little is known about the behaviour being studied or an interpretist approach is adopted and the goal is qualitative data e.g. in anthropology on culture study the researcher interacts with the subjects if fully submerged it results in participant observation e.g. joining a gang as a member to observe it. Neutrality -depends on the a. Type of observation, b. Training level c. Control for reactivity 1. Structured questionnaires are more neutral than unstructured because behaviours are structured before 2. Training level- the more the training the more neutrality. More training clarifies the categories 3. Control for reactivity- can be controlled through 4 strategies i. videotapes ii. one-way mirrors iii. time with observer iv. participant observation videotapes can be used but leads to inhibition on the part of the subject one –way mirrors –consent should be obtained from subject spending time with subject makes them comfortable reactivity can be reduce by participant observation Applicability: better when other techniques cannot be used accurately Logs and Journals can be structured or unstructured a kind of self-observation –the client records his/her behaviour or the researcher records the client’s Unstructured logs are useful for qualitative data but logs can collect quantitative data too. Neutrality: can be limited esp. is unstructured can be enhanced: using of more structured journals and logs also to record many behaviours as possible Applicability -useful if detailed personal experiences are needed -when subjects are literate Scales can be used in questionnaires and interviews or structured observation good for collecting quantitative data scales are composite measures consisting of a number of items Uniform and tested extensively are called Likert scales most commonly used in social science the respondent responds to five of possible responses e.g. strongly agree, agree, no opinion, disagree and strongly disagree etc Fig 9.4 family-crisis oriented personal evaluation scale ( F-COPES) When we face problems or difficulties in our family, we respond by: Strongly disagree Moderately disagree Neither agree nor disagree Moderately agree Strongly agree 1 Sharing our difficulties with relatives 1 2 3 4 5 2 Seeking encouragement and support from friends 1 2 3 4 5 3 Knowing we have the power to solve major problems 1 2 3 4 5 4 Seeking information and advice from persons in other families who have faced the same or similar problems 1 2 3 4 5 5.) Seeking advice from relatives ( grandparents 1 2 3 4 5 6. Seeking assistance from community agencies and programs designed to help families in our situation 1 2 3 4 5 7. knowing that we have the strength within our own family to solve our problems 1 2 3 4 5 8. Receiving gifts and favours from neighbours ( e.g. food, taking mail etc) 1 2 3 4 5 9 Seeking information and advice from the family doctor 1 2 3 4 5 10 Asking neighbours for favours and assistance 1 2 3 4 5 11 Facing the problems “head-on” and trying to get solutions right away 1 2 3 4 5 12 Watching television 1 2 3 4 5 13 Showing that we are strong 1 2 3 4 5 14 Attending church services 1 2 3 4 5 15 Accepting stressful events as a fat of life 1 2 3 4 5 16 Sharing concerns with friends 1 2 3 4 5 17 Knowing luck plays a big part in how well we are able to solve family 1 2 3 4 5 18 Exercising with friends to stay fit and reduce tension 1 2 3 4 5 19 Accepting that difficulties occur unexpectedly 1 2 3 4 5 20 Doing things with relatives ( get-togethers, dinners etc0 1 2 3 4 5 21 Seeking professional counselling and help for family difficulties 1 2 3 4 5 22 Believing we can handle our own problems 1 2 3 4 5 23 Participating in church activities 1 2 3 4 5 24 Defining the family problem in a more positive way so that we do not become too discouraged 1 2 3 4 5 25 Asking relatives how they feel about problems we face 1 2 3 4 5 26 Feeling that no matter what we do to prepare , we will have difficulty handling problems 1 2 3 4 5 27 Seeking advice from a minister 1 2 3 4 5 28 Believing if we wait long enough, the problem will go away 1 2 3 4 5 29 Sharing problems with neighbours 1 2 3 4 5 30 Have faith in God 1 2 3 4 5 1 2 3 4 5 shows the F-copes scale for measuring Family Coping though recommended to use existing scales , it is not always practical as it depends on the variable being measured e a scale for Family Coping may not work for family cohesion There are three types used a. target problem scales b. goal attainment scales c. rapid assessment scales Target Problem Scales -useful for tracking changes in a client’s target behaviour -useful also when actual outcome are difficult to identify -steps: identify problem, applying intervention and then repeatedly rating the changes in the problem. Goal attainment Scale - Aka GAS -it shows the achievement of outcome and are used both to set client goals and assessing if goals were met - involves four steps a. identifying problems b. specifying areas in which change is needed c. making specific predictions for a series of outcomes levels for each area d. by a set of date , scoring the outcome as they occur ( i.e. 5 outcomes possible, least –most favourable) Fig 9.5 Rapid Assessment Instruments - Aka RAI: a standardised series of structures questions administered to client to collect data in practice evaluations -they are short and easy to administer -easy to complete Ex: multi-problem screening inventory MPSI (Hudson 1990) which is computerised Neutrality can be neutral Applicability useful when quantitative data is needed useful for multifaceted concepts useful when time is limited Secondary data - This is the data collected for purposes other than the present study e.g. data collected for another project study - Computers make it easy to collect secondary data -useful when conducting needs assessments - There 2 strategies when using secondary data 1. rates under treatment 2. social indicators Rates under treatment Use existing data from agencies to determine the needs of a community. The problem is that the use of present services does not show unmet needs. Social indicators Use demographic data from public records to predict a community’s needs e.g. facilities in a community, housing pattern, crime patterns etc Vignettes can be used .these are hypothetical situations either drawn from a source or developed by the researcher for the purpose of eliciting certain responses from the participants in the study. Indirect sources: -refers to information that was collected for another purpose but can be used for research e.g. case records, newspapers, media reports. - best handled under content analysis - content analysis is a way of coding information in a systematic quantifiable form Neutrality secondary data has limited neutrality indirect sources can be biased because they are not collected for research and will be missing information on the research question direct source are more neutral but exact form of questions needed be asked Applicability a. when useful data are available b. when the definition of secondary data variables and form of the questions are the same c. for needs assessments d. for a limited budget e. historical study – because it offers lots of data Who Collects the Data? the decision depends on the type of research question asked not only the researcher who can collect information also the client e.g. through journals and logs, mailed questionnaires, scales , checklists it is useful to use the client in single –system studies reactive effects can be a problem too when the client collects the data on his/her behaviour or self monitoring Combining Data Collection Methods Methods and instruments are encouraged to be used together e.g. quantitative and qualitative combining measures enriches the study and ensures a maximum number of dimensions of the problem under study Triangulation: refers to using combination of data collection methods, theories, researchers and research methods. it can enhance validity ( interpretive studies). Determining Reliability and Validity: important to assess reliability and validity of a measuring instrument before using it how they assessed depends on the approach and whether qualitative or quantitative data is to be collected quantitative data collection instruments are more reliable and valid e.g. standard interview, questionnaire, observation schedules and scales qualitative instruments are less reliable and valid Fig 9.2 characteristics of data collection methods Unstructured Mailed questionnaire Participant observation Standardized observation Logs Face-to-face administered standardized scales High response rate Yes No n/a Yes May be Yes Anonymity assured No Yes No No No No Low reactivity effects No Yes Maybe Maybe Yes Yes Illiterate subjects Yes No Yes Yes No No Semilegal or illegal behaviour No Maybe Maybe No No No Large sample or limited funds No Yes No No No Yes In-depth “thick description” Yes No Yes No yes No Reliability refers to the consistency of a measure shows the extent to which a measure reveals actual differences in the problem being studied than differences in the instrument itself e.g. a wooden ruler – reliable to measure a table Plastic ruler – unreliable to measure table because it would stretch There are two ways to assess reliability of an instrument 1. assess sources of error 2. the degree to which its reliability has been tested SOURCES of ERROR There are four major sources 1. Retrospective information 2. Data collection conditions 3. Structure of instruments 4. Undefined variables Unclear definition of variables: difficult to define because many social work terms are vague if not clearly defined and operationalized a variable’s measure is unreliable i.e. the outcome can be interpreted differently by different social workers questions might have two different meanings to 2 different people e.g. Do you often use public transport in the city? “Often” can be defined differently by different people this is a big problem in interpretive studies where the variable are not clearly defined and operationalized Use of retrospective information this information is collected mainly through subject recall through questionnaires or interview these data is almost always distorted case records are an example of retrospective data and usually reflect the idiosyncratic recording practices of the individual social worker. the social worker may be biased in the cases he chooses to report- leading to unreliability Variations in conditions for collecting data the respondent may respond differently to a male or female interviewer ( reactive effect0 other factors include the ethnicity and age of the interviewer the location may also be a factor Structure of the instrument this has to do with aspects of the instrument Testing Reliability it is important to know what others have done as reliability tests so you may use them on your own instruments it is tested by getting 2 or more measures of the same thing and seeing how close they agree .Four methods to establish reliability 1. test-retest 2. alternate form 3. split half 4. observer reliability Test-retest means administering the instrument to the same set of people on separate occasions the results are then compared .if they are similar the instrument’s reliability is high Problem: the first testing influences the second i.e. in the second testing subjects may be less motivated, less interested or repeat answers from the first testing it is not a good method to test instruments that are strongly affected by memory and repetition Alternate form Here different but equivalent forms of the same test are administered to the same group of individuals within a short time and then compared Problem: developing of the equivalent tests can be time consuming may also encounter the same problem as test-retest Split half here items on the instrument are divided into comaparable halves eg dividing a scale so the first half can have the same score as the second part if the 2 halves are equal then the instrument is reliable Problem: ensuring the 2 halves are equivalent Observe reliability involves comparing administrations of an instrument done by different observers and interviewers to be effective the observers need to be thoroughly traines NOTE: the comparing of these results which usually uses some kind of correlation coefficient; a static that measures the extent to which they are related .it can range between 0.0 to 1.0 .1.0 means the instrument is perfectly reliable, 0.8 means it is reasonably reliable. Table 9.3 Criteria for assessing the reliability of measuring instruments Is the variable clearly defined Is retrospective information avoided Are there controlled conditions under which the data are collected Is the question format closed Are reliability tests used? If so, is the correlation coefficient greater than 0.5? If the answer is yes to most of these questions , then the instrument is probably reliable the validity of an instrument reflects the extent to which you are measuring what you think you are measuring it is different from reliability unlike reliability no as straightforward because there are different types of validity each tested differently Three types of validity 1. criterion validity 2. content validity 3. construct validity Criterion describes the extent to which a correlation exists between the measuring instrument and another standard eg to validate an instrument developed to assess a program that helps pregnant teenagers succeed in high school, a criterion such as SAT scores might be used as a comparison. score similarities indicates criterion validity content concerned with the representativeness of the content of the instrument the content in the instrument need to be relevant to the concept being measured construct describes the extent to which an instrument measures a theoretical construct. a measures may have a criterion and content validity and still not measure what it should is difficult to establish because social work variables are difficult to define and are theoretically vague therefore construct validity considers both the instrument and the theory e.g. to measure aggression in preschooler one theory here would be that aggressive children will not be achieving well in class. if the instrument does not reflect this dimension then it has no construct validity solution: to define the construct in small concrete and observable behaviours Table 9.4 1. was the instrument tested for criterion validity 2.Was the instrument tested or content validity 3.Was the instrument tested for construct validity 4.Is the variable defined as clearly and concretely as possible If the answer is yes to most of these questions , then the instrument is probably valid ( that is , if the findings from the tests support the validity of the instrument) Criteria for assessing the validity of quantitative measuring instrument Feedback Important to test the validity of quantitative data collected in interpretive studies. Interpretive studies do not have clearly defined concepts, the intent may well be to elaborate these concepts. the participants should be let to verify the data The feedback can be formal (focus groups and community meetings) or informal (meetings and informal gatherings with the participants. THE AGENCY AND DATA COLLECTION it is better to choose less time consuming instruments like rapid assessments instruments, case records and self –observation the social worker may not be responsible on deciding on the data collection or the instruments. ETHICAL ISSUES IN COLLECTING DATA Three issues are involved in ethical issues 1. potential harm to subjects 2. anonymity and confidentiality 3. justification of the research Harm to participants NASW Code of Ethics (1997) 1. Social workers in evaluation and research should be carefully consider possible consequences and should follow guidelines developed for the protection of evaluation and research participants. Appropriate institutional boards should be consulted. 2. social workers engaged in evaluation and research should protect participants from unwanted physical or mental distress, harm, danger or deprivation these are difficult to observe or avoid because many questions can be comfortable to subjects e.g. questions on income, childhood abuse etc federally funded projects have to be reviewed by human subject committees or institutional review board Anonymity and confidentiality NASW Code of Ethics (1997) Social workers in evaluation or research should ensure the anonymity or confidentiality of participants and of the data obtained from them. Social workers should inform participants of any limits of confidentiality and when any records with research data will be destroyed. Anonymity: -means that the researcher cannot identify a given response with a given respondent interviews can never be anonymous anonymity can increase objectivity in the responses Confidentiality: -the researcher knows the identity of the respondents and their responses but does not disclose this information. Justification of the research NASW Code of Ethics (1997) states Social workers should never design or conduct evaluation or research that does not use consent procedures, e.g. forms of naturalistic observation and archival research, unless a review justifies it because of prospective scientific educational or applied value and unless equally effective alternative procedures that do not involve waiver of consent are not feasible. social workers should also ensure the research is needed and justified. HUMAN DIVERSITY IN COLLECTING DATA There are three issues a. selection of data collection methods for diverse populations b. relevance to diverse populations of the content of the data collection method c. the application of the data collection method to divers populations Selection of data collection methods for diverse populations -extent to which data collection methods may or may not be applicable to certain groups within a population needs to be considered - different cultures may regard different collection methods differently - some authors (Davis 1986) suggest that women’s different ways of thinking require different approaches to data collection techniques; male research methodology – emphasizes the abstract and formal and is easily quantifiable and positivist approach is easily used here. The female approach emphasizes connection and is more suitable to interpretive approach and qualitative data. Relevance to diverse population of the content of the data collection method words or phrases in the questionnaires or interview may have different interpretations by the subject from what the researcher intended this can stem from the researchers insensitivity to the subjects cultural group e.g. in some groups mother may include mother-in-law. this leads to validity and reliability problems Example: Briggs, Tovar, and Corcoran (1996) assessed the validity and reliability of the children’s Action Tendency Scale with Latino youth. They concluded that it may be usefully employed with Latino sixth and seventh graders.( 191) Application of the data collection methods to diverse population involves the way in which the instrument is administered may not be sensitive t the needs of diverse populations to get reliable and valid information , questions should be relevant to the group under study interviews should be conducted by people sensitive to the population under study e.g. be familiar with the group’s language, gender role and intergenerational role expectations in order to engage the subject in the interview and obtain valid and reliable data. Read More