Research Methods in Management - Case Study Example

Outline I. Constructing the Argument A. Introducing the Topic B. Discussion of Research Methods C. Conclusions and Implications II. Research Methods in Management A. Reliability and Validity a. Experimental Designs b. Quasi-Experimental Designs c. Non-Experimental Designs B. Data Collection Methods a. Survey Method b. Focus Group Method c. Interview Method III. Conclusions and Implications a. Importance of Research in Management b. Benefits to staff and consumers c. Benefits to business strategy Introduction Corporate managers find themselves being trained in the skills and competencies of a researcher as well as a leader, giving them insight into staff and process evaluations, as well as giving managers the ability to design research projects, such as into customer or staffs satisfaction. This paper will review empirical processes of research design, collection, analyses and interpretation suitable for 21st century strategic business management. Firstly, reliability and validity of qualitative and quantitative methods will be presented. Secondly, data collection techniques will be discussed. Next, methods of analysis of data will be highlighted. Finally a conclusion will synthesize the main points to demonstrate the importance of knowledge of research methods in management. Validity and Reliability in Qualitative or Quantitative Research Design In experimental research there is the explicit assumption that the universe functions according to causal laws (Creswell, 2003). The purpose of an experimental design is to establish the cause-effect relationship between sets of variables, by way of isolating assumed casual factors, and controlling suspected confounding or extraneous variables. It is hypothesized that an independent variable causes changes in a dependent variable, and that alternate hypotheses can be provided by other factors that are able to influence the results. The design uses random selection procedures to recruit a sample and randomly allocates participants to two or more groups (i.e., treatment group/s and a control group) (Neely, Gregory & Plats, 2005). Due to these random procedures, experimental methods allow for high external validity (generalization of results to a wider population), as the sample is more likely to be representative of that population. Alternatively, a quasi-experimental research design does not use random allocation of participants to groups, instead they are self-selecting (e.g., they have cancer or they do not have cancer) (Bryman, 2002). The quasi-experimental design is used in studies that are unable to control the independent variable, or when it is considered unethical or unfeasible to attempt to control the IV. The two main types of quasi-experimental designs are: 1) the non-equivalent control group; and 2) the pre-post design. Non-equivalent control group designs have both a treatment and a control group, whereas the pre-post design has no comparison group, as each participants serves as their own control in regards to their pre-test data. Due to the lack of random allocation the results of quasi-experiments cannot be generalized to a wider population with as much confidence as with an experimental design. There is also the non-experimental design in which no treatments (i.e., independent variable/s) are given to participants (Bryman, 2002). There is no random selection or random allocation of participants, and so the results of the study are unable to be generalized at all, as no causal relationships can be predicted. These designs tend to be used to investigate naturally occurring phenomenon in which the independent and dependent variables vary without researcher intervention. The advantages of experimental research methods are that the use of quantitative levels measurement (i.e., numerical data), random selection and allocation procedures, and a controlled environment, allow for higher confidence in the results, as well as greater generalizability of the results (Creswell, 2003). The results are more likely to be reliable, in that the study can be replicated and similar results found over time. It is also more likely to have high internal validity, so that the researcher can be confident that the results obtained are a true representation of what is actually occurring. Furthermore, the experiment is more likely to have high external validity, which means that the results can be generalized, allowing them to be applied to other settings and possibly other populations. However, the limitations of experimental methods are that there must be a large enough sample for the results to be representative and so reliable and valid (Creswell, 2003). And, the methods are often intrusive, so that using a lab-based approach requires creating an artificial situation to obtain high internal validity. In contrast, the quasi-experiment can provide a more real-world context in although a drawback of this is regression to the mean effects, so that alternative explanations of the results can be provided for results (Mason, 1997). Also, the method may not represent the true differences between groups in regards to variables that are not measured and that may actually be the reason that the results are the way they are. As such, the quasi-experiment has several risks for its internal validity. Whilst non-experimental methods can contribute a wealth of information in regards to the description of participant experiences on a more subjective level, they lack reliability and validity as they do not identify causal reasons that for the results that occur (Mack et al., 2005). The 21st century has revealed the importance of combining qualitative and quantitative designs, as a mixed-methodology balances the strengths and weaknesses across the different methods, utilizing each relevant to their merits. Data collection choices Turning now to popular data collection methods in business research, survey research is one of the most important areas of measurement. As a statistical tool a survey measures quantitative and qualitative data about characteristics in a population. A set of questions (items) are responded to by participants, and this provides demographic, attitudinal and in some cases, observational data (Rogelberg, 2002). It is highly recommended that a pilot test be done using a small sample of separate users before the actual survey is carried out on a larger sample. This aids the primary investigator in checking the survey for problems and re-wording or deleting items (Mitchell & Jolley, 2004). There are many advantages to the survey method including; Collecting information from a large sample. In turn this enhances the questionnaires reliability, validity, statistical significance and power. Diverse types of information can be collected; surveys are flexible. Investigation of attitudes, values, beliefs, and past behaviours. Controlling for many types of error (i.e., English as a second language, age, gender, ethnicity, exaggeration). Easy to administration. There are however, disadvantages to the survey method; The investigator is reliant on the participant's motivation, honesty, memory and ability to respond. Closed questions, may have a lower validity for affective items. Non-responses can cause error, and so the sample is not entirely random. Another popular research method used by managers is focus groups. Focus groups are a form of group interviewing but it is important to distinguish between the two. Group interviewing involves interviewing a number of people at the same time, the emphasis being on questions and responses between the researcher and participants. Focus groups however rely on interaction within the group based on topics that are supplied by the researcher (Mason, 1997). Hence the key characteristic, which distinguishes focus groups, is the insight and data produced by the interaction between participants. Overall the goal of the focus group is to engage the respondent's to share their attitudes, feelings and experiences in a phenomenological way. In contrast to interviews, the focus group seeks to measure the views and emotional experiences within the group context. Also, during the focus group it is the participants who are in control, the moderator being present simply to guide the proceedings towards discussion of particular topics. The focus group enables a researcher to gain a large amount of data in a short period of time (Mack et al, 2005). A focus group is suited to the initial stages of research, and so is suited to the evaluation of a new website. It is not unusual for the focus group method to complement other data collection methods, in this case, surveys, interviews and documentation. Unfortunately, focus groups are limited in their ability to generalize their results (Bryman, 2004). This tends to be because only a small number of people are able to take part in the group otherwise the group becomes unwieldy. As such, it is unlikely that participants are representative of a wider population. In contrast, a semi-structured in-depth interview can provide a human element to the research process (Bryman, 2004; Shank, 2002). It is a flexible method in that it uses a guide of questions as a framework, whilst allowing the participant to have a free flow of conversation. As such, it provides an opportunity for the participant to talk in-depth about a topic (Mason, 1997). This type of interview provides the researcher with insight into the experiences of participants, from the participant's point of view. The interview is a one-to-one process and allows the researcher to gauge the participant's interpretation of the relationships they see between events, people and social values (Bryman, 2004). The researcher asks neutral questions and listens actively, does not approve or disapprove of answers, and does not encourage the participant to provide particular answers, instead facilitating the participant to follow up on previous answers (Padgett, 2004). Strengths include the gaining of a personal perspective as opposed to a group interpretation of experiences (Shank, 1995). The interview process also allows the researcher to observe the conversational nuances, body language, casual comments and contradictions of participant interpretations (Mason, 1997). The use of audiocassettes to tape record the interviews can increase the accuracy of reporting, transcription and the final interpretation by the primary investigator (Penslar, 1995). However, due to the subjective nature of the interview process, it is inevitable that the information collected will also contain biased opinions and feelings (Shank, 1995). Also, interpretation of the data will be subjective, and so exposed to researcher bias (Patton, 2002). Alternatively, the gender, age, social status or ethnicity of the primary investigator may influence responses inadvertently (Mason, 1997). Hence, reliability for the semi-structured in-depth interview will be low: there is unlikely to be consistency in the responses across the participants; there is a lack of standardisation of the interview process across the sample, in that it is semi-structured and so new questions arise based on the response of the participant; and there is a large amount of subjectivity in the interpretation of the response categories by the primary investigator. Thus, the results of this study will not be able to be generalized to a wider population (Patton, 2002). Analysis Techniques Tests of normality to ensure that the data meets the assumptions for higher order tests would include the use of histograms, stem-and-leaf plots, box plots, normal probability plots and detrended normal plot. Other normality tests would include Kolmogorov-Smirnov test, using a Lilliefors significance level, or the Shapiro-Wilks test. There would also be tests for skewness and kurtosis. The histogram would list vales on a vertical axis to indicate the frequency of cases. The horizontal axis would have values at the midpoints of their ranges. The stem-and-leaf and box plots are similar descriptive statistics, being able to provide information about the actual values in a distribution providing more information than a histogram alone. The stem-and-leaf displays along its length rows that correspond to the number of cases that fall within a particular interval. It represents each case with a numeric value that corresponds to the actual observed value. A box plot summarises information about a distribution but does not plot actual values, instead it plots summary statistics such as the median, 25th and 75th percentiles and the upper boundary of the 75th percentile. The median is shown as a horizontal line through the box. The smallest and largest observed values are represented as horizontal lines at either end of the box, also known as whiskers. A normal probability plot has each observed value paired with its expected value in a normal distribution. If the sample is normal then cases will fall close to the straight line. To plot the deviations of values from the straight line there will be no pattern of clustering, instead they should cluster around the horizontal line of a detrended normal plot. The Kolmogorov-Smirnov is the product of normal and detrended probability plots, a significance level greater than .05 ensures normality. The same goes for a Shapiro-Wilks test if the sample is less than 100. Skewness and kurtosis refers to the shape of the distribution, and can be used for interval and ratio data. If the distribution is normal than these values will be close to zero. Also, measures of central tendency (mode, median and mean) as well as variability (standard deviation) can also be used to determine normality. Thematic analyses (i.e., finding dominant themes from the interview), can be used for interviews and focus group data analysis (Buckley & Chapman, 1996; Huff, 1991). Following the interview/focus group the written notes of the interviewer/facilitator are summarized, and assessed for their relevance to the questions asked. Comments on observations made during the interview/focus group can also be included. The taped interviews can be transcribed verbatim onto a word document. Two raters, who are unaware of the research hypotheses, independently identify key themes in the transcripts. Inter-rater reliability is checked, and themes with at least a moderate agreement (r = .60) are included for further analyses (Shank, 2002). The key themes are then organized into broad categories and specific sub-categories and labels provided respectively (Patton, 2002; Shank, 2002). Frequencies and percentages can be obtained via basic descriptive analysis of all variables. The Chi square test can be used for all categorical variables (e.g., gender, job role). Spearman's correlation can be used for ordinal data that does not have a normal distribution. Continuous levels of measurement that have a normal distribution can be analyzed with Pearson's r correlation, t-tests, ANOVA or multivariate tests (e.g., for age, number of days off work) (Mitchell & Jolley, 2004). Correlation tests indicate the relationship between two variables in a linear manner. Partial correlations provide a single measure of linear association between two variables whilst adjusting for the effects of a third or more other variables. T-tests determine whether there is a significant difference between two sets of scores. For three or more sets of scores ANOVAs are used. Basically, the ANOVA aims to derive two different estimates of variance of the population from the data and then calculates a statistic from the ratio of these estimates. The between-group variance is a measure of the effect of the independent variable combined with error variance. The within-group variance is the error variance alone. Factor analysis is a multi-variate test to reduce data to a small set of underlying factors that summarise the essential information within the variables. Multiple regression extends on bivariate correlation, producing an equation that represents the best prediction of a dependant variable from several independent variables. A MANOVA is an extension of an ANOVA that uses multiple dependant variables. Conclusion In conclusion, the topic of research methods in management is of importance because empirical and systematic evaluation of an organization provides qualitative and quantitative data that can contribute to planning strategies and operating processes, and to providing insight into employee and customer satisfaction. A manager who has a clear understanding of the principles and assumptions involved in research design, collection, analyses and interpretation, can confidently make business choices that are informed by up-to-date, relevant, reliable and provide accurate information. References Bryman, A (2004) Social Research Methods (2nd ed.). Oxford University Press, Oxford. Buckley, P. J. & Chapman, M. (1996) "Theory and method in international business research". International Business Review, 5(3), 233-245. Creswell, T. (2003) Quantitative, Qualitative and Mixed Methods. New York: Sage Publications. Guion, I (2001) Conducting an In-Depth Interview. Family Youth and Community Sciences Department Document, FCS6012. Huff, Darrell (1991; first printed 1954) How to lie with statistics. New York: W. W. Norton & Company. Mack, N, et al. (2005) Qualitative Research Methods: A data collector's field guide. North Carolina, Family Health International. Mason, J (1997) Qualitative researching: An introductory text. New York, Sage Publications. Mitchell, M. L., & Jolley, J. M. (2004) Research Design Explained. Boston: Wadsworth Publishing. Neely, A., Gregory, M., & Platts, K. (2005) Performance measurement system design: A literature review and research agenda. International Journal of Operations & Production Management, 25(12), 1228 - 1263. Padget, D K (2004) The qualitative research experience. Thompson Learning: Southbank. Patton, M Q (2002) Qualitative Research and Evaluation Methods. Thousand Oaks, Sage Publications. Penslar, R. L. (1995) Research Ethics: cases and materials. Indiana University Press, Bloomington. Rogelberg, G. (2002), Handbook of Research Methods in Industrial and Organizational Psychology. New York: MacMillan. Shank, G. D. (2002) Qualitative Research: a personal skills approach. Columbus: Merrill/Prentice Hall. Bibliography Alvesson, M., & Skoldberg, K., (2000) Reflexive Methodology: New Vistas for Qualitative Research. New York: Sage Publications. Aczel D. A. & Sounderpandian J. (2005) Complete Business Statistics, 6th edition, New York: McGraw Hill. Bailey, J. S., & Burch, M. (2002) Research Methods in Applied Behavior Analysis. Sage Publications. Bernard, R. (2000) Social Research Methods. Sage Publications. Buckley, P. J. & Chapman, M. (1996) "Theory and method in international business research". International Business Review, 5(3), 233-245. Bryman, A. (2004) Social Research Methods (2nd ed.). Oxford University Press: Oxford. Creswell, T. (2003) Quantitative, Qualitative and Mixed Methods. New York: Sage Publications. Ghauri, P & Gronhaug, K. (2005) Research Methods in Business Studies. Prentice Hall. Gliner, J. A., Morgan, & G. A. (2000) Research Methods in Applied Settings: An Integrated Approach to Design and Analysis. Lawrence Erlbaum Associates. Guion, I. (2001) Conducting an in-depth interview. Family Youth and Community Sciences Department Document, FCS6012. Huff, Darrell (1991; first printed 1954) How to lie with statistics. New York: W. W. Norton & Company. Mack, N, et al. (2005) Qualitative Research Methods: A data collector's field guide. North Carolina, Family Health International. Mason, J. (1997) Qualitative researching: An introductory text. New York, Sage Publications. Maxim, P. S. (1999) Quantitative Research Methods in the Social Sciences. Oxford University Press. Mitchell, M. L., & Jolley, J. M. (2004) Research Design Explained. Boston: Wadsworth Publishing. Neely, A., Gregory, M., & Platts, K. (2005) Performance measurement system design: A literature review and research agenda. International Journal of Operations & Production Management, 25(12), 1228 - 1263. Padget, D. K. (2004) The Qualitative Research Experience. Thompson Learning: Southbank. Patton, M. Q. (2002) Qualitative Research and Evaluation Methods. Thousand Oaks: Sage Publications. Penslar, R. L. (1995) Research Ethics: Cases and materials. Indiana University Press: Bloomington. Rogelberg, G. (2002) Handbook of Research Methods in Industrial and Organizational Psychology. New York: MacMillan. Shank, G. D. (2002) Qualitative Research: a personal skills approach. Columbus, Merrill/Prentice Hall. Read More