Validity, Reliability and Generalisation in Business and Management - Essay Example

VALIDITY, RELIABILITY AND GENERALISATION] Introduction Management research is essential because it contributes to theory. The methodology that is used in management research should be justifiable in terms of yielding more accurate and believable results. Therefore, validity, reliability and generalisation should be considered when carrying out research because they play a very significant role in business and management research, as critically established in this essay. This essay critically evaluates the role of validity, reliability and generalisation in management research. The essay also explains how validity, reliability and generalisation assist with the design of a questionnaire aimed at finding out why post graduate students choose to study HRM. In addition, the essay provides recommendations, which include a series of questions that could be incorporated into a questionnaire aimed at finding out why post graduate students choose to study HRM. 1. The Role of Validity, Reliability and Generalisation in Business and Management Research In management research, validity, reliability and generalisation should be addressed in research with a lot of caution because it is tricky to distinguish them precisely. For instance, validity is defined as the relationship between test results on research with other objectives that the study seeks to achieve or measure (Wainer & Braun, 2013: 40). Therefore, researchers should clearly outline their research objectives. On the other hand, Schensul, LeCompte and Schensul (1999: 271) define reliability as the consistency of research results and the ability of such results to be replicated by other researchers. It is worthy to note that a measure may be reliable, but such a measure’s reliability does not ensure its validity (Rubin & Babbie, 2010: 87). It is imperative that business and management researchers should consider reliability and validity separately. This is because reliability is about consistency while validity is about truthfulness in measures (Jackson, 2013: 90). On the other hand, seeking to ensure reliability in management may distort the purpose of a study. This is because a researcher will design a measurement tool or process that will ensure the results obtained from such a study will be replicated by other researchers. This leads to a researcher’s slight deviation from the original purpose of the study, which may in turn, adversely affect validity. Separately, generalisation may be used in businesses and management research to contribute to theory. Therefore, in consideration of generalisation, management research should be designed so that it is properly conversant with theory to contribute to the already established theory (Rowley, 2002: 20). For instance, case study research uses an already developed theory, which business and management researchers compare with the findings of a case study. Therefore, when a sufficient number of cases are found to support the theory, then such research findings can be replicated (Rowley, 2002: 20). However, due to the changing times and variables, generalisation may not be given its due significance in some studies such as carrying out a study for the first time, or research on a topic which has never been addressed before. Therefore, seeking to ensure generalisation in a study may alter the main objectives of the study. Validity and reliability enable business and management researcher to observe human behaviour. Most business and management research is carried out on human behaviour (Gallagher, 2013: 102). For instance, market research is concerned with consumer behaviour while human resource research focuses on employee behaviour, in terms of job satisfaction and supervision. Therefore, when validity and reliability are incorporated in the research process, human behaviour may be observed with a greater degree of accuracy because appropriate measurement tools are used. Measurement instruments for behavioural research ought to be valid and reliable (Drost, 2011: 105). However, validity, reliability and generalisation may not lead to effective management research pertaining to human behaviour as human behaviour cannot be measured accurately because it fluctuates with circumstances such as motivation. Therefore, reliability may not be achieved, though the findings of such a study may be true or valid. Reliability is essential in business and management research because it enables researchers to ascertain the consistency of measurement tools used in research. According to Drost (2011: 108), reliability enables researchers to find out whether measurement is consistent over time or whether it is stable in various conditions to contribute to valid findings, inferences and conclusions. Validity enables business and management researchers to determine the meaningfulness of the components of research. Therefore, it enables researchers to determine whether what they are measuring while carrying out research is what they intended to measure. Various questions of validity are asked by researchers to support or establish the validity their measurements, in spite of the fact that completely certain responses to such questions cannot be found (Drost, 2011: 114). For example, a researcher can ask questions such as; does a relationship between two variables exist? Is the relationship between two variables causal or not? For instance, if a researcher studying employee satisfaction just before Chrismas when all the employees have been awarded bonuses, he should query whether those results are reliable. Validity is about having the greatest chance of getting findings on what research seeks to find out, correctness or accuracy while reliability is the ability of research findings to be replicated. It enables business and management researchers to evaluate case studies properly so as to improve quality, reliability and trustworthiness (Andersen, Hansen, & Stern, 2012: 1). Validity, especially external validity may be used by business and management researchers to determine whether their assessment scores’ relationships replicate interactive, high or low relationships with other measures of behaviours. Therefore, researchers are able to use rationale for intended and unintended consequences evaluation (Joughin, 2009: 97). It is however, important to note that management researchers find it hard to distinguish the type of validity that is being required. Some management researchers may not have sufficient scientific knowledge in designing measurement processes that achieve a specific type of validity. Therefore, validity may not be ‘real’, especially face validity because it may be affected adversely by the subjectivity of the researcher. Based on this, a researcher may make a measurement tool or procedure to appear that it is valid because it cannot be quantified, but in the reality the research may not be valid. On the other hand, generalisation is about how studied cases or research findings can fit other situations. Therefore, business and management researchers have to design their research process to allow research findings to be generalised to the population. According to Folk and Guenther (2013: 2), generalisation is the ability of research findings to be applied to other population to a greater extent. When findings have a high degree of reliability, business and management researchers are able to study behaviour with confidence. In quantitative research, reliability facilitates the enhancement of quality of research because it enhances explanations. In qualitative research, reliability generates understanding to increase the quality of research. Qualitative research involves seeking of valid explanations about phenomena, facilitating the significance of reliability. Therefore, researchers have to take into consideration, the aspects of validity and reliability while analysing results so as judge the quality of their research appropriately (Golafshani, 2003: 601). In consideration of validity and reliability, business and management researchers are capable of ensuring that they verify consistency of data by examining raw data thoroughly and employing rigorous research approaches to ensure that their research is of good quality (Golafshani, 2003: 601). However, this may require that researchers have to tailor data to ensure consistency, which may be contrary to the aim of carrying out research to obtain factual and unaltered data. In business and management research, researchers should use a valid measure. Therefore, validity as a research element helps researchers to determine the valid measure that they can use in their research. In addition, reliability and validity can be used in questionnaires by business and management researchers by phrasing questions so that the questions actually measure what they are intended to measure (Searle, 2002: 56). Therefore, it worth noting that the concepts of validity, reliability and generalisation determine the research design that researchers employ in business and management research (Quinton & Smallbone, 2006: 126). Validity, reliability and generalisation determine the sampling method to be used when collecting data both quality and quantitative (Brown & Remenyi, 2004: 343). However, in their bid to achieve validity, at times, management researchers are forced to design a measure that disguises its purpose so as to ensure that it measures what it intends to measure. For instance, when collecting information pertaining to prejudice and impartial treatment in the work place, based on race, it is expected that the subjects or informants may alter their responses, depending on how they believe the interviewer and other people perceives them, which means that there is no point of designing such a measure. In addition, when a researcher uses closed questions, validity is lowered. 2. How the Role of Validity, Reliability and Generalisation in Business and Management Research assists with the Design of a Questionnaire aimed at finding out why Post Graduate Students choose to study HRM The role of validity, reliability and generalisation in business and management can assist in designing questionnaires for qualitative research is imperative in research design. For instance, incorporation of the concepts of validity, reliability and generalisation, may assist in designing questionnaires for data collection. The design of a questionnaire aimed at finding out why post graduate students choose to study HRM can be influenced by validity, reliability and generalisation in various ways. First, a questionnaire should be a valid measure of the research. Therefore, in consideration of validity, reliability and generalisation, the type of questions to be included in the questionnaire is determined. These questions should be framed to ensure that the respondent can only give one genuine answer (Kumar, 2003: 63). Validity, reliability and generalisations assists in designing the components of the questionnaire that seeks to find out why post graduate students choose to study HRM, in terms of the questions to be asked. Consideration of the concepts of validity, reliability and generalisation makes the researcher to take sufficient caution in determining the layout of the questionnaire and the way in which questions will be asked to increase the level, accuracy and completeness of the responses. Success of a questionnaire is determined by appropriate design of questions, and these should be formulated prior to the presentation of the questionnaire for data collection (Foddy, 1996: 232). Great caution is required when formulating questions pertaining to the choice of post graduate students to study HRM so as to collect accurate and complete responses. Therefore, the questionnaire should have questions that can be easily understood by the respondents to increase the accuracy and completeness by which informants will respond. According to Becker (2002: 30), a questionnaire that incorporates the concepts of validity, reliability and generalisation should be formulated to have clear and simple questions, which will be understood by all the respondents. Validity, reliability and generalisation can assist in the design of a questionnaire aimed at finding out why post graduate students choose to study HRM by enhancing and determining the development and selection of questions which will provide information that will facilitate an analysis that is related to the purpose of the research directly. Therefore, in consideration of validity, reliability and generalisation, researchers can formulate relevant and analysis facilitating questions regarding the reasons of post graduate students choosing to study HRM. In consideration of validity, reliability and generalisation, it will be possible to obtain the truth of the situation under study by designing questions that relate to the subject and employing appropriate methods. This will enable the research to get knowledge, explanations and understanding (Taylor, Sinha study HRM is sought and the concepts of validity, reliability and generalisation can aid in finding out a whether the research is able to find out what it claims to be assessing through the research method employed and the questions included in the questionnaire (Taylor, Sinha, & Ghoshal, 2006: 3). Validity, reliability and generalisation concepts can aid in designing a questionnaire aimed at finding out why post graduate students choose to study HRM by assisting researchers to ascertain whether the questions included will enable the study achieve its objectives (Taylor, Sinha, & Ghoshal, 2006: 3). Validity, reliability and generalisation will also assist in designing a questionnaire aimed at finding out why post graduate students choose to study HRM by providing the criteria by which the researchers have to judge their choice of the research methods. Validity, reliability and generalisation can also enable researchers to use valid method so as to get true differences between individuals or groups. In this case, the truth about why post graduate students choose to the ways in which to carry out interviews or use the questionnaire to get valid information (Taylor, Sinha, & Ghoshal, 2006: 3). By considering the concepts of validity, reliability and generalisation, researchers can be able to revise the questionnaire, in a bid to improve it so that the research’s validity and reliability requirements are met (Bashir, Afzal, & Azeem, 2008: 37). This will lead to the use of an appropriate questionnaire aimed at finding out why post graduate students choose to study HRM. 3. Recommendations and a Series of Questions that could be incorporated into a Questionnaire aimed at finding out why Post Graduate Students choose to study HRM It is important for the researcher who is responsible for the design of the questionnaire to be thoroughly familiar with the questions to be asked (Iarossi, 2006: 27). This requires that the researcher should know the exact objectives of the questions and the type of information sought by the study. In this case, the objective of the study is to establish why post graduate students choose to study HRM while the type of information sought may be opinions and facts pertaining to the reasons as why post graduate students choose to study HRM. Given that a good question ought to be relevant and accurate, it is important that the questions are designed and worded in a way that makes it easier for a respondent to provide accurate information (Miller, 1998: 94). Also, the style in which the questions are asked and arranged in a sequence should motivate the respondent and allow him or her to recall (Iarossi, 2006: 27). While carrying out a qualitative study in business and management, researchers should prioritise identifying questions which will generate a high rate of screening and improving them, before concluding the questionnaire because this will be beneficial in reducing errors in surveys (Foddy, 1996: 186). Business and management researchers should ensure that they make the questionnaire an interesting one to the respondents by making the questions interesting (Dörnyei, 2003). Researchers should also remove items that may pose difficulties to the respondents in answering, as well as, those that consume much of the respondents’ time. To achieve this, it is imperative for business and management researchers to have an idea about what the research seeks to find out or analyse so as to design the questionnaire to accomplish the aims of the study (Foddy, 1996: 40). Appropriate questions which can lead to collectionof accurate information cannot be formulated when the researcher does not know the research objectives. The main aim of carrying out a study is to acquire information. Therefore, business and management researchers should ensure that they tailor questions in such way to suit the context of the respondents so as to increase the accuracy in responses (Adèr, Mellenbergh, & Hand, 2008: 194). Also, the order in which researchers arrange their questions is very important. Most relevant questions should come first, while the least significant questions will be last (Adèr, Mellenbergh, & Hand, 2008: 195). Where there are unpleasant questions, they should be put at the end of the questionnaire. Therefore, the first question should be non-objectionable and salient. This means that the question should apply to every respondent, be easy to answer and be interesting (Adèr, Mellenbergh, & Hand, 2008: 195 and Fanning, 2005: 5). Avoiding questions written in a complex way will encourage respondents to read the questions fully, rather than rushing to provide an answer before they complete reading the question (Survey Monkey, 2011: 5) (Becker, 2002). Adams and Cox (2008: 18), emphasise on the importance of designing questionnaire which the respondent can easily understand, interpret and complete because this is what increases the accuracy of responses. Reliability and validity should be considered during the design process of a questionnaire (Adams & Cox, 2008: 18). It should not be so long because this will make the respondent fill it hurriedly so as to finish hence the responses may be less accurate. Respondents may fail to read the questions comprehensively and as a result they may fail to interpret complex questions (Adams & Cox, 2008: 19). Therefore, questionnaires should use a language that is simple and can be easily understood by respondents (Malhotra, 2013: 190). Only one item should be asked by each question to avoid any discrepancies in the responses (Office of Planning and Institutional Assessment, Pennsylvania State University, 2006: 4). According to Oracle (2012: 3), questions should be simple and straight forward. Business and management researchers should ensure that they do not include built-in assumptions while formulating the questions to be included in the questionnaire (Taylor-Powell, 1998: 4). This will help to reduce bias and will ensure that the data collection instrument used, which is the questionnaire, is valid and reliable. Also, business and management researchers should try as much as possible to avoid leading questions which lead to bias (Survey Monkey, 2011: 6). The questions to be included in a questionnaire should not offer any clue or hint to the respondent on how she or he is supposed to respond to the question. Some of the questions that should be included in a questionnaire aimed at finding out why post graduate students choose to study HRM include; What is the name of your institution? Are you a postgraduate student, or do you intend to study a postgraduate course? What postgraduate course are you studying, or are you intending to study? If you are studying HRM, did you make the choice to study HRM yourself? What were the reasons that made you choose to study HRM? If some other person advised you to study HRM, what reasons did s/he cite as to why you should study HRM? If you are studying some other course, why do you think post graduate students choose to study HRM in your institution? Given opportunity, will you consider studying HRM? What are the reasons why you would choose to study HRM? The aforementioned questions will enable the researcher to know the institution to which the respondent belongs. This will help the researcher to determine whether responses came from various locations to increase the reliability of the research. The questions will also enable the researcher to know whether the respondent is a postgraduate student studying HRM or another course or a student intending to study a postgraduate course. By asking respondents who are currently studying HRM, why they chose to study HRM, the researcher will get valid and reliable responses. There is a higher degree of expectation that these responses will be valid because they are most likely to be true, given that the questions are open. Valid and reliable responses can also come from post graduate students who are not studying HRM, but may know why their colleagues choose to study HRM in their institution. Therefore, the researcher has to seek such students’ opinions by asking them why they think those post graduate students who are studying HRM chose it so as to increase the validity and reliability of research findings. These questions are relevant and will enable the researcher to gather the required information. Conclusion From the critical evaluation, it has been established that validity, reliability and generalisation are vital elements that should be incorporated in management research. It has also been noted that it is quite tricky to establish distinction among these three elements. Therefore, sufficient caution should be taken when carrying out research, which incorporates these elements. The elements of validity, reliability and generalisation should be considered separately in management research, but researchers should ensure that their findings meet the three criteria. It has also been noted that validity, reliability and generalisation play a role in designing a questionnaire, which is aimed at finding out why post graduate students choose to study HRM. Finally, by taking validity, reliability and generalisation, into consideration, it is possible to formulate accurate questions for a research questionnaire, for collecting information on the aforementioned topic. References Adams, A., & Cox, A. L. (2008). Questionnaires, In-Depth Interviews and Focus Groups. In: Cairns, Paul and Cox, Anna L. eds. Research Methods for Human Computer Interaction, Cambridge, UK: Cambridge University Press, 7–34. Adèr, H. J., Mellenbergh, G. J., & Hand, D. J. (2008). Advising on Research Methods: A Consultants Companion. Huizen: Johannes van Kessel Publications. Andersen, O. W., Hansen, H., & Stern, E. (2012, August ). Case Studies in Development Evaluation: Validity, Generalisation and Learning Highlights of an international Workshop, Copenhagen May 2012 [Online] available from [13 February 2014] Bashir, M., Afzal, M. T., & Azeem, M. (2008). Reliability and Validity of Qualitative and Operational Research Paradigm. Pakistan Journal of Statistics and Operational Research , Vol.4, No.1, 35-45. Becker, A. (2002). Network Analysis and the Organisation of Events. New York: Diplomarbeiten Agentur Press. Brown, A., & Remenyi, D. (2004). European Conference on Research Methodology for Business and Management Studies. Reading: Academic Conference Publications. Dörnyei, Z. (2003). Questionnaires in Second Language Research: Construction, Administration and Processing. New Jersey: Routledge Press. Drost, E. A. (2011). Validity and Reliability in Social Science Research. Education Research and Perspectives, Vol.38, No.1, 105-123. Falk, I., & Guenther, J. (2013). Generalising from Qualitative Research: CaseSstudies from VET in Contexts [Online] available from [13 February 2014] Fanning, E. (2005). Formatting a Paper-based Survey Questionnaire: Best Practices. Practical Assessment, Research & Evaluation , Vol. 10, No. 12, 1-14. Foddy, W. (1996). Constructing Questions for Interviews and Questionnaires: Theory and Practice. Cambridge: Cambridge University Press. Gallagher, K. (2013). Skills Development for Business and Management Students: Study and Employability. Oxford : Oxford University Press. Golafshani, N. (2003). Understanding Reliability and Validity in Qualitative Research. The Qualitative Report, Vol.8, No. 4, 597-607. Iarossi, G. (2006). The Power of Survey Design: A Users Guide for Managing Surveys, Interpreting Results and Influencing Respondents. Washington, D.C: World Bank Press. Jackson, S. (2013). Research Methods: A Modular Approach. New York: Cengage Learning Press. Joughin, G. (2009). Assessment, Learning and Judgement in Higher Education. Dordrecht: Springer Press . Kumar, A. (2003). Research Methodology in Social Science. New Delhi : Sarup & Sons Press. Malhotra, N. K. (2013). Questionnaire Design and Scale Development [Online] available from [13 February 2014] Miller, G. J. (1998). Handbook of Research Methods in Public Administration, Second Edition. Madison: CRC Press. Office of Planning and Institutional Assessment, Pennsylvania State University. (2006). Using Surveys for Data Collection in Continuous Improvement. Innovation Insight Series, No.14, 1-7. Oracle. (2012, March ). Best Practices for Improving Survey Participation [Online] available from [13 February 2014] Quinton, S., & Smallbone, T. (2006). Postgraduate Research in Business: A Critical Guide. Thousand Oaks: SAGE Publications. Rowley, J. (2002). Using Case Studies in Research. Management Research News, Vol. 25, No.1, 16-27. Rubin, A., & Babbie, E. (2010). Essential Research Methods for Social Work. Belmont: Cengage Learning Press. Schensul, S. L., LeCompte, M. D., & Schensul, J. J. (1999). Essential Ethnographic Methods: Observations, Interviews, and Questionnaires. Walnut Creek: AltaMira Press. Searle, A. (2002). Introducing Research and Data in Psychology: A Guide to Methods and Analysis. New York: Routledge Press. Survey Monkey. (2011). Smart Survey Design [Online] available from [13 February 2014] Taylor, B., Sinha, G., & Ghoshal, T. (2006). Research Methodology: A Guide for Researchers in Management and Social Science. New Delhi: Prentice-Hall of India Press. Taylor-Powell, E. (1998). Questionnaire Design: Asking Questions with a Purpose. Program Development and Evaluation, 1-29. Wainer, H., & Braun, H. I. (2009). Test Validity. New York: Routledge Press. Read More