Strengths and Weaknesses of the Published - Research Paper Example

Business Research Methodology Introduction: This is a critical analysis of recently publish journal paper on business research methodology. This analysis will focus on the strengths and weaknesses of the published paper. The paper chosen here is – “Applying best-worst scaling to wine marketing by Eli Cohen” The methodology used in the paper is focused on quantitative analysis using a rating scale. Rating scales are widely used by marketing researchers and analysts to study consumer preferences. Rating scales could however have many problems in the use although these problems could be largely avoided with BWS, best-worst scaling method. The steps to design and analyze a BW study would be related to the overall efficacy of the method and how the method could be used to explain and understand buying trends, marketing issues, consumer spending and behavior and consumer preferences and one of the examples used in the study has been related to the consumer preferences and buying behavior among Australian and Israeli consumers who choose wine in a retail store. The comparative buying behavior has been provided and consumer preferences of both the countries are compared and highlighted with the best worst scaling method. The Best worst scaling method also known as the BWS method could be understood best with evidential and empirical study when the BWS has been used to study consumer behavior or spending in a specific case study. Considering the necessity of evidential and empirical case study, the comparison between Australian and Israeli consumers has been drawn considering their wine purchase behavior and the study also demonstrates the steps to design and analyze a Best-Worst scaling study which is one of the popular methods to study consumer buying behavior, preferences and spending patterns. The BW rating scale provides rating as well as rankings and the study in this case has been conducted with the help of a questionnaire and the results have been analyzed. The BWS approach has been considered as especially suitable and applicable to wine businesses and especially when multicultural comparisons are drawn as in this case, statistical methods are applied for determining the best and worst scores for an attribute and choice probabilities are determined. The BW scale aids in comparison of attributes using B_W and B/W scores and a discrimination or differentiation of the attributes analyzed is also more effective. Probabilities or attributes are easy to understand with this method and the graphical presentation makes the study rather clear and concise. The analysis with the BW scaling system is rather simple and easy to understand and follow. The BWS is a form that researcher can use for research and market surveys and questionnaires and attain information with the method of scaling. In the particular study analyzed, the BWS method determines wine cues which consumers choose in a retail store. The strengths and weaknesses of the study are determined following an analysis of the methods used and the manner in which data have been collected and analyzed across two different cultures studying consumer preferences and behavior dealing with wine selection in retail stores. Analysis: One of the most popular and typical forms of scaling is the Likert type scale which is a rating scale on which survey respondents or participants are asked to mark or tick their preferred rating for a particular attribute. Each response category is then labeled or marked and the rating for each attribute is explained from high to low rating measures or endpoints. In some cases scale categories are described with labels such as important or fair or good etc. Survey respondents may not use ratings the same way across respondents, and hence, the meaning respondents associate to the categories influences the perceived distance between categories although these distances may be objective to the researcher, they may be quite subjective to the respondents and depends on their individual perception (Crask and Fox, 1987). Cohen writes “treating these adjectival descriptors or categories as equal interval scales may generate different conclusions than if they are treated as ordinal scales. Furthermore, people may limit their responses to certain parts of a rating scale (Couch and Keniston, 1960; Bachman and O’Malley, 1984) and different parts of the scale are used more often than others by different cultures” (Cohen, 2009, p.1752). The above passage suggests that Cohen already admits the possible lack of validity in this method of scaling as despite using a scaling method that is supposed to highlight the objectivity of a study, the scaling method and the responses obtained through the method could be largely dependent on individual perception and the way individuals interpret these scales. As aptly suggest, ‘the distance between four and five for one person may be different that another’s’. Apart from subjective perceptions, results obtained in rating scale methods could be subject to other kind of biases s seen in different population from various ethnic or regional backgrounds. There may be ratings among individuals which are too similar or too different although the similarities and differences may not be objectively measured. The study has suggested that resident of different countries will differ significantly in their responses although Cohen (2003) has claimed that segmentation studies in international markets show differences which may be due to differences in scale usage rather than to real differences in consumers’ preferences. Cohen seems to have blamed the differences as due to differences in scaling and methods as used I different cultures rather than subjective consumer preferences suggesting that this method itself could be objective if scales are used properly and that consumer preferences could be objectively measured with the same scale and same scaling method and techniques. However it is controversial, how far Cohen’s claim is true as subjective differences in preferences could happen at individual or ethnic levels and the objectivity of the scaling method is questioned so any objectivity that the study would claim would largely depend on how far these objectivity claims could be established by researchers and what are the procedures to suggest that the scaling method is in fact objective. Considering that different subjects from similar or dissimilar backgrounds are subject to a range of biases, their perceptions might have an impact on scores and the ratings systems used or approved by them. The relative importance of each attribute is derived based on the average score given by the respondents and thus reliable conclusion cannot be drawn on the relative importance of attributes that are measures on the scales as respondents may widely vary in their relative importance factor as some tend to give more importance to certain attributes and thus subjectively give a higher or lower rating as the case may be than other respondents. “Individuals might like a particular attribute and depending on their preferences they might rate the specific attribute on a scale. As Cohen has used citation from other studies, ‘some individuals truly might like (or dislike) nearly every attribute or combination of them and they rate them as ‘‘important’’ (or ‘‘not important’’). Such responses do not provide adequate discrimination to help managers identify real priorities (Finn and Louviere, 1992)” (from Cohen, 2009; p.1754)”. However the rating scales are adequately used in situations or studies when the number of attributes is relatively small and attributes are required to be ranked in terms of their specific characteristics, there could be wide variations in the ratings depending on how individuals view the traits and especially the characteristic and this personal perception could vary widely. Ranking of attributes could be done by simple scaling, rating or marking process and in this study, the rating scale has been successfully used although the significance of this is also based on subjective perceptions not that just of the respondents but also of the researcher. Ranking of attributes are subject to researcher and respondent or participant biases and Cohen suggests that such pitfalls are avoided if the ranking tasks are simplified by using paired comparisons and this could be one of the most reliable methods of ranking. ‘‘even a child who is unable to understand a rating scale could perform a series of paired comparisons reliably’’ (Cohen and Orme, 2004). In paired comparisons, respondents are simply asked to choose which is ‘‘more’’ important (the other one is the ‘‘less’’ important) of two items or which is a better version or preferable among two items that are presented in the study. Assuming n items, the number of possible pairs is n(n_1)/2. The disadvantage of the paired comparison is that if there are many items then possible number of pairs will increase significantly and this can create confusion in the study making it a poor or ineffective method. Thus paired comparison is only effective for a small number of items that have to be compared. In the case of comparing wines band consumer preferences in Australia and Israel if the number of consumer items that are to be paired and compared are low, then paired comparisons could well be the best method as it is easy to evaluate, tabulate and analyze. According to Cohen, the ‘‘BW’’ method is an extension of paired comparisons. The BWS method models cognitive processes and respondents identify the highest and least or best an worst characteristics of attributes presented to them or the best and worst attributes are marked on a scale and the design is usually for three or more items. This method of BW scaling seems to have some advantages where respondents identify and compare the best and worst attributes on a scale and this scaling method may have certain advantages when compared with other rating based measures. The method used by Cohen for the study of wine preference and purchase in Australia and Israel draws on comparison between consumer preferences and behavior in these two countries. The BW approach, also known as maximum difference scaling, was developed by Louviere and Woodworth (1990) and is an extension of Thurstone’s (1927) random utility theory for paired comparisons (Cohen 2009). Respondents can only choose one most and one least preferred item in each choice set (Cohen, 2003). According to Cohen, this method overcomes the issue of many items having similar importance weights or items that could have similar ratings are avoided as the clear least preferred and most preferred choices are indicated for each choice set so there are no duplications and trade offs. According to Cohen, BW also avoids the problem of rating bias, as there could be only a single method by which the least and most preferred items or attributes could be adequately chosen and this would be independent of the cultural background of the respondents thus suggesting more objectively and less confounding variables in the study of the preferences and choices. Cohen thus chose this method and has justified why the BW scaling method with its lack of possible biases and effects of cultural differences would be the most suitable method to study cross cultural similarities and differences in consumer habits, preferences and behavior and in this case the behavior is based on the nature of wine buying and preferences for specific types of wine across Australia and Israel. This is thus a largely comparative study based on cultural variations and wine buying and consumer preferences across two countries with very different culture, history and preferences. The BWS methodology has been successfully applied to understand consumer preferences or buying behavior in healthcare or to study country related difference sin attitudes regarding human rights, animal rights or child labor (Cohen 2009). However Cohen’s study dealing with wine marketing could be considered as unique. After collection of data on preferred choices and attributes, the data are subject to statistical analysis which is then used for further interpretation of the BW method and its applicability of studying cross cultural wine preferences as understood from this study. For assessing the quality of any wine the properties that are considered would be price, brand, region, medal, variety and intrinsic cues such as taste, bouquet and other sensory characteristics of the wine (Cohen 2009, p.1760). The brand is one of the most important attributes that consumers assess while purchasing any product or service (Gordon, 2002) and in case of wine purchase specifically, the region and the origin of the wine is an important cue in wine choice and influence wine purchase (as cited in Cohen 2009). The 13 attributes which were identified from previous studies and chosen to represent the range of choice cues in wine selection are given here: Attribute: 1 Promotional display in-store 2 Grape variety 3 Origin of the wine 4 Information on the shelf 5 Alcohol level below 13% 6 Matching food 7 Information on back label 8 Medal/award 9 An attractive front label 10 Brand name 11 Someone recommended it 12 I read about it in a guide 13 Tasted the wine previously The number of times as an item is chosen as the best possible attribute and the number of items it is chosen as the worst possible attribute in determining wine preferences are subtracted to get the choice sets of preferences for attributes. In this case a large number of items could be studies despite the paired comparison method as this study seems to move beyond the traditional pitfalls of paired comparisons. There could be multiple or repeated comparison when respondents are asked to rate on various different combinations but comparing each item with each other item and frequently although increases the internal validity of the survey and makes the comparison method more comprehensive and effective, could make the entire study longer and more repetitive for the respondent. This is best avoided and in Cohen’s study the attributes are initially identified with the help of literature review after which the design of the study is chosen, presented and finally results and analysis are presented to suggest consumer choices for wine. In Cohen‘s study, it has been realized that the decision of purchasing wine depends on the consumption situation (Hall and Lockshin, 2000) so the subjects were asked to choose the ‘‘best’’ (most important) and ‘‘worst’’ (least important) reason why the subject purchased a bottle of wine and why a specific bottle or wine was preferred during the process of purchase. The research question used by Cohen was thus what were the best and what were the worst attributes that consumers used to determine their choice of wine when buying wine and in this case the cross cultural study has been done between consumers in Australia and consumers in Israel and comparisons were made on their worst and best reason for buying a specific type of wine. For this study data were collected in Australia during 2006 and the subjects were selected by convenience sampling outside retail wine stores and from an online panel. The respondents were asked to choose the ‘‘best’’ (most important) attribute and the ‘‘worst’’ (least important) choice set from the attributes presented and the results indicated the best or highest to lowest scores. Attributes such as ‘tasted the wine previously’ and ‘someone recommended it’ as also ‘grape variety’ and ‘origin of the wine’ were some of the best indicators for which customers chose a specific wine. Information on the shelf, An attractive front label, Promotional display in-store, or alcohol level were least indicative as best cues for making choices on a particular wine. From the study, Cohen suggests that - The results indicated the most important attribute for choosing or preferring a wine variety is ‘‘tasted the wine previously’’ for Australian consumers. The next important attributes that most influence Australian consumers are ‘‘recommendation’’ and ‘‘grape variety’’. More Australian consumers consider the ‘‘origin of the wine’’ as important while ‘‘information on the shelf’’ for example is less important for most consumers, and ‘‘alcohol level’’ is also the least important for Australian consumers. The cultural similarities and differences have also been studied in this research with BW scaling and scoring method and “tasting the wine previously” has the highest or best score or attribute for both Australian and Israeli consumers and “I read about it” seem to have the same relative importance for people of both the countries, “the origin of the wine” seems to be of much higher importance to Australian consumers. This however is the BW scaling interpretation obtained from data analysis of quantitative measures obtained with the scales and scores when data was collected from respondents who purchased wines in retail stores or online. The BW scores thus provided extensive quantitative measures and scores which could be used as objective data to determine consumer preferences in terms of numbers and these numbers or data scores were ranked to understand the relative importance of attributes and the best or worst and highest or least important attributes that affected consumer decisions and preferences while buying wine. Strengths and Weaknesses: This study by Eli Cohen has several advantages and disadvantages in the sense that there would be major strengths as identified in the method of scaling itself as this scaling method seems to be relatively simple and easy to understand, follow, tabulate and interpret. Even the statistical procedures are simpler and being based on paired comparison methods, there is the advantage of not having to deal with many different choices or ratings when ranking the importance of a product or a choice. As seen in case of methodology and data collection for Cohen’s study of wine preferences, the attributes are presented to the consumers or respondents using previous research which were used to determine what would be the attributes for the scaling systems. The consumers were then asked to indicate their preferences for the best or highest importance and worst or lowest importance related attribute and considering the simple method o data collection, the process is highly efficient and clear ideas and opinions of respondents are obtained through the data collection procedure. The data analysis is also relatively simple when compared with other traditional scaling methods as there is a simple statistical analysis and ranking method that would determine which are the most preferred when compared to which are the least preferred attributes when considering wine purchase and as determined or marked by consumers. The BW method tends to show better discrimination and distinct high –low best-worst scores rather than the traditional scaling and rating methods that may have five point scales and attributes that are marked from very good to very bad on a five point scale. Thus the BW method is not just more accurate but less time consuming and comparatively simple when considered in terms of traditional scaling methods that may have complicated statistical analysis methods. The BW method is also useful in the sense that consumers from different cultural backgrounds could be effectively compared on the basis of their responses and this is again relatively simple as only the rankings of the best and worst attributes will have to be considered and compared in both the cases. The BW method of scaling is now one of the most preferred methods of research in marketing preferences and consumer behavior studies and especially when the studies involved comparing cross cultural data or consumer behavior. Some of the disadvantages identified for the study has been already discussed and these are when several different attributes have to be compared or when there is no specific best and worst attribute for a specific product choice. Individual perceptions may vary and so do cultural perceptions and the ethnic and individual differences in cognitive processes or personal preferences could affect the objectivity of such measures and validity of the tests which may then have to be cross checked or several attributes will have to be measured more than once to ensure that the validity of the study is maintained. The BW method is largely a method that is seen as a progress or advancement of previous or traditional scaling methods such as the Likert type scale and the procedure being more simple with just best and worst attributes, this sort of scaling method is highly preferred among researchers in marketing methodology and research studies. Although there are some disadvantages of the BW method used, the advantages are significant to enhance the popularity and usefulness of the method. Bibliography: 1. Bachman, J.G. and O’Malley, P.M. (1984), ‘‘Yea-saying, nay-saying, and going to extremes: Black– White differences in response style’’, The Public Opinion Quarterly, Vol. 48 No. 2, pp. 491-509. 2. Cohen, S.H. (2003), ‘‘Maximum difference scaling: improved measures of importance and preference for segmentation’’, Sawtooth Software Conference Proceedings, 3. Crask, M.R. and Fox, R.J. (1987), ‘‘An exploration of the interval properties of three commonly used marketing research studies: a magnitude estimation approach’’, Journal of the Marketing Research Society, Vol. 29 No. 3, pp. 317-39. 4. Couch, A. and Keniston, K. (1960), ‘‘Yeasayers and naysayers: agreeing response set as a personality variable’’, Journal of Abnormal and Social Psychology, Vol. 60, March, pp. 151-74. 5. Finn, A. and Louviere, J.J. (1992), ‘‘Determining the appropriate response to evidence of public concerns: the case of food safety’’, Journal of Public Policy and Marketing, Vol. 11 No. 1, pp. 12-25. 6. Gordon, W. (2002), ‘‘Minding your brand manners’’, Marketing Management, Vol. 11 No. 5, September/October, pp. 18-20. 7. Goodman, S. (2009), ‘‘A multi-country comparison of consumer choice for wine in retail stores’’,International Journal of Wine Business Research, Vol. 21 No 1 8. Saunders, M, Lewis, P & Thornhill, A 2003, Research methods for business students, 3rd edn, Pearson Educational, Essex. 9. Sequim,WA. Cohen, S. and Orme, B. (2004), ‘‘What’s your preference?’’, Marketing Research, Vol. 16, pp. 32-7. Read More