Assessment of Consultative Research for Playful Times Toys - Case Study Example

Assessment of Consultative Research for Playful Times Toys Overview This report is an assessment of the consultative study conducted by Hatfield Market Research Consultants on behalf of Playful Times Toys. There are a number of serious problems with the consultants’ report which are described in detail in the following sections. Secondary data used to establish a background for the primary research seems to be outdated and not entirely clearly relevant to the study. The primary research itself, while extensive, is not supported by evidence of the statistical analysis, and appears to omit several key details. Finally, the conclusions and recommendations offered by the report are limited and do not suggest alternative courses of action. Some recommendations on how to best compensate for these shortcomings are offered as a conclusion to this assessment. Relevance of the Secondary Data and its Analysis The purpose of secondary or desk research is to identify necessary information which has been reliably provided by earlier research, allowing the researcher to focus on more specific questions in new primary research. Secondary research, however, has three main disadvantages: it is often dated, or not as recent as might be needed; the questions answered by the prior research might not be completely relevant to the current market research; and since the researcher has no control over the timing or exact make-up of the study described in secondary data, there is a degree of uncertainty in the accuracy of the information provided. (Patzer, 1995: 14) The secondary data presented in the report gives a general snapshot of the overall toy market in terms of sales, the changes in personal income which relate to the potential of the market, and the details of toy purchases by age group. This is all generally relevant information. But since no connection is made between income level and toy purchases, the usefulness of the income data is not clear. As is discussed below, the connection between income and toy purchases was not made in the primary research, either. And there is a question of validity in terms of the age of the information presented, the most recent of which is from 2004. In the years since then, there have been numerous innovations in toys and significant changes in economic conditions, and the data presented may no longer accurately reflect current realities. Assessment of Research Sampling The sampling used in the larger part of the market research was based on a reasonable assumption about the target population – mothers with children in school – but there are several points which can be questioned as to whether the eventual sample group collected was the most appropriate for developing the information needed by Playful Times. The quantitative research is described as being a simple random sample, which of course means that a group of subjects is selected at random from a larger population and that any member of the larger population has an equal chance of being selected. (Easton & McColl, 1997) More details should be given to demonstrate that the sample was indeed sufficiently random to avoid the problems of sampling distribution that arise in quota sampling, which is actually what this survey research sounds like it was. Hatfield Consultants are careful to point out that 1,250 survey responses were gathered and that this sample size conforms to ‘industry norms’. That is probably a sufficient number in the sample; Punch (2003, p. 93) for example deems a sample size as small as 350 to be sufficient for this kind of relatively small-scale research. But Punch also points out that it is better to have a smaller body of good data than a larger one of poor data. (2003, p. 41) As Brace (2004, p. 36) points out, other information required by a good survey may not be immediately obvious for a particular research question, and implies that the resulting analysis will be wanting if this other information is not covered by the questionnaire. In Hatfield’s research, several important pieces of information appear to be missing, such as: More specific demographic data, such as income group, the number of children per family, and the specific areas in which the surveys were conducted. The only demographic information gathered was on age groups. The target sample is described as “mothers with children in school.” Are mothers with children in school the only toy-buyers in families? Considering that the target market for Playful Times Toys is, in its most general sense, all buyers of toys, there are a number of other descriptions of the individuals in that large group whose activity and opinions might impact the research results: fathers who buy toys, parents with children who are not yet in school, adults who buy toys for children other than their own (such as nieces, nephews, or grandchildren), adults who buy toys for institutional use (child-care providers, teachers, or health-care providers), and even children who are able to buy their own toys. Overall, despite the large size of the sample, it appears to be limited in focus. This limitation to a particular population sample is not prescribed by the stated research objective. While this does not necessarily mean that the wrong conclusions will be drawn from the research data, it does increase that risk. In terms of the qualitative research conducted, this appears to be somewhat more reliable. The research was conducted in the general way ‘mystery shopping’ is done, with researchers carefully maintaining anonymity and appearing as ‘normal’ visitors to the child-care facilities to prevent bias in the information collected. (Wilson, 1998) Rather more specific demographic information is provided in the description of the sample of the qualitative research than in the quantitative survey, which is good, but the number of people included in the focus groups – 16 total in the two sessions – is too small to draw any firm conclusions related to this information. Assessment of Survey Data Collection and Analysis The most glaring omission in the report is the lack of presentation of the statistical analysis, which ought to have been included in an appendix to confirm the validity of the findings presented in the main report on pages 4-6. The report states that “Data analysis was carried out using Excel to provide appropriate statistical tests,” but which statistical tests were deemed “appropriate” are not specified. One reason this is a significant concern is because of the findings that describe relatively low percentages of toy buyers among the different age groups in the survey, ranging from a low of 5% to a maximum of 35% of those surveyed. This is troublesome because the described target sample – mothers with children in school – would seem, as described in the previous section, to have a built-in bias. (Huff, 1954) Yet the number of toy-buyers in the survey sample, an average of only 17.5% across all age groups, is small, and no information is provided as to the actual numbers of respondents in each group. So there is a question of validity in the results, both for the survey overall and in each of the age groups; much of this could have been resolved by presenting a little more detail on how the findings were generated. In terms of the qualitative research, very little information as to what the participants were asked in the focus groups or in the interviews with children is provided. The reason this information is important is to demonstrate that the responses were not as a result of acquiescence bias, where the respondents simply agreed with assertions made by the interviewers regardless of the nature of the questions asked. (Schriescheim & Hill, 1981) For example, the report describes researchers asking young children whether they would like a teddy that talked, and states that the children responded very positively to this. It is frankly hard to imagine any child responding negatively, unless, of course, the child just did not happen to like teddies in the first place; therefore, the value of information such as this is a bit suspect. Assessment of the Report’s Conclusions and Recommendations The consultants’ report offers four conclusions: The market for toys is declining slightly. That may be true, but as was discussed above this conclusion is drawn from secondary data that is several years old. It is also unclear what relevance this bit of information has on the recommendations. Having developed this conclusion, the consultants seem to feel it is important and should be considered as part of Playful Times’ planning, but in what way the information should be used is not explained. Playful Times has relatively low brand awareness among consumers. This information is both very relevant and important, and is reflected in the consultants’ recommendations. Among consumers who are aware of the Playful Times brand, it has a good reputation. This is also valuable information, in the sense that it confirms that Playful Times’ present products and marketing are effective, at least to the extent that consumers are aware of the brand. Taken in connection with the information that a relatively low percentage of toy-buying consumers are familiar with the Playful Times brand, however, an appropriate recommendation would have examined in greater detail why the brand appeals strongly to some consumers, and suggested ways in which these positive attributes could be communicated to a wider market. An electronic teddy bear would likely succeed in the market. Since one of the survey questions specifically asked respondents whether they would purchase an electronic teddy bear, this conclusion appears to be valid within the context of the research conducted. The concerns about the strength of the research, however, as described above, tend to reduce the confidence Playful Times can take in the resulting recommendation that an electronic teddy bear be produced and sold. The recommendations are limited overall, and essentially suggest only the production of the electronic teddy bear and an increased brand awareness campaign. The brief for the consultative study, as described by the consultants themselves in making their recommendations, was that they should identify possible courses – plural – of action and make recommendations. The report does not offer or assess any alternative actions, nor specific details of how the recommended action should be carried out apart from suggesting a television campaign. Critique of the Report Presentation Because the provided report is in written form rather than a verbal or verbal/visual presentation, some of the conventions of formal research writing should be followed, such as (The Research Training Initiative, 1996, pp. 12-13): Clear and traceable referencing to the secondary data provided. An explanation of the possible limitations of the research conducted. Explanation of the relevance of the findings to the original question and the conclusions. While a detailed discussion of the statistical analysis would not be appropriate for a presentation of the report (Wilding, 2010), as was noted above it could easily be included as an appendix to the written report to verify the findings. Likewise, the references to the secondary data should also be provided. The relevance of the findings to the original question is not entirely clear, since the question or questions the study is meant to answer are not clearly described. Instead, a lengthy, conceptual discussion on the origin of toys is provided by way of an introduction. While criticising this part of the report may be in part a criticism of style, one must still wonder how much the management of a company that develops and produces toys needs this mostly hypothetical history explained to them, and the relevance of the discussion to the problem to be examined is certainly not clear. Conclusion and Recommendations The overall and somewhat unfortunate conclusion that must be offered is that the report not be accepted. This presents obvious difficulties to the management of Playful Times Toys because time and money has already been spent on this project, but because the likely results of any consultation will be further investment in product development, production, and marketing, it is important that the consultation provide findings and recommendations that are sound and reliable. There are simply too many areas in the Hatfield Consultants’ report that are unclear and reduce confidence in their validity. In order to avoid large additional expense and time waiting for the production of an entirely new consultation, however, as much use as possible should be made of the work which has already been done with an eye towards enhancing the detail of the original findings. As noted, the finding that only 36% of respondents were aware of the Playful Times brand is helpful; a follow-up study of a qualitative nature, perhaps using focus groups in the same manner as the consultants’ research should be conducted to concentrate on what brand aspects of Playful Times are appealing. This was done to some degree in the original study, but the sample size of 16 people is simply too small to draw reliable conclusions from it. The results of a follow-up study can then be used to better focus subsequent marketing and brand awareness initiatives. Finally, more energetic secondary research using updated information should be conducted to gain a realistic current insight into the toy market. Correlating income levels, toy purchases, and other demographic information can help to guide Playful Times in the development of toys that meet market demand. The electronic teddy bear may indeed be one of these, but considering the investment required, it is best to do the study necessary to be certain. References Brace, I. (2004) Questionnaire Design: How to Plan, Structure and Write Survey Material for Effective Market Research. London: Kogan Page. Easton, V.J., and McColl, J.H. (1997) Statistics Glossary, v. 1.1. UK: STEPS Consortium [Internet], available from: http://www.stats.gla.ac.uk/steps/glossary/sampling.html. Huff, D. (1954) How to Lie with Statistics. New York: Norton. Patzer, G.L. (1995) Using Secondary Data in Market Research. London/Westport, Connecticut: Quorum Books. Punch, K. (2003) Survey Research. The Basics. London: Sage. The Research Training Initiative. (1996) Presenting Research Findings. University of Central England in Birmingham. [Internet] Available from: http://www.biad.uce.ac.uk/ research/rti/rtrc/pdfArchive/PRF01.PDF. Schriescheim, C.A., and Hill, K.D. (1981) ‘Controlling Acquiescence Response Bias by Item Reversals: The Effect on Questionnaire Validity.’ Educational and Psychological Measurement, 41(4): 1101-1114. DOI: 10.1177/001316448104100420. Wilding, K. (2010) ’11 Tips on How to Present Research Findings.’ National Council for Voluntary Organisations [Internet], 16 September 2010. Available from: http://www.ncvo-vol.org.uk/networking-discussions/blogs/209/10/09/16/11-tips-how-present-research-findings. Wilson, A.M. (1998) ‘The Use of Mystery Shopping in the Measurement of Service Delivery.’ Managing Service Quality, 8(6): 414-420. DOI: 10.1108/09604529810235123. Read More