Qualitative Data Analysis - Assignment Example

Chapter Seven: Qualitative Data Analysis 7.1 Introduction The final part of the survey, Questions 19 and 20, sought managers’ opinions “in their own words” of their perceptions and observations of the use and effectiveness of strategic management tools. Following the completion of the survey, in-depth interviews were conducted with 18 managers, with the focus of the interviews on three main objectives. First, the interviews sought to follow up on the open-ended responses to Question 20, as these suggested a number of issues in which more information was needed. Second, the interviews were seen as an opportunity to directly address the research questions to the managers participating in the study. And finally, the interviews sought to gather more information on and clarify a number of issues revealed by the descriptive and quantitative analyses. 7.2 Perceptions from Survey Question 19 7.2.1 Summary of Responses Question 19 asked survey respondents, “Do you feel your SMT gives your organisation a competitive advantage, and if so, in what way?” and provided a number of suggested responses. In response to Question 19, the response was overwhelmingly positive, with all but two of the participants (sample, N=213) responding affirmatively. The second part of the question suggested a number of ways in which competitive advantage was supported. The respondents were permitted to select more than one response for this question, and about one-fourth of the managers did so (N=213, total number of responses 260, average 1.22 responses/N). “Improvements to products or services” gained the highest number of responses (82), followed by “More efficient use of resources” (68), “Better plan management” (46), “Attracting or retaining workforce talent” (37), “Better communications” (26), and “Other”(1). “More efficient use of resources” can be defined in terms of cost savings, time savings, or improved productivity without additional financial, material, or human resources. “Improvement in attracting or retaining talent” is fairly self-explanatory, but the connection between the use of strategic management tools and that outcome is not, so this factor was explored further in follow-up interviews. The same is also true of “product or service improvements”; the correlation of the use of SMTs to some degree of positive outcomes in these areas can be perceived readily, but how the use of the SMT leads to those outcomes requires further investigation. Perceptions of “better communications” and “better plan management” are perhaps more obviously linkable to strategic management tools, because the tools are specifically designed to address the factors directly. The single response of “Other” was explained as “success and failure indicator”; this was followed up in the open-ended response to Question 20 as “providing a way to easily compare expected/required performance with actual performance.” From the results of Question 19, the indication is that SMTs are widely used with the expectation of seeing a positive impact in firm outcomes, i.e. better service delivery or product quality. Process efficiency is also a key priority, with more efficient use of resources and better plan management – examples provided the respondents in the question included lower costs, better supply chain management and improvements to production management – being indicated as being positively affected by the use of SMTs. 7.2.2 Competitive Advantage Perceptions for CSF and Balanced Scorecard Users 7.2.2.1 Firm Characteristics In connection with the responses to Question 19, the most common “package” of strategic management tools the data on SMT use revealed – CSF analysis or Balanced Scorecards as primary tools supplemented with SWOT analysis (126 managers, N=213) – was analysed in detail to determine if there were some common perceptions of what ways this particular combination of tools benefitted firms. A somewhat greater number of managers (70, N=126) indicated Critical Success Factors analysis (CSF) as their firms’ primary SMT compared to the number (56) using Balanced Scorecards. A majority of the responses (74, 58.7%) were from firms defined as ‘large’ by the survey (firms with more than 500 employees), while medium firms (100-500 employees) accounted for 27.8% of the responses (35, N=126), and managers small firms with fewer than 100 employees (17) comprised 13.5% of this sample group. 70.6% of the managers (89, N=126) were from independent firms, while 27% (34) and 2.4% (3) represented firms that were Saudi Arabian subsidiaries of foreign firms and wholly foreign-owned firms, respectively. The majority of the managers (109, N=126) were employed by firms operating entirely within Saudi Arabia, with just 17 (13.4%) in firms with locations both inside and outside the country. Most of the firms (113, N=126) likewise operated multiple business locations, with only 13 (10.3%) of the managers indicating their firms had a single location. In terms of type of business, the sample (N=126) was fairly evenly-distributed, with managers representing investment (21) and building firms (20) being the largest segments of the group, followed by energy companies and food companies (17 each), real estate firms and petrochemical companies (10 each), financial firms (8), insurance firms (7), and cement manufacturers (5). 7.2.2.2 Manager Characteristics The characteristics of the managers in the CSF/Balanced Scorecards sub-group (N=126) were virtually to those for the entire survey sample (N=213); there were slightly higher proportions of managers at the General Manager/location manager (31.7% for the sub-group versus 28.6% for the full sample), and location department head/Assistant General Manager (36.5% for the sub-group versus 35.7% for the full sample), but slightly lower numbers of C-level, Division/Department Head, and supervisory-level managers. One key difference between the full sample (N=213) and the sub-group of managers using CSF or Balanced Scorecards is in the length of experience (expressed as “years in current position” in the survey), with the sub-group (N=126) in general being in their positions for a shorter amount of time; 49.2% of the managers in the sub-group (N=126, 65 responses) indicated they had been in their current position for between five and ten years, compared to 44.6% (N=213, 95 responses) for the full survey sample, while a smaller number (16, 12.7%) in the sub-group had been in their positions for 11 to 15 years than in the full sample (36, 16.9%, N=213). To summarise, the firms using CSF analysis or Balanced Scorecards as their primary strategic management tool were in general larger firms with multiple locations. The managers responding were at levels below C-level, most in their current positions for 10 years or less, and who participate in planning at the division, location, or department level. 7.2.2.3 Pattern of Perceptions among CSF/Balanced Scorecard Users The responses to Questions 13 through 18 of the survey were also analysed for the sub-group (N=126) of CSF/Balanced Scorecards users, and a pattern emerged amongst the responses: All of the responses were either “Positive,” “Very Positive,” or “Extremely Positive” (selections 3, 4, or 5 on the scaled questions), and all fell into a pattern wherein the range of “Positive” responses was 11% to 17% of the total; the range of “Very Positive” responses was between 40% and 43%; and the range of “Extremely Positive” responses was from 42% to 46%. By contrast, the responses for the full survey (N=213) ranged from 21% to 30% “Positive”; 13% to 24% “Very Positive”; and 6% to 12% “Extremely Positive”. Not only is the range of responses much narrower in the sub-group, the responses about the perceived effects of strategic management tools are significantly more favourable. In terms of the responses to Question 19, the CSF/Balanced Scorecard users (N=126) all indicated “more efficient use of resources,” “product and service improvements,” and “better plan management” as benefits, while two-thirds (85 of 126) also cited the SMT combination’s benefits in attracting or retaining workforce talent. Somewhat surprisingly given the unanimous perception of the two tool combinations’ (CSF and SWOT or Balanced Scorecards and SWOT) contribution to better plan management, only about 20% (15 of 126) of the managers also cited “better communications” as a benefit. When asked in follow-up interviews (see the following section) why they felt the particular combination of CSF analysis or Balanced Scorecards with the supplemental use of SWOT analysis was effective, the managers described a common perspective towards SWOT, but two different perspectives towards the benefits of the other tools depending on which one they used in their firms. SWOT was described by all the managers as a relatively easy and effective way to describe the overall business environment, and has a primarily external orientation. As one manager put it, “SWOT establishes the context in which our business operates, and is easy to update when the current situation changes.” The difference between CSF analysis and Balanced Scorecards appears to be that firms or managers who prefer a more external perspective find CSF analysis more useful, and rely on it to relate firm or department objectives to best practises within their business sector. On the other hand, those firms or managers who have a more internal focus – for example, several managers who expressed a higher priority for performance or process efficiency objectives – prefer Balanced Scorecards. 7.3 Insights from Survey Question 20 The final question of the survey invited an open-ended response, asking managers “Overall, what would you describe as SMT's benefits or shortcomings in facilitating planning and performance management?” The responses to the question varied in length and detail, and a number of survey participants provided no response at all; in order to make an assessment of the perceptions expressed in the responses, points raised in each were noted and then organized according to common keywords or phrasing in the responses. Through this analysis, several common perceptions emerged: 1. Where strategic management tools have failed to deliver expected results, a likely cause is the failure to include all the elements required by the SMT. The implication here is that managers are wary of “modifying” strategic management tools, having had the experience of seeing the tools not live up to expectations because steps or critical information was omitted. 2. Lack of sufficient information or training amongst all those using or affected by the use of a strategic management tool will lead to inefficiency and poor results. This perception was expressed in surprisingly strong terms by at least 17 of the managers who responded to Question 20, with statements like, “There is a disparity in the level of education and culture among managers leading to a lack of application of strategic tools as required”; “Must educate the concerned staff to know the strategic tools and how to apply them before starting their implementation”; “Lack of familiarity with middle levels of managers to use strategic management tools” being given as reasons why problems had been encountered with the use of SMTs in the managers’ past experience. One respondent went further to explain that the real issue, from his point of view, was that upper-level managers were not sufficiently trained or equipped to conduct the training and familiarisation needed by lower-level managers and staff to work within the parameters established by the strategic management tool. 3. The most-cited advantage of the use of strategic management tools was their usefulness in defining and quantifying objectives and required action steps. A common theme among the responses was the advantage provided by SMTs in defining goals and objectives, with several of the respondents citing the importance of aligning departmental objectives with overall firm objectives. Another factor cited by several respondents was the positive impact of SMTs on employee motivation and productivity; the common opinion amongst these respondents was that clarity of plans and objectives was a benefit to employee performance provided by the use of SMTs. A third factor that appeared to be a common priority was the use of SMTs as a means of assessing the firm’s performance in comparison to competitors; not surprisingly, the managers who expressed this point of view were those who indicated their firms used the SWOT-CSF combination, which appears to encourage a strong, outward-looking perspective. 7.4 Managers’ Insights from Semi-Structured Interviews 7.4.1 Profile of the Interviewee Group 18 managers agreed to participate in in-depth, semi-structured interviews. The managers represented a fairly even cross-section of the firm types, with 4 managers representing real estate firms, 3 each from the energy and building and construction sectors, 2 each from financial services, investment, and petrochemical firms, and one manager each from insurance and food companies. 10 of the managers were from independent firms, while 7 represented firms that are Saudi subsidiaries of foreign companies, with one other manager employed by a joint venture firm. Most (11 of 18) of the managers were from large firms, with 3 and 4 managers representing medium and small firms, respectively. One of the interviewed managers was a CEO; 3 each were enterprise-level department/division heads or location General Managers; 7 were location-level department heads or assistant general managers; and 4 were at the supervisory level. The group of interviewees reflected the relatively short experience in terms of years in their position seen in the sub-group of CSF/Balanced Scorecard users, with 14 of the 18 having 10 or fewer years in their positions, 3 with 11-15 years’ experience, and one (the single CEO in the group) with longer than 16 years. The group of 18 interviewees were somewhat atypical of the full research sample of 213 managers in that they had tried more strategic management tools than the average (3.5 tools, N=18 compared to 2.2 tools, N=213), and there were more current Key Performance Indicator users in the interviewee group than there were Balanced Scorecard users, although like the full survey sample, CSF users were the largest group. There were 8 CSF users amongst the interviewees, 4 KPI users, 3 who considered SWOT their primary current SMT (also an exception to the general pattern of the full research sample), only 2 who used Balanced Scorecards, and 1 manager who preferred Benchmarking. As the group of interviewees was quite small and the objective of the interviews was to obtain management insights on the use of SMTs in general – rather than identifying or analysing specific tools’ characteristics – these variations between the interview sample and the full survey sample are not critical. 7.4.2 Results from the Structured Portion of the Interviews The interviews were conducted individually with the managers in person, except for three managers who, due to scheduling constraints, agreed to interviews by telephone. While the interviews were as open-ended as could be allowed, they were focused on three main areas as described earlier: Perceptions offered in response to Question 20 of the survey, issues raised by the analysis of the survey results, and the main research questions for this study. In terms of the interview group’s responses to Questions 13 through 18 of the survey, the results were positive to about the same degree as the results for the overall survey sample. The pattern of responses for these 18 managers was the same for all of the questions except Q17. SMT effectiveness in the areas of information gathering, development of firm objectives, development of department objectives, and performance monitoring (Q13 through Q16), as well as improving the ease or efficiency of planning processes (Q18) were rated “Very Positive” by 10 managers (N=18), “Extremely Positive” by 7, and simply “Positive” by one manager. The results for Q17, overall firm performance in the absence of SMTs or the use of different SMTs than the ones used currently, were less favourable, but only by the slightest degree, as 2 managers rated SMTs effects as “Positive”, and only 6 as “Extremely Positive.” In statistical terms – as an illustration only, keeping in mind the low number of interview participants – SMTs effectiveness for Q17 had a mean rating on a 0-to-5 scale of 4.22, whereas for the other five questions, the mean rating was 4.33. 7.4.2 Further Insights on Question 20 The more open-ended portion of the interviews sought the managers’ further insights on the positive and negative aspects of strategic management tools revealed by the responses to Question 20 of the survey: First, that strategic management tools can perform poorly if insufficient information is provided, or steps in the process are bypassed; second, that performance outcomes can be less than expected if managers and employees are not sufficiently familiar with the strategic management tools being used; third, that strategic management tools are beneficial in terms of defining objectives and action steps; and finally, because of these definitions, strategic management tools have a positive effect on employee motivation and productivity. On the subject of the advantages of SMTs being reduced by incomplete information, the managers felt that it was not so much a negative judgment of whatever tools were being used, but rather a problem of the planning process in general. One respondent in particular noted the challenge of gathering sufficient relevant information about external factors affecting the firm and therefore, hindering the function of its SMT (which was CSF analysis in this case). When it was pointed out that the interviewees had given relatively high marks to SMTs in terms of gathering information, several of the managers explained the apparent discrepancy by stating that SMTs had strengths in organising information and identifying which information was needed, but that actually gathering or providing that information as inputs to the SMT was a separate process, which evidently varies in effectiveness. Another significant issue revealed by the survey was the negative impact on results from a lack of understanding or training in the use of SMTs. Again, the problem was seen by managers as a possible flaw in organisation or perspectives towards planning processes. The common perception was that SMTs had less than the desired effect because they were not well understood by employees or managers at lower levels in the firm. Relating to this, these managers cited the need to include the entire organization in planning processes for the best results. The need for flexibility in SMTs was cited by several managers, who saw a risk of plans becoming too rigid and outdated if the SMT did not allow for adjustments to changing conditions. Two managers did point out, however, that problems they had encountered (with using SWOT analysis, specifically) might not be attributable to the tool itself, but rather “bad habits” managers and other employees developed by becoming accustomed to it. When asked to elaborate on this observation, one manager explained, “SWOT only tells you certain things. If it is used at the same time as something else, like CSF or Benchmarking, it works just fine. The problems are created, though, when people think of it in step-by-step terms. They do the SWOT analysis, and then move on to the next method, rather than doing them together. What happens then is, the information that’s used for the next step is just what the SWOT says, and that’s not usually enough.” On the topic of employee motivation and improving productivity, the managers who use CSF, KPI, and Balanced Scorecards (which were 14 of the 18 interviewees) had similar views even if they used different tools. The key common factor was the establishment of quantifiable performance targets, which all three tools do in different ways. One manager (a CEO of a real estate firm using CSF analysis as a primary SMT) explained that, “There is a remarkable difference in employee performance when there are clear performance targets than when there are not. Using CSF, the tool doesn’t always give those targets automatically, but it tells us what our broad goals are, and that makes it easier to ‘work backwards’ and determine the steps that need to be taken to reach those goals. And of course, those steps are where we can develop targets, which is just a way to make sure everyone is working towards the same goals.” 7.4.3 Issues Raised by the Analysis of Survey Results As described in the previous chapter, the results of quantitative analyses did not provide conclusive answers as to whether particular strategic management tools were better than others overall, whether some firm types were more successful in applying strategic management tools effectively than other types of firms, or whether particular strategic management tools could be matched to particular firm types for best effect. While the opinions of the managers being interviewed did not necessarily provide conclusive answers on these questions, they did provide some insights that suggested strategic management tools are perhaps more properly assessed in the context in which they are being used – the type of firm using them, the particular business sector and market that determines that firm’s external environment, and what orientation the firm has towards dealing with its competitive environment, for example an internal perspective that might be better served by a tool like Balanced Scorecards, or an external perspective better-suited to tools like KPI or Benchmarking. One manager, whose comments were largely confirmed by two others in similar circumstances – i.e., working in firms wherein four or more different strategic management tools had been tried during the managers’ tenures – suggested that the process of trying different tools actually helped to focus the firm’s planning process. “Our industry (a telecom firm) is very dependent on being aware of what the latest trends are amongst competitors,” he explained. “We were relying on CSF analysis to help our strategy and planning, but for us, it was too inward-looking; our critical factors were all things that happened within the company, without a formal way of connecting them to what was happening in our market. We use KPI analysis now, and our planning seems to be much more usefully focused.” 7.4.4 Insights on the Research Questions The final part of the in-depth interviews addressed the research questions of this study: 1. Which strategic management tools are used by the surveyed organisations in Saudi Arabia? 2. How are these strategic management tools used? 3. To what extent are strategic management tools used without modification? 4. To what extent are the tools effective in addressing organisational performance? 5. Which of these tools are most effective? 6. To what extent do organisations use the same tools consistently over time? 7. To what extent does the organisational context contribute to the effectiveness or failure of strategic management tools? The objective in asking these questions of the interview participants was to compare their insights with the conclusions that could be drawn from the results of the primary survey research; this comparison serves as an informal validity test of the main research effort’s outcomes. The feedback from these questions also provides a greater amount of qualitative insight to help explain some results that are somewhat uncertain; for example, there is some evidence to suggest the ‘organisational context’ has an impact on the perceived effectiveness of strategic management tools (and some evidence that suggests it does not), so the discussion of the matter in an open-ended interview provides some clarity as to the direction and degree of that impact. In terms of which strategic management tools are used by Saudi firms, the six tools selected for the study do in fact represent the options for virtually all the firms; no manager in the sample group (N=213) indicated that an SMT outside those selections was being or had been used as a primary strategic management tool. The interviewees did disclose informally using or experimenting with a number of other tools for the sake of curiosity and expanding their knowledge; the tools mentioned in this context were the Boston Consulting Matrix (1 instance), PEST or PESTEL analysis (3 instances), SMART objectives (1 instance), Value Chain analysis (1 instance), Porter’s Five Forces analysis (2 instances), Gap analysis (1 instance), and spreadsheet or “what-if” analysis (3 instances). In every one of these instances in which a different tool not part of the study was mentioned, however, it was characterised as being only a “partial” or “casual” tool, perhaps helpful in terms of organising information or generating ideas, but not considered a formal planning or strategy management activity. For example, the three managers who took note of “what-if” or spreadsheet analysis all explained that it could be useful in an accounting context as a forecasting or budgeting tool – it might be applied as a test of strategies and targets, but only after those had been developed through a more “regular” planning process utilising the broader tools examined by the research. The use of the “main” strategic management tools covered by the study and evidently preferred by Saudi firms takes one of two forms, although with a bit cross-functional purpose. Based on the responses from the interviewees, firms that primarily rely on Balanced Scorecards or CSF analysis – which constitute the majority of the research sample and therefore the majority of Saudi exchange-listed firms – prioritise performance management, while those that primarily use SWOT analysis, KPI analysis, or Benchmarking prioritise strategic planning. When this dichotomy was presented to the interviewed managers for their insights, however, the consensus was that those categorisations are somewhat simplistic, and that in order for an SMT to be effective, it must provide functionality for both priorities. The interviewed managers were virtually unanimous in agreeing that strategic management tools were not significantly modified from their “standard” form, nor should be. While most of the managers felt that there should be some “flexibility” built into the tool – such as is available in most tools anyway, since the input of important factors, indicators, or targets is according to the user’s discretion – the problem of a lack of familiarity or skill in using existing SMTs among lower-level managers or workers recommends against altering the tools. As to the questions of how effective strategic management tools are towards improving firm performance generally and which tools were most effective in doing that, opinions were mixed, with the managers understandably favouring their own choices as being the most effective. The general consensus was that strategic management tools do contribute significantly to positive firm performance, but under certain conditions: That the tools are “the right fit” for the firm’s circumstances, that they are properly applied, and that the entire firm – or unit within the firm, as the case may be – is fully capable of using the tools. The question of using the same tools consistently over time was also felt to be related to the “success conditions” described by the managers; changing strategic management tools frequently was seen as disruptive. And finally, in the view of the interviewed managers, the organisational context is all-important in determining whether or not a strategic management tool will succeed or fail in its application. The example provided by the telecom manager in the previous section is illustrative of this; CSF analysis did not address that particular firm’s competitive environment to the degree that the firm needed it to, and KPI analysis turned out to be a better fit. Other firms, however, reported satisfactory outcomes using CSF, so clearly a judgment that KPI analysis is a superior tool – outside of the context of that one telecom firm – would be debatable. 7.5 Conclusion When asked to describe the advantages or disadvantages of the use of strategic management tools, the respondents provided answers that revealed a few common perceptions. SMTs were seen by the majority of respondents to have positive impacts on their firms’ product or service delivery, and to positively impact planning and process efficiency. SMTs were also viewed as having a positive impact on employee motivation and performance. The biggest problems or challenges cited included the negative effect of incomplete information inputs to the SMT, and a lack of familiarity and integration of lower levels of the organisation with the SMT and its application. In terms of the research questions, the key takeaways from the managers’ interviews were that firms are not inclined to modify the conventional forms of strategic management tools, but rather choose tools that best fit their particular needs; that other types of tools are used for specific applications, but are considered secondary or supplemental to primary “strategy” tools; that in general the use of strategic management tools is considered better management methodology than not using them; and that the relative effectiveness of any strategic management tool in promoting positive firm outcomes is heavily dependent on the organisational context, both in terms of the firm’s external market and competitive environment, and its internal attributes. Read More