The Critical Success Factor of Data Warehousing Implementation: Key Challenges for a Company Like Offco - Research Paper Example

Introduction There has been a marked increase in the popularity of Data warehouse particularly for large organizations over the past decade. Not surprisingly, 95% of the Fortune 1000 companies have implemented or plan to implement data warehouse (Wixon and Watson, 2001). The aim of developing data warehouse is to provide a single platform for accessing the organization data across different business unit and IT systems which in turn help management to improve the efficiency of the decision making process (Boyer, 2007). Transactional data from operational systems are extracted, transformed and loaded to the data warehouse; consequently, data warehouse is a time-variant collection of data (Inmon, 2005). The data can be grouped so as to analyze the behaviour of consumers. Therefore, managers are able to optimise strategic plans thereby reducing costs and possibly through improved understanding of consumer behaviour attain and sustain increased sales. The present essay will initially introduce two well-known approaches to developing data warehouses created by Kimball and Inmon before moving on to a comparison of the advantages and disadvantages of these approaches. Based on the derived understanding it will attempt to suggest which approach may offer greater potential for Offco. Company. Thereon the essay will move on to identifying the key challenges for a company like Offco associated with the process of establishing such data warehouses as well as attempting to instruct regarding possible solutions to overcome those challenges. The essay shall end with explanation of critical success factor of data warehousing implementation and summarizes all important points in conclusion. The Two Approaches to Data Warehouse Modelling There are 2 well-known approaches of building data warehouse developed separately by Inmon and Kimball. Since, understanding the fundamental principles of both approaches will help the organization choose the one that is more suitable given its strategy-specific goals, the present segment of the essay will attempt to introduce the concepts and to compare the basic features of the approaches to one another to the degree of detail affordable given the scope of the present essay. Inmon’s Top-Down Approach Inmon’s approach or the top-down approach is based upon the traditional entity-relationship diagram (ERD) to build data warehouses. In top-down approach, data is extracted from operational database and then it is cleaned in the standard format to ensure data consistency before it is loaded into the atomic data warehouse. After that, data is extracted from atomic data warehouse and installed into the departmental databases which depend on a given department’s needs. Users then query and create reports from the departmental database (Breslin, 2004) Kimball’s Bottom-Up Approach Kimball’s approach or bottom-up approach has been developed by using the process of dimensional data modelling which has the advantage of featuring lesser complexity compared to ERD. Even though, users are not IT experts, they still can easily understand the logic of this approach (Kimball and Strehol, 1995). While top-down approach begins with creating enterprise data warehouse and then develops end-user application, bottom-up approach begins with smaller tasks of creating multiple data marts which are in essence small-scale replicas of the aggregative data warehouse and then incrementally integrate them into enterprise data warehouse (Watson et al, 2002). A key tool to integrate various data marts is called “data bus”. It sets a standard for all data marts to conform such as key column names, attribute definition, and attribute values (Breslin, 2004). In this approach it is this Data bus that ascertains data consistency before loading the data into each data mart and it leads to successfully integration of the many data marts into a single enterprise data warehouse. A Comparison of the Immon’s and Kimball’s Approaches A comparison of the two approaches in terms of characteristics, advantages and disadvantages are presented in the following table. Inmon’s Approach Kimball’s Approach Characteristics Uses the entity-relationship diagram (ERD) to develop logical model Uses the dimensional modelling ( star-schema diagram) to develop the framework for the data warehouse (Chenoweth et al,2003) This approach is shaped by pure data model This approach is shaped by processing requirements (Inmon,2005) Data modelling is subject- or data-driven Data modelling is process oriented (Turban et al,2008) Relational model is best in long-term for building data warehouse Dimensional model is good for short-term because there is a limited scope for data warehouse (Inmon,2005) Relational database is normally used to support transactional processing Dimensional data mart is normally used to support decision making (Chenoweth et al,2003) Relational database intents to capture all of the information needed for the transaction processing required to run the organisation (Chenoweth et al,2003) The star-schema intents to answer all of the relevant questions to support decision making in each business unit Relational database has a large number of normalized tables The dimensional data mart has only one or a few non-normalized tables which enhance the performance and maintain an intuitive user interface (Chenoweth et al,2003) IT professionals are primary audience End users are primary audience (Turban et al,2008) It is appropriate for very large bases of data It is appropriate for smaller bases of data (Inmon,2005) Advantages and Disadvantages Highly Flexibility is one of the major strengths of this approach Efficiency of access and delivery of data to end users is the most important strength of this approach (Inmon, 2005) It takes longer time to get answers to queries and is not optimized for performance for any user (Inmon, 2005) It is quicker response times and easier for end users to query (Turban et al,2008) It is good for indirect access to data It is good for speedy and efficient direct access to data It is time-consuming to develop and may fail to deliver within a reasonable timeframe (Watson et al,2002) It requires less time to develop because data mart can be built quickly because of its limited scope Data warehouse is more expensive and requires larger amounts of resources Data mart requires lesser amounts of initial cost because of being smaller in scale and less complex. The process of building enterprise-wide data warehouse is likely to face more technical problems and thus requires specialized software and experienced team (Wixom and Watson, 2001) Building data mart poses much lesser technical problems because it contains data for only one business process, though the organization may face problems with the process of integrating multiple data marts (Watson et al,2002) It is more difficult to design because data warehouse is developed for the whole organization at once It is easier to design because it is less complex and focuses on one business process at a time It has greater risk because it is huge and complex project. Being a smaller project is thereby less risky and more manageable (Ang and Teo,2000) It takes long time to complete the project and to evaluate return on investment Benefits can be obtained earlier (Ang and Teo,2000) Data is compatible because all data derives from single data warehouse There is a possibility of incompatibility in data or redundancy of data because of multiple data marts The suitable approach for Offco Ltd. An empirical study by Ariyachandra and Watson (2008) where 454 companies were asked to respond to the surveys regarding the effectiveness of the particular data warehouse which was used by these companies, surprisingly, found no significant differences in preferences for enterprise data warehouse approach or data mart approach and both were thus found to be evenly efficient and effective. However, apart from the data warehouse methodologies, an organisation should consider factors such as the availability of resources, the urgency of the need for the warehouse, management’s strategic view of the warehouse, compatibility with existing systems etc as well. Given the particular context of Offco, it does seem that Kimball’s approach would be more suitable for it. The remainder of this section of the essay clarifies the statement. Firstly, at present Offco.’s operations are relatively dependent upon operations and interactions between various business units. Each unit has their own decision-support environment and uses multiple systems and platforms; consequently, no one knows what the others are doing with customers in other business units. The CEO has decided it is time to change the company’s operations in this regard. The company goal is to become highly customer-centric and present a single face to customers. This is where the data warehousing methodology has come into the picture. It integrates various data from different systems into one unified database and offers a single view of the entire business. Nevertheless, the company cannot afford a 2-3 year lag to build an enterprise-wide data warehouse particularly given the highly competitive markets of the present age. The company rather needs a system that could deliver rapidly even if in phases as well as ensure quick returns to the company (Watson et al.,2002). These needs do make Kimball’s approach more suitable. Secondly, senior managers in Offco are still hesitating with the benefits of data warehouse. Building data marts which will show quick benefits and returns on investments will help convince these people to embrace the data warehouse initiative. Third, Kimball’s approach begins with analyzing user’s requirements. It will increase manager’s satisfaction because they will be able to use the system in accordance with the company specific needs. Fourth, user friendliness is crucial for building the data warehouse in this case because it will encourage widespread use of the system across the organization (Boyer, 2007). There is no point in building an expensive system that no one uses or no one is able to exploit its abilities effectively and fully. User-friendly designing is the Kimball’s philosophy (Breslin, 2004). Fifth, the Aug and Teo(1999) study showed that 90% of users use pre-canned queries and reports when only 10% codes new queries. Kimball’s approach concerns about the possible question that users may ask in the beginning and prepares the answer beforehand. This method also leads to quick response time which necessary for decision support (Chaudhuri and Dayal, 1997). Sixth, it is recommended to start small and focused rather than build large-scale data warehouse because such huge efforts always come with greater risks and high percentage of failure in the project. While incrementally built data marts enable the project team working toward their goal in attaining more manageable size at lesser risk (Wixom and Watson, 2001). Seventh, Gardner stated that cost is one of essential factors to be considered. Kimball’s approach actually offers better allocation of resources as it does not require huge lump sum investments initially. Rather than investing huge amount of money at once, Kimball’s approach is less expensive and thereby allows the company to spend the remaining money in other beneficial alternatives. Besides, top management are likely to support the project that invest less and give return quickly; hence, it is likely that managers will be satisfied with this approach. Finally, if this approach is used to develop the data warehouse, users will be able to access information for decision making easily without much reliance on the IT department (Ang and Teo, 1999). In addition, Kimball’s approach allows both IT professionals and end users to participate throughout the development process which in turn increases the likelihood of user acceptance when the project is completed (Breslin, 2004). Both top management support and acceptance from users are key elements to successfully build a data warehouse. Therefore, we can conclude that Kimball’s approach is more appropriate for Offco. Key Challenges and Solutions Building such a huge project of enterprise-wide data warehouse always comes along with challenges. Nevertheless, understanding and identifying such challenges in the initial phase would make the company better equipped to face these. Key challenges that are likely to be faced by Offco Ltd. have been classified into three groups according to the spheres they are likely to emerge in - business, organizational and technical challenges. Business Problem A possible challenge that can emerge in the business sphere is that senior management teams may not fully support data warehousing initiatives as they don’t have a clear understanding about the benefits that data warehouse can deliver while they are perfectly aware of the cost implications. They may thus question whether the data warehouse will enable the organization achieve their goals to be a customer-centric organization or not. Historically, the organization strategy focuses on product-oriented strategies which involve high-quality of manufactures and logistics (Fenton, 1998). Nevertheless, the organizations have realized that only improving product quality is not enough to compete in today’s market, the organizations are therefore moving forward to focus on customer centric policies. It is necessary to target the right customers and then offer company’s products to these targeted customers. CRM strategy has come into picture because it involves using existing customer data to improve company’s product and service as well as build long-term relationships with customers (Couldwell, 1999). A study by Payne and Frow (2005), outline a process-based CRM strategy framework which consists of 5 components, namely strategy development process, value creation process, multichannel integration process, information management process, performance assessment process (see figure 1) Strategy development process is the process of setting business strategy as well as its customer strategy. The management team needs to decide which strategy is suitable for the organization and how to develop that strategy. In addition, the managers require examining both existing and potential customer so as to form a segmented customer class in the appropriate way. Consequently, extensive customer data needs to be analyzed and tools such as the data warehouse are necessary. Value creation is the process of determining what value the organization can offer to its customer and what value the organization can receive from its customers. Multichannel integration process is the process of determining which distribution channels the organization will use and if there are many channels through which a customer can interact with the organization, how to create and present a single fact to customers. Performance assessment process is the process of measuring the success. Lastly information management process involves the collection of customer data to generate customer insight about their needs and their behaviour. Key material for analyzing these is an enterprise-wide data warehouse, because it collects and stores vast amount of customer data from different sources. Managers are able to find the relationships and patterns among large amount of customer data by using data warehouse tools. Therefore, data warehouse is one of the competent tools that enable the Offco to achieve their customer centric organization goal. Organizational Problem The most common reason for data warehouse effort failure is weak sponsorship from top management (Wixom and Watson, 2001). The data warehouse initiative at Offco encountered lack of management support from both CEO and various independent business units as well. Top management has viewed data warehouse as a technology initiative which is involved with IT solution and IT people. However, it is important to realize that data warehouse project is not simply an IT implementation, it needs collaboration from many parties and commitment from top management is critical to the success of the data warehouse initiative. In order to solve this issue, urgent meetings between the CEO and management from various business units need to be arranged so as to explain the benefits of data warehouse. To get the support from CEO, the implementer needs to connect and link the business goal to data warehouse project. Main objective of Offco is to be a customer centric organization which requires the management team to understand customer’s needs and their habits based on past experience of customer’s information. At present Offco has various business units which have their own decision-support environment. In addition the operation is decentralized – the company has multiple systems and data-sharing is limited between different units. The management cannot see the whole picture of the entire organization because the customer data is scattered in different locations. Data warehousing initiative will however, enable the management team analyse data at enhanced ease because large quantities of data from diverse sources are collected and stored in a single area and the data can be grouped in the way to help decision support for managers, which in turn increases sales for the organization. Moreover, the management teams are to be welcomed to participate in the meeting and suggest alternatives. They can share their ideas regarding what they want to know about the customer, what aspect they want to analyse and what they expect from data warehouse. Understanding user’s requirement and expectation will help to create data warehouse with increased customization and effectiveness as well as increase the satisfaction of the different management teams. Getting involved in developing process will increase the ownership of the system and consequently the management teams are likely to become involved, committed and thus support this data warehouse project. Technical Problem At present, Offco is running 4 systems concurrently - legacy sales order entry system, ERP system, marketing information system and customer care system. Consequently, data has been dispersed in different systems and there are inconsistencies in definitions which lead to technical problems of integrating various data into one single data warehouse. To eliminate this issue, various integration technology tools has been used such as Enterprise application integration(EAI), service-oriented architecture(SOA), enterprise information integration(EII) or extraction, transformation, and load(ETL) to clean data into same format and then load into enterprise data warehouse (Turban et al,2008). Furthermore, Offco have processed customer orders of more than 1 million per month and there is a potential of large increasing amount of data in the future. In order to prevent the upcoming problem, implementers need to aware of the scalability of data warehouse to ensure that it is able to deal with changing needs and expectations. (Ang and Teo, 2000) Critical success factors Based on the empirical study by Wixom and Watson (2005), research model for data warehousing success has been developed in Figure 2. The authors tested the hypothesis that there are seven critical success factors of data warehouse implementation, namely management support, champion, resources, user participation, team skills, source systems and development technology. There critical success factors were tested against the possibility of the data warehouse success and success in six dimensions of data warehouse been identified as critical factors for aggregative success. These are organizational implementation success, project implementation success, technical implementation success, data quality, system quality and lastly perceived net benefits. The Organizational implementation success factor evaluates the level of success in dealing with organizational issues such as change in management, user resistance and adequacy of support. Project implementation success evaluates the level of success in completing the data warehouse project on time and within budget. Technical implementation success evaluates the level of success in overcoming technical problems. Data quality success evaluates the extent of the quality of data to support decision making, including data accuracy, completeness and consistency. System quality success evaluates the system’s flexibility, integration, quick response time and reliability. Perceived net benefits evaluate the overall benefits from data suppliers to stakeholders and organization. 111 organizations responded to the surveys and the authors conclusions are summarized as follows: Management support is recognized as an essential factor for organizational implementation success. Widespread and consistent sponsorship from management teams can overcome political resistance as well as encourage participation across organizations. Users are likely to conform to management’s expectations and tend to accept the system already supported by top managements. Nevertheless, a surprising finding from Hwang and Xu’s research (2008) is that top management support has no impact on data warehousing success. They claim the top management has significant influence on the decision to implement data warehouse but it is no longer a critical success factor thereon. Champion are groups of people who promote and support the project and also resolve political issues within the organization. The author believes that champions will help the data warehouse project achieve both objectives of organization and project implementation goals. Nevertheless, the survey results have proved that champions have no impact on these objectives and therefore the role of champions is excluded from the key success factors of data warehouse implementation. Resources which include money, time and people are deemed as one of critical success factors to complete the project since data warehouse project is huge thus requiring significant investments of of money and time. The availability of resources helps the organization to cope with organizational obstacles and also lead to high level of organizational commitment. User participation arises when roles and responsibilities are assigned to users. When users are part of the implementation process, they understand what benefits the data warehouse provides and thus are likely to accept the data warehouse when it is completed more willingly. Nevertheless, the survey showed that user participation by itself was not sufficient to encourage users to accept the data warehouse project. Team skills include both technical skills and interpersonal skills of members of the data warehousing team. The ability to perform difficult tasks and well interact with users have been identified as one of the major fundamental requirement to complete the complex data warehouse project. Source system refers to the quality of data from source systems. It was proved that standardized data from diverse source systems will ease the process of extracting cleaning and loading information into the data warehouse. In such situations fewer problems occurred and thereby resulted in unhindered success in implementing the system. In addition, Hwang and Xu (2008) supported that poor data quality led to high data warehouse project failure rates. Development technology refers to the software, hardware, methods and programs used in building the data warehouse. The results show that better development tools support lead to higher rates of technical implementation success. In summary, the critical success factors identified are management support, resource availability, team skills, quality of source system and development technology tools. These factors have significant influence on the possibility of success of data warehouse implementation by persuading the organization to accept data warehouse, completing the project on time and within budget as well as overcoming technical problems that may surface. Conclusion Thus what emerges from the detailed discussion of the two alternative methodologies of building data warehouses is that both approaches have different advantages and disadvantages. However, both these approaches cater different advantages and thus the appropriateness of either depends upon the specific goals of the organization. For Offco. company, it was recommended to implement a data warehouse based upon Kimball’s method because data marts are easily built, less risky and require less lump-sum investments and thus are suitable given the specific needs of the company. Various challenges that may emerge during building the data warehouse such as business problems, organizational problems and technical problems were discussed and the solutions to these were addressed. Finally, the essay evaluates the critical success factors based upon the Wixom and Watson (2001) study. It was found that the success factors mainly comprise of management support, resource availability, team skills, quality of source system and development technology tools. References Ang, J. and T. Teo (2000). ‘Management issues in data warehousing : insights from the Housing and Development Board’, Decision Support System, Vol.29, pp.11-20 Ariyaghandra, T. And H. Watson (2008) ‘Which Data Warehouse Architecture is Best?’, Communications of the ACM, Vol.51(10), pp.146-147 Boyer,K. (2007) ‘Enterprise Data Warehouse Challenges’, DM Review, December 2007, pp.22-23 Breslin, M. (2004) ‘Data Warehousing Battle of the Giants: Kimball and Inmon Models’, Business Intelligence Journal, winter 2004, pp.6-20. Chenoweth, T., Schuff, D. and R. Louis (2003) ‘A method for developing dimensional data marts’, Communications of the ACM, Vol.43, No.12, pp.93-98 Couldwell, C. (1999). ‘Loyalty Bonuses’, Marketing Week, 18 February, pp.14. Fenton, D. (1998) Data Warehousing: Enabling the Strategic Shift to a Customer-Centric Business, [Online], Available from: http://www.tdan.com/view-articles/5038 [Accessed 17 Mar 2009] Gardner, S. (1998) ‘Building the Data Warehouse’, Communication of the ACM, Vol.41(9), pp.52-60 Hwang, M. And H. Xu (2008) ‘A Structural Model of Data Warehousing Success’, Journal of Computer Information Systems, Fall 2008, pp. 48-56 Inmon, W. (2005) Building the Data Warehouse, 4th edition, Indiana: Wiley Publishing. Kimball, R. and Strehlo, K. (1995) ‘Why decision support fails and how to fix it’, Sigmod Record, Vol. 24(3), pp.92-97 Payne, A. and P. Flow (2005) ‘A Strategic Framework for Customer Relationship Management’, Journal of Marketing, Vol.69, pp.167-176 Turban, E., Sharda, R., Aronson J. and D. King (2008). Business Intelligence: A managerial Approach.New Jersey : Pearson Education. Watson, H., Wixon, B., Buonamici, J. And J. Revak (2002) ‘Sherwin-Williams’ Data Mart Strategy: Creating Intelligence across the Supply Chain, the Association for Information Systems. Wixom, B. and H. Watson (2001) ‘An Empirical Investigation of the Factors Affecting Data Warehousing Success’, MIS Quarterly, Vol.25, No.1 , pp.17-41 Bibliography Chaudhuri, S. and U. Dayal (1997) ‘An Overview of Data Warehousing and OLAP Technology’, ACM Sigmod Record, Vol.26(1), pp.65-74 Cunningham, C., Song, Y. And P. Chen (2006) ‘Data Warehouse Design to Support Customer Relationship Management Analyses’, Journal of Database Management, Vol. 17(2), pp. 62-84 Hurlock, J. (2006) ‘Overcoming the Implementation Challenges of a Credit data Warehouse’, The RMA Journal, Vol. 88(10), pp.50-51 Johnson, L. (2004). ‘Strategies for Data Warehousing’, MIT Sloan Management Review, Spring 2004, pp.9 Kimball, R. and M. Ross (2002) The Data Warehouse Toolkit: The Complete Guide to Dimensional Modeling , 2nd edition, New York: John Wiley and Sons. Moore, R.(1999) The Challenges with Data Warehousing, [Online], Available from: http://www.information-management.com/issues/19990101/232-1.html [Accessed 16 Mar 2009] Rizzi, S., Lechtenborger, J., Abello, A. and J. Trujillo (2006) ‘Research in Data Warehouse Modeling and Design: Dead or Alive?’, Proceedings of the 9th ACM international workshop on Data warehousing and OLAP, pp.3-10 Sen, A. And A. Sinha (2005) ‘A Comparison of Data Warehousing Methodologies’, Communication of the ACM, Vol.48(3), pp.79-84 Read More