What Is a Data Warehouse and Its Importance - Coursework Example

Data Warehousing Introduction Data warehousing is a very powerful tool in today’s’ competitive business environment. “Virtually everything in business today is an undifferentiated commodity, except how a company manages its information. How you manages information determines whether you lose or win” - Bill Gates (quote taken from the Book ‘Data warehouse project management’, 2000 – Sid Adelman and Lerissa Terpeluk Moss). Effective and efficient management of information depends on how good and efficient the data warehouse in an organization is made. Data warehousing is the base for introducing any business intelligence tool in the organization. What is a data warehouse? The data warehouse is a silo of data gathered from multiple data sources that is coming from wide-ranging sources and forming an all-inclusive database by filtering the old and new data based on business logic and the need of the organization. Data warehousing is the concept of using data from the data warehouse for further processing and getting business intelligence information in an organization. In a data warehouse the flow of data and information has to be continuous, systematic and structured in a manner so that decision makers at every level in the organization can do data mining, query the data to get desired answers, systematically use it for further processing in their decision support system. Data warehousing is an information technology based system that integrates data from other business processes and business occurrence, filters and stores it in a systematic manner and allows the business verticals to use these data or information effectively. Data warehouses are traditionally setup to update the operational data on a daily basis. There will be search engines, queries and filters inbuilt into the data warehouse system. That is why it is also termed as ‘operational data stores’. The importance of data warehousing and why organizations must train their project managers in data warehousing concepts and uses Data warehousing is one of the basic requirements for implementing an effective Enterprise Resources Planning (ERP) or Manufacturing Resources Planning (MRP) in any organisation. The success of any business strategy system and tools are dependent on the maximum control over the data or information flow. The control over the business –data is possible through the maintenance of a centralized data store. A centralized data store can be readily accessed by the management and various departments for effective decision making and planning the operational schedule. An ideal MRP system will provide the managers and departments with updated stock information and activity schedules that help in formulating effective production strategies in addition to facilitating improved visibility of the entire operations. Businesses are often viewed as composition of various functions. Each function requires information to plan, operate, and monitor results. Hence the key aspect of business function is information and its accurate and timely availability to the different departments within the organization. The organization can ensure this through the development of an effective information system based on an effective data warehousing model. Business intelligence has been defined as the “processes, technologies, and tools needed to turn data into information, information into knowledge, and knowledge into plans that drive profitable business action” (Loshin, 2003). There are various business intelligence tools used by the management today and the most important this is data warehousing. For knowledge management and business analytic tools we need systematic and accurate flow of data. The business intelligence tools provide the companies with the ability to respond more accurately to changes in the business environment enabling them to take advantages of market opportunities or plan for contingencies in advance. The essence of business intelligence tools lies in their capacity to integrate data from various sources and compile into meaningful reports allowing easy access to managers that in turn assists in formulating effective business strategies providing the business with the necessary competitive edge in the market. Key benefits of adopting business intelligence within business enterprises involve increasing cost effectiveness, improving competitive advantage, timely business decisions, efficient utilization of resources, synchronizing different departments within the organization and increased revenues. The marketing department of a company uses the historical sales data to project the future sales graphs and tables that is further used by the team to plan marketing strategies. This is an example how business intelligence is used and the role of data warehouse. The need of data warehousing is greater than it was ever before due to changes in the world economy and business operational conditions. A business house can react, adjust and adapt to the changes only if it has access to the relevant data in timely manner. Business intelligence is the key for change management and change management is easy and feasible only when we have concepts in place for data mining and data warehousing. “Many organisation assume that if they put their most experienced project manager on a data warehouse project noting will go wrong – yet we find a most experienced project manager struggling with their data warehouse project because they treat data warehouse like a traditional system – which is not” ‘Data warehouse project management’, 2000 – Sid Adelman and Lerissa Terpeluk Moss). We realize that traditionally concepts of data warehousing is not taken seriously and till very recently it has not been given the due importance that it should have got. Now suddenly the business environments have changed. When the data warehousing is considered as a difference system that is needed to sustain in the competitive market place, we find that there is lack of knowledgeable people in the industry who understands the concept and can setup and run the data ware house. History of data warehousing The modern history of data warehousing traces its origins to the 1980s when IBM introduced business data warehouse concept. In 1985 Procter and Gamble introduced ‘business intelligence’ system to link sales information and retailer scanner data. This system was developed by Metaphor Computer Systems Inc. (Hayes, 2002) After the year 1991 the data computing saw a much revolutionized approach that were brought in data warehousing techniques. During this period introduction of client-server technologies have been done. Software packaged for storing, slicing, and transmitting the data in whatever way the uses desired and wanted were designed and developed. Distributed computing became the trend of the day. All companies did researches and development in this area and most of them were using these techniques in the data warehouse models. A data warehouse model with Client-Server technology was the dominant method. Now also most of the models are based on this architecture. Between 1990 and 2000 many books and tool kits were published by experts to make the users and architects of data warehouse familiarize with the techniques and approaches of building and maintaining data warehouses. These developments have brought standardization and new dimensions in the data warehouse concepts. In early days of development several On-line Analytical Processing (OLAP) tools were made available to the business processing community to interrelate with various segments of the data in a warehouse. Even in today’s context OLAP is a very significant tool for data warehousing. It helps the warehouse to deliver the data or information to the intended user in appropriate time and when and where desired. The introduction of e-commerce and internet in the business computing has made the data warehousing almost at par with the pace of business growth. Delivery over internet is cheaper in comparison to OLAP tools. But there are some other draw backs like data-security and duplication of data that is associated with the internet delivery system. Now at present situation the rapid advancement in techniques and technologies for data-storage, data-retrieval and data-flow have been changing the models and concepts of data warehousing. At one side the business needs are changing rapidly because of the globalization of the economy and dependency of business on market driven forces at the other side the technologies are making it possible to fulfil all needs of business intelligence through data warehousing. The last two decades have seen a major change in the business environment and how the businesses are done. And also new, improved and cost effective technologies are being developed and introduced on a regular basis. These changes in business and technology have also been making a great impact on the data warehousing concepts and its modelling. The data warehouse models are changing very fast to be able to cope with the changing business intelligence needs. Models and architecture of data warehousing The architecture of a data warehouse is designed and adopted depending on the need of the organisation. Simple data warehouse architecture consists of mostly 4-layers. Operational Database layer: for storing the operational data of the organisation. Mostly ERP data are stored in this layer. Accessing layer: for retrieving the operational data and transforming it, refining it and re-loading it again in the warehouse. This layer is between operational layer and information layer. This layer will house the tools for retrieve, transforming and loading the data. Metadata layer: for tagging and indexing the data in categories, sub-categories and other criteria so that retrieval is faster and easy. Information access layer: To access the processed data, analyse it and take out the Management Information System reports. The tools for implementing business logic and business intelligence reside in this layer and it acts depending on the need of the management. For setting up a data warehouse one must chose the proper tools, technologies, and methodologies. There are various components that are required for building the warehouse. The tools and techniques for data-usage, data-management, hardware maintenance and software for data storage and mining are the major items that will determine the efficiency of feasibility of a data warehouse. Technology deficiency can have an adverse impact on the data warehouse and organisation can incur losses due to the ineffectiveness of the data warehouse and later cost of replacement. The data warehouse must be built in such a way that it can adapt to the real time business environment and it must be capable of scaling up in quickest possible time so that varied business needs are accommodated in time and without much of the cost escalations. The data warehouses are becoming enormously bigger and storing huge amount of data in the warehouse. To make it efficient and decentralize a concept has come up in data mining called ‘data-mart’. Data marts are the smaller part or the compartment of the data warehouses that caters to a particular section or division of the business. For example a data-mart made for Sales department will maintain the data only for the sales department and all the capability of the data warehouse will cater to the business need of the sales department only. Current trends and future The internet and social entrepreneurship is the buzz word of business intelligence concepts today. Internet is able to provide incredible impact on the data warehousing. Internet is the core of the e-business both in the area of B2C and B2B segments. Businesses are able to get information on market trends, user’s behaviours, and all that is required for the business intelligence and growth. The individual and businesses are able to gain access on information through internet, then drilling it down to intranet and then spreading it through extranet thus strengthening the operations and capability of their data warehouse. Internet has become the essential part of the data warehouse. The new tend is that the business intelligence needs data not only from their internal operations but also from outside of their business from external occurrence like operations of competitor’s and associates. A balanced approach are being taken by the companies to design and maintain the data warehouse in such a manner so that it can take the full advantage of internet, client-server technologies and large data-computing and data-storage environments. An isolated data does not help but an integrated data helps in today’s fast and rapid pace of businesses. The knowledge of IT professionals is growing days by day in the area of data warehousing concepts and techniques. There are experts available who have designed and are instrumental in spreading the best practices in data warehousing. Even the small and medium companies are dependent on data warehousing for business analysis and business development. The last two decades have seen the tremendous growth in the area of data warehousing and it is envisaged that the next decade will be better for data warehousing. Data warehouse is not only used for storing and retrieving data bust it also helps in keeping a backup of our operational data. It is one of the secured ways of keeping your data backup. A company operating in one part of the world keeps its data warehouse in the other part of the world. This is effective in making the data safe from any kind of unforeseen eventuality. Data warehousing is a very significant concept for the commercial business. Another new trend that the data warehouse business sector has been witnessing is the sharing of the data with the industry. There are large corporate companies and consultancy organizations that are thriving on the concept of ‘knowledge is power’. Knowledge management and sharing is their ball game. These knowledge companies are earning a large amount of business and revenue through sharing the data and intelligent data by managing large data warehouses, making it presentable and packaging it in such a manner that it fulfils the business needs of third party companies. Businesses are trying to outsource their data warehouse needs. As per the survey conducted by Gartner recently Question1. Are corporate executives using information in a strategic manner as a company? • 75% Agree or Strongly agree Question 2. Is the business using information in a strategic manner? • Less than 50% Agree Question 3. Do you have a means of measuring the effectiveness of the use of information? • Less than 20% Agree (Source: Yoh, 2009 - Data Warehousing Trends and Hot Jobs) The industry is maturing and so the business houses. We need to increase more expertise in the data warehousing techniques, concepts and models. Conclusion Business decisions are taken with the help of data and information available in the context and companies are rapidly adopting innovative tools and techniques to facilitate the process of data collection and conversion into relevant meaningful information. The effectiveness of implementing data warehousing within business organizations can be improved when it is customised to fit with the organization’s strategy and business objectives. There are vast potentials of data warehousing features that can be utilised by business enterprises for enhanced performance and accuracy of information flow across various departments. References: 1. Hayes, Frank 2002, The story so far, Available from www.computerworld.com/s/article/70102/The_Story_So_Far?taxonomyId=009 2. Yo 2009, Data warehousing trends and hot jobs – A white paper for buyers of consulting and contract services, Available from http://www.yoh.com/yoh_about/yoh_news/datawarehousingtrendshotjobs.pdf 3. Adelman, Sid & Moss, L.T. 2000, Data warehouse project management, Addison Wesley Longman. 4. Loshin, David 2003, Business Intelligence – the savvy managers’ guide, Morgan Kaufmann Publishers, USA. 5. Agosta, Lou 2008, Trends in Data Warehousing for the Second Half of 2008, Available from http://www.b-eye-network.com/view/8373 6. Baker, Adrienne 2009, What’s hot and not: Trends in the data warehousing, Available from http://www.information-management.com/news/data_warehouse_software_as_a_service_open_source_mpp_smp_analytics-10016330-1.html 7. Baldwin, James R. 1997, The data warehouse: an overview. 8. Schroeck, Michael J 2001, Data warehousing: The past 10 years have been quite a ride, Information Management Magazine, Available from http://www.information-management.com/issues/20010201/3007-1.html 9. Albert, George 2000, The importance of data warehousing, Available from http://www.hinduonnet.com/businessline/2000/05/03/stories/150339m6.htm 10. Haris, Muhammed 2009, Data warehousing: importance and security, Available from http://www.saching.com/Article/Data-warehousing--Importance-and-Security-/3073 11. Reed, Michael 2008, A definition of data warehousing, Available from http://www.intranetjournal.com/features/datawarehousing.html 12. Agosta, Lou 2006, Data warehousing trends for 2007, Available from http://www.information-management.com/news/1069307-1.html Read More