Future of Business Intelligence, Data Classification and Prediction - Coursework Example

Compare and contrast some aspects ification, using at least three examples of different approaches and their strengths and weaknesses Introduction Technology is advancing with light speed, the technology, particularly with reference to computing, has undergone rapid changes during the last thirty years. As technological has advanced the cost of computing has decreased particularly with reference to hardware. Data storage space has become less expensive and data collection tools have become more and more sophisticated. Data warehousing concept has evolved from early days of data collection to archiving of data from where data is increasingly being analyzed for pattern recognition and identification of knowledge that can be used for knowledge discovery in databases (Norton 1999). Soumen et al (2009) agree that data quantity surrounding us is humongous and the amount of information bombarding us is increasing. Making sense of this increasing data volume requires data mining skills and techniques that have evolved with increase in computing power. An allied discipline is Competitive intelligence which is a discipline used for improving market standing, improving strategic thinking – seeing through morass of disinformation and market disruptions and interpreting events without getting emotionally swayed with “pregnant” data. It is about analyzing an opportunity or threat before it has materialized (Reviews 2007). Data Mining overview Finding patterns in data is a common way of analysis. Scientists want to discover the pattern and use the patterns for developing theories that can be extended beyond the concerned data in allied fields. This helps the scientists predict what will happen in newer situations. Intelligence is thus about using the available information in an efficient manner based on picture which may or may not be perfectly clear and exploiting the gleaned intelligence for making strategic decisions. Data mining calls for electronic data storage and using of specified search for pattern identification. Global data doubles by in every 20 months, and increased availability of machines that can digest and process such data have increased opportunities for data mining. Intelligently analyzed data may our only redemption in making sense of the growing data volume. Bits and pieces of information lead to understanding of the big picture. Data mining is using the existing data to solve problems and discovering patterns in data. Consumer shopping data might help in eliciting likely reason for customer loyalty and churn. Also data analysis on same database may identify the reason why customer may be attracted to other product or service thus allowing development of special targeted offers. Data is only useful if it can be analyzed or mined, intelligence is about knowing the identity and acting on signals that analyzed data portrays – it is also acting on information before everyone else sees the same picture (Soumen et al 2009). Getting information from data (Chethan n.d.). Data mining can be divided into two separate types including predictive data mining which using existing variables in the data set predicts the unknown variables while descriptive data mining focuses on finding patterns that can be interpreted by analysts (Chethan n.d.). Data has to be cleaned for removing inconsistencies and extraneous information, often data requires consolidation from multiple sources and streamlined to provide pertinent data for the concerned task. Later the data may be transformed so appropriate mining techniques can be applied. This step is usually followed by evaluation and then presentation of gleaned knowledge. Classification of data is used to predict the class of data so that the mode derived can be used for derivation of attributes about the data. Classification data can be represented using IF-Then rules, decision trees or neural networks. Decision tree is like a tree with multiple branches each representing outcomes. Neural network consists of collection of neuron type processing units with weighted connections. Classification models are used for categorical data that is discrete or unordered while prediction models use continuous data. Thus missing numeric data can be predicted or forecasted using these methods. Regression Analysis is one statistical technique while other techniques that help in identification of trends may be useful as well. Another technique is cluster and factor analysis. Cluster analysis groups objects with high degree of intraclass similarity while being not similar to objects which are members of other clusters. Similar outliers who are markedly different from the class of data being studied are useful in fraud analysis while evolution analysis is useful for analyzing change of behaviour of data over time. This type of analysis is useful for time related data. Data mining systems can be classified according to multiple criteria, including data models – different types of data may require different handling as the kind of analysis may cover the dimensions of characterization, association and correlation as well as different kinds of analysis as presented above. Data mining system can also be distinguished on the basis of levels of abstraction of knowledge that will be mined - sophisticated data mining systems will incorporate multiple data mining techniques (Jiawei 2006). Future of Business Intelligence Business applications will evolve as per needs of the times and will become commonplace as organizations will need BI technologies to survive and grow. Effective differentiation and growth will require utilization of BI and will grow to leaders within their dominion. The future BI will shift from a centralized model to distributed system where knowledge capturing is via distributed pockets or islands. BI will also be more pervasive and demand for information and analysis will be in real time perspective. Every process will become interlinked with BI so decisions will become automated and whenever human intervention is required more contextual information will be available for the knowledge worker (“The Future of BI” 2005). Business More Intelligently Data mining tools are useful for classification – for determining what inherent characteristics differentiate one group from another. Data mining is very useful for predicting what will happen but the tools fail to determine when the things will happen if they do. Making specific forecasts about specific group of customers’ future behaviour and especially predicting what the customers are likely to do is not going to be a very easy problem for data mining. Ascribing attributes about future behaviours using past behaviour as a predictor essentially means squeezing all randomness out of human behaviour – something which is not entirely possible. This limitation of data mining in trying to encapsulate human behaviour will always remain. Data models may become more complex and variegated but the limitation still stands. The concept about one to one marketing can’t extended to mass market as it is humanly not possible to distinguish each customer rather it will only be individual clusters that aim to represent the respective individuals. This concept aims to nullify the randomness of individual customer and the effort required to gain insight into nature of individual customer is far less than incremental effort required for that kind of detail. Probability models can provide very accurate forecasts about net life of customer or amount of purchases that he would do over the coming year. Hence basic probability model can be used to capture basic behaviour then use data mining can be used to determine how and why the groups of customers with specific behavioural tendencies are different from each other. Probability model helps to identify tendencies and explain these tendencies as a function of the variables outputted from data mining engine to determine behavioural tendencies (Fader 2007). Data Classification and Prediction Classification means assigning specific labels to analyze particular class of data; these categorical labels have got no bearing or order as that would be attributable to data that can be numerically ordered. Numeric prediction would be incorporated into deducing supposed expenditure of a customer at sale period. Predictors can be used where base values are continuous. (Jiawei 2006) Data classification is explained by the above diagram, where training data is processed by classification algorithm. Test data is used to estimate the accuracy of classification rules and if the accuracy is acceptable then the rules can be applied for classification of new data points or “tuples” or measurements about n dimensional attribute defining a particular variable. Classification and prediction are two forms of analysis that can be used – classification can be used for classes while prediction models continuous valued functions. For effective analysis data needs to be cleaned and normalized for removing extraneous values and handling missing values. Comparison of different classification and prediction methods are yet to determine superiority of one method over others (Jiawei 2006). Principles of web mining We have considered the need and focus of determining data mining benefits. Data mining can be predictory (decision support), generative (generating new or improved designs) or explanatory (scientific analysis) . Web based mining is used for data captured via web server. Access log has record for all files and items requested, time spent on each page and form of activity conducted. Usage statistics are typically not considered data mining. Classification and prediction as we have seen are used for discrete and continuous data respectively while clustering is useful for grouping like minded items. Association rules describe relationships like in purchase analysis it shows items bought together. Take the case of Amazon which based on customer’s interest in particular products shows association of other products that other customer have bought thus trying to influence the customer into buying stuff which they did not originally intend. Sequential pattern analysis is analysis of data captured over certain time period. This helps in identifying and extracting repetitive patterns – web mining is especially suited to this type of analysis. Time series is another form of analysis which allows the analyst to analyze changing behaviour. Summarising the individual traits into generalized analytic model helps in evaluation. After data cleaning which removes the irrelevant entries rules and pattern identification is done. The first level of pattern identification has to be limited within specific threshold limits so that important patterns are first identified. Interpretation of the patterns and rules have to be done within analytic model – here web logs and interactivity languages are useful in formulating rules and user pattern of interaction. Web mining is limited to activities performed under control of web server so user activities beyond the concerned site are not captured (Claus 2004). (Claus 2004) Web usage mining for educational multimedia As seen above web usage mining extracts patterns and rules from access logs. Session statistics show how user interacted with the web based material under the purview of the web server. The goals and aspirations of the web designer or end user manager may be compared with user behaviour – more specifically usage of particular resources and how they were used. Session classification extracts the focus of learner, identifying the main learning objective of the user. Behavioural pattern identification are used to identify what a user might do after certain sequence of activity. These patterns can be both reflection of how instructor intended for the system to do and abstraction of student’s learned behaviour. Time series are measurements over set time spans aiming to elicit evolution or adaptation of user’s learning behaviour. Learning styles show how a user learns – user’s interaction with educational environment is captured in models like Kolb’s learning style inventory which proposes assessment of similarities and differences in learning styles of students and teachers. According to Kolb in Experiential learning each mode of learning and adaptation require different abilities – learning activities can be captured through four dimensions concrete experiences, reflective observation, analytic conceptualisation, and active experimentation (Claus 2004). (Claus 2004) Conclusion As Soumen et al (2009) states data mining usage may allow analysis on data repository with analysis extending beyond the original scope of data. In this way the analysis may sometimes offer insights which may not be correct as the data represented results which focused on different premise. It becomes a case of knowledge management and knowledge based resources and how these are used. Information overload is as damaging as lack of it as too much information may lead to action paralysis. It is always useful to employ filters to screen data and separate critical from non essential. Focus on low cost solutions and keep a hand on the nub of the market, getting the feel of the consumer and how he or she would behave would enable the company to survive amongst corporate and competitive chaos. Monitoring competition effectively can help highlight areas of concern before these take on damaging proportions (Reviews 2007). References 1. Claus Paul. Data Mining Technology for Evaluation of Learning Content Interaction. International Journal on Elearning. Oct-Dec 2004. pg.47 2. Deepak Pareek. Business Intelligence For Telecommunications. Taylor & Francis Group, LLC. 2007. 3. Soumen Chakrabarti, Earl Cox, Eibe Frank, Ralf Hartmut Güting, Jaiwei Han, Xia Jiang, Micheline Kamber, Sam S. Lightstone, Thomas P. Nadeau, Richard E. Neapolitan, Dorian Pyle, Mamdouh Refaat, Markus Schneider, Toby J. Teorey, Ian H. Witten. Data Mining Know It All. Morgan Kaufmann Publishers. 2009 4. Jiawei Han. Micheline Kamber. Data Mining: Concepts and Techniques. Morgan Kaufmann Publishers. 2006 5. Chethan.M. Data Mining. Retrieved from www.scribd.com/people/documents/789936-chethan-m 6. REVIEWS: Anticipating change, Feb 25, 2007. Retrieved From http://www.dawn.com/weekly/books/archive/070225/books6.htm 7. The Future of BI. http://computerworld.com/printthis/2005/0,4814,104589,00.html. 2005 8. M Jay Norton. Library Trends. Urbana: Summer 1999. Vol. 48, Iss. 1; pg. 9 9. Kurt Thearling. An Introduction to Data Mining. Retrieved from www.thearling.com 10. Peter Fader. Business, More Intelligently. CIOInsight. 2007. Bibliography 1. Jay Liebowitz. Strategic Intelligence. Taylor & Francis Group, LLC. 2007. 2. Anne Walker, Kathleen Millington. Business intelligence and knowledge management. Information Outlook. Washington: Aug 2003. Vol. 7, Iss. 8; pg. 38 Read More