Data Mining and Data Fusion for Direct Marketing - Term Paper Example

DATA MINING AND DATA FUSION FOR DIRECT MARKETING Data mining and data fusion are the exploration and analysis, by automatic or semiautomatic means of large quantities of data in order to discover meaningful and potentially useful information, such as patterns, knowledge rules, regularities, etc. There are also many other terms which carry a similar or slightly different meaning, such as knowledge discovery, knowledge mining from databases, knowledge extraction, data analysis and so on. In contemporary business context, the wide spread use of bar codes for most commercial products, the computerisation of many business and government transactions, and the advances in data collection tools have provided us with huge amounts of data at unprecedented rates. Data is generated by transactions that form the foundation of many industries, such as retail, manufacturing, utilities, transportation, insurance, credit cards, and banking. In addition to these internal data, external data sources also provide demographic, lifestyle, and credit information on retail customers, and credit, financial, and marketing information on business customers. Data mining is a very useful tool to analyse business data and to use it to identify key customers and in turn increase business opportunities by targeted marketing. From the direct marketing perspective, data mining and data fusion provide a necessary means to collect and analyse customers' data in order to utilise direct marketing strategy most effectively. Data mining and data fusion are the exploration and analysis of large quantities of data in order to discover meaningful patterns and rules (Berry et al., 2004). Used in various simpler forms in earlier times, mining and fusion have made way for businesses to finally make some sense out of all the data that they have accumulated for years (Rudd, 2000). Several books and journals have been published to strengthen the cause of these tools to improve relations between customers and generate more business. Some other areas where data mining and data fusion have been extensively used include credit scoring, direct marketing, sales forecasting, insurance, manufacturing, telecommunications, web-mining and text mining. An area pertaining to the scope of this paper is the application of data mining and fusion in direct marketing. Mining and fusion are useful tools in almost all aspects of a business and direct marketing. It helps in building supporting systems for day-today business. It is useful in forecasting trends, it is used in decision making processes. It can be used in strategic planning of the course of action to be followed (Berson et al., 1999). A significant application in this area is the usage of data mining and fusion as tools in customer marketing and effective promotion (Berry et al., 2004). Customer acquisition is another such application. Statistical modeling using data mining and fusion are useful in effectively segmenting customers so that suitable marketing efforts can be carried out (Berson et al., 1999). Estimation of customer profitability is another such vital application of data mining. Determining customer segments help establish better marketing and services practice (Berson et al., 1999). Data mining and fusion thus help businesses to a very great extent in managing customers and helps maximise the tenure of relationship with customers, optimise the transactions or business carried out, increase profits associated (Berry et al., 2004). Mining is an activity that processes raw data or information recorded on a transactional basis. Earlier in smaller companies, this 'mining' process was carried out by people who dealt directly with customers. Now with companies that have billions of customers, a system has to be established to carry out these activities. Mining is a follow-up and is closely connected with data warehousing (Berson et al., 1999). If useless data has been collected and warehoused, the mining process will yield no better results. At the same time, having too little data would affect the results gravely, making it inconsistent. Warehousing thus provides an efficient start to the mining process as opposed to depending on huge bulky systems that are very dynamic in nature (Berson et al., 1999). Data mining as the name suggests mines for information. It does not create any new data, but rather uses the existing data to relate several factors together to put forth a predictive model that will help read into customer behaviour (Saarenvirta, 1998). In the recent times, data mining has become a very powerful business tool that is supported by three significant components namely, data or information, high computing power and choice of several mining algorithms or solutions (Thearling, 1995). With latest technology comes the ability to process more complex data and give results that are much more accurate (Seifert, 2004). The infrastructure needed for data mining is fairly simple and manageable. There are several good software in the market that serve as an efficient means to mine data. Data mining activity can be classified into two types - verification oriented and discovery oriented. Verification-oriented mining helps verify or disqualify user's hypothesis about the data. It normally involves conventional statistical methods and analysis. Discovery-oriented mining identifies new rules and patterns from existing data, without any manual statistical analyses. Most recent applications of data mining have all been of this type. Several scenarios are possible in which both these methods are used (Maimon et al., 2000). In direct marketing, data mining techniques can be used to segment customers into meaningful groups. Firstly, the data that is used to "mine" comes from previous historic data about customers. This data could be available in abundance (Groucott et al., 2004). Particularly for large organisations, the data is available in excess and could be warehoused for years. Mining helps extract only required data from this information system, so that analysis is carried out only on relevant data. Before carrying out any segmenting activity, the data is first filtered out and then subjected to some analysis to determine any correlation between various attributes in the customer data. This will be very useful in eliminating the need for excessive attributes, which would weaken the analysis (Groucott et al., 2004). Once this cleaning is carried out, the data can then be segmented using some of the following techniques: (1) Clustering - this is a fairly popular tool that is used in segmentation. This method focuses on creating and managing groups or clusters whose members least differ from each other in attributes. A more acceptable clustering technique used for this purpose is iterative in nature like K-means clustering. However, the kind of data and the purpose for segmentation is very important in determining if clustering will be a useful tool. (2) Neural Networks - this technique is a far more reliable method to be used in segmentation. Although, mostly limited to researching at first, this tool has now been built into existing mining systems and has thus gained appreciation. The technique predicts the behaviour of the customer, based on previous behaviour, to which it assigns weights depending on its importance. Once this model is trained and validated sufficiently, it could be a near-accurate tool that can be used in (3) Hotspot Analysis - known to be a more efficient clustering technique, hotspot analysis classifies consumers into segments based on their attributes, such as behaviour, profitability, demographic data, etc. In addition to this, hotspot analysis identifies maximum response attributes that would help achieve a better result than clustering that focuses heavily on patterns in data (Groucott et al., 2004). In direct marketing, algorithms of data mining can be utilised to reveal patterns of consumers in order to segment those consumers who are likely to respond to the product or service. From practical perspective, if banking institution plans to develop a co-brand credit card in partnership with a mobile communication company (typical example of co-brand projects in banking industry), the following data mining algorithm should be utilised: (1) Obtain the database of all customers, among which X% are clients actively using their credit cards; (2) Conduct data mining on the dataset; Analysis on geo-demographic and financial information on the database; Conduct data pre-processing, namely fix the missing values, prepare contact information etc. Divide the database on training and testing sets; Conduct an application of learning algorithms to the training set (3) Analyse the patterns and numbers found during testing test phase; (4) Utilise the patterns obtained from testing test phase analysis to forecast likely clients among the customers who do not actively use their credit cards; (5) Conduct a rollout (start direct marketing campaign) to the likely clients. In case, a banking institution decides to offer a new financial product to be promoted to its clients in the database (thus, none of the clients are consumers), a pilot study is conducted, in which a small part (traditionally, less than or 5%) of the client base becomes a target for direct marketing campaign. Data mining is conducted during the pilot study in order to receive information on likely-consumers in the whole client base. From the critical perspective, these two illustrated example are typical and similar from the data mining viewpoint except that the dataset for mining in the second example is smaller. Benefit data mining provides for direct marketing can be evidently derived from continuation on scenario discussed above. Banking institution has a choice of conducting direct marketing mail campaign or mass marketing mail campaign, as illustrated in Table 1.1. The whole database consists of 1,200,000 clients. In direct mailing, only one fifth of the clients identified as likely consumers by data mining (which costs $170,000) are picked to receive the promotion package in the mail for bank monthly statements. The mailing cost is thus decreased dramatically. Simultaneously, the response rate can be improved from 1% in mass mailing to 3%, which is considered a realistic improvement for a 20% rollout in direct marketing. In this example, direct marketing campaign yields the net profit while mass mailing - a loss. It is evident that the importance of integrating good data mining and data fusion techniques with direct marketing is critical to the effectiveness of promotion and overall success of campaign/research. From the critical perspective, as revealed in several practical examples data pre-processing is vital, because even simple observations of data could yield useful insight about clients' attributes, which are critical for modeling. REFERENCES Berry, M., Linoff, G., 2004. Data Mining Techniques : For Marketing, Sales and Customer Relationship, Wiley, Second Edition, April Berson, A., Smith, S., Thearling, K. 1999 Building Data Mining Applications for CRM, Mc Graw Hill, December. Groucott, J., Leadley, P., Forsythe, P. 2004. Marketing: Essential Principles, New Realities, Kogan Page Publishing Maimon, O., Last, M.. 2000. Knowledge Discovery and Data Mining, Kluwer Academic publishers Rudd, O. 2000. Data Mining Cookbook, Wiley Publishing, November 2000 Saarenvirta, G. 1998. "Mining Customer Data", DB2 Magazine, Fall. Seifert, J. 2004. "Data Mining: An overview", Congressional Report Service for Congress, December. Thearling, K. 1995. "From Data Mining to Database marketing", DIG White Paper, Retrieved from < http://www.thearling.com/text/wp9502/wp9502.htm> April 15, APPENDIX Table 1.1 A comparison between direct mail campaign and mass mail campaign. Mass mailing Direct mailing Number of customers mailed Cost of printing, mailing ($0.71 each) Cost of data mining Total promotion c ost 1,200,000 $852,000 $0 $426,000 (20%) 240,000 $170,400 $40,000 $125,200 Response rate Number of sales Profit from sale ($70 each) 1.0% 6,000 $420,000 3.0% 3,600 $252,000 Net profit from promotion -$6.000 $126,800 Read More