Regression Modelling and Analysis - Term Paper Example

 Abstract Regression Analysis is a major part of every business and is an important tool for business statistics. The straight line regression method has been focused upon. The paper provides complete explanation of the various regression methods, detailed explanation of the regression line using the least square methods and also the interpretation of the regression line. The paper provides a complete, clear and concise explanation of all the aspects of regression modelling and regression analysis. Introduction: Regression modelling and analysis forms an important aspect of business statistics. The paper begins with a detailed explanation of regression analysis and then focuses mainly on the straight line regression method. The formula used to derive the regression line using the least squares methods are also presented followed by a discussion on the importance of fixing the causation prior to a regression analysis. The interpretation of the regression line forms an important part of the analysis and is explained with an illustration for better understanding. It is necessary to establish the ‘goodness of fit’ of the regression line using the coefficient of determination to identify to what extent the regression line explains the movement of the variables. Time series analysis and forecasting is also discussed in detail and further research recommendations are provided in the conclusion part of the paper. Regression Analysis: Regression analysis involves modelling and analysing the relationship between a dependent variable and one or more independent variables. Regression modelling and analysis is primarily conducted to observe or forecast the change in the dependent variable when one or more of the independent variables are changed. Regression differs from correlation - which establishes whether there is any association between two variables (Chatterjee and Hadi, p. 24). Correlation does not imply causation whereas for regression modelling and analysis, it is required to establish the direction of relationship or in other words, the causation. Identifying the direction of relationship or causation establishes the dependent and independent variables. When there is only one independent variable the relationship of that variable with the dependent variable is said to be linear and the regression conducted to identify the numerical connection between the variables is based on linear regression model. When there are multiple independent variables involved, multiple regression model is applied to identify the numerical relationship between the dependent and independent variables. The basic concept of regression is to fix a best fitting line or curve to the data points so that it represents the relationship between the variables (Chatterjee and Hadi, p. 100). This report covers the linear regression model, i.e., the relationship between the two variables are linear. This is also termed as straight line regression as the resulting equation obtained from the analysis is in the form of a straight line. Straight Line Regression: The numerical relationship between the variables is established in the form of a straight line equation. The normal straight line equation is of the form: Y = a X + b In this equation, Y represents the dependent variable, whereas X represents the independent variable. The slope of the equation is given by ‘a’ and it represents the sensitivity of the dependent variable to changes in X. In other words, when X increases by one unit, Y changes by ‘a’ units. The Y – intercept of the equation is given by ‘b’ and it represents the value of Y when the value of X is equal to zero. Fixing the Straight Line – Least Squares Method: The values of ‘a’ and ‘b’ in the regression equation can be computed using the least squares method. The principle of least squares method is to fix the regression line so that the sum of the squares of the perpendicular distances of the data points from the line is kept to a minimum (Mendenhall and Sincich, p. 34). The steps taken to derive the values of ‘a’ and ‘b’ are as follows: 1. The distances of the data points from the regression line Y = aX + b are computed. 2. These distances are squared and the sum of these squares is found. 3. This results in a second order equation with variables ‘a’ and ‘b’. 4. The first differential (dY / dX) is equated to zero to find the values of ‘a’ and ‘b’. 5. To ensure that the values represent the minimum distance, the second differential is found (d2Y / dX2) and verified that it is negative. From the above steps, the equation to find the values of ‘a’ and ‘b’ are derived as: Causation: It is evident from the above equation that the X and Y variables cannot be interchanged. If the variables are interchanged, the resulting equations will be completely different and the regression model will not be reliable. Hence it is essential to establish the direction of relationship and fix the dependent and independent variables before estimating the regression line. For instance, when studying the relationship between the number of hours spent watching television and the grade obtained by the student, it is evident that the grade obtained (Y) is dependent on the number of hours on TV (X) (Mendenhall and Sincich, p. 44). In this case, X causes Y or in other words, Y is dependent on X. Hence X is the independent variable and Y is the dependent variable. The regression analysis conducted to identify the relationship is termed as regression of Y on X. It is imperative to note that interchanging the variables X and Y will be meaningless and will yield useless information. Hence the causation has to be fixed when modelling a regression analysis. Interpretation: As discussed earlier, in the straight line regression method, the resulting equation is in the form of Y = aX + b. It is necessary to interpret this equation properly in order to derive useful information out of the analysis (Gelman and Hill, p. 35). Interpretation of a regression line is explained below with the help of an illustration. Consider the two variables, number of hours (X) watching television daily and the grade obtained (Y). The grade point considered here is based on a maximum score of 5. A sample relationship between the two variables is taken into account and the regression line is then fixed based on the least squares method. Assume that the regression equation obtained is: Y = 4.5 X – 0.25 The following can be inferred from the line obtained from the regression analysis: 1. It is possible to estimate the grade of a student based on the number of hours spent by the student in watching television every day (Gelman and Hill, p. 70). 2. When a student watches no television, it is estimated that he / she will get a grade point of 4.5 (out of 5) as represented by the ‘a’ or the Y – intercept. 3. When a student increases the number of hours by 1 unit, his grade goes down by 0.25, as represented by ‘b’ or the slope of the regression line. Coefficient of Determination: Regression analysis also involves identifying whether the results are reliable. Coefficient of determination is computed to identify the extent to which the regression line explains the movement of variables or the ‘goodness of fit’ of the regression line (Gelman and Hill, p.45). Coefficient of determination is just the square of the Pearson’s correlation coefficient (R) which is computed as follows: The R value can range from - 1 to + 1 and is a measure of the degree of association of the two variables, with ‘0’ representing no relationship, a positive value representing a direct relationship and a negative value indicating an indirect relationship. Squaring the R value gives the coefficient of determination. Hence R2 value can range from 0 to 1 and hence it is represented as a percentage. For instance, an R value of - 0.6 will result in a coefficient of determination of 60 % indicating that 60 % of the movement of the variables are explained by the regression equation obtained using the least squares method and 40 % of the movements are unexplained or unknown. An R value of + 1 or -1 will result in 100 % coefficient of determination indicating that the fit of the regression line is perfect (Mendenhall and Sincich, p. 55). Interpolation and Extrapolation: The regression line can be used to obtain the values of dependent variables both within the range of the data points (interpolation) and outside the range (extrapolation). When interpolating or extrapolating based on the regression line, it is essential to ensure that the equation is of good fit (around 90 % fit) so that the values obtained can be considered to be reliable. Presence of outliers in the initial regression analysis can also adversely affect the results of interpolation and extrapolation (Chatterjee and Hadi, p. 45). Hence these outliers have to be identified and excluded from the analysis so that the most common behaviour of the variables are modelled and extreme and rare ones are neglected. Time Series Analysis and Forecasting: Regression modelling and analysis is mainly conducted to study the relationship between variables and also use the relationship to forecast or estimate the dependent variable by changing the independent variables. Regression has various applications in business administration. The regression model can be applied to estimate sales or demand using historical data and considering time as the independent variable. The resulting equation obtained can then be extrapolated to forecast the sales or demand at a future time period (Chatterjee and Hadi, p. 67). The time series models are always ‘univariate’, i.e., involve only one variable. In the case of time series analysis, the causation is ‘by - passed’. In other words, the causation exists in some other form, but is overcome by utilising time period as the independent variable. Time series analysis eliminates the need to understand the reason for the movement of the variable and is purely focussed on forecasting for the future based on historical data. Conclusion: Thus the regression modeling and analysis are discussed in detail in the report along with some illustrations. It is evident that the main application of regression is time series analysis. This can be taken a step further by de – seasonalization of the time series by identifying regular seasonal movements within each time period (Chatterjee and Hadi, p. 39). Classical decomposition can be applied to separate the parts of the time series and then analysis can be conducted to obtain the regression line. This will enable better forecasts and also explain the variable movements with respect to time in a better and detailed manner. ­­­­­­­­­­­­Further References Box, George E. P., Gwilym M. Jenkins and Gregory C. Reinsel. Time Series Analysis: Forecasting and Control . Wiley, 2008. Chatfield, Chris. The Analysis of Time Series: An Introduction, Sixth Edition. Chapman and Hall/CRC, 2003. Downing, Douglas and Jeffrey Clark. Business Statistics. Barron's Educational Series, 2010. Hamilton, James Douglas. Time Series Analysis. Princeton University Press, 1994. Levine, David M., Timothy C. Krehbiel and Mark L. Berenson. Business Statistics: A First Course . Prentice Hall, 2009. Shumway, Robert H. and David S. Stoffer. Time Series Analysis and Its Applications: With R Examples. Springer, 2006. Works Cited Chatterjee, Samprit and Ali S. Hadi. Regression Analysis by Example. Wiley-Interscience, 2006, 21- 115. Gelman, Andrew and Jennifer Hill. Data Analysis Using Regression and Multilevel/Hierarchical Models. Cambridge University Press, 2006, p. 31 – 74. Mendenhall, William and Terry L. Sincich. A Second Course in Statistics: Regression Analysis. Prentice Hall, 2003. p. 25- 56. Read More