Line of Best Fit Squares Regression LIne - Coursework Example

Line of Best Fit/ Least Squares Regression Line RAJESHWARI DEVERAPALLI 7/27/2007 This paper gives the understanding of the line of best fit and its approach to linear regression, where and how they are applied with examples and the different models of regression with uses and purposes. INTRODUCTION PAGE TO BE INCLUDED BY CUSTOMER Understanding the Line of best fit and linear Regression Many statistical quality control problems need the linear relationship between two or more variables. To find the linear relationship between two variables that is variable x and variable y we plot a graph taking the variables of x on the horizontal bar and the variables of y on the vertical bar .When we plot a graph we obtain the dots which tend to cluster along a well defined direction which suggests a linear relationship between the two variables x and y. When the dots of a scatter diagram tend to cluster along a well defined direction and a line can be drawn which gives the linear relationship between the two variables x and y is called the line of best fit. Such a line of best fit for the given distribution is called the linear regression. In general, the feature of linear regression is to find the line that best predicts y from x or the line that predicts x from y, linear regression does this by finding the line that minimizes the sum of the squares of the vertical distances of the points from the line. The term "regression", is mainly used to find the relationship between two or more variables used in different context. The method was first used to examine the relationship between the heights of fathers and sons. The two were related, and found that a tall father tended to have sons shorter than him; a short father tended to have sons taller than him. The height of sons regressed to the mean. The term linear regression is also referred to as the lines of regression. Types of regression models: There are two types of regression models. 1) Simple regression model 2) Multiple regression model Simple regression model is of two types a) Linear simple regression model. b) Non linear simple regression model. Multiple regression model is again of two types a) Linear multiple regression model. b) Non linear multiple regression model. There are two types of linear regression models 1) Simple linear regression model 2) Multiple linear regression model Simple linear regression In simple linear regression model we have the relationship between the variables is a linear function. Equation of the form y= a + b x is a linear equation with dependent variable y and independent variable x. When we draw a graph of this equation we obtain points which cluster in a particular direction which gives us the linear relationship between the two variables apart from the error or residual. We have the y-value of the data set and we have the y-value given by the equation y = a x + b (remember a is slope, x is x-value, and b is y-intercept). In order to calculate the line of best fit we make use of the principle or method of least squares.In general the goal of linear regression is to find the line of best fit which gives or predicts the value of y from x or the value of x from y. It can be done by minimizing the sum of squares of the vertical distances of the points from the line. It assumes that the data is linear and finds the slope and intercept that make a line which is straight and best fit for the data. Multiple linear regression In multiple linear regression models the variable is influenced by many factors. When we have a variable being compared to lot of factors or lots of comparisons are to be made .For example if the IQ quotient of the students of a class are to be compared we first find the most talented person of the class and then compare him with all the other students in order to test the IQ quotient of all the students. This can be done by least squares method. The multiple linear regression equation is of the form y=a1+a2x2+a3x3+a4x4+-------------------- We again solve this equation by the same method of least squares where multiple equations are to be solved to obtain the curve of best fit. There are different methods of solving linear regression models. The most effective approach usually used to solve the regression models is by the method of least squares method or curve fitting method. The principle of least squares This principle provides a more elegant procedure for fitting a unique curve or line for the given data. In general to solve a given equation of the form y=a+bx+cx2+----------+kxm-1 Eq-A We fit a curve to the given set of n data points (x1,y1),(x2,y2),----------------(xn,yn) Now we have to determine the constants a, b, c, -------k such that it represents the curve of best fit. Substituting the values of (xi,yi) in Eq-A, we get n equations from which the set of n constants can be found this happens only if n=m. If n>m, we obtain n equations which are more than the m constants and cannot be solved for these constants. At a particular point x=xi we get an experimental or observed value and the corresponding value on the curve given by Eq-A which is the expected value or calculated value. The difference between the expected and the observed values is called the residual or error given by the equation yi - µi = ei at x=xi where µi=a+bx+cx2+--------+kxm-1 .The errors so obtained will either be negative or positive. To give equal weightage to each error, We square each of these and obtain their sum E2=e12+e22+---------+en2.The curve or line of best fit is that for which the ei`s are small as possible or the sum of squares of the errors is minimum is called the principle of least squares. The linear regression model is used for calculating the line of regression of y on x or the line of regression of x on y for which we use the method or principle of least squares. Notation and naming convention The notations used in calculating the linear regression model are 1) xi = x1, x2, x3,--------------- the x series 2) yi = y1, y2, y3,----------------the y series 3) The sum of all the x terms, = 4) = The sum of all the y terms= 5) n is the total number of entries in the data 6) a and b are the variables to be determined 7) =The summing of all the values which are obtained by multiplying each x term with each y term. 8)  is the average of all the y terms and  is the average of all the x terms. 9)  is the sum of all the deviations of x from the average of x . 10)gives the sum of all the deviations of y from the average of y.= 11)  The deviations of x from the average of x is multiplied by the deviations of y from the average of y for each data and then is added. = 12) gives the sum of squares of all the x terms 13) σx is the standard deviation of all the x terms 14) σy is the standard deviation of all the y terms 15) r is the coefficient of correlation and is defined by r=  16) Sx = σx2) is called the standard estimate of x. And Sy= σy2) is called the standard estimate of y. Method for solving a linear regression problem We consider the following equation yi = a + b xi Eq- 1 Where yi`s are dependent variables and `s are independent variables Now we have to determine the constants a and b so that Eq- 1 gives for each value of x, the best estimate for the average value of y accordance with the principle of least squares. We first calculate the total of dependent variables and then calculate the total of all the independent variables=  Then we obtain equations of the form  Eq-2  Eq-3 The normal equation for the unknown value a is obtained by multiplying the equations by the coefficient of a and adding. The normal equation of b is obtained by multiplying the equations by the coefficient of b and adding. Solve these normal equations as simultaneous equations for a and b. Solve Eq-2 and Eq-3. Substitute the values of` a and b in eq-1which gives us the line of best fit. Now we calculate the averages  Eq-4 Where  and  Where  is the average of all the y terms and  is the average of all x terms.  Eq-5 This shows that the average of x and average of y lie on Eq- 1. Sifting the origin the means of x and y Eq- 3 takes the form  Eq-6  Eq-7 Since the deviations of the values of x from the mean of x is 0. Therefore solving Eq-3, Eq-4, Eq-5, Eq-6 and Eq-7 we get  Eq-8 Thus the line of best fit becomes  Eq-9 This is the equation of line of regression of y on x. Its slope is called the regression coefficient of y on x. Application of how linear regression is used Linear line of regression analysis can be applied for finding the regression of sales on IQ scores of marketing men which can be seen from the following example. The data on marketing men, their scores and sales of each marketing person is given in table 1. Table 1: Data of marketing men’s sales and their IQ scores Marketing men 1 2 3 4 5 6 7 8 9 10 IQ Scores 40 70 50 60 80 50 90 40 60 60 Sales in $ 2.5 6.0 4.5 5.0 4.5 2.0 5.5 3.0 4.5 3.0 To analyze the regression of sales on IQ scores we have to calculate the mean of IQ scores and sales which are represented by  = 60 and  = 4.5 and then calculate = 0.06 as per the calculations shown in the following table IQ scores (x) Sales (y) Deviation of x from assumed mean dx Deviation of y from assumed mean dy dx .dy dx2 dy2 40 2.5 -20 -2 40 400 4 70 6.0 10 1.5 15 100 2.25 50 4.5 -10 0 0 100 0 60 5.0 0 0.5 0 0 2.25 80 4.5 20 0 0 400 0 50 2.0 -10 -2.5 25 100 6.25 90 5.5 30 1 30 900 1.00 40 3.0 -20 -1.5 30 400 2.25 60 4.5 0 0 0 0 0 60 3.0 0 -1.5 0 0 2.25 From r and using Eq-9 the line of regression of y on x is given below  For x = 70 the line of regression of y on x is given by y=4.65. Since the difference between the estimated value and the calculated value is 0.6 it is concluded that there is a positive relation between the sales and the IQ scores of marketing persons. To calculate the second type of line of regression that is x on y we consider the following equation x = a + b y Eq-10 Again it is solved in the same manner as regression line of y on x by the method of least squares.  y Eq-10  y Eq-11  y Eq-12  y Eq-13  Eq-14  Eq-15  y Eq-16  is the equation of the line of regression of x on y. Now to obtain the line of regression of x on y and from the calculations done from the above example we have  For y = 5.5 the line of regression of x on y is given by x=.60.67. Since the difference between the estimated and the calculated value of x = 0.67 there is a positive relation between the sales and IQ scores. Now we have the geometric mean between the two regression coefficients is the correlation coefficient r and is given by r=r2 and calculate the standard error of estimate x and estimate y by using Sx = σx2) and Sy=σy2) Since the sum of squares cannot be negative we have r2 Read More