Partial Autocorrelation Functions - Assignment Example

Question No. 8 Box and Jenkins utilized sample autocorrelation function and partial autocorrelation functions to identify order of an ARMA (autocorrelated moving average) model. If y0, y1, ... yT-1 are a sequence of values from a process, the sample autocorrelation function is Rt = ct / c0, t = 0, 1, ..., T-1 Where Ct = 1/T ∑ (yi – y-bar)(yi-t – y-bar) Is the empirical autocovariance at lag t and c0 is the sample variance. The sample partial autocorrelation pt at lag t is the correlation between the two sets of residuals obtained from regressing yi and yi-t on the set of values in between them. Box and Jenkin’s method has three steps, namely model selection, model estimation and model checking. Frequently Box and Jenkin’s is an iterative method and there may be competing candidates to describe a series. To achieve stationarity or remove trend two techniques are usually applied. The first one involves fitting either a parametric model or a spline function. In this case the ARMA model is applied to the residuals. Alternatively, Box and Jenkins recommended taking suitable differences of the process to achieve stationarity. Here the assumption is that the original series is ARIMA and the difference gives rise to the ARMA series. To determine whether the series has been reduced to a stationary series, one may look at the autocorrelations. For a stationary series, the autocorrelation sequence would converge to 0 quickly as lag increases. The time plot given in Figure 2 is already a stationary series as there is no evidence of any trend. Both autocorrelation plot and partial autocorrelation plot need to be looked at simultaneously. The partial autocorrelation become 0 at lag p+1 or greater when the process is AR(p). Strictly speaking the largest PAC is at lag 2 and the second largest at lag 24. These are the only two significant partial autocorrelations. When consider the ACF at lag 24, no significance is noted. However at lag 16 ACF is significant, but no corresponding significance is noted in PACF. Instead of looking at lags such far apart, it is better to see if smaller lags can be considered to identify the model. The order of the process may be taken to be 2. But to come to any definite conclusion an iterative process should be initiated. Question No. 10 In classical linear regression model the observation vector y ~ N(Xβ, σ2I). Want to test β = β*, where β is the parameter vector, X is the design matrix, I is identity matrix and σ2 is the common variance. Estimate of β, the unknown parameter is (XX)-1 Xy. The two components of the observation vector y, the predicted part X β-hat, and the residual y - X β-hat are orthogonal. They are uncorrelated and since they follow multivariate normal distribution, they are also independent. Any function of the predicted random vector and any function of the residual vector will also be independently distributed. Using (9) and (12) given in Lecture 5 and using the result that ratio of two independent chi-square variables divided by their respective degrees of freedom, follows an F distribution with proper d.f. the F-statistic for testing parameter of linear regression is obtained. Under the null hypothesis, the statistic given in (14) follows F(k, T-k). However, if β ≠ β*, then the value of F-statistic is too large, hence reject the null when observed F is large. To interpret it geometrically, we may think of two hyperplanes, one in the direction of the parameter estimate and the other, in the direction of the residuals which is orthogonal to the estimates. When β = β*, the direction of the hyperplane incorporates the specified value of β, and the departure from it is small compared to the dispersion of the residuals. The ‘explained variation’ is smaller compared to the ‘unexplained variation’. However, when the null hypothesis is not true, the actual hyperplane is different from the assumed hyperplane. Hence the ‘explained variance’ is large. When testing hypothesis about a subset of β, the procedure to develop the test statistic is basically the same. Consider two models: Y = X1 β1 + e (M1) Y = X1 β1 + X2 β2 + e (M2) The first one assumes β2 = 0. To test this hypothesis that whether β2 is at all required AFTER fitting a regression model with β1 only, fit both models and obtain residuals from both fitting. Then compute Residual Sum of Squares (RSS) from both models, take difference of RSS(M1) – RSS(M2). This also follows a chi-square distribution with df (df(M1) – df(M2)). The F statistic is F = [(RSS(M1) – RSS(M2)) / (df(M1) – df(M2)]/[RSS(M2) / df(M2)] It has df as df(M1) – df(M2), df(M2) Question No. 11. Let us assume X (height of father) and Y(height of adult son) follow bivariate normal distributions. Since height of son is dependent on height of father, X and Y cannot be independent. Any linear combination of X and Y will follow univariate normal distribution with suitable mean and variance. Consider E(X) = μ1 and E(Y) = μ2. Null hypothesis is H0: μ1 = μ2 against the alternative that Ha: μ1 < μ2, since significant increase in the height from generation to generation is being tested. The random variable Z = X – Y follows normal distribution with mean μ1 − μ2, and variance σ21 + σ22- 2ρ σ1σ2. Under H0, Z follows N (0, Var(Z) = σ21 + σ22- 2ρ σ1σ2) From a sample of size n, compute z-bar (sample mean) and sample standard deviation. If sample variance is denoted by S2Z, then the test statistic is z-bar / SZ (ratio of sample mean and sample std dev). This statistic follows a t-distribution with (n-1) degrees of freedom. Rejection region is on the right hand side, i.e. Reject H0 if the observed value of the test statistic is too large. The reason being, if son’s height is significantly more than father’s height, the value of the test statistic will be large. The critical point will be determined by the size of the test and sample size. Question No 12. Following the notations of (10) in Lecture 6, the system of equations may be written as Yc = (IM ⊗ T)α +( IM ⊗Xj )Bc + ξc (a) When Xj = X: Yc = (IM ⊗ T)α +( IM ⊗X)Bc + ξc When βj = β: Yc = (IM ⊗ T)α +( IM ⊗Xj ) β + ξc When βj = β, αj = α Yc = ιMTα +( IM ⊗Xj ) β + ξc If the disturbances are independently and identically distributed, then for (a) each of the equations are separately estimable and their efficient estimates will be the ordinary linear model estimates. β.j = X’j (I − PT )Xj−1X’j (I − PT )y.j and ˆαj = yj-bar − xj-bar. ˆ β.j , where I − PT = I − ιT (ι_T ιT )−1ι_T (b) Here all the regressors are the same. Efficient estimator of β is given by the full set of TM mean adjusted equations are to be taken together. The dimension pf PT above will be changed. (c) When further restrictions are incorporated, the estimate of regressors are given as above (part b) and estimate of α is y-bar – β-hat x-bar (all MT observations taken together, as if a MT dimensional vector is observed from the same system). To test hypotheses related to the above conditions, progressively restricted models are fitted to the data and the residual sums of squares are compared through chi-square and F statistics. If S is the RSS for the most general model, then F (all β’s are the same) = [(Sβ – S)/(M-1)K] / [S / (MT – M(K+1))], which follows F ((M-1)K, MT – M(K+1)) Sβ is the residual SS for the restricted model Consider Sγ to be the RSS for the restricted model where all β’s are the same AND all α’s are the same. F = [(Sγ – Sβ)/(M-1)] / [Sβ / (MT – (K+M))], which follows F(M-1, MT – K – M) In all cases rejection region is right sided. Such hierarchical testing is possible since with one restriction after another, the parameter space is getting reduced and the residual sums of squares are increasing. Read More