Heteroscedastic Regression Models

Readings and References:

• W. H. Greene, Econometric Analysis, 5th Ed., Chapter 11: Heteroscedasticity, Prentice-Hall, 1999.
• A. C. Harvey, "Estimating Regression Models with Multiplicative Heteroscedasticity," Econometrics, 1976, 461-465. (Paper)
• K.-P. Lin, Computational Econometrics: GAUSS Programming for Econometricians and Financial Analysts, Chapter 9: Heteroscedasticity, Lesson 9.4.

ll(b,a,s²|Y,X) = -½N (ln(2p)+ln(s²)) -½ ln(|W|) -½ (1/s²)(e'W^-1e) + S_i=1,2,...,N ln(|¶e_i/¶Y_i|)

The last term is the sum of log-Jacobians from e_i to Y_i over the entire sample. Since the variance term can be solved as s² = e'W^-1e / N for a given W, the concentrated log-likelihood function is

ll^*(b,a,s²|Y,X) = -½N (1+ln(2p)-ln(N)) -½N ln(e'W^-1e) -½ ln(|W|) + S_i=1,2,...,N ln(|¶e_i/¶Y_i|)

It is clear that the maximum likelihood estimation is in general not equivalent to the nonlinear least squares unless W = I, the identity matrix, and ¶e_i/¶Y_i = 1 for each i=1,2,...,N. If W is known and the log-Jacobians vanish, it is the GLS (Generalized Least Squares) problem that minimizes e'W^-1e.

Unfortunately, W = W(a) is not known and must be parameterized with a lower dimension a, which in turn is estimated together with the vector of model parameters b. The models with heteroscedastic and/or autocorrelated errors are the special cases of the general regression model in which W(a) is defined more specifically.

For simplicity, consider a regression model e = F(Y,X,b) = Y - f(X,b). Then for each data observation i, ¶e_i/¶Y_i = 1. We assume further that the heteroscedastic error e_i ~ normal(0,s_i²). The log-likelihood function is

ll(b,s_i²|Y_i,X_i) = -½ [ln(2p) + ln(s_i²) + e_i²/s_i²]

Summing over a sample of N observations, the total log-likelihood function is written as

ll(b,s₁²,s₂²,...,s_N²|Y,X) = -½N ln(2p) -½ S_i=1,2,...,Nln(s_i²) -½ S_i=1,2,...,N(e_i²/s_i²)

Given the general form of heteroscedasticity, there are too many unknown parameters. For practical purpose, some hypotheses of heteroscedasticity must be assumed:

s_i² = s² h_i(a)

where s² > 0 and h_i(a) is indexed by i to indicate that it is a function of Z_i. That is h_i(a) = h(a|Z_i), where Z is a set of independent variables that may or may not be coincide with X. Depending on the form of heteroscedasticity h_i(a), denoted by h_i for brevity, the log-likelihood function is written as

ll(b,a,s²|Y,X) = -½N (ln(2p) + ln(s²)) -½ S_i=1,2,...,Nln(h_i) -½(1/s²)S_i=1,2,...,N(e_i²/h_i)

Let e_i^* = e_i / Öh_i and substitute out the maximum likelihood estimator of s² with e^*'e^*/N, then the concentrated log-likelihood function is

ll^*(b,a|Y,X) = -½N (1+ln(2p)-ln(N)) -½ S_i=1,2,...,Nln(h_i) -½N ln(e^*'e^*)

The last two log-terms can be combined as:

ll^*(b,a|Y,X) = -½N (1+ln(2p)-ln(N)) -½N ln(e^**'e^**)

where e^** = e^*Öh, and h = (h₁h₂...h_N)^1/N. It becomes a weighted nonlinear least squares probelm with the weighted errors defined by e_i^** = e_iÖ(h/h_i).

Consider the following special cases of h_i = h(a|Z_i) = h(Z_ia):

1. s_i² = s²(Z_ia), Z_ia > 0
2. s_i² = s²(Z_ia)²
3. Multiplicative Heteroscedasticity: s_i² = s²exp(Z_ia)

   The corresponding concentrated log-likelihood function for estimation is

   ll^*(b,a|Y,X) = -½N (1+ln(2p)-ln(N)) -½ S_i=1,2,...,NZ_ia -½N ln(e^*'e^*)

   where e_i^* = e_i / exp(Z_ia)^½ for each observation i=1,2,...,N.

   Equivalently,

   ll^*(b,a|Y,X) = -½N (1+ln(2p)-ln(N)) -½N ln(e^**'e^**)

   e_i^** = e_iÖ(h/h_i), h = (h₁h₂...h_N)^1/N, and h_i = exp(Z_ia) for each observation i=1,2,...,N.

4. Exponential Heteroscedasticity: s_i² = s²P_m=1,2,...,M Z_im^a_m, where M is the number of variables in Z_i. This is equivalent to the multiplicative case if the variables in Z are logs. A special case, with a single variable, is

   s_i² = s² Z_i^a

   If a = 0, the model is homoscedastic; If a = 2, it is the case (ii).

Example

Y = b₀ + b₁ X + b₂ X² + e

Find and compare the maximum likelihood estimates based on the following hypotheses of heteroscedasticity (Program and Data):

1. s_i² = s² X_i²
2. s_i² = s² X_i^a
3. s_i² = s² exp(aX_i)