or
F(Yt,Xt,β) =
ρ(B)-1θ(B)ut
ρ(B) = 1 - ρ1B - ρ2B2 - ... -
ρpBp
θ(B) = 1 + θ1B + θ2B2 + ... +
θqBq
where B is the back-shift operator.
In the following, we assume F(Yt,Xt,β) = Yt - f(Xt,β) and
ut = θ(B)-1ρ(B)(Yt-f(Xt,β)) ~ nii(0,σ2).
We assume |ρ| < 1 for model stability. It is clear that
σ2 = Var(ut) = (1-ρ2) Var(εt).
Therefore, define u1 = (1-ρ2)½ε1 and ut = εt-ρεt-1, t=2,...,N.
The log-likelihood function of Yt derived from the normal density of ut is
ll(β,ρ,σ2|Y,X) = -½N(ln(2π)+ln(σ2)) + exp(-u'u/(2σ2)) + ∑ln(Jt)
with the following Jacobian transformation from ut to Yt (depending on ρ only):
| Jt(ρ) = |∂ut / ∂Yt| = | (1-ρ2)½| for t=1 | |
| 1 | for t>1 |
Therefore, the (exact) concentrated log-likelihood function is:
ll*(β,ρ|Y,X) = -½N (1+ln(2π)-ln(N)) +½ ln(1-ρ2) -½N ln(u'u)
Extension: AR(2)
The model is defined as εt = ρ1εt-1 + ρ2εt-2 + ut with the following proper transformation:
Again, we assume |θ| < 1 for model stability. The model is
ut = Yt - f(Xtβ) - θut-1
Notice that the one-period lag of error terms, ut-1, is used to define the model error ut. A recursive calculation is needed with proper initialization of u0. For example, set the initial value u0 = E(ut) = 0 (or alternatively the sample mean of ut), then
u1 = ε1 = Y1-f(X1,β) and
ut = εt - θut-1 =
Yt-f(Xt,β) - θut-1 for t=2,...,N.
Since each log-jacobian term vanishes in this case, the (conditional) concentrated log-likelihood function is simply
ll*(β,θ|Y,X) = -½N (1+ln(2π)-ln(N)) -½N ln(u'u)
This is the mixed process of AR(1) and MA(1). Using the variable transformations as of AR(1) and data initialization as of MA(1),
u1 = (1-ρ2)½ ε1
ut = εt - ρεt-1 - θut-1, t=2,...,N
the (conditional) concentrated log-likelihood function for parameter estimation is
ll*(β,ρ,θ|Y,X) = -½N (1+ln(2π)-ln(N)) +½ ln(1-ρ2) -½N ln(u'u)
This example demonstrates the nonlinear maximum likelihood estimation for three basic autocorrelated regression models: AR(1), MA(1), and ARMA(1,1). Based on the U. S. investment data from Greene's Table 13.1 (Data) formulate and estimate the three models of autocorrelation for a linear real investment relationship with real GNP and real interest rate:
Invest = β0 + β1 Rate + β2 GNP + ε
Consider the time series regression model (linear or nonlinear):
F(Yt,Xt,β) = εt
At time t, conditional to the available historical information Ht, we assume that the error structure follows a normal distribution:
εt|Ht ~ n(0,σ2t)
| where σ2t | = α0 + δ1σ2t-1 + ... + δpσ2t-p + α1ε2t-1 + ... + αqε2t-q |
| = α0 + ∑i=1,2,...pδiσ2t-i + ∑j=1,2,...qαjε2t-j |
Let υt = ε2t-σ2t, αi = 0 for i > q, δj = 0 for j > p, and m = max(p,q), the above GARCH(p,q) process may be conveniently re-written as an ARMA(m,p) model for ε2t. That is,
ε2t = α0 + ∑i=1,2,...m (αi+δi)ε2t-i - ∑j=1,2,...pδjυt-j + υt
This is the general specification of auto-regressive conditional heteroscedasticity, or GARCH(p,q), according to Bollerslev [1986]. If p = 0, then it is the GARCH(0,q) or simply ARCH(q) process:
σ2t = α0 + ∑j=1,2,...qαjε2t-j
ARCH(1) Process
The simplest case is q = 1, or ARCH(1), originated in Engle [1982] as follows:
σ2t = α0 + α1ε2t-1
ARCH(1) model can be summarized as follows:
F(Yt,Xt,β) = εt
εt = ut(α0 + α1ε2t-1)½
where ut ~ nii(0,1)
Then, the conditional means E(εt|εt-1) = 0 and the conditional variances σ2t = E(ε2t|εt-1) = α0 + α1ε2t-1
Note that the unconditional variance of εt is
E(ε2t) = E(E(ε2t|εt-1)) = α0 + α1E(ε2t-1).
If σ2 = E(ε2t) = E(ε2t-1), then σ2 = α0/(1-α1) provided that |α1| < 1. Therefore, the model may be free of general heteroscedasticity although the conditional heteroscedasticity is assumed.
The ARCH(1) process can be generalized (therefore the name Generalized Auto-Regressive Conditional Heteroscedasticity) to:
GARCH(1,1) Process
σ2t = α0 + α1 ε2t-1 + δ1 σ2t-1
This resembles the mixed auto-regressive moving-average process ARMA(1,1) as described in autocorrelation. Presample variances and squared error terms can be initialized with ∑t=1,2,...,N ε2t/N. The following parameter restrictions are necessary to preserve stationarity of the error process:
Another extension is ARCH or GARCH in mean (ARCH-M or GARCH-M model) which adds the heteroscedastic variance term directly into the regression equation (assuming linear model):
ARCH-M(1) or GARCH-M(1,1) Model
εt = Yt - Xtβ - γσ2t
σ2t = α0 + α1 ε2t-1 (or σ2t = α0 + α1 ε2t-1 + δ1 σ2t-1)
The last variance term of the regression may be expressed in log form or in standard error σt. For example, Yt = Xtβ + γln(σ2t) + εt. Moreover, constraints on the parameters in the conditional variance equation may be required to ensure the positivity of variances: α0 > 0, 0 ≤ α1 < 1 (or α1 + δ1 < 1, δ1 ≥ 0).
ll = -½N ln(2π) - ½ ∑t=1,2,...,Nln(σ2t) - ½ ∑t=1,2,...,N(ε2t / σ2t)
with the general conditional heteroscedastic variance GARCH(p,q) process:
σ2t = α0 + α1ε2t-1 + α2ε2t-2 + ... + αqε2t-q + δ1σ2t-1 + δ2σ2t-2 + ... + δpσ2t-p
The parameter vector (α,δ) is estimated together with the regression parameters (e.g., εt = Yt - f(Xt,β)) by maximizing the log-likelihood, conditional to the starting values ε02, ε2-1, ..., ε2-q, σ20, σ2-1, ..., σ2-p and satisfying the nonnegativity requirement for the estimated variances: σ2t > 0, t=1,2,...,N.
We note that the presample series: ε02, ε2-1, ..., ε2-q, σ20, σ2-1, ..., σ2-p may be initialized by the estimated (homoschedastic) unconditional variance:
1 / [1 - (∑i=1,2,...,qαi + ∑j=1,2,...,pδj)]
or by the estimated sample variance of residuals:
∑t=1,2,...,Nε2t/N.
The model of interest is
Yt = 100 [ln(Pt - ln(Pt-1)] = μ + εt
where Pt is the bilateral spot Deutschemark-British pound exchange rate. Thus Yt is the daily percentage nominal returns of BM/BP exchange. Test, identify, and estimate the appropriate GARCH(p,q) variance structure.