Instrumental Variables

For estimation of a classical linear regression model Y = Xβ + ε, recall the important Assumption 3:

E(ε|X) = 0

That is, E(ε_i|X) = 0, i=1,2,...,N. This implies:

E(ε) = E(E(ε_i|X)) = E(0) = 0
E(X'ε) = E(E(X'ε|X)) = E(X'E(ε|X)) = 0
Therefore, Cov(X,ε) = E(X'ε)-E(X')E(ε) = E(X'ε) = 0
For large sample, we assume a weaker form as:
plim(Xε/N) = 0.

There are many occasions such as omitted variables or errors in the explanatory variables, which result a clear violation of Assumption 3 for the classical linear regression model. The consequence is that the least squares estimator is biased, inconsistent and inefficient.

Instrumental Variables

A replacement for X, called Z, may be used to restore the consistency of least squares estimation. Z must satisfy the following conditions:

Z is not correlated with the model error ε: E(ε|Z) = 0 or plim(Z'ε/N) = 0
plim(Z'X/N) = Cov(Z',X) = Σ_ZX is finite and nonsingular
plim(Z'Z/N) = Var(Z) = Σ_ZZ is finite and positive definite
The number of columns in Z ≥ The number of columns in X

Z is called the instrumental variables for X. Z may include all or part of X variables as long as they are exogenous.

Instrumental Variable Estimation

The estimation consists of two steps as follows:

Formulate a multivariate regression as X = Zδ + u, and estimate the parameter vector of δ as:
d = (Z'Z)^-1Z'X, and
X^p = Zd = Z(Z'Z)^-1Z'X.
Using OLS to estimate the model Y = X^pβ + ε:

b = (X^p'X^p)^-1X^p'Y

= [X'Z(Z'Z)^-1Z'X]^-1 X'Z(Z'Z)^-1Z'Y

It is clear that the selection of instrumental variables is crucial for a successful estimation of the model parameters. In practice, in addition to the replacement for endogenous explanatory variables, the instrumental variables include the exogenous explanatory variables already in the model. Therefore, the instrumental variable estimation (IV) is summarized as:

Define W = Z(Z'Z)^-1Z'X, and note that W'X = W'W.
b = (W'X)^-1W'Y = [X'Z(Z'Z)^-1Z'X]^-1 X'Z(Z'Z)^-1Z'Y
Var(b) = s²(W'X)^-1 = s²[X'Z(Z'Z)^-1Z'X]^-1
where s² = e'e/(N-K) and e = Y - Xb.

Two Special Cases

If Z and X has the same number of columns, then
b = (Z'X)^-1Z'Y
Var(b) = s²(Z'X)^-1
If Z = X, then it is OLS.

Robust Inference of IV Estimator

If the classical assumptions of homoscedasticity and no autocorrelation are not satisfied, in conjunction with the problem of endogenous independent variables (i.e., lagged dependent variable), the linear regression model is presented as follows:

Y = Xβ +ε, ε|Z ~ iid(0,Ω), Ω ≠ σ²I
b = (X^p'X)^-1X^p'Y = [X'Z(Z'Z)^-1Z'X]^-1X'Z(Z'Z)^-1Z'Y
Var(b) = (X^p'X)^-1(X^p'ΩX^p)(X^p'X)^-1
where X^p'ΩX^p = X'Z(Z'Z)^-1(Z'ΩZ)(Z'Z)^-1Z'X

The estimation of Var(b) depends on the estimation of the consistent estimator of Z'ΩZ = Z'E(εε')Z. A robust estimate of the variance-covariance matrix can be based on Newey-West estimator allowing general heteroscedasticity and autocorrelation. That is,

Σ = Z'ΩZ = S₀ + ∑_j=1,...,J[1-j/(J+1)](S_j+S_j')
where
S₀ = (1/N)∑_i=1,...,Ne_i²z_iz_i'
S_j = (1/N)∑_i=j+1,...,Ne_ie_i-jz_iz_i-j'
Note: e_i = Y_i- X_ib, i=1,...,N.

Generalized Method of Moments

Consider the model: Y = Xβ +ε, ε|Z ~ iid(0,Ω), a more general estimation approach is the Generalized Method of Moments (GMM). The key assumption is the exogeneity assumption of the intruments:

E(ε|Z) = 0

This implies the moment functions E(Z_iε_i) = E(Z_i(Y_i-X_iβ)) = 0. GMM estimator of β is obtained to minimize the objective function which is the weighted quadratic form of the moment functions:

Q(β) = (Z'ε/N)'W(Z'ε/N)

where W is the weighted matrix which is symmetric positive definite. We have,

b = [X'ZWZ'X]^-1X'ZWZ'Y
Var(b) = [X'ZWZ'X]^-1[X'Z(WΣW')Z'X][X'ZWZ'X]^-1
Σ = E(Z'εε'Z) = Z'ΩZ = = S₀ + ∑_j=1,...,J[1-j/(J+1)](S_j+S_j')
as defined above, allowing for general heteroscedasticity and autocorrelation up to the J-th order.

If W = Σ^-1 (optimal weighted matrix), then it is the optimal or efficient GMM estimator:

b = [X'ZΣ^-1Z'X]^-1X'ZΣ^-1Z'Y
Var(b) = [X'ZΣ^-1Z'X]^-1

If W = (Z'Z)^-1, then it is IV estimator.

If W = I, then it is Minimum Distance (MD) estimator.

Instrumental Variable Specification Tests

Test for Endogenous Regressors
Does a specific regressor endogenous? Given the null hypothesis that one (or more) of X is exogenous (therefore we do not need instrumental variables), Durbin-Wu-Hausman (DWH) test is formulated as follows:
1. Let X = [X₁,X₂]; X₁ is exogenous and X₂ is endogenous. The first stage of 2SLS is the regression:
  X₂ = X₁α + υ.
  Let U be the estimated error terms, in which the residual endogeneity is stored.
2. The test for endogenous regressors is to run the auxilary regression with robust (heteroscedastistic-consistent) covariance matrix:
  Y = Xβ + Uγ + ε
  If E(ε|U,X) = 0, U is exogenous and has no effect on Y. In other words, X₂ is exogenous and the estimator of γ is 0.
3. DWH test is an F test with degrees of freedom K₂ and N-K-K₂, where K₁ = #X₁, K₂ = #X₂, K = K₁+K₂.
Test for Over-Identification
It is clear that the more Z is correlated with X, the more precise the instrumental variable estimator. Although the instrumental variable estimator is consistent but it is not unbiased. The extent of biasedness depends on the quality or validity of the instrumental variables used to remove endogeneity in the explanatory variables. Of course, the more good instrumental variables Z's, the better of the model estimates. But not all are valid instruments.
How many of good instruments will be enough? We need to know that the extra or excluded instruments (over the exogenous X's) will not violate the assumption E(ε|Z) = 0, the null hypothesis. Therefore, a simple Hausman test for over-identification is performed as follows:
1. Using L instruments Z, calculate the residuals from the instrumental variable estimates of the model: e = Y-Xb.
  Note that L > K, the number of columns in X.
2. Regress the residuals e on all Z's, and obtain R².
3. Then, NR² ~ χ²(L-K). A statistically significant test statistic indicates that the instruments may not be valid.
Many variations of over-identification tests are available in the literature depending on the estimation methods (LIML, GMM, etc.) and consideration of robustness of the estimators.
Test for Weak Instruments
The use of instrumental variables must be justified by first checking the correlations of instruments with endogenous variables. A good fit of the first stage of 2SLS is required. In addition, a reasonable partial R-square of endogenous variables (X₂, after controlling the effects of exogenous variables X₁) is expected.
More formal test for weak instruments can be found in J. H. Stock and M. Yogo, "Testing for Weak Instruments in Linear IV Regression," in Identification and Inference for Econometric Models: Essays in Honor of Thomas Rothenberg, ed. D. W. K. Andrews and J. H. Stock, 80-108, Cambridge University Press, 2005.
In brief, under homoscedasticity assumption, their test evaluates the bias of IV against that of OLS and provides a measure of size distortion for Wald test (for the zeros parameters of endogenous variables) at 5% level of significance.