Testing Instrument Strength and Validity
What You'll Learn
You'll learn how to diagnose whether your instrumental variable is strong enough to be useful and how to test if multiple instruments are actually valid.
Why This Matters
Remember from Two-Stage Least Squares that instruments help us isolate causal effects when we have endogeneity. But not all instruments are created equal. A weak instrument barely correlates with your treatment variable, making your estimates unreliable and biased. With multiple instruments, you can also test whether they're truly exogenous (the over-identification test).
Testing Instrument Strength: The F-Statistic
After running the first stage of 2SLS (where you regress the treatment on your instrument(s)), check the F-statistic on the instrument coefficient(s).
Rule of thumb: F-statistic > 10 suggests your instrument is sufficiently strong.
- F < 10: Weak instrument—your second-stage estimates may be severely biased
- F ≥ 10: Generally acceptable strength
- F > 20–30: Strong instrument
Think of it like signal strength on your phone: below a threshold, the connection (causal estimate) becomes noisy and unreliable.
Over-Identification Test (Multiple Instruments)
When you have more instruments than endogenous variables, you can test whether they're all valid using the Sargan or Hansen J-test:
- Null hypothesis: All instruments are exogenous (valid)
- If you reject (p < 0.05), at least one instrument is likely invalid
This only works with 2+ instruments—one instrument alone can't be statistically tested for validity (you must rely on theory and domain knowledge).
Practical Workflow
- Run first-stage regression
- Check F-statistic (strength)
- If multiple instruments, run over-identification test (validity)
- If either test fails, reconsider your instrument choice
Key Takeaway: Use the F-statistic to ensure your instrument is strong (F > 10), and when you have multiple instruments, apply over-identification tests to verify they're all valid—weak or invalid instruments undermine causal inference.