IV in Practice: Judges and Examiners
PP5001 · Week 4 · Martinmas 2026
Professor David A. Jaeger
From Oregon to Judges and Examiners
Last week: a lottery changed the probability of obtaining insurance.
This week: assignment to an official changes the probability of receiving treatment.
- Pre-trial detention: assignment to bail judges.
- Disability insurance: assignment to disability examiners.
What makes these administrative assignments useful for causal inference?
Two Sources of Variation in Treatment
Officials apply common rules but may assess evidence or borderline cases differently.
Treatment can vary because:
- Cases differ: the same characteristics can influence treatment and the outcome.
- Officials differ: some grant treatment more readily than others.
A leniency design uses differences in officials’ decisions together with an assignment process that makes their cases comparable.
Where Does Quasi-Random Variation Come From?
Imagine two officials receiving comparable applications from the same queue.
- One approves 40% and the other approves 60%.
- Assignment is unrelated to applicants’ characteristics or potential outcomes.
- Encountering the more lenient official raises the probability of treatment by 20 percentage points.
Assignment supplies independence. Differences in leniency supply relevance.
These are hypothetical rates illustrating the argument.
Understanding the Assignment Process
A lottery, rotation, or queue may make assignment as good as random within a group of cases.
Establish:
- Which officials could have received each case?
- Can applicants or staff influence the assignment?
- Do offices, shifts, or specialist teams receive different kinds of cases?
The comparison and controls must reflect these institutional details.
Comparable groups assigned to different officials can still contain systematically different treated and untreated people.
The IV Assumptions in Practice
- Relevance: does the official assigned affect treatment?
- Independence: why should assignment be unrelated to potential outcomes, conditional on the assignment controls?
- Exclusion: through which channels can the official affect the outcome?
- Monotonicity: do officials order treatment decisions in a common direction?
For each paper, connect the assumptions to the institution and the evidence.
Measuring Leniency
Let R_i=1 denote a favourable initial decision for person i.
Measure official j’s leniency using decisions on other cases:
Z_i=\frac{\sum_{k:j(k)=j(i),\ k\ne i}R_k}{n_{j(i)}-1}.
Here j(i) is the official assigned to person i, and n_j is the official’s number of cases.
Leaving out the person’s own decision removes its direct contribution to the instrument.
A Leave-One-Out Example
An official approves three of five applications.
| Approved |
2 |
2/4=0.50 |
| Rejected |
3 |
3/4=0.75 |
The useful variation comes from differences in underlying leniency across officials.
In the detention study, the authors also account for court and time, and exclude all cases belonging to the same defendant.
Pre-trial Detention
Dobbie, Goldin, and Yang (2018)
The Effects of Pretrial Detention on Conviction, Future Crime, and Employment: Evidence from Randomly Assigned Judges
Philadelphia and Miami-Dade: court records linked to tax records.
The following exhibits are from the published article.
The Policy Question and Data
For discussion
Read Sections I and II of Dobbie, Goldin, and Yang (2018).
Explain the policy problem, the treatment, and the court and tax data.
Why might released and detained defendants have different outcomes even without a causal effect of release?
Identify the outcomes relevant to a policy recommendation.
Defining the Treatment
In these regressions,
D_i=1\quad\text{if released within three days of the bail hearing.}
Z_i measures the assigned bail judge’s tendency to release other defendants, after accounting for court and time.
Outcomes include guilty pleas, conviction, rearrest, and employment.
Read the signs as effects of release.
Judge Assignment and Balance: Table 3
For discussion
![]()
Dobbie, Goldin, and Yang (2018). Upper portion. Continued on the next slide.
Table 3: Balance (continued)
For discussion
![]()
Dobbie, Goldin, and Yang (2018).
Table 2
For discussion
![]()
Dobbie, Goldin, and Yang (2018).
Exclusion and Monotonicity
For discussion
Explain why the separate assignment of bail and trial judges matters.
Consider other ways bail conditions could affect later outcomes.
Give an example in which two judges reverse their relative leniency across types of defendants.
Connect each argument to the relevant IV assumption.
Interpreting the Results: Table 4
For discussion
![]()
Dobbie, Goldin, and Yang (2018). Original table excerpt. Standard errors in parentheses, clustered by defendant and judge.
Table 5
For discussion
![]()
Dobbie, Goldin, and Yang (2018). Original table excerpt. Standard errors in parentheses, clustered by defendant and judge.
Disability Insurance
Maestas, Mullen, and Strand (2013)
Does Disability Insurance Receipt Discourage Work? Using Examiner Assignment to Estimate Causal Effects of SSDI Receipt
Applications and examiner assignments linked to benefit decisions and subsequent earnings.
The Policy Question and Comparison
For discussion
Read Sections I and II of Maestas, Mullen, and Strand (2013).
Explain the policy question, data, and why comparing approved and rejected applicants may be misleading.
Distinguish an initial allowance decision from eventual benefit receipt and explain the role of appeals.
Initial Decisions and Benefit Receipt
- Treatment, D_i: eventual receipt of SSDI benefits.
- Instrument, Z_i: the assigned examiner’s initial allowance rate among other applicants.
- Outcomes, Y_i: employment and earnings.
An initial rejection may be followed by an appeal and eventual receipt of benefits.
The initial allowance decision and the treatment are therefore distinct.
Examiner Assignment: Table 2
For discussion
![]()
Maestas, Mullen, and Strand (2013). p. 1814.
Table 5
For discussion
![]()
Maestas, Mullen, and Strand (2013). p. 1820.
Connecting the Two Stages
With one excluded instrument and one endogenous explanatory variable,
\widehat\beta_1^{IV}=\frac{\widehat\delta_1}{\widehat\pi_1}.
Use the same sample and controls in the reduced form and first stage.
For employment two years after a 2005 decision:
\frac{-0.057}{0.204}\approx-0.279.
What does this estimate mean, and for whom?
Interpreting Table 4: Panel A
For discussion
![]()
Maestas, Mullen, and Strand (2013). p. 1819. Original excerpt. Parentheses contain t-statistics. For a test of zero, \operatorname{SE}\left(\widehat\beta\right)=\left|\widehat\beta\right|/\left|t\right|. Standard errors are clustered by examiner. All specifications include the controls listed in the paper.
Whose Treatment Effect?
With two officials, a complier receives treatment under the more lenient official but not under the stricter official:
D_i^1=1,\qquad D_i^0=0.
The binary-instrument LATE is
E\left[Y_i^1-Y_i^0\mid D_i^1>D_i^0\right].
With many officials, a common monotone ordering supports a weighted average of effects across the margins changed by leniency.
Comparing the Designs and Policy Implications
For discussion
For each paper, give the strongest institutional argument for independence, a plausible threat to exclusion, and a description of the people at the treatment margin.
Propose a policy change for which the estimate is informative.
What further evidence would a broader reform require?
References
Dobbie, Will, Jacob Goldin, and Crystal S. Yang. 2018. “The Effects of Pretrial Detention on Conviction, Future Crime, and Employment: Evidence from Randomly Assigned Judges.” American Economic Review 108(2): 201–240.
Maestas, Nicole, Kathleen J. Mullen, and Alexander Strand. 2013. “Does Disability Insurance Receipt Discourage Work? Using Examiner Assignment to Estimate Causal Effects of SSDI Receipt.” American Economic Review 103(5): 1797–1829.
Goldsmith-Pinkham, Paul, Peter Hull, and Michal Kolesár. 2026. “Leniency Designs: An Operator’s Manual.” Journal of Economic Perspectives 40(3): 213–240.
