What the field argues about

The central construct-validity question. Gaddis (2017) showed that many names in classic audits signal class as strongly as race, so estimated race effects may bundle several treatments together. The validated names data in this guide is a direct response: measure perceptions first, then choose names whose signals match the inference you want. John and I pushed this further in the Mitterer reply, arguing the fix is rarely to partial the perceptions back out.

Heckman (1998) argued that callback gaps at the screening stage say little about discrimination in wages or careers, and that audits sample a nonrandom slice of the market. The response is that the callback is a real gate with real consequences, and that screening-stage discrimination compounds downstream. Naming the outcome as everyday discrimination, rather than discrimination in general, keeps the claim honest.

Quillian and colleagues (2017) meta-analyzed US hiring field experiments since 1989 and found no decline in callback discrimination against Black applicants over 25 years. Our own meta-analysis with Gaddis and Larsen found the gaps concentrated in hiring and housing. The accumulation of comparable audits is itself one of the method's biggest payoffs: it turns a one-off finding into a measurable trend. A newer meta-analysis with Gaddis and colleagues turns the same tools on intersectional discrimination, pooling how race and gender signals combine (Gaddis, Crabtree, and coauthors, forthcoming, Sociological Science).

Audits consume the time of employers, landlords, and officials who never consented. Crabtree and Dhima (2022) propose weighing the aggregate burden against the social value of detecting discrimination that is otherwise invisible. The framework does not dissolve the tension. It forces designs to minimize burden and justify their scale, which is the subject of the fielding page.

The design isn't only about American employers. In Hou, Liu, and Crabtree (2020, Journal of Comparative Economics) we sent more than 4,000 résumés to Chinese firms and found Muslim applicants about 50% less likely to hear back, with state-owned firms no better than private ones. In Hughes, Gell-Redman, Crabtree, and coauthors (2020, JEPS) the targets were local election officials, not employers. The logic carries to any gatekeeper who reads a name and decides whether to reply.