No Thoughts Just AI: Biased LLM Hiring Recommendations Alter Human Decision Making and Limit Human Autonomy
Kyra Wilson, Mattea Sim, Anna-Maria Gueorguieva, Aylin Caliskan
Audit your AI hiring tools for demographic bias before deployment. Don't assume human reviewers will catch or correct biased recommendations—they amplify them instead. If you're building resume screeners, expose the model's confidence scores and flag when recommendations diverge from baseline human preferences.
Hiring managers defer to AI resume screeners without questioning their recommendations, even when those systems encode racial bias across 16 different occupations.
Method: Researchers ran a 528-person experiment where participants screened resumes with simulated AI models that exhibited race-based preferences (detected through candidate names and affinity groups). The AI bias mirrored factual and counterfactual estimates from real-world systems. Participants shifted their own racial preferences to match the AI's recommendations, even for high-status positions where human judgment traditionally dominates.
Caveats: Study used simulated AI models, not production systems. Real-world bias patterns may differ in magnitude or direction.
Reflections: Do transparency interventions (showing AI confidence scores) reduce deference to biased recommendations? · Does bias amplification persist when reviewers evaluate candidates in batches versus individually? · Can training interventions help hiring managers maintain independent judgment when using AI tools?