How Humans and LLMs Read Gender into "Gender-Neutral" Physical Descriptions
Yingjia Wan, Lin Lin, Elisa Kreiss
Don't assume physical descriptions achieve gender neutrality. They carry systematic associations that readers interpret along gender lines. If you're generating character descriptions or alt-text, test against human gender associations—not just LLM outputs, which misalign predictably.
AI ethics guidelines recommend replacing gendered pronouns with "objective" physical descriptions ("short hair" instead of "he"). But do readers actually interpret these descriptions as gender-neutral?
Method: Physical descriptions carry structured gender associations among 304 US annotators rating 316 attributes. Associations were more consistent for women and men than non-binary identities. Sixteen LLMs partially recovered human associations but showed systematic biases: compressed rating distributions, weaker alignment for associations with men, and asymmetric abstention that disproportionately targeted the non-binary category. The best-performing proxy model was then used to analyze character descriptions in LitBank, revealing systematic gendered patterns.
Caveats: US-based annotators only. Cross-cultural gender associations may differ substantially.
Reflections: Do physical descriptions carry different gender associations across cultures and languages? · Can training interventions reduce LLM alignment biases for non-binary associations? · What alternative description strategies achieve actual gender neutrality in reader interpretation?