AI Systems Crush Benchmarks But Crumble in Deployment

Over 80% of deployed chatbots ship with at least one critical accessibility issue. 45% lack semantic structure entirely, making them unusable for screen readers.

Method: The team audited 106 production chatbots across healthcare, education, and customer service using four tools: Google Lighthouse, PageSpeed Insights, SiteImprove, and Microsoft Accessibility Insights. They found missing ARIA labels, broken keyboard navigation, and color contrast failures clustered in embedded widgets more than standalone apps. The multi-tool approach caught issues single audits missed—Lighthouse alone flagged only 60% of the problems manual audits revealed.

Caveats: Study focused on web-based chatbots only. Native mobile apps and voice interfaces weren't tested.

Reflections: Do chatbot frameworks introduce systematic accessibility debt, or is this a training gap among developers? · Which accessibility issues correlate with user abandonment in production? · Can automated testing tools be trained to catch the 40% of issues that currently require manual audits?

AI models solve problems correctly but generate explanations humans can't use. The reasoning improvements don't translate to better knowledge transfer.

Method: The KITE framework measures whether humans can learn from AI explanations by testing task performance after exposure. They ran large-scale experiments showing that models with higher reasoning accuracy don't necessarily produce explanations that improve human performance. The gap between model capability and human-usable knowledge is quantified through pre/post task accuracy deltas. They isolated explanation quality from raw model performance, revealing that current LLMs optimize for correctness, not teachability.

Caveats: Framework requires controlled pre/post testing. Hard to deploy in production environments with continuous user interactions.

Reflections: What explanation formats maximize human knowledge retention versus immediate task completion? · Can models be fine-tuned specifically for knowledge transfer rather than accuracy? · Does the knowledge transfer gap widen or narrow as model reasoning capabilities improve?

Workers want AI to augment specific tasks, but current systems automate entire workflows instead. The mismatch between worker preferences and AI capabilities creates overreliance and diminished agency.

Method: The auditing framework maps 1,016 occupational tasks from O*NET against current AI agent capabilities, then surveys workers to identify which tasks they want automated versus augmented. The gap analysis reveals systematic misalignment: workers prefer augmentation for judgment-heavy tasks like medical diagnosis, but AI agents are designed for full automation. The framework quantifies this tension by comparing technological feasibility scores with worker preference ratings across occupations.

Caveats: Framework relies on self-reported preferences. Actual adoption behavior may differ from stated desires.

Reflections: How do worker preferences shift after prolonged exposure to AI augmentation tools? · Which task characteristics predict successful augmentation versus automation? · Can organizations design hybrid workflows that satisfy both efficiency goals and worker agency?