AI Systems Project Confidence Users Cannot Verify

AI coding assistants promise productivity gains, but evidence from experienced developers working on real codebases—not toy problems—remains scarce.

Method: 16 developers with 5 years average experience on their projects completed 246 tasks in mature open-source repos, randomly assigned to allow or block early-2025 AI tools (GPT-4 class). This is the first RCT measuring AI impact on developers working in their own production codebases, not synthetic benchmarks. The study isolates tool effect from developer skill and project familiarity.

Caveats: Only 16 developers, all open-source contributors. Enterprise codebases with stricter review processes may show different patterns.

Reflections: How does AI impact vary between bug fixes, feature additions, and refactoring tasks in mature codebases? · Do productivity gains persist after 6+ months of continuous AI tool usage? · What's the effect on code review burden when AI-assisted code enters production?

LLMs and vision-language models answer medical queries with increasing confidence, but safety disclaimers that warn users against treating outputs as professional advice are disappearing.

Method: Tracked disclaimer presence across LLM and VLM model generations from 2022 to 2025. Found systematic decline in medical safety messaging as models improved in capability. Newer, more capable models paradoxically provide fewer warnings that outputs aren't vetted medical advice, even as users increasingly trust them for clinical decisions.

Caveats: Study doesn't measure whether disclaimers actually change user behavior or reduce harm—only their presence.

Reflections: Do users actually modify behavior based on medical disclaimers, or do they ignore them? · What disclaimer design patterns are most effective at preventing over-reliance on AI medical advice? · How should disclaimer intensity scale with query risk level?

Phishing warnings fail because they're static and vague. Users dismiss them without understanding the specific threat in front of them.

Method: Large-scale between-subjects experiment testing LLM-generated explanations for phishing warnings versus standard static messages. LLMs generate threat-specific explanations tailored to each phishing attempt's characteristics—explaining why this particular email is suspicious rather than generic 'be careful' messaging. Measures both detection accuracy and user perception of clarity.

Caveats: LLM explanations could be gamed by attackers who craft phishing attempts to generate benign-sounding explanations.

Reflections: Can attackers reverse-engineer LLM explanation logic to craft warnings that sound harmless? · Do personalized explanations improve long-term phishing detection skills or just immediate decisions? · What's the latency cost of generating explanations in real-time email clients?