Systems Work But Users Can't Verify Them

Designers hunting for UI inspiration waste hours filtering irrelevant results. Existing search tools miss semantic context like target users or app mood, and require metadata like view hierarchies that most screenshots lack.

Method: A multimodal LLM extracts semantic attributes directly from UI screenshots—no metadata required. It interprets queries like 'calming meditation app for seniors' and matches against extracted features including target demographics, visual mood, and interaction patterns. The system processes raw screenshots through vision-language understanding to build a searchable semantic index.

Caveats: Effectiveness depends on LLM's ability to infer user intent from visual cues alone—may struggle with niche design patterns or cultural context.

Reflections: Can this approach identify anti-patterns or accessibility issues during inspirational search? · How does semantic search quality degrade with stylistically unconventional or experimental UIs? · What's the latency cost of real-time LLM inference versus pre-computed embeddings?

Commercial dev tools are inaccessible to blind and low vision software professionals, forcing them to build custom workarounds. This creates a hidden tax: time spent on DIY accessibility tools instead of core work.

Method: Interviews with 30 blind and low vision software professionals revealed they build DIY tools for everything from screen reader-friendly code navigation to custom IDE plugins. These tools are brittle, undocumented, and break with every software update. The research maps the ecosystem of workarounds and identifies which commercial tool gaps force DIY solutions versus which reflect personal preference.

Caveats: Study focuses on software professionals; findings may not generalize to other technical roles or non-technical accessibility needs.

Reflections: What's the total productivity cost of DIY tool maintenance versus advocating for commercial fixes? · Can open-source communities sustainably maintain accessibility-focused forks of popular dev tools? · Which tool categories see the highest DIY abandonment rates due to maintenance burden?

LLMs pass explicit fairness tests but may harbor implicit biases that surface in decision-making. Alignment training teaches models to avoid biased language, not biased reasoning.

Method: Researchers created LLM agents with sociodemographically-informed personas and measured decision disparities in simulated scenarios. Even when models avoid explicitly biased statements, agents with different demographic personas make systematically different choices in identical situations—revealing implicit bias in the decision logic itself, not just the output text.

Caveats: Simulated personas may not reflect real-world decision patterns; findings need validation with actual user populations.

Reflections: Do implicit biases compound when LLM agents interact with each other in multi-agent systems? · Can adversarial training on decision outcomes reduce implicit bias without degrading task performance? · Which decision domains show the largest gaps between explicit fairness and implicit bias?