Component Accuracy Fails to Predict System Performance

Surgeons planning liver resections wrestle with competing safety constraints across cognitively distinct stages—anatomical discovery, plan refinement, surgery prep—but current 2D desktop tools force them into a single monolithic interface that obscures spatial relationships and scatters critical safety criteria.

Method: LiverPlan decomposes surgical planning into three stages, each with tailored VR techniques: context-preserving focus rendering for anatomical discovery, direct 3D plane manipulation with embedded real-time safety feedback during refinement, and explicit plane-vessel intersection visualization for surgery prep. A within-subjects study with eight hepatobiliary surgeons showed large-effect-size improvements in task completion time, perceived cognitive workload, and system usability versus a desktop baseline.

Caveats: Controlled task study. Real operating room adoption and long-term workflow integration remain unverified.

Reflections: Do stage-adaptive interfaces generalize to other surgical specialties with multi-stage planning (e.g., neurosurgery, orthopedics)? · What is the minimum VR fidelity required to achieve the cognitive workload reduction—could AR or high-resolution 3D displays suffice? · Does the shift from satisfying to optimizing safety criteria persist after surgeons return to 2D tools, or is it VR-dependent?

Social benefits in India require form completion, but beneficiaries often can't read or write. Frontline health workers enroll them one-by-one—a bottleneck that wastes stretched capacity and delays access to maternal and child health programs.

Method: FormBharo pairs LLMs with rule-based validation to fill structured enrollment forms over phone calls in Hindi. Form completion dropped by up to approximately 41 points when LLMs received error-prone real-speech transcripts instead of reference ones. Rule-based controls recovered many turn-level extraction errors, helping smaller, cheaper models match or surpass frontier models on form completion. Component performance didn't predict end-to-end performance: GPT-4.5 led turn-level extraction accuracy on reference transcripts (99.8%) but ranked lower on form completion.

Caveats: Piloted with one NGO for maternal health enrollment. Transfer to other form types and languages unverified.

Reflections: Do rule-based controls generalize to other low-resource languages with different phonetic error patterns? · What is the threshold of transcription error rate beyond which rule-based recovery fails? · Can end-to-end benchmarks like FormVoiceAgentBench predict real-world enrollment completion rates, or do human factors (trust, dropout) dominate?

Social media feeds optimize for immediate impulses, not reflective preferences. Existing alignment tools (configuration pages, in-feed controls) require user initiative and effort, so they're rarely invoked—leaving feeds misaligned as preferences evolve.

Method: Compass surfaces lightweight in-situ reflection prompts during everyday browsing and periodically simulates behavioral signals to directly manipulate feed content. A 10-day field study (N=15) embedded in YouTube Shorts found that Compass promoted more reflective and purposeful feed consumption, iterative preference adjustment, and stronger feed alignment versus a baseline without continuous support—without sacrificing the casual nature of browsing.

Caveats: Ten-day study with 15 users on YouTube Shorts. Long-term habituation and cross-platform generalization unknown.

Reflections: Do users eventually habituate to in-situ reflection prompts, reducing their effectiveness over months? · What is the optimal frequency of reflection prompts before they become intrusive? · Does continuous alignment reduce engagement metrics that platforms optimize for, creating a business model conflict?