Human-centric Intelligibility for Automated Vehicles: Aligning Explanations with Predictive Models of Passenger State

Abstract

Public acceptance of automated vehicles (AVs) is hindered by a lack of trust due to their unintelligible decision-making. This research bridges the gap between explainable AI development and passenger experience by investigating human-centric explanation systems for AVs. We developed and validated an on-road Wizardof-Oz evaluation platform to test AI-generated explanations safely. Through this platform, we created a comprehensive multimodal dataset capturing passengers' in-situ explanation demands during real driving scenarios, including physiological responses, vehicle dynamics, and environmental context. Our findings reveal that passengers need to understand what the vehicle is doing rather than why, with perception-based visualizations proving more effective than abstract attention heatmaps for building trust and situational awareness. We demonstrate that a predictive model using vehicle dynamics and physiological states can identify opportune moments for explanation delivery. As we extend towards generalizable modeling, this research aims to provide empirical guidelines for intelligible AVs and establish a framework for intelligible in-vehicle systems that dynamically align explanations with general and individual passengers' informational needs, fostering more adaptive and trustworthy human-vehicle interactions.