Integration of Data Fusion and Deep Neural Networks for In-vehicle Symbiotic HMI Design

This paper proposes an AI-driven framework for designing multimodal, symbiotic human-machine interfaces (HMIs) in modern vehicles. As automotive systems evolve toward higher levels of automation and user-centric design, the integration of data fusion techniques with deep neural networks becomes essential for enhancing driver safety and comfort. The proposed framework continuously monitors a range of inputs—including driver gaze, facial expressions, posture, and vehicle telemetry—and processes these signals through a hierarchical data fusion methodology. This approach involves raw signal preprocessing, feature extraction and selection, and decision-level integration using advanced deep learning models. Experimental validation using a high-fidelity driving simulator across varied scenarios (high-stress, relaxed, and frustration-inducing) demonstrates the framework’s capability to accurately detect driver distraction and drowsiness. The findings suggest that integrating sensor data fusion with neural network-based analysis can significantly improve the adaptability and responsiveness of HMIs.