Design and Evaluation of AI Safety Mechanisms in ADAS and Autonomous Vehicle Architectures
DOI:
https://doi.org/10.56472/WCAI25-120Keywords:
Index Terms, ADAS, Autonomous vehicles, AI safety, OOD detection, Bias mitigation, Simulation validation, Functional safetyAbstract
The progressive shift from human-driven to automated driver assistance and to fully autonomous vehicles has placed Artificial Intelligence (AI) at the center of automotive innovation. Advanced Driver Assistance Systems (ADAS) and Autonomous Vehicle (AV) platforms increasingly depend on AI for perception, decision making, and controls leading to enhanced vehicle safety and vehicle operational efficiency. However, the non-deterministic nature of AI, coupled with its dependence on training data and susceptibility to out-of-distribution (OOD) inputs, introduces novel safety hazards not encountered in traditional deterministic control systems [1] [2]. This paper aims to investigate the design and evaluation of safety mechanisms capable of detecting, mitigating, and recovering from unexpected AI behaviors in OOD scenarios for the AI system. The study considers layered safety architectures, continuous monitoring strategies, dataset lifecycle management, simulation-based validation, and performance metric analysis as part of an integrated safety framework. Key findings include that OOD detection techniques, such as Mahalanobis distance-based scoring, can significantly reduce misclassifications risk, although sometimes at the expense of operational coverage [3]. The integration of safety monitors into perception pipelines has been shown to improve system trustworthiness by identifying failure patterns before they escalate into hazardous decisions [4]. Furthermore, empirical studies reveal demographic performance disparities in pedestrian detection models, particularly under low-light or low-contrast scenarios, which highlights the importance of bias aware dataset curation [5] [6]. We conclude that achieving resilient autonomy demands a multi-layered safety approach: combining proactive monitoring, fallback control logic, diverse and bias mitigated datasets, rigorous simulation-based testing, and alignment with evolving regulatory standards. These measures form the basis for developing trustworthy AI systems capable of operating safely in unpredictable scenarios encountered during real-world driving conditions
Downloads
References
[1] ISO 26262, Road vehicles Functional safety, 2nd ed., International Organization for Standardization, 2018.
[2] ISO/PAS 21448, Road vehicles Safety of the intended functionality (SOTIF), International Organization for Standardization, 2019.
[3] K. Lee, K. Lee, H. Lee, and J. Shin, “A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks,” arXiv preprint, arXiv:1807.03888, 2018.
[4] M. Mohseni, M. Pitropov, L. P. Clarke, and T. H. Fung, “Monitoring and Improving Neural Network Safety Using Outlier Exposure,” arXiv preprint, arXiv:2412.06869, 2024.
[5] A. Wilson, S. Buolamwini, and T. Gebru, “Predictive Inequity in Object Detection,” arXiv preprint, arXiv:1902.11097, 2019.
[6] Reddit, “Bias in pedestrian detection AI under low-light conditions,” r/Futurology, Aug. 2023. Available: https://www.reddit.com/r/Futurology/
[7] “Advanced driver-assistance system,” Wikipedia, Jul. 2025. [Online]. Available:https://en.wikipedia.org/wiki/Advanced driver-assistance system
[8] European Commission, “Study on the effectiveness of advanced driver assistance systems,” Brussels, Belgium, Rep. No. MOVE/C2/SER/2017263, 2020.
[9] A. Kendall et al., “Learning to drive in a day,” in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), 2019, pp. 8248–8254.
[10] C. Chen et al., “Multi-view 3D object detection network for autonomous driving,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), 2017, pp. 1907–1915.
[11] S. Amershi et al., “Guidelines for human-AI interaction,” in Proc. SIGCHI Conf. Hum. Factors Comput. Syst., 2019, pp. 1–13.
[12] A. Doshi-Velez and B. Kim, “Towards a rigorous science of interpretable machine learning,” arXiv preprint arXiv:1702.08608, 2017.
[13] H. Zhao, J. Shi, X. Qi, X. Wang, and J. Jia, “Mask R-CNN based detection under adverse weather conditions,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. Workshops (CVPRW), 2019, pp. 0–0.
[14] Y. Li et al., “Generalized ODIN: Detecting out-of-distribution image without learning from out-of-distribution data,” arXiv preprint arXiv:2002.11297, 2020.
[15] B. Lakshminarayanan, A. Pritzel, and C. Blundell, “Simple and scalable predictive uncertainty estimation using deep ensembles,” in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), 2017, pp. 6402–6413.
[16] M. T. Ribeiro, S. Singh, and C. Guestrin, “Why should I trust you? Explaining the predictions of any classifier,” in Proc. ACM SIGKDD Int. Conf. Knowl. Discov. Data Min., 2016, pp. 1135–1144.
[17] J. Levinson and S. Thrun, “Robust vehicle localization in urban environments using probabilistic maps,” in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), 2010, pp. 4372–4378.
[18] National Transportation Safety Board (NTSB), “Collision between vehicle controlled by developmental automated driving system and pedestrian,” Rep. NTSB/HAR-19/03, Nov. 2019.
[19] Reddit, “Radar vs camera performance in fog for AV perception,” r/SelfDrivingCars, Mar. 2023. [Online]. Available: https://www.reddit. com/r/SelfDrivingCars/
[20] J. Dickmann et al., “Automotive radar: The key technology for autonomous driving: From detection to environmental understanding,” in Proc. IEEE Radar Conf., 2016, pp. 1–6.
[21] N. Kalra and S. M. Paddock, “Driving to safety: How many miles of driving would it take to demonstrate autonomous vehicle reliability?,” RAND Corporation, 2016.
[22] M. Hafner et al., “Cooperative and adversarial driver models for testing autonomous vehicles,” in Proc. IEEE Intell. Veh. Symp., 2018, pp. 126– 133.
[23] D. Hendrycks and K. Gimpel, “A baseline for detecting misclassified and out-of-distribution examples in neural networks,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2017.
[24] L. Ma, F. Zhang, and X. Yang, “Perception safety monitors for autonomous driving: An uncertainty-aware approach,” in Proc. IEEE Intell. Transp. Syst. Conf., 2021, pp. 159–166.
[25] C. Koopman and P. Koopman, “Safety assurance cases for autonomous vehicle development,” SAE Int. J. Transp. Saf., vol. 9, no. 2, pp. 1–14, 2021.
[26] SAE International, “Taxonomy and definitions for terms related to driving automation systems,” SAE J3016, 2021.
[27] K. Ohn-Bar and M. M. Trivedi, “Looking at humans in the age of selfdriving and highly automated vehicles,” IEEE Trans. Intell. Veh., vol. 1, no. 1, pp. 90–104, 2016.
[28] J. Koopman, P. Koopman, and M. Wagner, “Remote human supervision of autonomous systems,” in Proc. IEEE Int. Symp. Syst. Eng., 2019, pp. 1–8.
[29] M. Kendall and A. Gal, “What uncertainties do we need in Bayesian deep learning for computer vision?,” in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), 2017, pp. 5574–5584.
[30] C. E. Dickerson, “Redundancy and cross-checks in safety-critical automotive computing,” in Proc. SAE World Congr., 2020, pp. 1–9.
[31] S. Shalev-Shwartz et al., “On a formal model of safe and scalable selfdriving cars,” arXiv preprint arXiv:1708.06374, 2017.
[32] T. Buolamwini and G. Gebru, “Gender shades: Intersectional accuracy disparities in commercial gender classification,” in Proc. Conf. Fairness Accountability Transparency, 2018, pp. 77–91.
[33] Y. Wang et al., “Enhancing autonomous driving perception with synthetic data,” in Proc. IEEE Intell. Transp. Syst. Conf., 2020, pp. 1–7.
[34] J. Raji et al., “Closing the AI accountability gap,” in Proc. ACM Conf. Fairness Accountability Transparency, 2020, pp. 33–44.
[35] M. Mitchell et al., “Model cards for model reporting,” in Proc. Conf. Fairness Accountability Transparency, 2019, pp. 220–229.
[36] A. Beutel et al., “Data decisions and theoretical implications when adversarially de-biasing,” in Proc. Conf. Fairness Accountability Transparency, 2019, pp. 50–59.
[37] P. Koopman and M. Wagner, “Challenges in autonomous vehicle testing and validation,” SAE Int. J. Transp. Saf., vol. 4, no. 1, pp. 15–24, 2016. [38] M. Winner et al., “Virtual testing for autonomous driving,” in Proc. IEEE Intell. Veh. Symp., 2018, pp. 207–212.
[38] Z. Yan et al., “A comprehensive survey of digital twinPart 2: Roles of deep learning and blockchain in the industrial internet of things,” IEEE Trans. Ind. Informat., vol. 17, no. 6, pp. 4422–4442, 2021.
[39] ISO 21448, “Road vehicles Safety of the intended functionality (SOTIF),” International Organization for Standardization, 2022.
[40] G. Pokharkar, “Scenario-Based Validation for SAE Level 2+ Features Using Simulation-in-the-Loop (SiL) Systems,” International Journal of Innovative Research and Creative Technology, vol. 11, no. 4, pp. 1–8, Jul. 2025, doi: 10.5281/zenodo.16883284.
[41] E. Thorn, S. Kimmel, and M. Chaka, A Framework for Automated Driving System Testable Cases and Scenarios, Report No. DOT HS 812 623, National Highway Traffic Safety Administration, Washington, DC, Sep. 2018.
[42] R. Bishop, “A survey of intelligent vehicle applications worldwide,” IEEE Intell. Veh. Symp., pp. 25–30, 2020.
[43] M. Aly, “Real time detection of lane markers in urban streets,” IEEE Intell. Veh. Symp., pp. 7–12, 2008.
[44] N. Bojarski et al., “End to end learning for self-driving cars,” arXiv preprint arXiv:1604.07316, 2016.
[45] A. Lambert et al., “Over-the-air software updates for connected and autonomous vehicles: Challenges and opportunities,” IEEE Trans. Intell. Transp. Syst., vol. 22, no. 4, pp. 2103–2116, 2021.
[46] P. Koopman, “A case study of safety-critical embedded software,” IEEE Des. Test, vol. 29, no. 3, pp. 20–25, 2012.
[47] HLDI, “Real-world benefits of crash avoidance technologies,” Highway Loss Data Institute, 2020.
[48] Road & Track, “ADAS systems can change how people drive,” 2023.
[49] Euro NCAP, “Assessment protocolHighway Assist,” 2022.
[50] UNECE, “UN Regulation No. 157Automated Lane Keeping Systems (ALKS),” 2021.
[51] A. R. P. James et al., “Legal and institutional frameworks for autonomous vehicles in the EU,” Eur. Transp. Res. Rev., vol. 12, 2020.
[52] A. Dosovitskiy et al., “An image is worth 16x16 words: Transformers for image recognition at scale,” arXiv preprint arXiv:2010.11929, 2020.
[53] C. Michelmore et al., “Uncertainty quantification for deep learning in autonomous driving,” in Proc. ICRA, pp. 208–214, 2020.
[54] W. Samek et al., “Explainable AI: Interpreting, explaining and visualizing deep learning,” Springer, 2019.
[55] S. Vora et al., “Bias in visual AI: Evaluation and mitigation for pedestrian detection,” arXiv preprint arXiv:2103.03450, 2021.
[56] Thirunagalingam, A. (2024). Bias Detection and Mitigation in Data Pipelines: Ensuring Fairness and Accuracy in Machine Learning. Available at SSRN 5047605.
[57] Maroju, P. K. (2024). Advancing synergy of computing and artificial intelligence with innovations challenges and future prospects. FMDB Transactions on Sustainable Intelligent Networks, 1(1), 1-14.
[58] L. N. R. Mudunuri, V. M. Aragani, and P. K. Maroju, "Enhancing Cybersecurity in Banking: Best Practices and Solutions for Securing the Digital Supply Chain," Journal of Computational Analysis and Applications, vol. 33, no. 8, pp. 929-936, Sep. 2024.
[59] Settibathini, V. S., Virmani, A., Kuppam, M., S., N., Manikandan, S., & C., E. (2024). Shedding Light on Dataset Influence for More Transparent Machine Learning. In P. Paramasivan, S. Rajest, K. Chinnusamy, R. Regin, & F. John Joseph (Eds.), Explainable AI Applications for Human Behavior Analysis (pp. 33-48). IGI Global Scientific Publishing. https://doi.org/10.4018/979-8-3693-1355-8.ch003.
[60] Sehrawat, S. K., Dutta, P. K., Bhatia, A. B., & Whig, P. (2024). Predicting Demand in Supply Chain Networks With Quantum Machine Learning Approach. In A. Hassan, P. Bhattacharya, P. Dutta, J. Verma, & N. Kundu (Eds.), Quantum Computing and Supply Chain Management: A New Era of Optimization (pp. 33-47). IGI Global Scientific Publishing. https://doi.org/10.4018/979-8-3693-4107-0.ch002
[61] S. Panyaram, "Connected Cars, Connected Customers: The Role of AI and ML in Automotive Engagement," International Transactions in Artificial Intelligence, vol. 7, no. 7, pp. 1-15, 2023.
[62] Mohanarajesh Kommineni. (2022/9/30). Discover the Intersection Between AI and Robotics in Developing Autonomous Systems for Use in the Human World and Cloud Computing. International Numeric Journal of Machine Learning and Robots. 6. 1-19. Injmr. - 1=
