BehAVExplor: Behavior Diversity Guided Testing for Autonomous Driving Systems
Testing Autonomous Driving Systems (ADSs) is a critical task for ensuring the reliability and safety of autonomous vehicles. Existing methods mainly focus on searching for safety violations while the diversity of the generated test cases is ignored, which may generate many redundant test cases and failures. Such redundant failures can reduce testing performance and increase failure analysis costs. In this paper, we present a novel behavior-guided fuzzing technique (BehAVExplor) to explore the different behaviors of the ego vehi- cle (i.e., the vehicle controlled by the ADS under test) and detect diverse violations. Specifically, we design an efficient unsupervised model, called BehaviorMiner, to characterize the behavior of the ego vehicle. BehaviorMiner extracts the temporal features from the given scenarios and performs a clustering-based abstraction to group behaviors with similar features into abstract states. A new test case will be added to the seed corpus if it triggers new behav- iors (e.g., cover new abstract states). Due to the potential conflict between the behavior diversity and the general violation feedback, we further propose an energy mechanism to guide the seed selec- tion and the mutation. The energy of a seed quantifies how good it is. We evaluated BehAVExplor on Apollo, an industrial-level ADS, and LGSVL simulation environment. Empirical evaluation results show that BehAVExplor can effectively find more diverse violations than the state-of-the-art.
Wed 19 JulDisplayed time zone: Pacific Time (US & Canada) change
10:30 - 12:00 | ISSTA 6: Testing 1Technical Papers at Habib Classroom (Gates G01) Chair(s): Karine Even-Mendoza King’s College London | ||
10:30 15mTalk | Synthesizing Speech Test Cases with Text-to-Speech? An Empirical Study on the False Alarms in Automated Speech Recognition Testing Technical Papers Julia Kaiwen Lau Monash University Malaysia, Kelvin Kai Wen Kong Monash University Malaysia, Julian Hao Yong Monash University Malaysia, Per Hoong Tan Monash University Malaysia, Zhou Yang Singapore Management University, Zi Qian Yong Monash University Malaysia, Joshua Chern Wey Low Monash University Malaysia, Chun Yong Chong Monash University Malaysia, Mei Kuan Lim Monash University Malaysia, David Lo Singapore Management University DOI | ||
10:45 15mTalk | PhysCov: Physical Test Coverage for Autonomous Vehicles Technical Papers Carl Hildebrandt University of Virginia, Meriel von Stein University of Virginia, Sebastian Elbaum University of Virginia Link to publication DOI Pre-print | ||
11:00 15mTalk | BehAVExplor: Behavior Diversity Guided Testing for Autonomous Driving Systems Technical Papers Mingfei Cheng Singapore Management University, Yuan Zhou Nanyang Technological University, Xiaofei Xie Singapore Management University DOI | ||
11:15 15mTalk | Building Critical Testing Scenarios for Autonomous Driving from Real Accidents Technical Papers Xudong Zhang Institute of Software at Chinese Academy of Sciences, Yan Cai Institute of Software at Chinese Academy of Sciences DOI | ||
11:30 15mTalk | Virtual Reality (VR) Automated Testing in the Wild: A Case Study on Unity-Based VR Applications Technical Papers Dhia Elhaq Rzig University of Michigan - Dearborn, Nafees Iqbal University of Michigan at Dearborn, Isabella Attisano Villanova University, Xue Qin Villanova University, Foyzul Hassan University of Michigan at Dearborn DOI | ||
11:45 15mTalk | Concept-Based Automated Grading of CS-1 Programming Assignments Technical Papers Zhiyu Fan National University of Singapore, Shin Hwei Tan Concordia University, Canada, Abhik Roychoudhury National University of Singapore DOI |