Guiding Greybox Fuzzing with Mutation TestingACM SIGSOFT Distinguished Paper
Greybox fuzzing and mutation testing are two popular but mostly independent fields of software testing research that have so far had limited overlap. Greybox fuzzing, generally geared towards
searching for new bugs, predominantly uses code coverage for selecting inputs to save. Mutation testing is primarily used as a stronger alternative to code coverage in assessing the quality of regression tests; the idea is to evaluate tests for their ability to identify artificially injected faults in the target program. But what if we wanted to use greybox fuzzing to synthesize high-quality
regression tests?
In this paper, we develop and evaluate Mu2, a Java-based framework for incorporating mutation analysis in the greybox fuzzing loop, with the goal of producing a test-input corpus with a high
mutation score. Mu2 makes use of a differential oracle for identifying inputs that exercise interesting program behavior without causing crashes. This paper describes several dynamic optimizations implemented in Mu2 to overcome the high cost of performing mutation analysis with every fuzzer-generated input. These optimizations introduce trade-offs in fuzzing throughput and mutation killing ability, which we evaluate empirically on five real-world Java benchmarks. Overall, variants of Mu2 are able to synthesize test-input corpora with a higher mutation score than state-of-the-art Java fuzzer Zest.
Wed 19 JulDisplayed time zone: Pacific Time (US & Canada) change
13:30 - 15:00 | ISSTA 8: Fuzzing 2Technical Papers at Habib Classroom (Gates G01) Chair(s): Marcel Böhme MPI-SP; Monash University | ||
13:30 15mTalk | Guiding Greybox Fuzzing with Mutation TestingACM SIGSOFT Distinguished Paper Technical Papers Vasudev Vikram Carnegie Mellon University, Isabella Laybourn Carnegie Mellon University, Ao Li Carnegie Mellon University, Nicole Nair Swarthmore College, Kelton OBrien University of Minnesota, Rafaello Sanna University of Rochester, Rohan Padhye Carnegie Mellon University DOI Pre-print Media Attached | ||
13:45 15mTalk | Rare Path Guided Fuzzing Technical Papers Seemanta Saha University of California at Santa Barbara, Laboni Sarker University of California at Santa Barbara, Md Shafiuzzaman University of California at Santa Barbara, Chaofan Shou University of California at Santa Barbara, Albert Li University of California at Santa Barbara, Ganesh Sankaran University of California at Santa Barbara, Tevfik Bultan University of California at Santa Barbara DOI | ||
14:00 15mTalk | Finding Short Slow Inputs Faster with Grammar-Based Search Technical Papers DOI | ||
14:15 15mTalk | Fuzzing Embedded Systems using Debug Interfaces Technical Papers Max Eisele Robert Bosch; Saarland University, Daniel Ebert Robert Bosch, Christopher Huth Robert Bosch, Andreas Zeller CISPA Helmholtz Center for Information Security DOI Pre-print | ||
14:30 15mTalk | GrayC: Greybox Fuzzing of Compilers and Analysers for CACM SIGSOFT Distinguished Paper Technical Papers Karine Even-Mendoza King’s College London, Arindam Sharma Imperial College London, Alastair F. Donaldson Imperial College London, Cristian Cadar Imperial College London DOI | ||
14:45 15mTalk | Fuzzing Deep Learning Compilers with HirGen Technical Papers Haoyang Ma Hong Kong University of Science and Technology, Qingchao Shen Tianjin University, Yongqiang Tian University of Waterloo, Junjie Chen Tianjin University, Shing-Chi Cheung Hong Kong University of Science and Technology DOI |