Top Highlights

• Findings . To the best of our knowledge, we￿nd the￿rst evi- dence that debugging can actually be automated and is no subjective endeavour. In our experiment, di￿erent practitioners provide es- sentially the same fault locations and the same bug diagnosis for the same error. If humans disagreed, how could a machine ever produce the “correct” fault l...
• On average, participants rated an error as moderately di￿cult to explain (2.8) and slightly di￿cult to patch (2.3). On average, participants spent 32 and 16 minutes on diagnosing and patching an error, respectively. There is a linear and positive relationship between perceived di￿culty and the time spent debugging
• The middle 50% of consolidated bug diagnoses references three to four contiguous code regions, many of which can appear in di￿erent functions or￿les. In other words, practitioners often reference multiple statements to explain an error.
• While 282 out of 290 (97%) of the submitted patches are plausible and pass the provided test case, only 182 patches (63%) are actually correct and pass our code review.
• Out of 290 submitted patches, 100 (34%) exclusively a￿ect the control- ￿ow, 87 (30%) exclusively a￿ect the data-￿ow, while the remaining 103 patches (36%) a￿ect both, control- and data-￿ow.

Domain