When code review fails, use biometrics to measure code quality
What a time to be alive! Researchers from the University of Zurich recently showcased a system which relies on biometrics to anticipate the quality of code developers produce. The paper was presented at the 38th International Conference on Software Engineering in late May.
“Finding and fixing code quality concerns, such as defects or poor understandability of code, decreases software development and evolution costs.” This is how Sebastian C Müller and Thomas Fritz from the University of Zurich, Switzerland began the abstract of their project titled Using (bio)metrics to predict code quality online. The two informatics researchers acknowledged that relying on code review is “a common industrial practice to identify code quality concerns,” but warned that this method has two major flaws: it is costly and it only takes place after a code change is already completed.
10 lines of code = 10 issues.
500 lines of code = “looks fine.”
— I Am Devloper (@iamdevloper) November 5, 2013
Code review may be a key industry practice, but it’s not flawless. The researchers used Ward Cunningham’s quote “every minute spent on not-quite-right code counts as interest on that debt” to emphasize that “delaying software quality concerns [defects or poor understandability of the code] increases the cost of fixing them.” Müller and Fritz’s goal is to use biometric sensing to overcome these disadvantages and lower the development cost by identifying code quality concerns online while the developer is still working on the code.
To prove whether biometrics is a better way to measure code quality than code reviews, the researchers performed a field study with ten professional developers in a Canadian company over a period of two weeks. They gathered biometrics such as heart rate variability and more traditional metrics such as code complexity and churn.
Müller and Fritz asked developers to evaluate the perceived difficulty of the code elements they were working with after every committed change and repeatedly throughout the study. They also gathered quality concerns identified in peer code reviews of the committed changes.
The results show that biometrics helped to automatically detect 50% of the bugs found in code reviews and outperformed traditional metrics in predicting all quality concerns found in code reviews.
To examine the generalizability of their approach, they conducted a second study with five developers in a Swiss company over a period of a week and learned that some (but not all) of their findings can be replicated. The results of both studies indicate that developers’ biometrics may help identify “difficult places in the code and in turn quality concerns and thus might be used to lower the overall software maintenance cost.”
Amongst other results, our study shows that biometrics outperform more traditional metrics and a naive classifier in predicting a developer’s perceived difficulty of code elements while working on these elements. Our analysis also shows that code elements that are perceived more difficult by developers also end up having more quality concerns found in peer code reviews, which supports our initial assumption. In addition, the results show that biometrics helped to automatically detect 50 percent of the bugs found in code reviews and outperformed traditional metrics in predicting all quality concerns found in code reviews.
Although the paper shows the benefits of using biometrics to measure code quality, it also notes the privacy concerns. The researchers claim that “more research is needed to investigate a feasible solution.”