Thoughts on code quality and testing

Writing code isn't easy. Writing good quality code is difficult. Testing code quality is even more difficult. So what makes for good quality code and how do we test for it? The short answer is: it really depends.

As a developer, I often discuss what constitutes well-written code with other developers. Sometimes we end up agreeing, but usually it turns into a very lengthy discussion. Aspects which are thought to determine code quality include:

• following SOLID principles;
• unit testing and high code coverage (code coverage tells you the exact quality of your code);
• being able to release code for production;
• high customer opinion of the product;
• low bug rates.

All of the above aspects have merit, but they could just as easily be wrong given that quality is subjective. Consider the following I have recently been working on:

• Application A: +/- 100000 LOC, high complexity, follows SOLID principles, over 15K unit tests, code coverage >98%, released every month, 0 bugs over the last year, 75 known issues.
• Application B: +/- 10000 LOC, medium complexity, pragmatic approach to SOLID principles, some unit tests (+/- 300), the customers have a high opinion of the product and a new version is released to production each week. 10 bugs over the last year, but 0 known issues.

There are advantages and disadvantages of each application’s quality aspects.

Application A has fewer bugs, but more work needs to go into rearranging code in order to allow for new features. Application B, on the other hand, has a higher bug rate, but can quickly accommodate new features and fixes for the bugs that do occur.

Code quality continuum
These application examples are at opposite ends of the ‘code quality continuum’, as I call it.

This continuum defines the quality aspects that constitute good quality code for each application and provides a measurable scale that allows for an objective view of each individual aspect. Besides acting as a scale, the continuum also provides the following: the rationale for each aspect, the advantages and disadvantages of being at either end of the scale and guidance on how to objectively verify the position on the scale. An example is given below.

The aspects I am most interested in are: separation of concerns, unit test count, code coverage percentage, code coverage exclusion percentage, release frequency and bug regression. Other concerns might be expressiveness of code, comment use, etc. For now, though, I will settle for the aspects I mentioned first.

Some of the aspects I mentioned above have no direct link to the actual code. However, in my experience, they are indicators of a software product’s general quality. These aspects are also often thought to constitute good code quality.

The table below provides possible definitions for several aspects.

 

If we take this table of code quality aspects and apply it to the applications mentioned above, we could come up with the following table.

 
From the table above, we can conclude the following:

• Both applications score roughly the same on average, though Application B scores slightly higher.
• Due to the known issues in its code base, Application A scores poorly on bug regression.
• Application B scores reasonably well on the unit test count.

Which application has the higher quality? Given the amount of data, it is still not possible to determine which application has the higher quality. The table only indicates how each application can improve on each scale aspect. To benchmark applications against each other, we would have to broaden the number of aspects on which to test each application, so as to increase the data set.

However, it is probably safe to say that an application scoring high on all aspects is of better quality than an application that scores lower.

In a real-world situation, these numbers might be used as quality gates to promote an application to production.

Conclusion
Code quality is subjective and can only be made objective if it is broken down into individual aspects. Each aspect needs to be assessed independently. In the future, I may look into tooling that facilitates the assertion of these quality aspects.