Software quality analysis is essential to shipping high-quality software. You look at insights to be sure that your project is on track. Is the number of bugs increasing or decreasing? Are we adding a lot of complexity to the project? To answer those questions, we need a reference, something to compare with. You may think, we can retrieve our history to compare our project to itself and analyze the trends. But that option is much too expensive to be worth and will be incomplete. That is why you should start quality monitoring as soon as possible.
Monitoring software is about quality trends
Software quality is all about monitoring trends. Take the two charts below, the one on the left shows the evolution of the technical debt of an open-source project in the last 7 versions, and on the right, only the last version is displayed. The chart on the right gives you some information about the quantity of the technical debt but you have no reference. Therefore, it is difficult to assess if you should put efforts into decreasing the technical debt. While on the left, the technical debt is increasing in a manageable way as your version and codebase increase.
Now that we understand the relevance of having the history of our project, you may wonder if even at today’s date you could re-create that version’s history. In the next paragraph, we tell you about our experience in doing so.
Our project experience
Context
We have a continuous quality process to monitor source code and tests. For a special analysis, we needed to add a data provider that analyses java classes. We also had the requirements that the data provider had to be added on specific tags of our development process. Learn more about data providers with our article on data silos.
Technical pitfalls
This new provider is running through Apache Maven, at compilation time. Apache Maven is a build automation and management tool. Of course, we did not have these results in our continuous integration chain. Thus, we had to modify our Maven configuration and backport the changes to the first version we wanted to build. At first glance, it did not look that bad. It is just a new phase to add to the build process in order to automatically run our data provider. Since we had made other major changes to our build configuration, we decided to backport them too because our most recent configuration was better.
Here the headaches started. For that specific case, we created a new branch at the desired first version and started merging all our Maven configurations. The Maven configuration can be managed at the package level with the Project Object Model or commonly call POM. Thus, several packages mean several POMs, so you must go through all of them one by one and merge the changes according to the version you are creating.
Heureka!
We managed to compile and build the first version of the history. We got all the files we needed, test results, and bug finder to import into our software quality monitoring tool. Then, we thought we are almost done. In fact, the hardest part had to be done: We must merge upward, and there will only be minor changes. That step is also very error-prone; you may forget to change one version name and Maven will yell at you that it cannot compile your project!! Then goes debugging, and of course, you must iterate for all the versions you want to re-build. I hope you don’t have too many.
Other examples
Some tools you may use through an API may only return the current endpoint. For example, Jira Software offers an API with the current state and the history of your issues. If you don’t save your Jira history you will have to deduce the state of your issues in the past to get a consistent history.
Imagine trying to run all the tests of your continuous integration, it would take a huge amount of time and you even may need a dedicated server for some time.
I think you got the point, going back in time with software quality monitoring is a very harsh task, therefore, you should avoid it. When you have to re-run what your continuous integration does best it is painful. These tasks are automated for a reason!
Some nuances
In most cases, you don’t need to go as far as creating a new branch and merging the Maven configuration to have a complete and precise history. For us, it was a deep investigation with a precise goal in mind. Nonetheless, it demonstrates the importance of thinking twice about the tools you will use in your project monitoring as soon as possible. Indeed, adding them later will have consequences on your quality trends.
As a Chinese proverb says, “the best time to plant a tree was 25 years ago, the second-best time is now”, the latter applies to software quality.
Start quality monitoring now
Software quality monitoring will help you ship higher-quality projects. Comparison with the previous version is one of the most relevant analyses you can perform on your project. To achieve it, you need a history of your projects in your quality monitoring tool. As we learned it will be a very harsh and time-consuming task to rebuild the project history with the same level of detail as the current version. Thus, you should start using a quality monitoring tool as early as possible. In a previous post, Flavien Huynh has shown how to set up quality monitoring in just one day.
Starting is very important if you can think about all the data you want to aggregate at inception. Especially, you will increase the quality of your monitoring and trends will be more consistent. Nonetheless, you will still be able to add new pieces over time or as you refine your software quality processes.
Real use case – software quality monitoring explained
You want more background input? Check out the video with more details:
- How to reach continuous quality monitoring? 5:24
- Quality gates used to assess OKR 7:36
- An exciting first week of monitoring 10:15
- Benefits reached after one year 11:50
- Focus on tests, technical debt and requirements 14:03
- Focus on the review process 19:56
- Are our quality gates efficient? 22:20
- How to improve our process? 24:38
Get started with your projects
- Squore product information: Analytics for software projects monitoring
- Examples: Quality check of open source projects with Squore
