Software quality temporal monitoring: Back in time

Flavien Huynh

Let’s continue our exploration: by monitoring software quality with a tool, you should benefit from the big quality concepts without the headaches. As we saw already, we have to navigate through a lot of data, with great detail, coming from several origins.
And that’s not all, you’ll also need to go back in time!

Why use time in quality monitoring

Software quality standards often recommend to check code metrics against thresholds, and more dimensions can be added, like safety-dependent thresholds. There are many metrics, dealing with a lot of categories (complexity, comments, coverage, etc), enough to produce useful dashboards, and relevant alerts.

So with this approach, we are describing the project’s current quality status in great detail, it is like a photograph we can zoom in and out at will.

But you surely guessed we can go further to monitor software quality: temporal monitoring.
Knowing how good a project is today is important of course, but knowing that you are improving or deteriorating is even better, that’s why quality metrics are not only useful on demand, but also need to be produced regularly, following the project’s lifecycle.

In essence, software quality needs regular monitoring: it is not an act, it is a habit.

Going back in time, and in the future

What more can we do now that quality data is available for today, but also yesterday?

Evolution tracking
The first obvious action is to track quality evolution.
“Is 70% code coverage good?” is not an easy question. If you previously had 20% or 100%, the answer will definitely not be the same, and produce satisfaction or alarm.
It is important to know if quality objectives are reached, but also how.
Trend analysis and prediction
Going one step further, temporal data can also provide knowledge on probable outcome.
For example, tracking the code complexity can help detect growing and plateau phases. If we can’t see the growth curve flattening when the project should have reached a certain maturity, then an action must be taken.
Analyzing the time behavior of quality objectives can help anticipate problems.

Goals and Milestones monitoring
Software project lifecycle is typically split into milestones, each having quality goals.
Using regular trend analysis with the project’s planned quality can help check if goals for the next milestone are going to be met in time, or at all. It is much more relaxing to work knowing the progress being made towards the objectives.
Early milestone quality tracking can help avoid a tunnel effect.

See it in action

In a previous post, we presented our analysis of several GitHub open source projects, an ongoing showcase of software quality monitoring.

Software quality temporal monitoring starts from the moment you first rate a project.
Depending on your process, the project is rated for a version or release, each at a specific date, for which the whole rating details are kept (incrementally, or else the stored data would explode!)

For example, if a project is rated each time a patch is issued, you will get a version for each, and be able to check the trend, verify the impact of a given patch, etc.
We can consider two types of project versions:

The draft version, which is the latest available version (there can be only one)
It typically represents the work in progress, and can be updated as teams are working
Note that in the online showcase there are no drafts, since we only take into account releases available in GitHub.
The baseline versions, which are all stored versions (there can be as many as needed).
They represent a full quality state of the project for a given moment in time.

Here in the “project portfolio” view you can access:

The breakdown of managed projects, in categories
Each project’s versions are accessible: you can visit the past in one click

That’s it for now. Now that we have filled the data space and project time, we need the most important part of the software quality monitoring experience: you!

Next time we’ll explore the user side of things.

Further readings

Contact me

Flavien Huynh

Flavien Huynh works as a Senior Field Application Engineer at Vector. He is in charge of assisting customers reaching efficient and relevant monitoring solutions, by way of project management, training and technical support. Originating from the research field, he wrote several articles on the topic of Software Project monitoring, and managed R&D projects with research partners.

Legal Notice

This text is the intellectual property of the author and is copyrighted by coderskitchen.com. You are welcome to reuse the thoughts from this blog post. However, the author must always be mentioned with a link to this post!