Valérian de Thézan de Gaussan · Data Engineering for process-heavy organizations

On the importance of tests in software.

Did you know that 50% of developers never conduct any tests?

Most developers only test sporadically. This is the startling reality of the software industry today. I’m not even mentioning the use of methods like TDD, as a vast majority are unaware of what Test-Driven Development is.

In the quest to build software, we encounter myriad challenges: understanding client needs, designing solutions, establishing infrastructure, and much more. But all these efforts are futile if the code quality is compromised.

How can we make sure we implemented the right thing without tests? How can we ensure features are correctly implemented? How do we guarantee code doesn’t regress over time?

Testing is the backbone of quality; and quality, in this industry, is everything.

A story

During a routine day, an e-commerce platform’s data pipeline received an influx of customer orders. Unexpectedly, one order arrived with incorrectly formatted data - a seemingly small glitch. However, this minor error cascaded into a major issue. The pipeline, unprepared for such irregularities, triggered a series of exceptions, rapidly overwhelming the system. The back-end, unable to process this anomaly, came to a grinding halt, disrupting operations and halting all transactions.

This story is sadly an everyday story. It is something that could have been prevented.

How do you prevent this?

While it is mathematically impossible to test all pieces of data that could enter the pipeline, as there is an infinity of it, it is possible to think about all the cases, as they are finite. In the example above, this is a validation issue, so there are two cases: the data can be either valid or invalid.

To understand what is “valid”, business rules around the data must be discovered. In the case of an ecommerce solution, a valid order could be an order with a non-empty amount, a customer linked to it, etc.

  • I can start by creating a few incorrect data object: a null object, an empty object, an order with no amount, etc.
  • I will also gather a few valid orders.
  • Then, I write a few tests to test the validation rule. Each test feeds one of the invalid or valid objects into the pipeline, and check if the behavior is correct.
  • Thanks to these tests, I can make sure I implement the validation rule properly.

The tests have to run almost instantly, in a few milliseconds. That way, whenever I make a change to the code, I can just press a button and verify that everything is still in order.

They live with the codebase, so that if another developer comes alors, he or she will have access to the tests, and also make sure that any change performed will not cause regressions.

Yet, 50% of developers never conduct any tests.

And in the remaining 50%, the majority rarely conducts them.

I’m not guessing these figures : they are based on rigorous research. For detailed insights, check the source, an IEEE paper.

If you’re looking for meticulous, quality-driven software engineering, reach out to me. Let’s build software that’s not just functional, but robust and reliable.