Three Reasons To Win The Testing Trophy

In 2015, London was shaken to its foundation by Bloomberg’s software failure that affected over 300,000 traders worldwide. The same year, the Royal Bank of Scotland paid a £56 million fine because of a software glitch that had blocked their clients’ access to the bank’s services. In 2017, British Airways had to cancel over 1,000 flights due to a major IT breakdown. And perhaps coffee lovers from the US and Canada can recall the day when Starbucks ended up giving out coffee for free because its system refused to process payments.

The above examples show that one can’t underestimate the importance of software testing. Properly conducted tests reduce your stress levels and save you money, regardless of whether we are talking about e-commerce, finance, transportation, or even a coffee shop business.

Testing is an entire philosophy. Each app is unique, with its own business goals, complexity, and risks. That is why the idea behind testing should vary from project to project. This idea changes according to how much attention each type of test - static, unit, integration, and end-to-end - takes within a project. The amount of time and effort spent on each level can be graphically presented in various shapes. In this article, we are going to have a closer look at one of the most debated shapes - the testing trophy.

This figure was considered in depth by Kent C. Dodds in 2018 in an Assert(js) Conference. He developed the thought from the famous tweet of Guillermo Rauch, the CEO of Versel and co-founder of successful JavaScript projects in the US. The saying is: ‏Write tests. Not too many. Mostly integration. And there is a great deal of sense in this concise claim.

The traditional testing methodology is often presented as a pyramid. This concept was first described in Mike Cohn’s publication Succeeding with Agile. There, unit tests form the foundation, and they make up the largest part of the triangle. Then come integration tests in the middle, and end-to-end tests are on top.

Despite its short life-span in the software testing world, the pyramid evolved significantly and took the shape of an ice-cream cone, an hourglass, and even a honeycomb. Finally, two years ago, the trophy saw the light of day.

In such a form, most of the attention is focused on the central part, that is, integration tests. The middle layer can be broadened or shrunk according to the testing needs. This, in turn, also broadens the scope of the end-to-end tests. So for what reasons is such a model of testing proposed as a worthy alternative to the good old triangle?

#1 It provides the best combination of speed, cost, and reliability

No one would argue that the higher up the traditional or the newly proposed figure you get, the longer it takes to perform testing.

The static layer is the simplest one. It resides in catching type, stylistic, and formatting errors using linters, error formatters, and type checkers. Unit tests are relatively quick to write and easy to run. On this level, the work of separately taken units is verified. The end-to-end tests check the app from a user’s perspective. They are the slowest to write and run. The integration tests that verify the harmonious functioning of several modules together are in the middle. Thus, they are run at the best speed possible.

The next issue is cost. Each project has a limited budget allocated to testing, and you don’t want to go over it, do you? So the best option would be to focus on tests that are not too expensive and, at the same time, consider the design and provide the appropriate level of assurance. All this you find in the integration tests. End-to-ends require too much in terms of money, and unit tests are cheap. So why not just stick with the latter? The answer is, of course, reliability.

Each type of testing offers benefits. Simple unit tests deal with simple problems. In other words, they can check the proper operation of separately taken parts but can’t guarantee the same about the combination of units. The units are like parts of a motor. They might function all right individually, but when you put them together, the car won’t start. So unit tests can’t guarantee that the product as a whole is going to work as intended. The integration tests will.

Thus, integration tests provide the best possible combination of pace, cost, and reliability.

#2 It is highly profitable

The bigger the problem is, the more it will cost you in the future. If you fail to test a corner-case bug on the unit level, which one out of a million of your customers will run across, it’s not critical. But if the checkout button in the app doesn’t work, that is an issue that will cost you thousands of dollars.

On the other hand, the more complex the tests are, the more returns you will have on them. And that is for several reasons. As mentioned above, integration and end-to-end tests check critical cases that would hinder customers when using your product. Therefore, they give a higher level of confidence that the app is functioning just as the developers meant it to. Additionally, integration tests cover several components at once. So you can skip some unit tests but include them in the integration tests.

Therefore, integration tests are easier to write than end-to-end tests, and they are more stable than unit tests. Given all the pros and cons, integration tests have the highest ROI.

#3 It takes into account the architecture of the software

Another advantage of integration tests is that they are flexible enough to handle code changes. In contrast, unit tests show themselves have shown themselves to be rather architecture-unfriendly. What does this mean? For example, if you decide to introduce new components to the code on the unit level, you’ll have to rewrite the whole unit test suite. So integration testing puts stress on the design, which makes changing the code easier. In his article Test Shapes, Jeff Nyman concludes that the above leads to treating the code at the appropriate level of abstraction.

Conclusion

Testing is like painting a landscape. You use a fine brush for small details like branches or leaves. And you paint the background with a wider brush. The more layers your picture has, the more tools and techniques you apply.

The same is true for testing. And although it is reasonable to spend most of the time at the integration level due to its optimal cost and high efficiency, you need other tests as well. So be flexible, and combine different types of tests to deploy a high-quality product. And while implementing all these mechanisms, always keep the goals of your business in mind.