Skip to main content

Ralsina.Me — Roberto Alsina's website

Posts about python, programming

The lost opportunity in test coverage

Dis­claimer: This is a bit of a rant, but it's a friend­ly rant :-)

When peo­ple look at code cov­er­age, they are read­ing it wrong.

Suppose you have a class, something stupid, like your own implementation of a stack, called Stack. Because you are not a total monster, you have tests in your code right? In fact, you are claiming that you are doing TDD (Test Driven Development), or at least you like TDD, or you would like the idea of TDD, or, let's be honest here, you just say you are doing TDD, but what you do is you sprinkle the tests you feel are needed, which is largely OK, I am not going to judge you, you freak.

And then you add test cov­er­age check­s, and it says: 80%

What most peo­ple feel when they see that is dread. They see that 80% and feel "OM­FG, my tests suck! I don't have enough! If even 100% cov­er­age is not enough then this 80% means my code is an un­sta­ble piece of garbage!"

Well, no.

Whether your code is good or not is in­de­pen­dent of test­s. Tests give you the abil­i­ty to know if your code is crap or not... some­times. What tests re­al­ly give you (if they are not to­tal garbage in them­selves) is the con­fi­dence that you can change your code with­out sig­nif­i­cant­ly af­fect­ing the be­hav­iours the tests are test­ing.

So, if your tests of Stack ensure that:

  • Stack.push puts the element at the top
  • Stack.pop gets the top element
  • Your stack can hold as many el­e­ments as your re­quire­ment de­fines (may be in­finite)

Then what you implemented is a stack. Period. It works. It's fine. It may be inefficient, it may be ugly, who knows, but tests are not going to give you good taste. All they are going to do is ensure that Stack is, indeed, a stack, and behaves like a stack, and that when you stick your mittens in it and change things inside it it stays a stack.

Yet, your cov­er­age is 80%.

Should you add more test­s?

No.

You should delete 20% of your code.

Since code is a li­a­bil­i­ty and the as­set is the code's be­haviour, then that's what the first D in TDD is for.

Test Driv­en De­vel­op­men­t.

Use the tests to de­fine the be­hav­iour you wan­t. Then add code to im­ple­ment that be­hav­iour.

Don't chase use­less stats like cov­er­age.

If cov­er­age is not 100%, con­sid­er your test­s.

Is there be­hav­iour you want that is not rep­re­sent­ed as a sce­nario in a test?

If yes: then add test­s.

If not: re­move code.

And us­ing "cov­er­age is low" as an op­por­tu­ni­ty to delete code in­stead of adding tests is some­thing a lot of de­vel­op­ers mis­s.


Contents © 2000-2023 Roberto Alsina