Isak Berglind

Isak Berglind

Stop lying about the benefits of testing

Isak Berglind • September 5, 2021

php testing

First of all, I'm sorry for the click-baity title. I am a huge proponent of testing!

However, I think some people exaggerate the benefits, and claim that if you just write tests for your code, you can deploy safely and confidently, even on Fridays, because hey! You've got the tests, right?

I do not think that is true.

The tests are only as smart as we are. If we’ve missed some edge case (which, let's be honest - we usually do) and not written a test to make sure it's handled, the tests will not save us. Instead our app will break just as much as an untested app would.

A test can also be incorrectly implemented. Let's say for example we’re testing a function that should return true if a certain condition is met, and false if it doesn't. We test that it does in fact return true when it should, but we forget to prove that it returns false in all other cases. We now have a bug in the system that we thought we had tested.

Another issue that might arise is that the environment is not properly cleaned up and reset between tests. Some database rows, environment variables or similar might have spilled over from a previous test, causing the next test to pass falsely.

Writing good tests is hard. Sometimes we screw it up and write a bad test, just as we accidentally write bugs in our software.

Even though we've done our best to make sure everything is tested, most probably our new feature will have bugs in it. As I see it, the biggest benefits of testing come as the app matures. If we write a test whenever we find a bug, proving that it's fixed, the code base will become more robust each day and it's features will be documented in the tests. If we developed a feature without tests, six months from now, will we still remember all the edge cases and test them manually? Likely not. Not to mention code someone else wrote.

Tests are awesome! Test your software! It will make your life easier, the app more solid and give you more confidence that it does actually do what it's supposed to. Just be humble to the fact that even tested software has bugs. Don't deploy on fridays, even if the code is tested.