It Hurts When I Do This
There seems to be a division in the software development community over how granular code tests should be. Specifically, there are those who believe that tests should only utilize the public interface of a class, and those who believe that every separate piece of functionality should be tested independently of others.
There is common ground: everyone wants to write better code. Tests are simply a tool used for building better code. So the division is not one of outcome, merely granularity.
At iContact we have a goal of 100% code coverage. On the back-end development team, we use both unit tests (testing each method in isolation) and functional tests (testing the entire stack top to bottom.) We have had the discussion over how isolated tests should be. The answer we have come up with is "as isolated as we can possibly make them." This includes testing the innards of a class, not just its interface.
The Doctor Will See You Now
Picture the following scenario. You go to the hospital complaining of sharp stomach pain. The doctor checks your eyes, ears, nose and throat. He taps your knee with the little rubber hammer thing. He asks you questions to determine your lucidity, has you stand on tiptoe and touch your nose with your finger tips. After passing all these tests, the doctor declares that he can't find anything wrong with you, and sends you on your way.
How well do you think a doctor could diagnose your problem if they were confined solely to your "public interface": your five senses and anything the doctor can tell just be looking at and speaking to you.
Instead, we expect that doctors will have tools and equipment to look deeper then your public interface allows: MRI, x-rays, blood analysis and more. In the safe environment of the doctor's office, it is perfectly acceptable for the doctor to bypass your body's natural defenses in order to help fix you.
In your everyday life, however, the story is different. You don't expect, or desire, that any person you meet on the street can take samples of your blood, blast you with x-rays or perform a colonoscopy. It is only proper that others deal with you solely through your public interface.
Your production environment is like everyday life for your code. Your code interacts with other software and systems that don't want or need to know how your code works internally. In this environment, the public interface is sufficient.
However, in a development environment, code should be considered "in the doctor's office." Any tools the developer has should be brought to bear in diagnosing problems, identifying their root cause and fixing them. That includes the ability to circumvent the code's public interface and examine its internals.
When a doctor makes a diagnosis, he needs to be as specific as possible. He doesn't just say that your digestive system isn't working; he identifies the exact organ, or even a smaller piece of that organ, that is faulty. He knows what it is supposed to be doing, and what it actually is doing. Testing your whole digestive “stack” from input to, ahem, output only tells him that something is wrong, not where the problem lies. In order to fix the problem, it must be isolated. Preferably, it should be isolated to the smallest possible area, in order to prevent fixes from doing widespread damage to other levels of the stack.
This Will Only Hurt for a Moment
When we first started testing our class' internal methods, our modus operandi was to create wrapper classes; classes that inherited protected methods and overwrote them as public. The wrapper methods simply called parent's protected methods. Tests could be written against the child's public method and return the same results as the parent's protected method.
The maintenance of these child classes was a nightmare. During development of new code and refactoring of old code we had to remember to not only update our tests to reflect changes, but also to update our wrapper classes to account for new parameters, new default values and new (or removed) methods. The extra work was actually a disincentive to meeting our granular testing goal.
And so dynamic shunts were born. We threw out wrapper class files and replaced them in our tests with calls to our dynamic shunt library.
The shunt library works by using reflection to gather metadata about how individual class method signatures are built. It then constructs a wrapper class definition containing the exact same method signatures, including default values and pass-by-reference parameters. The only difference is that the wrapper class methods are all defined as “public”. The class definition is read into memory and becomes available for tests to instantiate and call methods on.
Take Two and Call Me in the Morning
The upshot to all this is that the wrapper classes are no longer defined statically in a file. They are created on-the-fly at runtime and disappear when the process ends. If the class being shunted changes, the shunt automatically picks up those changes on the next run. We can change the class method signatures, add and remove methods at will, update the tests, and run! No more maintenance of wrapper classes. Now there is no more excuse for not testing class internals as well as interface.
The evolution of our shunt library is ongoing. We are currently exploring ways to test the private and final methods hidden deep within our legacy code (before refactoring them into plain protected methods, of course.) We hope to make testing of non-public methods a community best practice. Dynamic shunts are a huge step towards removing obstacles, and therefore excuses, to doing truly granular unit testing.
The PHP shunt library and usage information is available on Google Code.
No comments:
Post a Comment