A frequently asked TDD question
Today I and Tommaso facilitated the kickoff of a new project for a new team. One question that came up was:
We know that before the project is finished, we will have to profile all buttons and links so that users will not see options they are not authorized to use. Should we keep this in mind as we write the code today? Or should we rigidly adhere to YAGNI and defer all work on profiling until the profiling story is chosen by our customer?
This question comes up quite often. An answer that is often heard by agilists is
Never! Today you should only work for today’s story! The profiling story might never be chosen, after all, and even if it is chosen, you will refactor your code to accomodate the new functionality. YAGNI, my friend!
Tommaso rightly objected that this point of view is wrong. When the moment of link profilation comes up, the application is usually at a late stage of development. Refactoring all links and buttons to include profilation takes a lot of work. And we knew this functionality was coming, so we can’t even invoke the “customer changing their mind” excuse.
Agile development can only work if we can keep down the cost of adding functionality even late in the development cycle. This should apply to unforeseen changes, and even more so to features that we always knew were needed. If a change requires major reworking, then we clearly did something wrong.
What do I suggest then? Should we build in infrastructure right at the beginning for all major features? Should we do design upfront to protect us from this sort of mistakes?
Well, there is not a clear-cut answer. Naive reading of the TDD book will lead you to believe that just applying the TDD mantra
- Red—Write a little test that doesn’t work, and perhaps doesn’t even compile at first.
- Green—Make the test work quickly, committing whatever sins necessary in the process.
- Refactor—Eliminate all of the duplication created in merely getting the test to work.
Kent Beck, Test Driven Development by Example
will lead you slowly but surely, almost automatically, to a well-written system that makes it easy to add changes at all times. Well, there is a huge misunderstanding here. There is nothing “almost automatic” in this process. If you read carefully the book, it will also say things like “Our designs must consist of many highly cohesive, loosely coupled components”.
Just how highly cohesive and how loosely coupled our systems must be, is something that is left to the judgment of the reader. Which in most cases is not skilled enough to imagine just how cohesive and decoupled people like Kent Beck think when they say “cohesive and decoupled”. Most people don’t understand how serious people like Kent Beck are when they say “eliminate all of the duplication”.
Because if you’re not fanatical about removing duplication, about striving for code that is highly cohesive and decoupled, there is little chance of achieving code that is easy to change over time.
My answer is that if I am serious about removing duplication, then I will *not* have code like <a href='foo'>bar</a> written in more than one place; let alone in hundreds of places. I will have a single function, a single method, a single place where the html link is generated. Then when the time comes to add profilation, or HTTPS, or Ajax, or anything else that affects how my links should work, then I will have a *single* place to change. The change will not be too expensive.
If I don’t think I can work with the level of discipline it takes to really remove duplication, then perhaps I’ll be better off stopping pretending I’m really doing TDD, and start doing some design upfront.
March 19th, 2010 at 01:01
Hi Matteo, agree that just following blindly the TDD mantra will not always lead to develop a design that consist of many highly cohesive, loosely coupled components: TDD is not an excuse to turn off the brain and to stop learning.
Instead if I understand correctly, I disagree when you say
> There is nothing “almost automatic” in this process.
XP is explicitly defined [1] to create a set of generative rules [2] that do have some automatic effects that works in combination with the ability of team members to learn from this and adapt coding to it [3].
Indeed when I read about CAS in [3] I recognized many concepts referenced in XP Explained and Cockburn make it clear too in [1].
What I can add is that TDD with mock objects can be very effective to guide the team toward the correct understanding and application of the design principles, when the team do it right and learn from it.
[1] Agile software development: the business of innovation, Highsmith-Cockburn, 2001, http://alistair.cockburn.us/Agile+software+development:+the+business+of+innovation
[2] Generative Systems, http://en.wikipedia.org/wiki/Generative_systems
[3] Complex adaptive systems (CAS), http://en.wikipedia.org/wiki/Complex_adaptive_system
March 19th, 2010 at 08:42
Hi Luca,
when you say “the ability of the team members to learn from this and adapt” surely we cannot think this as something “almost automatic”, can we? :-) And when you say “when the team do it right and learn from it” again, how do we know that we’re doing it right? How can we know how “right” is?
March 19th, 2010 at 21:16
AutomaGic no, but automatic maybe, let me try with examples.
1- a team not doing tdd have training about solid principles and the demeter law, than the team is requested to produce code that conform to these principles
2- instead the same team have training about tdd with mock objects: about the practices and smells prescribed by the practice and the refacorings to remove duplications and clarify code intent. than the team is requested to write code driven by tdd with mock objects
in the example -1- the learning and application of the principles is non-automatic: say it is intentional.
in my experience i have observed many devs and 1 team succeeding into learning the principles and still failing to produce code that adhere to the principles
in the example -2- there is nothing explicit about the principles (this is were i mis?-use the word “automatic”) still the practices prescribed by tdd with mocks lead to remove and prevent many violations of the principles.
in my experience have observed 1 team that so produced code more adherent to the principles without an explicit policy to do so. then team observed the characteristics of the code (more easy to understand and change) and some member got interested into the principles, studied them and began to apply them in a more aware, intentional and extensive way.
if i understood your point, you suggest to apply tdd and also as in -1- teach design principles and ask the team to follow them.
while my point is to apply tdd with mock objects and support the observations of the effects on the code produced and support any curiosity about the principles from team members
is this correct? in your experience have you observed dynamic similar to the examples -1- and -2- or totally different ones?
if you arrived here, thank you for reading all this long comment.
March 20th, 2010 at 10:25
[…] commented on my previous post. My answer has got so long that it became a full […]
March 22nd, 2010 at 13:21
The TDD mantra has done more damage than good.
April 1st, 2010 at 20:53
“The problem with the Big Design Upfront is the Big, not the Upfront.”
Regarding the link example, view helpers in web frameworks do exactly this. They let you change the structure of all the links in the application simply by rewriting a route. :)
April 2nd, 2010 at 20:50
[…] (Red-Green-Refactor in detail) What is “simplicity” in programming? How to Manage Programmers A frequently asked TDD question Scrum Gathering: Community of Practice Testing Exceptions in JUnit 4.7 Top 10 PHP Techniques That […]
April 8th, 2010 at 08:34
Giorgio: the problem with frameworks is that they let you do everything that the framework thinks you should do. Once you need to do something that is outside what is considered proper by the framework authors, you’re out of luck. If all you need to do is to change the structure of the URLs in your internal links, Rails routes are wonderful. If what you need to do is to apply authorization rules to links so that they are active only if you’re authorized to use them, then Rails link_to won’t help you. You might do it by monkey-patching link_to, but I’m not comfortable with this sort of things. In Java you might do it by rewriting the framework class and placing it higher in the classpath than the framework original class. But it sucks to have to do this. The proper solution is to find all the places where you used link_to and change it to my_link_to. This sucks also, but it sucks a bit less, IMO :-)
April 18th, 2010 at 19:14
[…] collegue Antonio Ganci posted an article (Italian) that describes exactly what I meant in A Frequently Asked TDD Question and its followup. It’s about what you can accomplish if you’re really good in software […]