TDD and FUD
James Coplien wrote about something like “TDD is a cult”. The whole post manages to speak ill about TDD, without giving any concrete reason why TDD is bad. I agree with other parts of the talk, like the terrible decline in the quality of technical education in University (none of my young collegues knows about regular expressions, much less about finite-state automata; what kind of CS degree is that?!?). Other parts of the post look cloudy and misinformed; how can Coplien say that he is “packaging my learnings about TDD from the past 20 years” when the concept is barely 8 years old?
Sure, people have been doing automated tests for far longer than that, but the idea that you can build a whole system using the tests as a design guide is certainly original. The fact that it can actually work is surprising, yet in my limited experience it can.
TDD deteriorates the architecture, says Coplien. Clearly he has not read about the refactoring part? If you’re not actively improving the architecture you’re not doing TDD. So what is the point? That if you don’t know the technique you won’t get good results? Thanks a lot, I didn’t need Coplien to know this. I never said TDD is easy, but you can learn it, and it’s an amazingly effective way to get a good architecture. What is the risk here? That ordinary programmers will learn how to do architecture effectively? That would leave people like Coplien without a job :-) Gasp! Maybe that would force people like him to write programs again! The horror!
February 4th, 2008 at 12:06
Sigh, it’s interesting to stumble across postings like this on the web, that are wrong on so many points. It’s sad to find this from a “contract teacher” who is in a position to convey such “insights” to students. With the greatest humble respect for the author, let me offer another perspective.
On the very back cover of his book, Beck explains that the idea of writing tests to drive the code is not new. Maybe you’re a young buck, but we were doing this in FORTRAN back in the 1970s. Particularly when doing bottom-up design, it is great to build a set of APIs that are fully tested, and then build a layer on top of that. It works great when procedures are your primary organizing principle and when procedures are relatively decoupled (meaning there is little global data). Been there, done that, bought the T-shirt. In an OO environment, it sucks. Every procedure shares data that are global to it and other member functions. The essence of system form is in its object-ness rather than its procedure-ness. When using TDD, you build a bottom-up FORTRAN program in a Java wrapper. Whoopee.
Dan North, tired of TDD being “full of dead ends,” has moved on to BDD.
Read XP Refactored and the arguments (for which we have data) that unit testing in fact increases your bug density.
Marko Taipale and Ari Tannien (write to me and I’ll put you in touch with them) did an experiment to try out TDD in a major web gaming product in EGET in Finland. After six months they found that TDD brought them to the architecture they could have anticipated (and did anticipate) at the beginning. He writes, “Yep, we built a gaming platform with TDD and pretty much discovered that TDD offers _nothing_ for architecture.” Further, “An indication of TDD causing design problems for us was code smells that would not go away with refactoring. The code is clean and refactored and whatnot, but still something feels wrong.”
In my own company, we traced several problems with the GUI (basically issues of modality and lack of monotony — the two greatest interaction design failures) to TDD. Had the team taken a domain view — which is what Trygve Reenskaug holds to be the foundation of OOP — we would have captured the user’s conceptual model in the code.
In a large client of ours (which has 3 million customers) an XP project crashed and burned early on for lack of a sound architecture — an architecture that was supposed to owe to TDD. They went to the boss and asked to re-factor the code; the boss was impressed with the astuteness of his Agile little bunch. They then did a “refactoring sprint” that lasted as long as the entire project had taken up to that point. (Hey, what happened to constant refactoring? Oh, they had been doing that, too. Read below.)
Jeff Sutherland, inventor of Scrum, tells that people at his company, PatientKeeper, don’t do unit testing but use system testing instead. They, er, seem to be doing pretty well.
I could go on.
Clearly you have not read about the refactoring part. Refactoring is a disciplined activity that preserves functionality while making the code cleaner. I know how to do that within a class or within a class hierarchy. To argue that functionality is preserved while editing across class categories is pretentious. Such changes are not refactoring; they are just coding. Most often, they are just hacking. And the kinds of tests that one writes to scaffold a class definition can’t catch failures that arise in such changes. Also, see what Marko and Ari say above. And the constant refactoring being done in our client’s code only cleaned up stuff within architectural units, not between them. Since architecture is about what happens between the parts, moreso than within them, refactoring buys you very little for architectural improvement.
Write programs again? I work for a large financial client here in Copenhagen where I am working on a complex multi-threaded application. I’d guess I did several tens of thousands of lines of code last year and the project continues. Going into system test, the system has had five bugs.
So your post was wrong on the count of TDD being three years old. It was wrong about refactoring. It was wrong about my coding practice. I could go on, but just wanted to say enough that people reflect on your credibility to make these arguments. Let me suggest you go back to my ‘blog at http://www.artima.com/weblogs/viewpost.jsp?thread=216434 where I reflect on emotionally charged and substance-poor TDD arguments. The above posting certainly qualifies. The ‘blog isn’t about TDD at all but about the inability of people to substantiate their arguments with something other than “it feels good to me.” If you want to read what I feel about TDD, rather than about the ‘blog theme of unthinking so-called professionals, read either the article I published in Better Software in November and December last year, or the interview of me in December’s Lean magazine. I’m sorry that I need to take up arguments with an esteemed lecturer at Università dell’Insubria in a ‘blog where he hasn’t done a basic literature search, substantiated his arguments with logical progression, offered any empirical evidence, or put forth any theory. Maybe you can start doing the work of an academic or, at least, of a professional. O, the horror! :-)
February 4th, 2008 at 16:22
Hi Jim,
Thank for taking the time to comment here. I think it’s cool that you still write code, as opposed to just writing “architecture”. I didn’t mean to sound disrespectful, I was just mocking you. I don’t think I need to do a full literature search on your recent interviews just to comment on your blog; I still think your post is not substantiated, and it sounds emotionally charged to me. Anyway, it’s way less substantiated than your comment here.
I disagree on the fact that TDD existed before Beck. TDD is not just automated testing; it’s writing the test first, and refactoring, and doing it incrementally. Show me where this was written before Beck. I don’t think BDD is substantially different from TDD. I’d say it’s an incremental improvement on BDD.
I disagree also that TDD leads to procedural code wrapped in OO, or that refactoring doesn’t scale beyond the confines of a single class. In the experience of my team this is certainly not true. I wish I could do experiments to measure the effectiveness of TDD versus not doing it; but I mainly work as a developer, so I don’t do much in the way of research. I can only say that “it works for me and my team.”
Lastly, my University is a small one in a small town, yet I think I and my collegues do a decent job of educating young computer professionals. The work they do after they graduate makes me proud, even though I don’t claim it’s just thanks to my teaching.
February 4th, 2008 at 16:45
And, Jim: I didn’t meant that *you* can’t do TDD. I don’t know either way about that. I meant that if you don’t refactor, you’re not doing TDD, in which case you have no base to claim TDD doesn’t work.
February 18th, 2008 at 18:54
Just to quote Kent Beck from the acknowledgment in TDD By Example:
“Finally, to the unknown author of the book which I read as a weird 12-year-old that suggested you type in the expected output tape from a real input tape, then code until the actual results matched the expected result, thank you, thank you, thank you.”
February 21st, 2008 at 00:11
Matteo, I agree with Jim that TDD doesn’t improve the architecture… hey, also wearing an helmet while you’re working on a scaffolding doesn’t improve house architecture!
But when I’m doing my work (i.e. write code as good as I can) I like the warm protection of a good suite of UT.
February 23rd, 2008 at 11:28
Uberto, I disagree. I think TDD is primarily a tool for exploring and finding good design. The UT protection is a beneficial side effect, but it is not the primary reason for doing TDD. I mean, TDD as in the book by Kent Beck. If you are just doing unit testing, then you enjoy the protection while working, but it’s not all there could be to testing. TDD is about using the tests to drive the design, in an incremental way.
Now that does not mean that TDD gives you good design automatically! It still takes skill and intelligence and care and attention, just like when you do design without TDD.
Some people are good at design, Jim Coplien is certainly one of them. Ordinary programmers like me have a lot to gain from TDD, it helps us get to a good design. Not automatically, not guaranteed. But it improves our chances, a lot.
March 8th, 2008 at 13:36
Mmmh… let me to reformulate my phrase.
TDD doesn’t improve the architecture, but it’s a very good tool to improve design.
In other words, you need to study in order to improve your ability on design. TDD in itself doesn’t drive anybody toward a better design, it drives you (you Matteo) because you previously knew what a good design is.
I experienced very bad design created with TDD by people who doesn’t understand the design principles.
July 28th, 2008 at 09:17
I think one of Jim’s messages is that the trouble with TDD, as it is generally practised, is that it focuses on units (classes).
If you want to rearchitect and move functionality from one class to another you have to change the tests at the same time as the code. Bang goes your safety net.
The tests can actually act as a deterrent to the kinds of large-scale change that are needed to keep an architecture clean and simple, since making large-scale changes will involve unpicking your safety net while you work.
July 30th, 2008 at 15:11
Hi David,
this is a point; but is that your experience? Do you really find that it is the case? So what you are advocating is having acceptance/integration tests as safety nets in addition to unit tests? I guess so, given your background; and I agree. It’s not clear to me that Jim’s original post really ment this.