Test behaviour not syntax
Another thing that happened when I was last coaching a team on XP techniques. We were working on a real application, and one thing that we had to do was to serialize a data structure to XML for publishing on a web service. There are a ton of different ways to do that, but whatever you do, don’t write tests like this:
@test public static void serializesToCorrectXml() {
DataStructure structure = new ....;
XmlSerializer s = new XmlSerializer();
String xml = s.serialize(structure);
assertEquals("<?xml version=\"1.0\"?><foo><bar>zot</bar></foo>",
xml);
}
Can you see what’s bad with this code?
I can name a few things:
- It doesn’t scale. When the data structure grows, the xml string becomes big and difficult to handle
- It’s brittle. A tiny, insignificant difference in white space will break the test.
- It does not express intent. Look at the name of the test. We could not come up with anything better than this.
The crux of the matter is that we don’t care at all what the xml string is. What really matters is what we do with it, and in this case what we do is build the XML in one process, and reconstruct an equivalent data structure in another process. We should test that the behaviour works, not the details of an intermediate structure.
@test public static void serializesAndRebuildsFoobars() {
DataStructure structure = new DataStructure("foo", "bar");
XmlSerializer s = new XmlSerializer();
XmlParser p = new XmlParser();
assertEquals(structure, p.parse(s.serialize(structure)));
}
Now this tests the behaviour we’re interested in. Note that this test is robust, as changes of the intermediate representation will not break it. We don’t really even care what the serialized data type is: we could change it to byte array, or anything else, with no need to change this test.
This sort of tests supports incremental development. If my data structure was, for instance, a list, I could write a nice sequence of tests:
- serializesAndRebuildsEmptyLists
- serializesAndRebuildsSingletonLists
- serializesAndRebuildsListsOfManyElements
- …
One final note: let the pair who writes the serializer also write the parser. This will save a lot of arguments :-) Vertical slices of functionality always work best.
June 15th, 2010 at 22:21
well, if you’re lucky enough, both client and webservice will be under your control, so yes: focus on data (structure) sent/received, to be sure they’re as expected.
but is far more common a scenario where webservice aren’t under your control, they represent and integration point with external systems: that is, XML document syntax _is_ what you’re trying to test. for example, (I can think of at least 2 cases that happened to me in the last 6 months) switching from one parsing library to another or simply testing interoperabily (such as Java/.NET).
of course, not as much domain logic there ;) but still, webservices aren’t supposed to encapsulate domain knowledge, so focusing on syntax would be good enough.
June 16th, 2010 at 08:34
Hi Jacopo,
My answer got so long that I made a new post about it :-)
June 16th, 2010 at 08:37
[…] « Test behaviour not syntax […]
June 22nd, 2010 at 08:06
Matteo,
this time I partially disagree with your point. Well, if it’s obviously a bad practice to test the raw XML string since, as you pointed out, it would create a really brittle test, I don’t feel comfortable with your solution.
First of all, I entirely agree with Jacopo’s annotations. But what I would underline here is that you wrote what I call an “ever green test”. I mean that your test could be successful even when both XmlSerializer and XmlParser are wrong, if the mistakes in the two classes are symmetric. Think, for example, to the case of the broken XML prolog you showed: if both classes use a broken prolog, they will talk each other, but they will never talk to anyone else. Or you can think of both classes serializing and de-serializing decimal numbers using comma as a decimal separator, as in the Italian convention (I really did this mistake, so I’m pretty an expert ;-) ).
You said that in your use case the two classes need to talk to each other and not to the rest of the world, so my annotations could be of no interest, anyway I think we need to take care of “ever green tests”.
June 22nd, 2010 at 15:50
Hi Franco,
thanks for your criticism!! Yes, I see how that could be a problem. The prolog problem I would fix by having the parser use an XML parser, so that invalid xml would cause an exception. But I could get that also by testing the XML against XPath queries, as Jacopo suggests in the next post.
YMMV, but I still think it’s valuable to write code to extract the information from the xml and check this information. The extraction code could be a separate class, or private testing methods inside the test class.
I see how it would be valuable to *also* have a check that the syntax conforms to what I expect. But that would be a *testing* test, not a TDD test. I think that tests with serialize-then-parse would be better for driving the design.