Answering Jacopo’s comment
In a comment to my post about testing data serialization, Jacopo says:
well, if you’re lucky enough, both client and webservice will be under your control, so yes: focus on data (structure) sent/received, to be sure they’re as expected.
but is far more common a scenario where webservice aren’t under your control, they represent and integration point with external systems: that is, XML document syntax _is_ what you’re trying to test. for example, (I can think of at least 2 cases that happened to me in the last 6 months) switching from one parsing library to another or simply testing interoperabily (such as Java/.NET).
of course, not as much domain logic there ;) but still, webservices aren’t supposed to encapsulate domain knowledge, so focusing on syntax would be good enough.
Hi Jacopo,
Even in your case, the users of your service will be interested in doing something with your data, not in its syntax.
Consider this modified test: there’s an error in it, but it’s not easy to see.
@Test public static void serializesToCorrectXml() {
DataStructure structure = new ....;
XmlSerializer s = new XmlSerializer();
String xml = s.serialize(structure);
assertEquals("<?xml version=\"1.0\"><foo><bar>zot</bar></foo>",
xml);
}
Can you see it? The xml prolog is broken. Now if your code passes this test, it will break in production with your paying customers.
Consider this other test:
@Test public static void serializesArray() {
XmlSerializer s = new XmlSerializer();
String xml = s.serialize(new String[] {"A", "B"});
assertEquals("<list><el>A</el><el>A</el></list>", xml);
}
this one is also wrong, for a different reason. It’s syntactically valid, but does not contain the right information.
Even when I am not writing the client side, I would still write a client for my tests. The client should do something with my data that is similar to what a typical real client would do. At the very least, it should parse the XML to make sure it’s valid! That would take care of the first broken test. Then I would make the client extract the information from the serialized form, and make sure it contains the same information.
@Test public static void serializesArray() {
String [] anArray = { "A", "B" };
XmlSerializer s = new XmlSerializer();
XmlClient c = new XmlClient();
String xml = s.serialize(anArray);
assertArrayEquals(anArray, c.parse(xml).toStringArray());
}
June 16th, 2010 at 21:22
hi Matteo, thanks for the reply :)
from your previous post I thought your team mates were supposed to test-drive client code, to sent XML to webservice. so, examples I was thinking about were mainly related to that scenario: a (more or less) documented “contract” in terms of
* message syntax
* message data content
testing through a full serialization/deserialization loop would (probably) be too much, and not focused on task at hand: sent data to remote service.
asserting strict xml content would fail, as you noted, being fragile. but testing specific values, such as
assertContains(“A”, xml);
would probably be _good enough_. then, a final integration test, connecting to a remote service (or a fake local one, but through HTTP) would cast behaviour on stone, to be much more confidend.
what do you think?
June 16th, 2010 at 21:25
well, snippet has been stripped ;) originally was:
assertContains(“<el>A<el>", xml);
June 17th, 2010 at 09:08
Hi Jacopo,
what other tests would help you develop and design your code incrementally, as opposed to simply checking that it works?
June 17th, 2010 at 20:38
hi Matteo,
quoting your original post:
“This sort of tests supports incremental development. If my data structure was, for instance, a list, I could write a nice sequence of tests:
* serializesAndRebuildsEmptyLists
* serializesAndRebuildsSingletonLists
* serializesAndRebuildsListsOfManyElements”
this still stands for syntax-checking tests (sticking with client code), where you’d probably assert, incrementally:
assertDoesNotContain(“<el>", s.serialize(emptyList));
assertContainsTimes(1, “<el>A<el>", s.serialize(listWith("A")));
assertContainsTimes(2, “<el>A<el>", s.serialize(listWith("A", "A")));
or even better, with XPath:
assertCount(0, “//el", s.serialize(emptyList));
assertCount(1, “//el[@text='A']", s.serialize(listWith("A")));
assertCount(2, “//el[@text='A']", s.serialize(listWith("A", "A")));
I was just stating that serialization/deserialization tests do not focus on intended goal for that piece of code: send XML data to a webservice.
when you say:
"Note that this test is robust, as changes of the intermediate representation will not break it. […] we could change it to byte array, or anything else, with no need to change this test"
it was not clear to me why would you change such representation, given XML message forwarding was the original scope.
but you're probably right: I would probably use a subset of coarse-grained (integration) tests for driving development of such piece of code.
anyway, what feedback did you collected from last team you joined as a coach? did they find test-driving XML generation code useful?
June 17th, 2010 at 21:00
The XPath tests are in my opinion better than the ones with assertContains .
The original goal for my team was to transmit a datastructure over the wire. The format of the serialized data structure was not important, as long as it could be parsed reliably.
What I mean to say is that even if your job is to develop the server side only, the value from your web service comes from being able to deserialize the data structure and do something with it. This is why I may write a deserializer anyway.
The team I was working with was happy with testing against a literal string, though. It will take a bit of time until they realize the limitations. The tests they have, crude as they may be, are still good enough for them to drive development.