Extreme Enthusiasm

English summary: during the last Italian Agile Day the one concrete action that was proposed during the retrospective was to post what concrete successes we would be able to obtain in the next two months. This is my contribution.

Che cosa è cambiato fra il 21 novembre 2008 e il 30 gennaio 2009?

Tommaso Torti ha proposto, durante l’ultimo Agile Day, di raccontarci di lì a due mesi che risultati concreti abbiamo ottenuto nel miglioramento della nostra vita lavorativa.

Una cosa che ho fatto è stato di ridurre i miei impegni di prevendita e altro, e concentrarmi con più attenzione nelle vicende del nostro team in Sourcesense. Il team a questo punto è cresciuto a 12 persone (oggi 13), ed è stato impegnato in questo periodo su due clienti.

(more…)

I’ve been trying for some time to put my finger on the uneasy feeling I have when I talk with or read from good, skilled, mainstream Java programmers. I think my uneasiness comes from a fundamental, basic, philosophic difference in values. I value simplicity. Of the 5 (or 7) XP values, this is the one that for me does not even need explaining. I think “surely everybody values simplicity, don’t they?” In fact, it seems not everybody does.

It’s always difficult for me to keep my blood pressure down when we’re talking about Maven, for instance. Maven solves a difficult problem, namely the problem of mantaining a collection of Java application modules and libraries across different source code repositories and servers. As a result, Maven is extremely complex. From the complexity come the many problems that Maven has: it’s slow; it’s difficult to understand; it’s difficult to debug when things go wrong. For that matter, things often go wrong with Maven, again as a result of it being so complex; in fact, complexity leads to fragility.

Given this preamble, I think you understand why I regard Maven’s complexity as a major, show-stopping problem. But, to my unending surprise, Maven fans don’t regard this as a problem. They regard Maven’s complexity as a minor problem, if even that. A necessary evil, perhaps, which they don’t think it necessary to spend much effort to reduce.

Now Maven is just an example; a controversial one at that, since in fact many good Java programmer dislike it. But the same arguments could be made against most if not all Java major frameworks; Spring comes to mind, and Hibernate, too. They’re way more complex than necessary; they fail for reasons that require divine, or at least guru, intervention to solve.

I have different values. I value simplicity. Less parts mean less things that can go wrong, less things to understand. I like it when things work correctly from the start. I like it when the defaults allow me to use something in a basic way with no configuration at all. I like it when simple things can be done in a simple way, so that difficult things become possible.

We stand on the shoulders of giants. Me, I admire the works of the Algol patriarchs, Dijkstra, Wirth and Hoare. And I admire the works of the Unix patriarchs, Kernighan, Ritchie, Pike and McIlroy. And the works of the XP patriarchs: Beck, Jeffries, Cunningham. And the works of the Smalltalk patriarch, Kay. All of them valued simplicity.

Hoare, Wirth and Dijkstra deserted the IFIP Working Group 2.1 because they thought the specification for Algol 68 was way too complex; it ruined the beauty and simplicity of Algol 60. They produced a famous minority report about their disagreement.

Kent Beck explicitly named simplicity as one of the 4 (now 5) values of XP. The XP method is simple to explain; it does not try to write down every single detail of what I as a programmer should do. Yet it contains just enough guidance; it’s precise and clear enough.

The Unix patriarchs valued simplicity. The VMS patriarchs didn’t, and as a consequence neither did Windows NT. A clear example is the difference in the API for creating a new process. In Windows you have

BOOL WINAPI CreateProcess(
  __in_opt     LPCTSTR lpApplicationName,
  __inout_opt  LPTSTR lpCommandLine,
  __in_opt     LPSECURITY_ATTRIBUTES lpProcessAttributes,
  __in_opt     LPSECURITY_ATTRIBUTES lpThreadAttributes,
  __in         BOOL bInheritHandles,
  __in         DWORD dwCreationFlags,
  __in_opt     LPVOID lpEnvironment,
  __in_opt     LPCTSTR lpCurrentDirectory,
  __in         LPSTARTUPINFO lpStartupInfo,
  __out        LPPROCESS_INFORMATION lpProcessInformation
);

It has 10 parameters, many of which are structures with tens of other parameters. In Unix you have

pid_t fork(void);

It has zero parameters. All the things you can do with the complex and monolithic CreateProcess, in Unix you can do with smaller, simpler, modular calls.

It’s difficult to work together when we have a disagreement on fundamental values. For this reason it’s important that we recognize what values are important to us. Let’s talk about them, put them in the open, respectfully discuss about them. At least we will know what to expect of each other, so that working together in a fruitful way becomes a possibility, in spite of our differences.

Lately I was chatting with a collegue, an experienced agile developer who is just now beginning to work as an XP coach. He said to me “It’s difficult to picture what a coach does all day — it is much simpler when I am a developer.”

I also felt that way at times. When I first spent so much time away from the code, that I was not able to provide any significant technical help anymore, it hurt. There is also a danger that the coach drifts away from doing useful work for the team. There’s so much email to be done, so many things to coordinate, so many things that beg your attention. It’s important to keep your focus on the work of the team.

So, what is it that a coach is doing all the time? I wrote down a list, so that whenever I find myself doing something not on this list, I know I may not be doing my job well.

• Coach developers, customers and other collegues about agile values, principles and practices.
• Write iteration reports.
• Update and tidy up the team documentation (aka wiki gardening).
• Mantain data about team performance: stories, effort, code metrics.
• Train junior developers.
• Ensure the team keeps up the practices they gave themselves.
• Encourage developers. Invent or find new things to keep work fresh and fun.
• Provide feedback.
• Lead retrospectives.
• Pay attention. Observe the process. See what the developers are doing, and how they are doing it.

Anything else?

Purpose

To learn about the hexagonal architecture, which is a good way to structure an application, and how to shield your domain model from external apis and systems.

Prerequisites

This is not a basic exercise. I suppose you are already familiar with TDD and refactoring. You will need a computer with Java ≥ 1.5 and a Java IDE (I assume you will use Eclipse).

Ingredients

This kata can be done in two ways; if you want to try the refactoring way, then import the code in your IDE. If you want to try the TDD way, start with a blank Java project.

Problem: write a program that

1. Loads a set of employee records from a flat file
2. Sends a greetings email to all employees whose birthday is today

The flat file is a sequence of records, separated by newlines; this are the first few lines:

last_name, first_name, date_of_birth, email
Doe, John, 1982/10/08, john.doe@foobar.com
Ann, Mary, 1975/09/11, mary.ann@foobar.com

The greetings email contains the following text:

Subject: Happy birthday!

Happy birthday, dear John!

with the first name of the employee substituted for “John”

The program should be invoked by a main program like this one:

public static void main(String[] args) {
    ...
    BirthdayService birthdayService = new BirthdayService(
        employeeRepository, emailService);
    birthdayService.sendGreetings(today());
}

Note that the collaborators of the birthdayService objects are injected in it. Ideally domain code should never use the new operator. The new operator is called from outside the domain code, to set up an aggregate of objects that collaborate together.

Goals

The goal of this exercise is to come up with a solution that is

• Testable; we should be able to test the internal application logic with no need to ever send a real email.
• Flexible: we anticipate that the data source in the future could change from a flat file to a relational database, or perhaps a web service. We also anticipate that the email service could soon be replaced with a service that sends greetings through Facebook or some other social network.
• Well-designed: separate clearly the business logic from the infrastructure.

An optional complication

If you want to develop further the domain logic, you can take into account the special rule for people born on a 29th of February: they should be sent greetings on the 28th of February, except in leap years, when they will get their greetings on the 29th.

Testability

A test is not a unit test if:

• It talks to a database
• It communicates across the network
• It touches the file system
• You have to do things to your environment to run it (eg, change config files, comment line)
  Tests that do this are integration tests.

Integration tests have their place; but they should be clearly marked as such, so that we can execute them separately. The reason we draw this sharp distinction is that unit tests should be

1. Very fast; we expect to run thousands of tests per second.
2. Reliable; we don’t want to see tests failing because of random technical problems in external systems.

One way to make code more testable is to use Dependency Injection. This means that an object should never instantiate its collaborator by calling the new operator. It should be passed its collaborators instead. When we work this way we separate classes in two kinds.

1. Application logic classes have their collaborators passed into them in the constructor.
2. Configuration classes build a network of objects, setting up their collaborators.
   Application logic classes contain a bunch of logic, but no calls to the new operator. Configuration classes contain a bunch of calls to the new operator, but no application logic.

The hexagonal architecture

The traditional way to structure an application in layers is

+--------------+
| presentation |
|--------------|
|    domain    |
|--------------|
| persistence  |
+--------------+

The meaning of a layer diagram is that

1. a layer is a set of classes;
2. a class cannot reference classes in layers above; a class can only reference other classes in the same layer, or in the lower layers.

In other words, it should be possible to compile a layer without access to the source code of the layers above it, as long as we have the source code of the layers below.

The traditional three-layers architecture has many drawbacks.

1. It assumes that an application communicates with only two external systems, the user (through the user interface), and the database. Real applications often have more external systems to deal with than that; for instance, input could come from a messaging queue; data could come from more than one database and the file system. Other systems could be involved, such as a credit card payment service.
2. It links domain code to the persistence layer in a way that makes external APIs pollute domain logic. References to JDBC, SQL or object-relational mapping frameworks APIs creep into the domain logic.
3. It makes it difficult to test domaain logic without invoving the database; that is, it makes it difficult to write unit tests for the domain logic, which is where unit tests should be more useful.

The hexagonal architecture avoids these problems by treating all external systems as equally external. The system is not seen as a pipe with user interface as one end, and the database at the other. We model the system as a kernel of application code, surrounded by ports and adapters to the external systems.

Every external systems is hidden behind a facade that:

1. Provides a simplified view of the external system, with only the operations that we need to do with it.
2. Is expressed in terms of the domain model.

The domain model does not depend on any other layer; all other layers depend on the domain model.

+-----+-------------+----------+
| gui | file system | database |
|-----+-------------+----------+
|          domain              |
|------------------------------+

How can we make the domain independent, for instance, of the database? We should define a repository interface that returns domain objects. The interface is defined in the domain layer, and is implemented in the database layer.

The Facade-Adapter combo

The way to implement the hexagonal architecture is to abstract external systems and APIs with a Facade. A facade is a simplified view of the external system. For instance, there are a million of things that I could do with SQL and JDBC and my relational database. But my application only needs to do a few, specific things with the DB; for instance, retrieve all the employees that match some criterion. In that case, I can write a simple interface that exposes just that operation:

interface EmployeeRepository {
  List<Employee> findEmployeesBornOn(int month, int day);
}

Note that the interface is written in terms of domain objects: the Employee object belongs to the domain. A case could be made that the specification for the date (month, day) should also be a domain object; I leave that to you, the reader.

So that was the facade part. The domain logic will only deal with the facade, and can be tested thoroughly using stubbed and mocked versions of that interface. But what about the real implementation?

The code that talks with the real database (or flat file, in our case) implements the facade and correspond to the adapter pattern. The adapter can be unit tested as well, by mocking the external APIs, but this is usually not worth the effort. Mocking the JDBC APIs is in most cases very complicated. It’s perhaps more effective to just test it on a copy of the real database, or maybe with an in-memory version of the real database. There are no hard-and-fast rules here; there is a mixed bag of tricks. One thing most proponents of mock objects agree on is that you should only mock your own interfaces (see section 4.1 of Mock Roles, Not Objects).

Acknowledgment

The idea for the exercise of sending email I got from someone I overheard at an XP Day; I think it was Willem van den Ende but I’m not sure.

Thanks to the Orione team for beta-testing and commenting on this kata, in particular Marco Gulino and Roberto Albertini who performed it more than once.

References

The standard layered architecture is described in Patterns of Enterprise Application Architecture by Martin Fowler, and in Domain Driven Design by Eric Evans.

The Hexagonal architecture is due to Alistair Cockburn. Here is a video of Alistair explaining it.

My collection of resources on the Hexagonal Architecture

A pattern similar to the hexagonal architecture is the Onion architecture by Jeffrey Palermo.

Miško Hevery explains the adapter-facade combo in his post Interfacing with hard-to-test third party code.

Again Miško Hevery talks on Google Video on Dependency Injection, part of the Clean Code Talks Series.

The Dependency Injection pattern is one form of the Dependency Inversion Principle (pdf) by Bob Martin. Thanks to Jacopo for pointing out that this comment of mine was wrong.

Test Driven by Lasse Koskela contains plenty of tricks about how to do integration tests in Java.

Appendix: Useful snippets

How to convert a String to an InputStream:

new ByteArrayInputStream(string.getBytes());

This work is licensed under a Creative Commons Attribution-Share Alike 2.5 Italy License.

Update 2009/01/17: restored images, added CC license.
Update 2009/12/09: add link to my Hexagonal Architecture page
Update 2020/01/07: fix the url for Alistair Cockburn’s page, add video