Extreme Enthusiasm

Lunedì 18 aprile sono invitato dagli amici del Novara XPUG per una presentazione. Parlerò di “planning patterns and antipatterns”, una presentazione che ho proposto anche a Better Software 2011 e che conto di rodare in questa occasione :o)

Le informazioni sono su www.gallug.it . L’indirizzo è

Viale Beato Quagliotti, 1
28066 – Galliate (NO) – Italy

At lunch I had the familiar conversation about how TDD/pairing are fine unless you’re under pressure to deliver, and then …. you just do whatever it takes. Well, if you don’t think TDD/pairing are “what it takes,” then why do you /ever/ work that way?
Dave Nicolette

Same thing here. I was talking with a potential customer the other day and he said “for most projects here we don’t have time to write tests, we just code like hell to get them finished as soon as we can”. Well, but writing tests, for me, *is* the way to get it finished as soon as I can. The moment when we’re under pressure is the moment we should stick to good technique the most.

One example of this is a nice coding competition by Lasse Koskela and Markus Hjort at XP Days Benelux 2006. The objective of the game was to write a program to play Indian Poker (a silly, fun version of Poker) and win against other programs. The whole competition lasted less than three hours. As you can imagine there was not much time to waste :-) So what did we do? (I was pairing with a young chap from Philips who didn’t know much Java, so I, well, kept the keyboard all the time.) As I mentioned in my earlier report on the event, We wrote an acceptance test and then we used TDD. We did drive *all* code with tests. And we won :-)

How did we win? Well, for one thing, half the other programs crashed. Ours didn’t. We decided on a simple strategy to decide when to “stay” and when to “fold”. Since we finished early and we still had something like 10 minutes, we decided to add an option of raising if the cards looked very good for us. Still driving code with tests. And you know what? At the last round only two or three programs remained. Our bot won by raising!

Three hours, a new API, a new problem. I did what it takes: to use the best programming technique I know. I don’t have time for working in a sloppy way.

David Tchepak wrote a thoughtful post on TDD. I share what he says. I find useful this little summary:

The process of writing the test gives us all sorts of feedback.

• The setup is too long or complicated? We’ve probably got too many collaborators (or are violating the Law of Demeter for them) and can try encapsulating them behind a new abstraction.
• Too many scenarios or test fixtures on the same SUT? Our SUT probably has too many responsibilities.
• Too hard to isolate the behaviour we’re trying to test, or can’t find how to write an assert for the behaviour? Maybe the current API or abstraction is wrong, and needs to be broken down differently?

[...] Writing the test first gives us ample opportunity to respond to this feedback, to sketch out our design, before committing to it. There is no cheaper time to change your code than before it is written.

The article is full of little gems like this note:

If you can’t get TDD to help you with a problem, make a note of it before trying something else, and come back to it later. It is important to see if it was just a problem TDD was ill-suited for, or whether a gap in your knowledge has been exposed. You’ll never know the difference if you give up too easily on TDD. Pursuing these leads is what led me to discover mocking, IoC containers, conventions, BDD etc. And I’m far from finished finding gaps in my knowledge. ;)

I like this tone. He does admit that TDD will not always for us, yet he advises not to accept that as a fact. Every obstacle in our work should be a reason to sit down and think what we could have done differently. Just as a team should not accept bugs as an inevitable fact of life, but should treat each one as an indication that they must do something to improve their process, the same should apply to when we are “stuck” not knowing how to proceed, or when we don’t like how our code turned out to be. Think about it; what part in our process could have been improved so that this does not happen again? (And if this leads me to think “my process? what process??”, great! I’m on the right path to improve :-)

Being in bed with a little fever, I thought it might be a good time for a little easy work on functional programming in Java.

Haskell envy

One of the things functional programmers often use is algebraic datatypes. For instance, in Haskell you would define a binary tree as such:

type tree = Empty
          | Leaf of int
          | Node of tree * tree
  

This says that a tree is either the Empty tree, or a Leaf that carries an integer value, or a Node that carries two subtrees.

The next thing is to define functions, such as tree height; the nice thing is that the functions are then defined by cases. It’s kind of fun:

height :: Tree -> Int
height Empty = 0
height (Leaf n) = 1
height (Node l r) = 1 + max (height l) (height r)
  

This says that an Empty tree has height 0, a Leaf has height 1, and a Node has height that is the maximum height of its subtrees, plus 1. Now we could have fun defining all sorts of other functions, like the contour (the list of the integers of all leafs), search for a particular value, addition of elements, and so on.

Additionally, when working in languages like Haskell, the compiler makes sure that when we define a function by cases, the program will not compile unless we remembered to define all cases.

This sort of data structures have all sort of applications; an obvious one is representing syntax trees, that is the output of a parser. Another one is representing the syntax rules themselves.

We can do it!

What’s to do if you have to work in a simpler language like Java? Do we have to give up the fun of algebraic data types? Of course not! It’s possible to define such structures in Java, even though it will take one or two orders of magnitude more lines of code.

Step zero: define the structure

public abstract class Tree {}

public class EmptyTree extends Tree {}

public class Leaf extends Tree {
    public Leaf(int value) {
    }
}

public class Node extends Tree {
    public Node(Tree left, Tree right) {
    }
}

Now we can create our trees with the following deliciously clumsy syntax:

Tree tree = new Node(
    new Node(new Leaf(0), new EmptyTree()),
    new EmptyTree());

We could make it slightly less verbose with static functions.

Step one: define height

The easy way to add a function is to define it abstractly in Tree. The compiler then forces us to define it in all Tree subclasses (in all cases). First a test:

@Test
public void heightOfSimpleTree() throws Exception {
    Tree tree = new Node(
        new Node(new Leaf(0), new EmptyTree()),
        new EmptyTree());
    assertEquals(3, tree.height());
}

Then the implementation:

public abstract class Tree {
    public abstract int height();
}

public class EmptyTree extends Tree {
    public int height() {
        return 0;
    }
}    

public class Leaf extends Tree {
    public Leaf(int value) {
    }

    public int height() {
        return 1;
    }
}

public class Node extends Tree {
    private final Tree left;
    private final Tree right;

    public Node(Tree left, Tree right) {
        this.left = left;
        this.right = right;
    }

    public int height() {
        return 1+ Math.max(left.height(), right.height());
    }
}

(I told you this was going to be verbose. You were warned :-)

Step two: respect the OCP, son!

Now we could go on defining all kinds of functions on trees this way; but this would mean that for every new function, we need to change all of our classes. That won’t do; we don’t want to keep changing and adding to the same old files. We want to close the Tree classes for modification, but keep them open for extension; this is the open/closed principle. What can we do?

A pattern to the rescue. Have you ever wondered where it’s reasonable to use Visitor? Well, this is it. As usual, we start with a test:

@Test
public void heightWithVisitor() throws Exception {
    Tree tree = new Node(
        new Node(new Leaf(0), new EmptyTree()),
        new EmptyTree());
    Visitor height = new Height();
    assertEquals(3, tree.accept(height));
}

The implementation of visitor can be:

public abstract class Tree {
    public abstract int accept(Visitor visitor);
}

public class EmptyTree extends Tree {
    public int accept(Visitor visitor) {
        return visitor.visitEmptyTree();
    }
}

public class Leaf extends Tree {
    private final int value;

    public Leaf(int value) {
        this.value = value;
    }

    public int accept(Visitor visitor) {
        return visitor.visitLeaf(value);
    }
}

public class Node extends Tree {
    private final Tree left;
    private final Tree right;

    public Node(Tree left, Tree right) {
        this.left = left;
        this.right = right;
    }

    public int accept(Visitor visitor) {
        return visitor.visitNode(left, right);
    }
}

Note that each tree subcase provides the visitor with all the information they contain. This way we avoid getters, which is nice. The definition of Height is very similar to the Haskell example:

public class Height implements Visitor {
    public int visitEmptyTree() {
        return 0;
    }

    public int visitLeaf(int value) {
        return 1;
    }

    public int visitNode(Tree left, Tree right) {
        return 1 + Math.max(left.accept(this), right.accept(this));
    }
}

The nice thing is that now the Height function lives in its own little object. We were able to decouple it from the Tree classes.

We still can’t forget to define a case, as the compiler will force us to implement all three methods of Visitor.

Step three: generics

Of course, the big problem with this Visitor implementation is that it only works for functions that return integers. We can solve this with Java generics:

public interface Visitor<T> {
    T visitEmptyTree();
    T visitLeaf(int value);
    T visitNode(Tree left, Tree right);
}

public abstract class Tree {
    public abstract <T> T accept(Visitor<T> visitor);
}

public class EmptyTree extends Tree {
    public <T> T accept(Visitor<T> visitor) {
        return visitor.visitEmptyTree();
    }
}
// etc.

The other way to solve the genericity problem would be to define the Height visitor as a stateful object, that has a int result() method that is not part of the Visitor interface. But we’ll leave that to some other time.

A delightful little book on this very subject is A Little Java, A Few Patterns by Matthias Felleisen and Daniel P. Friedman. In fact I think I learned the meaning of Visitor by reading this book! The authors wrote a number of books on Lisp and Scheme in a similar vein; the first one was The Little Lisper and it takes a very unusual approach to teaching.

Hope it comes in handy!

Yesterday I presented “TDD from end to end” at Codemotion in Rome. My talk was about how to use TDD to drive the development of the whole system, from the GUI to the database. The slides contain a small solved exercise, which could be a good kata for Java servlet and JDBC skills. The techniques I showed are not my own! I’m showing things I learned from Francesco Cirillo and Carlo Bottiglieri.

The slides are in Italian, but they mostly contain Java code with English identifiers: 20110305-tdd-codemotion.pdf. Enjoy!

Questo sabato parlerò al Code Motion di TDD. Io sono alle 15.20 nella sessione “Tool and Processes.”

http://www.codemotion.it/programma-talks

After the fact, it’s easy to find better solutions. I present two examples from my experience with an ecommerce Rails application.

How to avoid joins, part I

We needed to make both product categories and discounts appear or disappear at specific times. The canonical solution seemed, at the time, to declare

class Schedule < ActiveRecord::Base
  belongs_to :schedulable, :polymorphic => true

  def active?
    Time.now.between?(self.start, self.end)
  end
end

class Discount < ActiveRecord::Base
  has_one :schedule, :as => :schedulable

  # ...
end

class Category < ActiveRecord::Base
  has_one :schedule, :as => :schedulable

  # ...
end

The schedules table contains the two columns “start” and “end”. The logic for a schedule is pretty simple: it’s “active?” if “now” is between “start” and “end”.

A lot of thought and discussion went into deciding when to save a Schedule object in the schedules table; we wanted to write

class Category < ...
  def active?
    self.schedule.active?
  end
end

but this presumes that there indeed is a schedule saved in the database for a_product, so it had to be

class Category < ...
  def active?
    self.schedule && self.schedule.active?
  end
end

and similar complication happened when adding or changing the schedule. Selecting all the active categories, got more complicated for we needed an extra join, and a left outer join at that. That impacted all queries on categories or discounts, in particular the free search query that really didn't need to get more complicated. We decided to add a post_create callback so that every new discount or category would always have an associated schedule. The decision to go with the "polymorphic has-one association" led to complification and increased coupling.

So what is a simpler, caveman's solution? Well, why not add "validity_start" and "validity_end" columns to both categories and discounts tables? Our Ruby code would become:

module Schedulable
  def active?
    Time.now.between?(self.validity_start, self.validity_end)
  end
end

class Discount < ActiveRecord::Base
  include Schedulable

  # ...
end

class Category < ActiveRecord::Base
  include Schedulable

  # ...
end

So all it takes to make a model "schedulable" is to include the Schedulable module, and add two more columns to the model table.

Analysis of the "caveman" solution:

• Duplication? In the definition of the two tables, perhaps a bit. But no duplication in Ruby code, and far less code to write.
• Denormalized? Actually no. A "schedule" is not something relevant to the business; it doesn't need an identity. It is not a business entity so it's proper that we implement it as a collection of attributes rather than with its own table.
• Good Rails design? I think it is. The decision to make Schedule a model was bad design, as a Schedule is not a business entity.

So the "caveman" solution actually is what a good data modeler would have chosen from the start. We were fascinated by how easy it was to use the "polymorphic association" to remove the duplication of the two extra columns, that we ended up complicating our life for no good reason.

Lesson learned: always consider what a caveman would do. He might be smarter than you!

Lesson learned: Rails is an effective way to put a web GUI in front of a database. Think like a data modeler. Think Entity-Relationship.

How to avoid joins, part II

Later in the same project, we needed to make the website respond in English or Italian. Rails 2 is well equipped for localizing the GUI out-of-the-box; but it will not deal with the problem to translate the properties of your model. In our case, we needed to translate the names and descriptions of products.

The canonical Rails solution at the time was to use the Globalize2 gem. It's actually a good gem, but in retrospect it was not a good fit for our problem.

Fact: we didn't need to support 1000 languages. Just 2. Maybe 3 or 4 in the next 5 years.

Fact: we didn't need to translate 100 attributes in 100 models. Just 3 properties in 2 models.

The Globalize2 gem adds a join to a "globalize_translations" table to every query. That didn't do any bit of good to the free search query, that was awfully complicated already! So while in theory Globalize2 is transparent, in practice you have to modify many queries to take it into account.

Once again, what would our friend the caveman do? You guessed it, add extra columns instead of a join. Replace columns "name" and "description" with "name_en", "name_it", "description_en", "description_it". You add a bit of drudgery to the schema definition, but your queries turn out to become simpler. Taking advantage of the fact that schema migrations in Rails are very easy, it would not be a big problem to add a new language by adding the few extra columns.

And all the Ruby code it would take to produce the globalized descriptions in the web pages would be to define

class Product < ...
  def globalized_name
    if self.attributes.include? "name_#{current_locale}"
      self.attributes["name_#{current_locale}"]
    else
      # fallback to Italian
      self.name_it
    end
  end
end
  

My solution is not transparent, but it's simple and easy to understand; while Globalize2 tries to be transparent but does not quite succeed.

Lesson learned: less magic please. If you can't achieve complete transparency, then go for being explicit *and* simple.

What I’ve been up to?

It’s been some time. This last eight months or so I haven’t been doing much software development. Most of my work has been in support. I’ve done consulting work.

This autumn I was hired to help a team with an application that was constantly crashing. The application was closed source, so we could not easily change it or trace how it worked. The first thing we did was to start monitoring stats. I wrote a simple monitor that tracks the health of a Tomcat process. This gives a feeling for what’s happening inside the application, for what happens in normal times and in times when it’s stressed. We kept a monitor open with baretail open on these log files and watched what was happening.

Every two or three days, the count of busy threads in Tomcat started rising, from the normal level of 2-3 up to the maximum of 150. At that point there was nothing to do but restart the server.

Lesson 0: set up watch dogs. It’s easy to write a custom status page sniffer. Nagios keeps an eye over many variables (memory, CPU load, failing services and the like.) New Relic is a very good all-round solution when you don’t know where to start.

The usual suspect in such cases is the database, and watching the process monitor on SQL Server revealed that many connections were blocking each other. We called in one consultant, then another, and finally a third one who knew how to deal with this problem.

Lesson 1: a consultant may know how to deal with your problem, or he may be out of his depth. You better make sure that any consultant you call in know what they’re doing. Warning signs is when they try stuff at random (I read in a blog that changing parameter XYZ could be helpful!) or just give you canned responses (uh, better install the latest service pack…)

The last SQL Server consultant really knew his stuff, and was able to solve the SQL Server hangups problems by cleaning up and optimizing the SQL Server installation. It turned out that this database has been running with no maintenance for years.

Lesson 2: if you run a business that depends on database-backed applications, you better have a full-time DBA looking over maintenance.

After the SQL Server tune up the application ran considerably better. But there were more problems on the horizon. We still had a few occasions when the application did block. I didn’t know what to do to get insight on what happened in those cases. I wanted a thread dump of the Java Virtual Machine, but on Windows it turns out it’s not an easy thing to do.

Lesson 3: Windows works. You can’t really say it doesn’t. And this is the most charitable thing I can say about it. All the same, Unix is so much better than Windows in every which way. It has always been.

In the end I found a neat utility called psexec that helped us solve that problem. We set up a script that checked the number of busy threads on Tomcat every 30 seconds, and created a thread dump for us. This showed that our Tomcat threads were busy trying to deliver a mail message to a SMTP server. A quick investigation found that the application was somehow configured to use a temporary, slow SMTP server nobody remembered about. We changed the configuration to point to a fast server and this problem was fixed.

The last failure mode we encountered happened rarely; at times the memory consumption in Tomcat started growing from the usual 100-150MB up to its configured maximum of 500MB, and stay there. At that point, the application was slowing to a crawl, and there was nothing but, again, reboot it. Try to imagine our feelings of helplessness while this was happening!

Our next tool here was to run JMap to get a memory dump, and then analyzing it with the Eclipse Memory Analyzer. The culprit seemed to be some data objects that we knew the application was allocating to contain the result of queries. Whenever a user was requesting a report of, say, 50K records, the app allocated memory to hold all of the result in memory, and saved it… guess where…. in the user session! Then it showed the first 100 rows to the user. Since that query was slow, the user was staring at a blank screen for a minute or two, and then reload (F5!), which caused Tomcat to perform another query. Each of these 50K rows queries was allocating 50MB of ram in the JVM. Boom, boom, boom, once this started it brought Tomcat to its knees.

Lesson 4: Don’t store more than a few bytes in the user session. Being stateless is the key to scaling. Which brings us to the next…

Lesson 5: Pagination is best done in the database. Caching is best done in the database. Filtering is best done in the database. Heavy computations with the data are best done in the database. Brush up on your SQL skills! If you’re doing business software, your app’s performance and capabilities will be dominated by how well you use your database.

We could not change the application code, but luckily there was a configuration parameter that limited the maximum number of returned rows. Setting it to 1000 solved the problem.

Lesson 6: whenever you perform a query, you must be certain that the number of rows it will return is bounded.

And so our job with this application was finished. We went on tackling the next… One amazing thing that I learned in the process is that placing Apache in front of Tomcat is not always gueranteed to work well. We had some really rough time when we discovered we had this devastating Apache bug, and later we found a different bug in a different app with the same setup.

Lesson 7: try to use as few pieces (processes, computers, things) as possible. Every piece you have may be a cause of failure. Whomever says “… and by using XYZ, we get ZYX for free” is a fool! Nothing comes for free.

One of the collegues who valiantly supported me in this adventure was so much overwhelmed by the stressful situation that he resigned. I don’t know what to think of this; certainly this was not the sort of job that we were trained to deal with. This was Operations, not Software Development. And this was the life of the consultant. Diving into a problem not knowing the solution, but with the confidence that we will find the means to find the means of solving it. This is the opposite of “you tell me what I must do and I do it”, which is an unfortunately common misinterpretation of “customer collaboration”.

Lesson 8: Act as a consultant, not as a contractor (as @p_pugliese would say). Find your own ways of gathering information, making plans, trying solutions. Find your own ways of checking if you’re making progress.

In the end, my collegue will have to decide for himself if this is a crisis that leads to coming back to this job stronger than before, or a signal that his deep desire is to do something else.

Mercoledì 1 dicembre al Milano XP User Group, ci sarà Francesco Cirillo a rispondere alle nostre domande su “10 anni di XP in Italia.” Dove? A Sesto San Giovanni, presso XPeppers. Francesco ha portato Extreme Programming in Italia, formando il primo team XP italiano, di cui ho la fortuna di avere alcuni membri come colleghi. Ma Francesco non si è limitato a importare in Italia le cose apprese da Kent Beck; Francesco ha aggiunto le sue idee originali, come la Tecnica del Pomodoro, la sua maniera molto metodica di misurare la performance dei team, il suo stile di design emergente che è un po’ diverso dal TDD standard.

Contribuisci alla serata postando le tue domande sulla pagina dell’evento.

OCP Dojo at the XP Days Benelux 2010

Last week I presented with Antonio Carpentieri The Open/Closed Principle Dojo at the XP Days Benelux 2010. 19 people attended; I’m happy that we received positive feedback and good criticism.

In the feedback for our session, someone wrote in the “what I like”:

• Refactor before implementing new stuff
• Can apply this on Monday

In the general feedback about the conference someone wrote, in the “Which (new) things are you going to apply or investigate further?”

• The Open/Closed Principle
• Pomodoro technique
• IFs are evil

I’m also happy that Marc Evers and Rob Westgeest did a demo of the OCP Dojo at the NLJug last month.