Extreme Enthusiasm

Ho avuto il piacere di essere relatore di Domenico Marini, che ha sviluppato una bella applicazione di tracking per i progetti di sviluppo Agili. L’idea per questa applicazione era nata quando il gruppo XP di Varese stava muovendo i primi passi, sviluppando un’applicazione come esercizio collettivo. Non c’era una maniera chiara per chi non fosse presente alle riunioni per capire come fosse il piano delle iterazioni, o a che punto si fosse con lo sviluppo. Avevo visto una pagina sul wiki pubblico dello XPUG con una sinossi chiara delle carte selezionate per ciascuna iterazione. Le carte erano state digitalizzate ed erano rappresentate come thumbnail, raggruppate per iterazione. Mi piaceva l’aspetto di quella pagina, e avevo suggerito a Domenico di realizzare un’applicazione che presentasse un’interfaccia utente simile, con una maggiore facilità di aggiornamento.

Domenico mi ha preso in parola, e ha realizzato molto bene la “vision” iniziale, grazie anche al lavoro di Federico Gobbo, corelatore della tesi, che è stato un cliente preciso ed esigente. Da vedere: il video della presentazione (David Heinemeier Hansson è stato un buon maestro… :-) Da scaricare il testo della tesi. Mi fa anche piacere reclamizzare il fatto che Domenico e il suo amico Massimiliano Pepe, hanno assorbito ed elaborato quello che abbiamo fatto nei primi due anni di Essap ed hanno aperto una società che sviluppa agilmente. In bocca al lupo!

James Coplien wrote about something like “TDD is a cult”. The whole post manages to speak ill about TDD, without giving any concrete reason why TDD is bad. I agree with other parts of the talk, like the terrible decline in the quality of technical education in University (none of my young collegues knows about regular expressions, much less about finite-state automata; what kind of CS degree is that?!?). Other parts of the post look cloudy and misinformed; how can Coplien say that he is “packaging my learnings about TDD from the past 20 years” when the concept is barely 8 years old?

Sure, people have been doing automated tests for far longer than that, but the idea that you can build a whole system using the tests as a design guide is certainly original. The fact that it can actually work is surprising, yet in my limited experience it can.

TDD deteriorates the architecture, says Coplien. Clearly he has not read about the refactoring part? If you’re not actively improving the architecture you’re not doing TDD. So what is the point? That if you don’t know the technique you won’t get good results? Thanks a lot, I didn’t need Coplien to know this. I never said TDD is easy, but you can learn it, and it’s an amazingly effective way to get a good architecture. What is the risk here? That ordinary programmers will learn how to do architecture effectively? That would leave people like Coplien without a job :-) Gasp! Maybe that would force people like him to write programs again! The horror!

Di ritorno dall’Agile Day 2007. Direi che la giornata è stata un grande successo: 250 partecipanti, numerose sessioni di buon livello su due track parallele. Al pomeriggio addirittura tre! Dato che Federico ha deciso alla fine di partecipare, portando una sessione nello stile degli XP Day belgi.

Io e il mio collega Tommaso abbiamo presentato la nostra esperienza su “Come dominare il codice ereditato”, ovvero come evitare di farsi prendere dall’ansia e dimenticare tutte le buone abitudini che abbiamo appreso. Soprattutto non dimenticare di scrivere test piuttosto che debuggare, e pianificare l’attività per feature visibili piuttosto che per task tecnici. Per chiunque si trovi in queste circostanze, e questo include in pratica tutti i programmatori, il libro aureo è il WELC.

E’ stato molto divertente, compresi i tragitti in macchina con gli amici del Milano XP User Group… abbiamo un sacco di idee per nuove cose da fare!

I’m back from XP Days in Mechelen, where I presented a session with Federico on the Pomodoro Technique. The presentation was well attendend (~30 people) and well received; a few participants were enthusiastic about applying the Pomodoro. It’s been fun, and two participants made an even funnier video summary of the Technique.

Quest’anno la presentazione iniziale all’Agile Day sarà tenuta da Tim Mackinnon, un agilista di lunga data e dai molti successi. Ho appena finito di leggere il suo The Slacker’s Guide to Project Tracking; or spending time on more important things…. Non c’è nulla che non si possa trovare in altre risorse, tipo l’ottimo Agile Estimating And Planning; però è un articolo semplice, preciso e informativo; e si basa sull’esperienza personale.

Here’s a quick and dirty report; I haven’t had the time to hyperlink it properly yet.

I came to Pisa by train on the evening of Friday 25 with my friend Federico. Federico had an appointment for dinner with some members of the Esperantist community, and he invited me to join them. (Federico is keen on Esperanto; he wrote his Master’s thesis on the subject.) I was a bit reluctant at first, being afraid of not being able to communicate… But I’m glad I accepted. We met in a pleasant little pizzeria “La Tana”; there was a Pisan guy, a young Catalan esperantist, and a young Russian esperantist woman. They were all very nice and fun to talk with; the Russian and the Catalan knew enough Italian to understand what I was saying, and I managed to understand the gist of their conversation. Esperanto is a very simple and regular language, that is designed to be easy to learn, to speak and to write. The vocabulary comes from west-European languages like English, German, French, Italian or Greek; so I could guess the meaning of most words. It was a very cosmopolitan evening. When we parted they told me “lernu Esperanton!”

They say that studying Esperanto for two days gets you to a level equivalent to a year of studying English. Most people think of English as the “universal language”, yet most speakers find it difficult to learn English. In fact English is quirky and arbitrary; spelling rules have an exception for every word. Pronounciation is difficult; most people can’t get the “th” sound in “this” right, and the proper sound for “a” in “play” is a bit of a mystery. I’m a Rails fan, so I’m perfectly prepared to believe that some technologies can be much better than others. So, what if some Zhamenhof guy invented a language “technology” that works much better than English for the purpose of multicultural communication? It might well be.

I’m typing in my hotel room; it’s Saturday morning and today is the second Railstoitaly day. Yes, the conference continues on a Saturday. I’m staying in a municipality-owned hotel that is a restored monastery of the 15th century. It’s beautiful, and so is Pisa in general. Everyone thinks about the leaning tower but the city would be well worth a visit even if it didn’t have the silly tower.

Yesterday the keynote was by Zed Shaw, the author of the Mongrel web server. His laptop wouldn’t work with the beamer so he had to improvise without his slides—We did get to see the original presentation the next day. Anyway I was not disappointed: the topic of his talk was programming ethics, “by which I do not mean ‘do not throw Chinese dissidents in jail’…”. Rather, he presented us with three rules that he follows when he’s coding. The first rule is “You cannot communicate sufficiently”, which means that even if you put a lot of care into making your code clear, intention-revealing and understandable, it’s still possible that your code will not be perceived as such by others. Just imagine what would happen then if you didn’t put care into making it understandable!

The second rule is “Your understanding is flawed”, which means that whatever I think, it’s never the whole truth. My understanding of any program is flawed; it may seem to me that there’s lots of useless crap in it, but it may well be that I don’t understand the original problem well. It’s a very powerful rule; when I remind myself of it, it helps me stay humble. It’s just too easy for a programmer who thinks himself smart to dismiss anything he don’t approve of with contempt; let’s not forget that my understanding is flawed, and I found I was wrong about things many times in the past. Programmers are too quick to say “the last guy to work on this program was an idiot”; yet our understanding is flawed, so we should assume that the last guy knew things about the problem, the program, etc., that we don’t.

The third rule Zed presented us is “Cause the least suffering”. Now this has a distinct Buddhist flavour to it; and I like it. Whenever we write code we have a huge potential to cause pain to others, or even ourselves, in the future. Think writing uselessly complex code, just to satisfy my ego. Think neglecting refactoring, just to “save a little time”. He was right in calling Java “the language of pain”; I’ve seen so much complex and downright unfriendly code in Java land; so many things that seem to be designed just to cause suffering (think EJB!) Later in the day David Heinemeier Hansson talked about the “merchants of complexity”, that is, the people who sell us those utterly complex application servers, that are so difficult to use and serve no useful purpose beyond hosting WAR files.

Zed said that Ruby and Rails have this zen-like quality of doing complex things with simple code. But in these days Java programmer come to Rails development, and tend to bring with them their bad habits, which boil down to the love of complexity.

Zed also said that Agile people tend to take the DRY principle too rigidly, coming up with very complex ways of removing duplication; yet it’s much more important to communicate clearly what the code is doing than removing every bit of duplication. Sure, we should not copy-and-paste code; but a little duplication is better than complexity.

Matteo’s note: There is much about what Zed said that resonates with XP values. XPers are always looking for code that “speaks”; they talk about “programming by intention”, which means that the code should explicitly show the programmer intentions; it’s not rocket science, it’s just a matter of choosing good names and extracting methods a lot. Often quoted rules for “code that is simple enough” are that, in order of importance, it:

1. Runs all the tests;
2. Expresses every idea you need to express;
3. Contains no duplicate code;
4. Has the minimum number of classes and methods.

(Ron Jeffries, Ann Anderson, Chet Hendrickson, Extreme Programming Installed, 2000) In effect these rules place “clear intent” over “no duplication”. When Agilists talk about the DRY principle they clearly are NOT saying that DRY is an end in itself; it’s merely a means to achieve Simple Design. The point is that takes time to become good at design. But this does not mean that you can’t have Simple Design without a lot of study. The “remove all duplication” rule is a very simple guideline that will make your code an order of magnitude better than most. It’s not all there is to say about design; it’s a guideline useful for getting at code that’s reasonably good. (End note.)

Zed is a good-humored, intelligent, friendly and overall nice guy. I’m not surprised to hear from him that people write to him, after attending his talks, and ask him advice about life in general :-) Which he modestly turns down: “I’m not coding Jesus!” I hope my summary of his talk was accurate enough.

The next talk was by James Cox, a collegue of Zed (I think?): “Scaling your app—a choose your own adventure game.” James had these very funny slides depicting a sort of treasure map; along the route there were lots of “monsters” named “MySQL”, “Ruby and Rails”, “Testing”; one for each topic. James explained each of them in the order chosen by the audience. We got quite a few good tips. The main theme was that you get scalability by following “measure; refactor; repeat”. All web apps need tuning, and before you tune you measure. He suggested that, in addition to monitoring our servers, we keep asking our users for how the app feels. (Matteo’s note: it may be that the site UI(User Interface) is broken, yet your users won’t tell you unless you ask.) As basic measuring tools he suggested Httpperf, AB(Apache Benchmark), and Flood.

About MySQL, one good tip is to execute the ”\s” command at the MySQL prompt; it gives all sort of information about the server status. He suggests to learn the advantages and disadvantages of the different MySQL table engines (MyIsam, InnoDB, DBD). He said to offload as much as possible to memcache.

Regarding the new Rails Query cache, he suggests to ditch it altogether; “caching for free” won’t work. It’s so much easier to use the ”||=” trick that the query cache does not make sense.

Explanation: in a model I can rewrite a method

def expensive_query
  Foo.find(...)
end

with

def expensive_query
  @foo ||= Foo.find(...)
end

The ”||=” thing is a Ruby (and Perl) idiom. It is the same as writing

def expensive_query
  @foo = @foo || Foo.find(...)
end

which means “if @foo already has a value, then return that; otherwise assign the result of Foo.find(...) to @foo and return it”. This has the effect of caching the result of the query in the current object; I may call expensive_query many times on the same object, but the query will be only run once. (End explanation.)

The same trick can be used with class variables (those that start with @@). In this case the query gets executed once, for all present and future instances.

Desi McAdam talked first about the DevChix nonprofit she’s founding, to help women get into the programming profession. Matteo’s note: I’m still not totally convinced. It is true that very few women are programmers, but that does not mean we should necessarily do all that’s in our power to get specifically more women in the profession. I don’t think there “should” be at least 50% of women, or even 10% of women. It might just be a fact that most women don’t care about programming, and that’s all right with me. On the other hand, I’m not a woman, and I don’t know what it’s like for a woman to like programming and live in a world of men. I know we men programmer can be condescending, or impatient (Desi talked of an environment where a woman can ask for help and not be told to RTFM.) When we are not downright embarassingly obnoxious with our sexist jokes.

One of the DevChix (proposed?) initiatives I find very sensible: to have an apprenticeship program, where someone takes responsibility for “raising” and mentoring a junior programmer. I think this is sorely needed in our profession, where everyone learns in isolation. I think this has some affinity with the ideas of Brian Marick, who wishes that a programmer’s competence be certified by their peers, in a “web of trust” of sorts.

Then Desi presented a summary of REST programming in Rails.

Federico Gobbo presented his and Massimiliano Pepe’s work on WikiMaps. It is a web application that marries Mind Maps with Wikis. Federico says that while established wikis are powerful tools, it’s difficult to get one started succesfully. In the early stages the absence of structure can lead to disorientation. Federico’s idea then is to get the topic of the wiki started with a mind map. The mind map acts as structured table of contents to the wiki. The WikiMap app lets you add a note to every node of the map. There is plenty of room for the note, that becomes in effect a full wiki page. One shortcoming is that an external tool (FreeMind) is needed to draw the map; it would be nice to have a way to draw the map within a browser.

Nicholas Wieland gave a fun presentation of how he managed to ship Zooppa in 40 days with 3 people. He said that one crucial element was to release as often as every day, so that the customers could see the app. This made up for the absence of formal requirements (and probably, I should add, for the fact that customers could not come up with requirements without seeing a working prototype first.) One other insight Nicholas gave us was to code starting from user interface drawings: take a picture of the interface and “make it work.” Also important was to keep it as simple as possible, without giving in to temptations like “we’ll code this little framework so that in 2009 we’ll…”

We went to lunch under a pouring rain.

After lunch Thomas Fuchs, co-author of Prototype and author of the Script.aculo.us Javascript library, entertained us before his speech with some nice C64 music. That got me in a nostalgic mood… That mood was reinforced by his choice of font for the presentation: the C64 font looks just like the screen of the dear old home computer. The half-forgotten but familiar pixelated character shapes triggered warm feelings. Thomas gave us a sneak peek of the newest Script.aculo.us goodness due soon in version 2.0. I’m not much of a GUI person so I can’t comment much on this. Some effects are meant to look like in Keynote presentations. Some new feature make it easier to combine more than one effect in one smooth animation. But probably the most important novelty is the introduction of a “sprocket” system (think plugin).

Paolo Donà presented his work on Rails Widgets; it’s a plugin that encapsulates common user interface elements, bundling together the necessary HTML, CSS and Javascript. Paolo said that working with Rails makes it so easy to develop the business logic, that HTML work becomes dominant. “What happened to me, Rails turned me into a web designer! It’s DHH’s fault!” Realizing that some tasks, such as tabbed navigation, require hours of work to get right, and tend to be needed in most if not all applications, Paolo decided to stop reimplementing the same UI elements every time. These widgets look like an interesting concept; I share Paolo’s feeling that I can do web design work, but I would rather write business logic.

Ettore Berardi talked about his experiences in full-text search. He researched how three popular full-text search mechanisms work and presented the pros and cons of each. The three tools he examined are Ferret, Solr and Sphinx. I’m a bit of a caveman when it comes to full-text search, as I always used the very limited MySQL built-in engine. I realize, listening to Ettore, that these engines are much more powerful (some things cannot be tuned in MyIsam full-text search unless you recompile MySQL.) They also look like they’re really simple to use, in Rails fashion.

The last event of the day was the Skype interview with DHH. How not to like David? He’s smarter than anyone deserves to be, yet he’s nice and friendly. He does not try to sound modest; he’s too proud of what he’s done. And yet he does not sound like he’s full of himself. I think he’s generous; many people are happier now just because someone like him came out and wrote Rails. You hear things like “now I’m having fun again at work”. I think it has a lot to do with David’s focus on beauty, pleasure and fun in what we do. He says (quoting from memory) “To a craftsman the feel of the scalpel in his hand is very important. It would be absurd to him to work all day with a tool that does not feel right. The same goes for web frameworks; what if we had to work all day, every day, with tools that feel bad to work with? What kind of life would that be?”

I asked two questions to DHH. The first was “will you be writing a third edition of ‘Agile Web Development with Rails’?” To which he replied, “Not anytime soon.” He said that the getting the second edition out was urgent, for Rails 1.2 made much of what was in the first edition of the book outdated. But Rails 2.0 does not make much of the 2nd edition book invalid. He said they could write a PDF update, but not a full 3rd edition for the moment.

My second question to DHH, a little toungue-in-cheek, was: “You did a great job getting Rails 2.0 out; but can you tell us what you think will be in Rails 3.0”? To which David said: “I don’t know!”. He said major new releases of Rails happen when there’s a new “big thing”. The “big things” that changed from Rails 1.0 to 2.0 were Ajax and REST. You can’t predict what the next big thing will be, but when it will happen, David says “we’ll make it work with Rails in the Rails way.” (All of this as I recall from memory.)

The day was not finished though: we had a social dinner at the centro sociale Rebeldia. How to say “centro sociale” in English? I don’t know. It’s a sort of squatter-clubhouse. This one had techno-sculptures made of old TVs and blinking leds, and the obligatory colourful graffiti. The people were friendly. On a blackboard in the bar room was written “No Italian lesson today”, which probably means someone gives free Italian lessons to immigrants; of course it’s a good thing to do. The Esperantist people from the day before joined us. We had a nice dinner, followed by a concert by an all-women group who sounded a bit like a punkish version of the Shadows. It was a nice feeling to be in such a place; it was just like going to alternative concerts so many years ago…

Stay tuned for the report of the second day.

Ho visto l’eccellente Ratatouille ieri. Un dettaglio che mi ha colpito: la bella Colette che istruisce l’imbranato Linguini nell’arte della cucina dice, buttando una bracciata di pentole sporche nell’acquaio,

Cos’è questo disordine? Che cosa farai con tutte queste pentole sporche quando arriva l’ora di punta? Tieni sempre in ordine la tua postazione: il disordine rallenta il lavoro!

Ricorda nulla? Vorrei avere un clip di questa scena per quando si discute del tempo “perso” a fare refactoring!

(Strip from http://xkcd.com/)

Che cosa significa la “*” in rember* e insertR*? In generale, data una funzione f, in programmazione funzionale f* significa applicare f uniformemente a una struttura. Se la struttura è una lista, f* restituisce una lista con tutti gli elementi trasformati da f. Se la struttura è un albero, il risultato sarà un albero.

Quindi la rember* non è altro che la reject applicata ad alberi (ovvero liste comunque innestate. Sì Ugo, alla fine queste liste sono piante :-).

Il prossimo passo è generalizzare: definire una funzione star come fattor comune di insertR* e rember*, in modo da poter scrivere

def rember_star(a, l)
  star l, lambda { |sym| sym == a ? [] : [sym] }
end

def insertR_star(old, new, l)
  star l, lambda { |sym| sym == old ? [old, new] : [sym] }
end

La funzione star prende una lista e una funzione; dunque è una funzione di ordine superiore. Cool stuff! La mia soluzione:

def star(l, f)
  if l.is_a? Symbol
    l
  elsif l.empty?
    []
  else
    f.call(star(l.first, f)) + star(l.rest, f)
  end  
end

Espresse così la rember* e la insertR* mi sembrano più chiare. La star esprime lo schema ricorsivo, le lambda descrivono cosa fare localmente.

Prossimo esercizio: risolvere in Ruby i problemi posti dal genio di Lists and Lists (Gordon Plotkin, 1996).

Vedi anche: Why Ruby is an acceptable LISP

C’è un libro bello e divertente che insegna a programmare in Lisp, che si chiama “The Little LISPer.” Per divertimento, proviamo a risolvere un esercizio da quel libro; ma da Rubyisti che siamo, lo facciamo in Ruby :-)

L’esercizio originale, a pag. 98:

What is (rember* a l), where a is cup, and l is ((coffee= cup ((tea) cup) (and (hick)) cup)?

Risposta:
((coffee) ((tea)) (and (hick)))

Questa rember* in pratica rimuove tutte le occorrenze del primo simbolo, in una lista comunque innestata. Una prima approssimazione potrebbe essere

def rember_star(a, l)
  l.reject {|x| x == a}
end

Infatti con questa passiamo questi test:

assert_equal [], rember_star(:anything, [])
assert_equal [], rember_star(:foo, [:foo])
assert_equal [:bar], rember_star(:foo, [:bar])
assert_equal [:a, :c], rember_star(:b, [:a, :b, :c])

Ma fallisce miseramente con questo:

assert_equal [[[]]], rember_star(:foo, [[[:foo]]])

<[[[]]]> expected but was
<[[[:foo]]]>.

perché reject non prevede di esaminare ricorsivamente gli array innestati. E poi la soluzione che vogliamo è una funzione definita ricorsivamente! Il punto è di rispolverare il vecchio muscolo della programmazione funzionale. Allora una prima versione, equivalente alla versione con reject:

def rember_star(a, l)
  if l.empty?
    []
  elsif l.first == a
    rember_star(a, l.rest)    
  else
    [l.first] + rember_star(a, l.rest)
  end
end

Nota: Ruby supporta Array#first ma, inspiegabilmente, non supporta Array#rest. Per fortuna Ruby è estendibile:

class Array
  def rest
    self[1..-1]
  end
end

assert_equal [], [1].rest
assert_equal [2, 3, 4], [1, 2, 3, 4].rest

Ora, per risolvere pienamente l’esercizio, quello che resta è ricorrere non solo sulla coda della lista ma anche sulla testa:

def rember_star(a, l)
  if l.is_a? Symbol
    l
  elsif l.empty?
    []
  elsif l.first == a
    rember_star(a, l.rest)    
  else
    [rember_star(a, l.first)] + rember_star(a, l.rest)
  end
end

Di conseguenza dobbiamo aggiungere una “domanda” all’inizio: altrimenti ci ritroviamo a eseguire :foo.empty?, che produce un errore. Se la “lista” è stata montata fino a diventare un simbolo, restituiamo il simbolo. In questo modo riusciamo a passare anche il caso di test iniziale:

assert_equal [[:coffee], [[:tea]], [:and, [:hick]]], 
      rember_star(:cup, [[:coffee], :cup, [[:tea], :cup], [:and, [:hick]], :cup])

La programmazione funzionale è divertente! Ecco un problemino ulteriore (sempre da The Little LISPer):

What is (insertR* new old l), where new is roast, old is chuck, and l is
((how much (wood)) could ((a (wood) chuck)) (((chuck))) (if (a) ((wood chuck))) could chuck wood)


Risposta: ((how much (wood)) could ((a (wood) chuck roast)) (((chuck roast))) (if (a) ((wood chuck roast))) could chuck roast wood).

Riesci a scrivere insertR* in Ruby? O nel tuo linguaggio preferito?

Aggiornamento: aggiunta immagine copertina

Finalmente annunciato l’Agile Day italiano; si svolgerà a Bologna il 23 novembre. E’ un evento importantissimo per la comunità XP e Agile italiana. Io ho imparato tantissimo in tutte le edizioni che ho seguito. E’ gratuito, ma siamo tutti invitati a dare un contributo per evitare che gli organizzatori si sobbarchino tutto il costo dell’affitto delle sale.

Vorrei portare un contributo ma non so ancora di preciso cosa. Probabilmente qualche tutorial di base. Io trovo che alle conferenze i tutorial siano fra gli eventi più utili.

Questo evento è analogo agli XP Day che si svolgono in Belgio (dove io e Federico parleremo della Tecnica del Pomodoro), a Londra, a New York, in Francia e in Germania. Approfittiamone per portare amici, colleghi, clienti e dirigenti; secondo me è difficile convincersi che Agile è una cosa che funziona, se non si vedono con i propri occhi gli agilisti all’opera… la comunità Agile è una cosa preziosa.