Project automation for Confluence plugins

January 4th, 2010

A while ago our team was working at customizing Confluence for one of our customers. I’d like to share a few tips on how to automate Confluence plugin deployments.

(It should not be necessary to point out that all repetitive operations should be automated. Read all about it in The Pragmatic Programmer and Pragmatic Project Automation.)

Our automation was mostly Bash-based. I think it’s important to write expressive code in any language. When you’re scripting with Bash, the main way to be expressive is to use functions. We keep our scripts in a $(project)/script directory, so that you can invoke them with a minimum of keystrokes. The script we use for installing our plugin on localhost is:

 1  confluence_url=http://localhost:8080
 2  confluence_install=~/confluence-2.9.2-std
 3  admin_password=admin
 4
 5  source src/main/bash/lib.sh || exit 1
 6
 7  quietmvn package || exit 1
 8  plugin_jar=$(ls target/ourplugin-*.jar)
 9  [ -f "$plugin_jar" ] || (echo "no plugin generated"; exit 1)
10  confluence_login
11  confluence_uninstall_plugin
12  confluence_install_plugin

Lines 1-3 set up some variables for future use.

Line 5 loads our library of functions in the current shell process. The “|| exit 1” bit means “if this command fails, then quit immediately”.

The “quietmvn” in line 7 invokes maven with a filter that hides useless warnings. We found that Confluence plugin builds generate a lot of warnings due to dependencies on Atlassian packages that have poorly written “pom.xml” files. This (according to Atlassian) is harmless; but then again, useless warnings are harmful, so we filter them out.

Getting back to line 7, this builds the jar package of our Confluence plugin.

Line 8 defines a $plugin_jar variable with the relative pathname of the plugin jar. The $(foobar) bit is a Bash command that executes the foobar command and returns the text output generated by the command.

Line 9 says “if the plugin file does not exist, then exit with a meaningful error message.” In Bash, the command [ -f foobar ] tests if file “foobar” exists.

Line 10 to 12 are the interesting bit. They invoke three Bash functions that do what their name implies. They depend on the variables we set up in lines 1-3. This allows us to use the same functions for both local and production deployments. And here is our precious Bash library:

 1  cookie_jar=/tmp/confluence-cookies.txt
 2  curl="curl --cookie-jar $cookie_jar --cookie $cookie_jar --output /tmp/curl-output.html"
 3
 4  function confluence_login() {
 5    echo "login to confluence"
 6    $curl -s -S -d os_username=admin -d os_password=$admin_password "$confluence_url/login.action"
 7  }
 8
 9  function confluence_uninstall_plugin() {
10    echo "uninstall plugin"
11    $curl -s -S $confluence_url'/admin/plugins.action?mode=uninstall&pluginKey=ourpluginkey'
12  }
13
14  function confluence_install_plugin() {
15    echo "reinstall plugin ($plugin_jar)"
16    if [ \! -f $plugin_jar ]; then
17      echo "ERROR: $plugin_jar not found"
18      exit 1
19    fi
20    $curl -F file_0=@$plugin_jar\;type=application/octet-stream $confluence_url/admin/uploadplugin.action
21  }
22
23  function quietmvn() {
24      mvn $* | grep -v '\[WARNING\] POM.*Not a v4.0.0 POM'
25  }

The three functions I was talking about use Curl to interact with Confluence as if our script was a web browser. This is not the way to automate installs that Atlassian recommends, which is to use Maven with some Atlassian plugins. We found that the Maven way was not as reliable as using Curl.

And this is all there is to it. It took some time to find the right way to invoke Confluence with Curl, but every minute spent was worth it. This small library of Bash commands allows you to perform installs and uninstalls of any plugin with maximum reliability. It takes away a lot of pain from Confluence plugin development.

Ricerca sales manager in Sourcesense

December 22nd, 2009

Summary: position in Sourcesense for a Sales Manager

Sourcesense sta cercando un commerciale per assunzione. Questa persona si dovrà occupare (fra l’altro) di sviluppare il business per il team Agile. Se pensi di essere in grado di spiegare ai potenziali clienti perché è vantaggioso comprare sviluppo e formazione da un team che è in grado di sviluppare software in incrementi settimanali, con il controllo e la sicurezza che ne conseguono, allora sei la persona giusta per noi!

Lettori di questo blog: se conoscete una persona adatta, per favore diteglielo.

Tutte le informazioni qui:

http://www.sourcesense.com/en/careers/006-sales

Work with asymmetric bounds

December 22nd, 2009

Let’s talk about ranges of integers. Quick: how many numbers are in the range 16 ≤ x ≤ 37, that is in the set {16, 17, …, 37}?

If you answered 21 then you got it wrong: the correct answer is 22. When you express a range inclusive of both ends, the formula to compute the number of elements is

size of [lower, upper] = upper - lower + 1

The “+1″ in the above formula is the source of many “off-by-one” or fenceposts errors. Can we get rid of the “+1″? Yes, there is a way. Use asymmetric bounds, including the lower bound and excluding the upper bound. The formula becomes

size of [lower, upper) = upper - lower

I'm using square and round brackets as a shorthand notation for ranges with inclusive and exclusive bounds:

  [a, b] = { x | a ≤ x ≤ b }
  [a, b) = { x | a ≤ x < b }

Another reason in favor of asymmetric bounds: how do you express an empty range? With symmetric inclusive ranges, you can't: the smallest range you can express contains one element. For instance [3, 3] contains only the number 3. With asymmetric bounds, you can express an empty range with, for instance, [3, 3), which is empty.

Asymmetric bounds work very well in conjunction with counting from zero. Java and C arrays are indexed from 0, so that the index range is [0, N). The upper bound is now equal to the number of elements! Expressing it with symmetric bounds gives [0, N-1], which is ugly because of the "-1".

If you need to iterate on a C or Java array, the standard pattern is

  int[] array = new int[N];
  for (int i=0; i<N; i++) { ... }

You see how the body is executed when i==0, through i==N-1, but not when i==N, since we are using i<N as a boundary condition. How many times is the loop is executed? Exactly N! It's now easy to see that the iteration is performed the correct number of times.

Asymmetric bounds can be concatenated easily. Suppose you have a function that paginates results on a web page. You are on the page which contains results 30, 31, ... 39. If you express the range with two parameters from and to, the page url would look like


http://example.com/results?from=30&to=40

And the links to the next and previous page would be

  <a href="/results?from=20&to=30">previous page</a>
  <a href="/results?from=40&to=50">next page</a>

where the bounds for the next or previous page are easily computed by adding or subtracting the page size from the current bounds. Compare with


http://example.com/results?from=30&to=39

  <a href="/results?from=20&to=29">previous page</a>
  <a href="/results?from=40&to=49">next page</a>

it's more difficult to check that the bounds are correct here: you have to think where the "-1" must be applied.

So, if you define "+" as the operation of concatenation of ranges, you get this nice little law:

  [a, b) + [b, c) = [a, c)

It's this sort of properties that make it easy to combine software objects together. This helps towards the goal of programming with lego-like bricks.

In conclusion: express ranges with asymmetric bounds! You will avoid off-by-one errors, get rid of "+1"s from your code, and make your programs more modular.

References

The book C Traps and Pitfalls by Andrew Koenig has a good explanation of asymmetric bounds.

XP Day London was, indeed, a blast!

December 10th, 2009

It was very, very, inspirational and fun. I ran another session of Birthday Greetings Kata, facilitated an open space on the Hexagonal Architecture. London is a city I love to visit, and the Church House location, is fantastic: just a few steps from Westminster Abbey.

And I had the authors sign my copy of GOOS (I believe every good book should have an acronym… and I don’t like GOOSGBT :-) )

My signed copy of GOOS

Thank you guys! I have a feeling this book will be a good companion in my re-discovery of OOP.

Entity-relationship-oriented programming

December 10th, 2009

How I agree with Dafydd Rees when he writes:

Most Java and C# programmers have no idea how to do object-oriented programming.

They’re really doing entity-relationship-oriented programming. Trouble is, they’re in the majority so they form a self-reinforcing group that take for granted knowledge and skills that they don’t even realise that they don’t have.

Aaand… I’m afraid I was in the E-R-oriented camp until not long ago. Now I’m beginning to see how how objects should work.

Back from XP Days Benelux, on to XP Days London

November 29th, 2009

Last week I attended the XP Days Benelux. It’s the fourth time I go there, and it’s been a mild shock to realize how often I was there. The first time I went there was because I had the pleasure to meet Pascal, who was kind enough to help us run the first Essap. It was a big bet for us to do something as big as a Summer School. Luckily, we had help from Pascal and Francesco.

I was just starting to get my bearings in the Agile world back in 2006. Pascal came to Essap and taught us about estimating, planning and executing a plan, with his and Vera’s ingenious XP Game. And that was not his only contribution… he was, like, a *real agilist* who had a long tradition and a strong community behind. In 2006 we had just started the Milano XP User Group. It was great to meet someone who had a much bigger experience of working with agility.

There is this peculiar thing about the Benelux Agile community. They are real cosmopolitans. They speak English easily, since their countries speak many different languages and they are used to speak English even among themselves. They really *live* the agile values. I mean, if you want to succeed with agile, you better start living your life with the agile values. The organization of the XP Days reflects this.

For instance, it’s not like in old-school conferences where you send you session proposal and then it’s either rejected or approved. At Xp Days you are supposed to send a first draft, then you receive feedback on your proposal, then you improve your proposal with the feedback. This reflects the value of feedback, and the principle that good things are not done in one shot, but iteratively.

In the room where the plenary sessions were held, the core values of this group of agilists were exposed prominently. See in the first picture here, they are those sheets of papers high up to the right of the projector screen. They are unreadable in the photo, but they were well readable if you were in the room. They were

  • courage,
  • openness,
  • focus,
  • respect,
  • committment.

One other example of agile values in action is the many forms of feedback that are encouraged. For every session, you’re encouraged to write your feedback on a small card for that session. At the closing of each day, people who attended each sessions are asked to tell everybody what they learned (in 60 seconds! The timebox is another agile principle.) When you leave the conference you are asked to write a feedback sheet for the conference in general. “Give the gift of feedback”, is what the organizers say.

So my trail in the world of agile has been very much influenced from the beginning by the Benelux agilists. It’s a great trail to be in :-) Over the years I got to meet many more friends there, and it’s great to meet new ones every year. Some, like Yves, Marc and Willem, continued Pascal’s tradition and came to help us organize Essap in 2008.

My and Antonio’s contribution to the XP Days this year was the session on the Birthday Greetings Kata. Thanks to all participants! We learned a lot of valuable feedback on how to improve this session, and I’m ready for the next stop, which will be in London.

Impara il TDD con il team Orione di Sourcesense

November 25th, 2009

Summary: my company, Sourcesense, offers a one-day course on TDD

Se ti interessa imparare il Test-Driven Development con sviluppatori che lo usano tutti i giorni, in un corso di una giornata, mani-sulla-tastiera, iscriviti al nostro corso. E’ il primo corso pubblico che facciamo; prima d’ora abbiamo sempre fatto corsi in casa dai nostri clienti. Speriamo di fare un pienone!

Quando: 27 gennaio 2010

Dove: a Sesto San Giovanni, via Venezia 23

Quanto costa: € 300,00 più iva; ma puoi spendere solo € 200,00 più iva se completi il pagamento entro il 27 dicembre.

Chi lo insegna: io (Matteo Vaccari) più un altro sviluppatore del team Orione di Sourcesense.

Tutte le informazioni, e il link per l’iscrizione, sul sito di Sourcesense:

http://www.sourcesense.com/it/agile/training/

Back from iad09

November 21st, 2009

So this was the sixth Italian Agile Day!

It’s been great to meet old friends, and make new acquaintances too.

Appreciations:

  • Peter Stevens, for sharing insightful tips and tricks on how to succeed with fixed price projects.
  • Alberto Provaglio, for the interesting insights on system dynamics.
  • Jacopo Romei, for the courage of sharing his not-so-successful experiences.
  • Gabriele Lana and Simone Genini, for helping me with the coaching workshop.
  • Simone Casciaroli, for convincing me to do a workshop instead of my old boring presentation, and working hard at preparing it. We’ll do another one some other time!
  • All the people that attended the workshop. It’s been overwhelming. Thanks for coming and contributing!
  • Alberto Brandolini, for teaching me that there’s a lot more to DDD than I thought.
  • Indrit Selimi, for taking the picture at the workshop.
  • Marco Gulino, for the self-sacrifice of giving up coming to the IAD, getting up at 4 in the morning *and* having to deal with a confrontational situation.
  • My Sourcesense collegues, for contributing many sessions again in 2009.
  • Marco Abis and the Bologna XP User Group, for all the work *and* for being great people to hang around with.

Thank you all! Let’s do even better next year!

How I setup confluence on Debian

November 13th, 2009

Pardon the dreariness of this post. I did this twice in a couple of days. I thought I might as well write down what I did, to save me time next time. Let me state first that this is an opinionated install :-) I don’t like bloated Tomcat installations with many applications on them. I prefer to have each application in its own place with its own server. So I will use the Confluence standalone package, which is integrated with Tomcat. When I will install Jira on the same box, I will download another standalone package.

Preparations

Connect to your server. Execute

  apt-get install apache2 ca-certificates
  apt-get install mysql-server

Create file /etc/mysql/conf.d/confluence.cnf with the following contents

  [mysqld]
  default-storage-engine=innodb
  lower_case_table_names=1

Then restart mysql with

  /etc/init.d/mysql restart

Edit /etc/apt/sources.list, appending the string ” non-free” to every line.
Then execute

  apt-get install sun-java6-jdk

Install Confluence

Download the Confluence standalone package for gnu/linux (not the one for
evaluation)

  wget http://www.atlassian.com/software/confluence/downloads/binary/confluence-3.0.2-std.tar.gz

Create the directory /opt/confluence. In that directory, expand the confluence
archive. Then create a symlink from /opt/confluence/confluence-app to the
actual directory that contains the confluence application

  ln -s /opt/confluence/confluence-3.0.2-std /opt/confluence/confluence-app

Then create directory /opt/confluence/confluence-data.

Edit the file confluence/WEB-INF/classes/confluence-init.properties within the
confluence distribution, and add the line

  confluence.home=/opt/confluence/confluence-data

Edit the file conf/server.xml and change the following:

  • Change 8080 and 8000 to something other than these, which are the default
    Tomcat ports and might be useful some other day. I use 8180 and 8100.
  • In the first Connector element, add the attribute address=”127.0.0.1″. This
    will make Tomcat bind to localhost only. I don’t want to make Tomcat
    directly accessible from the outside.
  • In the first Context element, change path=”" to path=”/confluence”.
    Otherwise I can’t get the reverse proxy from Apache to work.

Create a file /opt/confluence/startup-confluence.sh with the following contents:

  export JAVA_HOME=/usr/lib/jvm/java-6-sun
  su -m www-data -c /opt/confluence/confluence-app/bin/startup.sh

Change ownership of the whole /opt/confluence thing to www-data

  chown -R www-data.www-data /opt/confluence
  chmod +x /opt/confluence/startup-confluence.sh

Download the mysql driver

  wget http://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.10.tar.gz/from/http://mirror.switch.ch/ftp/mirror/mysql/

Extract the archive, and copy the jar it contains to
/opt/confluence/confluence-app/lib

Create the database and the user for confluence

  # mysql -uroot -p
  mysql> create database confluence;
  mysql> grant all on confluence.* to confluence@127.0.0.1 identified by 'secret';

Now if you didn’t do this when first connecting, exit from ssh and reconnect to create a tunnel, to be able to browse Confluence from localhost on the server.

  ssh -L1234:localhost:8180 root@myserver

Execute /opt/confluence/startup-confluence.sh, then point the browser to


http://localhost:1234/confluence

You should see the confluence configuration screen. Now just follow the wizard. Remember to choose “Custom installation”, “External database”, and “Mysql”.

When Confluence is up and running, we still have some things to do. Let’s configure Apache to serve Confluence in http://myserver/confluence . Edit /etc/apache2/site-available/default and add the following lines:

  <Location /confluence>
  	Order allow,deny
    Allow from all
  
  ProxyPass /confluence http://127.0.0.1:8180/confluence
  ProxyPassReverse /confluence http://127.0.0.1:8180/confluence

Then execute

  a2enmod proxy proxy_http
  /etc/init.d/apache2 reload

You should now be able to see Confluence at http://myserver/confluence

Now we must make sure that Confluence will start at system boot. Just call
/opt/confluence/startup-confluence.sh in /etc/rc.local. It’s simple, and it
works.

You may stop here unless you want to serve Confluence via ssl. In that case… let’s carry on. Execute

  a2ensite default-ssl
  a2enmod ssl
  /etc/init.d/apache2 reload

You should be able to access https://myserver/ now. Now move the proxypass configuration from sites-available/default to sites-available/default-ssl.

My impressions of the Emergent Design Workshop

October 29th, 2009

OK, so I did participate to the Emergent Design Workshop by Francesco Cirillo. This is the second time I attend a workshop with Francesco. The other one was about coaching and agile process management. This one is about the technicalities of making the Agile thing work for real in the code. It’s never easy to work with Francesco; if you do attend this workshop, be prepared to challenge everything you know.

In my particular case, I knew I didn’t know object-oriented design well. OK, I did read about the design patterns, and I did read some of Robert Martin’s writings, but never got really into this stuff. Yet somehow, I thought I could get away with not knowing this stuff deeply. This workshop changed this; now I realize more fully the amount of stuff I didn’t know, and why it’s very important for me to learn this.

And,… even more importantly… I learned to see why a certain kind of semi-procedural code disguised as object-oriented is not satisfactory; and it’s not fun. I gained a new set of eyes and a higher level of criticism for code. What I learned resonates with what I wrote earlier about “code that speaks.” It turns out I was on the right track there; the goal is to have code like Lego bricks; objects that you can combine together to obtain the desired results. Code that can withstand changes in specification, without becoming more complex. Above all, the thing I’m grateful to Francesco for is, to get back to the fun of working with software like objects.