Java anti-pattern: splitting the project into multiple projects
Don’t split your project into multiple separate projects unless absolutely necessary. By this I mean, don’t have a separate “model” project that produces a project-model.jar, an “infrastructure” project that contains data-access objects, a “web application” project that contains the user interface of the application, and so on. This breaking down the project in subprojects
- increases complexity a lot,
- makes integration longer and more difficult,
- is an obstacle to the evolution of the design,
- and results in a lot of work being spent on the build system.
With multiple projects it happens that you do work in one project, and the other projects don’t “see” your latest modifications. Perhaps your application works properly when tested within Eclipse, but fails to compile in the command line, because some of the jars are not up-to-date. This sort of things drives people to introduce things like Maven or Ivy in the project, and that brings a lot of additional complexity. Maven and Ivy are very complex tools. They tend to look for jars all over the world at every build, making integration slow, difficult to understand and unreliable. I’ve seen project that require from one to two hours just for integrating your changes into the repo, and that assuming that you’re skilled in working with the build process. All of this has nothing to do with producing value for the customer.
It’s just too easy to spend lots of time building complex infrastructure. Don’t do that: spend time thinking how to simplify things instead. Think about how to produce value for the customer. Think about how to obtain the desired result with less complexity.
Within the multiple-projects antipattern, there’s a second, minor antipattern, that is the trunk, tags and branches folders everywhere antipattern. By this I mean that each and every one of your sub-projects has its own triad of trunk, tags and branches directories. Even assuming that we’re going to need these three directories, they should appear only once, at the very top of the tree of subprojects. It you do it like this, the triad will not get in the way: I can checkout the trunk once, and I will have a single, complete and consistent tree to work with. If every project has its own separate triad, there’s no easy way to manipulate the whole tree of projects without including tags and branches. No easy checkout, no easy update, etc.
Now I hear you ask: “But what if I cannot keep all of my code in a single project? What if I have to produce two versions of my application? What if I have to produce more than one application sharing most of the codebase?”
Even so, dear reader, my advice does not change. I would generate multiple apps from the same single project. I would write a separate ant task to build each version of the app; my ant tasks would pick and choose the pieces that are needed for a particular app. Very simple. I know that this goes against the pattern “one artifact per project” pattern. So be it; I don’t think it’s a particularly valuable pattern, expecially since it leads the way to the introduction of complex build systems like Maven.
“But you’re going to end up with a messy, complicated, unreadable ant buildfile!” you say. I don’t think so, dear reader. There is a danger of making a mess of the ant buildfiles, but then again this is the usual danger of making a mess of your software. The same care and skill that is required to write simple, high quality source code must be applied to the buildfiles. In my experience, with some care, I can keep my ant buildfiles pretty lean.
“But I have a legacy system, full of separate projects!” In that and other cases, when it’s impossible to keep all the code in a single project, you can get by with a relatively simple buildfile. All you need to do in the buildfile is to explicitly invoke the build on all the projects it depends on. This will increase the complexity of your buildfiles, but not dramatically so. It’s made easier if you can rely on all of the subprojects being at predictable positions within your working directory. Otherwise, you can cope by defining environment variables or per-developer configuration files that help ant find the projects it depends on.
“But my situation is different! I need Maven!” Dear reader, I won’t tell you what you need in your situation. You are the only person in a position to know that. This advice worked for me. My bet is that in almost every situation Maven or Ivy could be profitably replaced by a simpler scheme, if you really care about keeping complexity down, and are willing to spend time and thought for that.
November 1st, 2008 at 18:53
Well, this time I couldn’t disagree more :-)
While point 4) is partially true (you have to work a bit longer on the build system) in the latest year I’ve been splitting projects for many reasons, and pros are far more than the cons. And I don’t have to use neither Maven or Ivy :-))
November 1st, 2008 at 22:42
Fabrizio: why don’t you share a few tips on how to do it right? :-)
November 3rd, 2008 at 11:05
We are now using separate projects as we inherited badly coupled code (e.g. why would you need a dependency on servlet.jar to test a class that only parses strings?) that is very difficult to decouple, no matter how much Feathers, so burned once shy twice.
No Maven, no cry… NetBeans transparently handles the increased complexity in build files, as all you have to do is to declare that your webapp depends on the model jar and so on and so forth. That also pushes us to consider design more properly, much more than having to rely on the way we deal with packages.
November 4th, 2008 at 00:14
Amen that. Sorry Fabrizio, to me this is definitely an antipattern – meaning that it seems to do more wrong than good on a global scale (your mileage may differ).
The deepest problem with the whole concept is that it usually degenerates into one “star” team writing the “infrastructure”, or “framework”, or some similar kind of bullshit, for the other less-skilled teams to consume. I work on such a team right now, and I still think that the cheaper way to improve productivity in most companies would be to take all the infrastructure guys out in the open and shoot them.
It takes a nearly non-human amount of humility to do a framework right. Most framework teams seem to just go out on a tangent and produce the kind of dog food that they wouldn’t eat themselves. I’m comfortable with saying that in my 10-years-plus experience, exactly 100% of the company frameworks I’ve witnessed were utter shit. (Yes, again, I was guilty of that kind of crime myself).
November 4th, 2008 at 12:49
how to manage multiple related projects ?
buildall.sh :
cd project1
ant
cp project1/dist/project1.jar project2/lib
cd project 2
ant
cp project2/dist/project2.jar project3/lib
….
very simple, very clear.
“But i cannot run sh in my windows pc!”
First solution: no windows at all
Second solution: cygwin
November 6th, 2008 at 09:25
I am a strong believer in divide-and-conquer. Complexity can be managed easily by dividing the project into smaller modules.
Let me take your points one by one:
1. increases complexity a lot,
Never. Project complexity is never increased by dividing source code into different modules. Each module is independently testable, and looking at any module you can immediately recognize clarity and purpose of it. Of course, if you say there are more integration points, yes, there are. But that does not increase complexity.
2. makes integration longer and more difficult,
Same as first point. Integration is not more difficult. It is easier infact. Just use a already well-tested library without working about the details of that module.
3. is an obstacle to the evolution of the design,
How can it be? When a project is modularized, only then design evolves. Monolithic folder structure creates fear in the minds of the programmers to approach the code itself due to perceived complexity.
4. and results in a lot of work being spent on the build system.
Before initiating a project/module, I spend considerable time configuring Maven (but in later versions of Maven this has reduced considerably due to archetype plugin). This I consider a one-time investment. When this is done correctly, all my team mates are able to consistently build the modules effectively.
This sort of things drives people to introduce things like Maven or Ivy in the project, and that brings a lot of additional complexity.
I use Maven. I have been using it since its inception. It does not increase complexity, but reduces it to a great extent. Using standard conventions I am not writing redundant Ant build scripts these days.
With multiple projects it happens that you do work in one project, and the other projects don’t “see” your latest modifications.
This is never the case if we are managing projects using Maven. We define a parent project with all the sub-modules inside it. Parent changes are always available to the developers.
November 6th, 2008 at 09:36
Hi Subash,
Thanks for sharing your experience. I’m not against modularization! I’m just saying that splitting your modules in separate Java projects does increase complexity.
Modularization is good, but once you split your code across multiple “projects” it becomes more difficult to do large refactorings. You have an incentive to keep your refactoring contained within a project, and lose the gains you might get from refactoring across them.
So I will keep my code in a single project until the time comes when it’s clear I will benefit from a split. It’s a tradeoff between simplicity and flexibility. More flexibility comes at a cost!
The thing about ant scripts, I agree that you risk having a lot of redundancy and cruft in them; yet there’s nothing to stop you from refactoring them and keeping them clean. Build scripts are code, just like your java code, and must be kept clean by refactoring too.
Perhaps when I’ll take the time to learn Maven well I will change my mind. What I’ve seen so far about Maven I don’t like. I’m not crazy about Ant either, but it suits much better my taste and opinions.
November 6th, 2008 at 18:14
I realize that saying “I don’t like it” is not saying much. So let me explain better. Maven lets you cope with projects with complex dependencies, at the cost of introducing a complex tool. I’d rather work at reducing complexity and dependencies. I think Maven is the wrong answer to a real problem. The good answer is “simplify!”
September 8th, 2014 at 14:28
The benefits of splitting any reasonably complex project into multiple sub-projects *far* outweight the cons, in my experience. Especially when you’re working with IntelliJ which lets you refctor and work simultaneously across all modules by respecting the directed acyclic graph of dependencies defined by your Maven POMs. Splitting projects down enforces separation of concerns by design, assisting unit testing, code reuse and finally… the most important point of all. We’ve all worked with that one developer who doesn’t understand MVC and wantonly pollutes even your lowest-level Model classes with high level application behaviour. Well, with a separate model project they can’t do that any more.
September 9th, 2014 at 13:16
Thanks for your comment. As you can see, the original post is from 2008, and it reflects my thoughts at the moment.
I am not completely against breaking down projects in modules, but I still think that it’s a dangerous move; it’s expensive to revert! and if you later discover that the current subdivision in modules does no longer serve your purposes, then you have strong resistance against refactoring the project towards a better structure. So I advise to break the project in modules only where there’s a demonstrated *need* to do so.
I agree about the argument about the collegue who breaks the design, but only in the sense that the module structure helps collegues understand the design. Other than that, I think that if a collegue unwittingly breaks my design, it is my fault! I should explain my design better. Design is an act of communication, and the effectiveness of communication is the responsibility of the speaker. When collegues break the design, perhaps it is a sign that the design is not good enough; it does not serve our purposes well enough.
January 26th, 2018 at 14:42
I also think that dividing too much could be a dangerous move, especially for simple projects. I see it as over engineering. It’s like adding inheritance right from the get go, just by assuming that the system will eventually grow. That’s an error in my humble opinion. Design must grow as the requirement expands. That’s the same when dividing a project into several Maven projects. There are tons of pros of applying the Divide and Conquer approach, but do you really need it? I think what matteo is trying to say is that in most cases the YAGNI principle will apply (you ain’t gonna need it).
I can see a real benefit in splitting a project into several modules when the ecosystem is getting large and that you can clearly see the system in several logical parts. As for creating a module for model, another one for peristence, another one for controllers, another one for views, etc… I think that doesn’t make sense and makes it difficult to maintain down the road.
Just my two cents.