Extreme Enthusiasm

True. So *assuming* frameworks are necessary takes away a valuable option.

— Matteo Vaccari (@xpmatteo) September 24, 2017

Context is king

Engineering is about making tradeoffs. It is about delivering useful solutions at an acceptable cost. As Carlo Pescio likes to say, a construction engineer knows the properties of materials; she knows when to use steel, concrete, glass or bricks. Every material has particular properties that make it more or less appropriate for achieving a desired effect.

In the world of software, application frameworks are a broad category of “construction materials” for application developers. What are, generally speaking, their properties? In which contexts are they useful or harmful? What kind of results are made easier, or more difficult, by using frameworks?

These are all relevant questions, IF you have a choice of using frameworks or not. All too often, developers assume that frameworks are necessary; the idea of not using one is not taken seriously. Indeed, not using frameworks can be scary. If I don’t use a framework, will my design be maintainable? Will my performance be good enough?

Like everything, it takes practice. I argue that it’s possible to have a very maintainable design and a rock solid performance without frameworks. When you learn to do it, you discover that it is not that difficult, and it does not requires you to write mountains of boilerplate code.

Once you know how to do stuff without frameworks, you can take an informed decision whether to use one or not. If you don’t know how, you don’t have a choice.

Frameworks are good for learning

Frameworks are a condensation of proven techniques and design solutions. You can learn a lot by learning frameworks. I, for one, learned a lot from Ruby on Rails. I think we should keep learning frameworks, just for the purpose of learning. When I was developing my first Mac application, back in 1990, I was struggling, as at the time I didn’t know any other Mac programmer. There was no Internet! Apple back then had an FTP server, that was maintained by an Apple employee in his spare time. So at some point I got hold of the MacApp framework; it was one of the first application frameworks to come out. It helped me in many ways; unfortunately, it increased the startup time of my app from 4 to 30 seconds. I was *so* pissed off by this added delay, but at the time, I thought I was better off keeping the framework. I remember I was constantly thinking “the authors of MacApp are very experienced Mac programmers, and they cristallized their knowledge in this framework. If only they had written a *book* instead! Then I could take the bits I need of their knowledge, and leave the ones I don’t need.”

Now I understand that writing an effective book is hard, expecially if your primary skill is programming and not book writing. I still think, though, that frameworks are good for learning techniques and design solutions.

Viewed in this lights, frameworks can be useful training wheels. For me, the goal is to be able to get rid of them and be able to do as good and even better. Once you know how to do that, then using frameworks can be a conscious choice.

Most frameworks are DB-centric

You can generally set up a barebones project much faster with a framework. It’s hard to beat Ruby on Rails on this; with `rails new` you create a working web app, and with generators you can quickly set up a CRUD REST system. Same goes with Spring Boot REST controllers.

Too bad that CRUD does not cut it. Organizing your app around your DB tables is not the way to design an app for a complex domain.

Rails and all the frameworks inspired by it, including Spring Boot, Symfony, Play, etc are all *DB-centric*. They assume that you will design your system primarily around an entity-relationship diagram. This leads to designing for data first, which leads to doing procedural programming. Let me stress this: DB-centric design leads to procedural programming, which is much different from object-oriented programming. Yes, there are “classes” and “objects” in Spring programs; however, “entities” in a Spring program are usually data structures without behaviour, while most behaviour is in service objects that have behaviour but no data. Which is exactly the opposite of object-oriented programming.

Like everything else, procedural programming has a place. It’s generally speaking easier to understand a procedural program than an object-oriented program. Even a bad procedural program is generally easier to understand and fix than a bad object-oriented program. However, procedural programming does not lead to a clean model of the domain, in the DDD sense.

Automating a complex domain in procedural code leads to complex procedural logic, with nested conditionals, and more often than not, with persistence code mixed in with business logic.

True object-oriented code, however, is good for automating complex domains, if you take care to keep persistence well separated from business logic. Moreover, true OO, free from persistence concerns, is good for TDD.

Talk is cheap

Talk is cheap. Show me the code.
Linus Torvalds

In fact, replacing what frameworks do for you may require less than 1000 lines of code. The things that are required of you are:

• knowing the fundamental technologies of the web: how HTTP works
• knowing some fundamental design patterns: the share-nothing architecture and MVC.
• knowing the fundamentals of databases: SQL and database design
• TDD

The way I do it, is by developing the application from scratch with TDD, focusing on delivering early an end-to-end scenario that is relevant to the customer. Perhaps a heavily simplified scenario, but still something that makes sense for the customer.

I cannot show you the code that went in projects done for paying customers. However, I have some code developed for trainings that I’m happy to share.

• My presentation “TDD da un capo all’altro” (TDD from end to end) contains, I think, enough snippets of code for a reader to reconstruct the app, showing how to TDD an app from absolute scratch.
• My training workshop Simple Design in Action contains successive stages of TDD-ing a web application, showing how to use embedded Jetty without Spring boot and persistence without ORM
• This revised version of the Video World project is the result of me and a few collegues removing several layers of frameworks from old ThoughtWorks training material. It shows how to do server-side rendering, with layouts.
• My version of the TodoMVC sample app shows a way to do frontend work without frameworks. I also have a video and a blog post about it.

The main thing is: these are not meant to be production code. They are versions, sometimes simplified, of stuff I did in production code. Even if I could share production code from real projects, it wouldn’t be wise to copy it in your own projects. Good software design is in how the code evolves; you cannot really see it in a snapshot of the code tree. What I suggest you to do is to try to TDD toy projects until you can do it without too much effort. You can steal some ideas from my training projects, but when you will be faced with real problems, you will have to find your own solutions.

When to use third-party code

Am I advocating that we write everything from scratch then?

This is a question worth answering. There is a world of third-party systems, libraries and frameworks out there: let’s classify along two axes, the first axis being how focused to a single task it is, versus how general-purpose.

The second axis is how easy or hard it is to reproduce the service it does to you; how easily you can do without it.

In the upper-left quadrant we have things that are focused on doing a single thing, but that are easy to reproduce in your own code. A good example would be the infamous left-pad function.

In the upper-right quadrant we have things that are focused, but difficult to reproduce. For instance, crypto libraries like openssl take a huge amount of skill and knowledge to write. Web servers such as Jetty also go up there. Yes, you can write a simple HTTP server in 10 lines of Java, but it’s not likely to be robust enough for production.

In the lower-right quadrant we have stuff that’s general purpose, yet difficult to reproduce in-house. Operating systems, programming language interpreter and compilers, DBMS. All system software goes there.

Now, what would go in the lower-left quadrant, where we find general-purpose things that are more or less easily reproduced in-house? If you have the skill, you can reproduce most services that application frameworks do for you without a lot of effort. Mind you, I’m not saying that I can easily rewrite Spring! I’m saying something very different: that I can reproduce with ease what Spring does for me.

• Routing http requests to the appropriate controllers: it can be done in-house, as simple as chain of IFs, or as sophisticated as a ~100 line router.
• Injecting dependencies in objects: I can very well instantiate objects explicitly and pass their dependencies in constructors. It does not take a rocket scientist.
• Automated CRUD “repositories”: thank you, I don’t need them. A real repository in the DDD sense is something that encapsulates custom persistence logic. I’m better off implementing it with a simple adapter over JDBC. Carlo Pescio suggests to use dbutils; I generally TDD my own.

Etc. etc. In this sense, Spring and similar frameworks are things that you can do without, if you want to.

Application servers such as JBoss too are general-purpose, also in the sense that they provide a set of different, not-that-much-cohesive services. Most of these services can be obtained in better ways:

• HTTP is best implemented with an embedded web server, as the popularity of Dropwizard and Spring Boot shows.
• Clustering as traditionally provided by application servers is best avoided. It is built on the assumption that web apps maintain expensive session state between one HTTP request and the other. In fact, only poorly designed web apps do that; well designed web apps are stateless or nearly stateless, and if this is the case, they are best served by non-clustered, independent servers. This, in fact, is the “share nothing” architecture.
• I argued in the past that distributed session management in Tomcat is broken. The same arguments is likely valid for most Java application servers.

For these reasons, I place JBoss and other JEE application servers in the lower-left quadrant: you can do very well without them.

                    focused on a single task
                               |
                               |
                     dbutils   |   Jetty
                    left-pad   |   openssl
easy to                        |                    difficult
do without  -------------------+------------------- to do without
                               |
                      Spring   |   Linux  JVM
                      JBoss    |   PostgreSQL
                               |
                               |
                        general purpose

Now what should be our strategy? Of course, everything on the right hand side should be bought and not done in-house. Unless the value of your system depends, let’s say, on advances in cryptography or DB technology that are not generally available, doing system software or crypto in-house is madness.

Stuff in the upper-left quadrant is a candidate for writing in-house. You can save some time by using dbutils over TDD-ing your own JDBC adapter. However, every dependency on an external library is a liability, as the left-pad incident shows.

Stuff in the lower-left quadrant is probably best avoided, if you have the skills. They have the highest impact on your design and reduce significantly your control over the fate of your system.

Other objections

You make the customer pay for reinventing the wheel

In the end, what really matters is to delivery quality software in reasonable time. I found that by avoiding stuff in the left half of the easy-focused diagram, teams are able to do so. If that involves some rewriting of common logic, so be it. It turns out that universal wheels such as Spring do not make me go faster.

By your reasoning, we shouldn’t use compilers or standard libraries

See the section above

You’ll write your own quirky framework and it will suck

No. I will not write my own framework. I write just the code that I need to solve this particular customer’s problem in this particular moment. I will spend zero time writing code with the expectation that it will be reused elsewhere, not even in other projects that I will do for this customer, not code that I expect to reuse in this same project next month.

I take YAGNI very seriously. I TDD the simplest code that solves the problem I have today. It’s counter-intuitive, but this makes me go faster. Every time I wrote code for tomorrow, I found I wasted my time.

This does not mean that I write the first thing that works and then move on. I tend to refactor quite a bit before I decide I’m finished with a bit of code. My code is generally very simple and clean as a result of quite a bit of effort. A side effect of this is that my code tends to be flexible and easy to extend.

I repeat: the simplest thing is often the result of sustained research and effort. It’s not the first thing and certainly not the easiest thing.

New developers will have a hard time understanding the codebase

That depends on how good my code is, isnt’t it? If you write simple code (see above answer), it shouldn’t be too hard to understand. On the other hand, even apps written with frameworks are not always that easy to understand.

Much wiser and skilled developers wrote the frameworks that you’re despising

Their circumstances may be different. Their programming habits and processes are probably quite a bit different from mine. I really don’t care how experienced, respected or influential framework authors may be; all I care about is the results I get. I evaluate the quality of a solution with respect to how well it helps me reach my goals.

Conclusions?

Engineering is about evaluating options. It seems to me that developers discard the “no-frameworks” option without giving it a lot of thought. In my experience, without frameworks, I was able to help team deliver high performance, high quality code in a short time.

Your circumstances may be different. Don’t trust me; think for yourself. Think critically about technologies and design solutions; and if you care about your craft, try to de-framework an app and see what happens!

This entry was posted on 24 September 2017 at 17:30 and is filed under Agile. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.