Best Practices setting up software projects - Part 1: The build tool

Within the last years developing, prototyping and designing software projects I collected a set of best practices for setting up and developing software projects. I wrote down a more general overview in my article about about defining key factors for successful software development.

The following (opinionated) principles are not limited to new projects. They are valid for yet created and legacy projects as well. Following them will make the maintenance of your projects and onboarding of new developers much easier and though, cheaper and faster.

I made my experiences mainly in the Java world, creating libraries, command line applications and web applications with Java or Groovy, using the Spring framework very often. So especially part 3 and 4 will be tied to the Java ecosystem. The examples may not, but the general principles will be valid for other ecosystems as well.

  • Part 1: The build and dependency management tool
  • Part 2: The version control system
  • Part 3: Context Definition and Dependency Injection
  • Part 4: Modularisation
  • Part 5: Documentation and the README
  • Part 6: Executable documentation and deployment prototype (aka "The Dockerfile", or similar)

Part 1: The build and dependency management tool

Dependency managenent and build configuration

Building your project and dependency management are two different tasks. You might want to use (or be using already) different tools. This is quite common (but not necessary) in front end projects, where npm and bower (npm for development dependencies, bower for front end, runtime dependencies) is used as dependency management and webpack, grunt or gulp as build tool.

In the back end world both tasks are commonly solved by the same tool.

Use a build tool

First of all: If your software needs to be prepared or build, yes, please use a build and dependency management tool (or more of them, if needed). Don't rely only on your IDE configuration[1]. There are several reasons for that.

  • It will save you time[2]. The build tool offers solutions to most issues you will have to address manually otherwise.
  • Independent instance of truth: The build of your software is ok, if it succeeds with your build tool, not if "it works on my machine".
  • Automation and Continous Integration: Build tools are, in contrast to IDEs, runnable by automated tasks on server systems. This enables Continuous Integration and Continuous Delivery.

Choosing a build tool

So which build tool you should choose? While it fits your needs, it does not matter. I encourage you to try out several ones, not only experimenting, but with real projects, too. With this experience you will be able to decide in the future which build tool suites best for a new project.

Besides that, consider these points choosing a build tool:

  • Does it fit in your technology stack? This means, for example, that you should use something able to communicate with Maven repositories if you are shipping your software in Maven repositories, and that it should be able to fetch your dependencies from where they are.
  • It is easier, especially if you are trying a new tool, if you know the technology or program language the tool is using. This will help you interpreting error messages, debugging and writing build scripts.

If you want to dive in deeper in build tools and their characteristics, read Neal Ford's article about hating or leaving Maven and the difference between composable and contextual tools.

The aim of this article is not a detailed overview over existing build tools (there are other articles specialized in this topic), it should encourage you to use a build tool as primary build authority and give you some hints how to choose one. Nevertheless I will share a few experiences I made with build tools.

Running your project

I met a lot of projects without any hint on how to start the software. Every developer had the run configuration saved in his IDE. This works... at least on "my machine" (from the developer's point of view).

If technically possible, provide a run task in your build tool configuration for development purposes. This is quite common[3] for Maven (e.g. mvn tomcat:run for web apps, Gradle (e.g. ./gradlew bootRun for Spring Boot applications, or npm build && npm serve, to assemble and serve front end applications).
Providing this, you will have the most important part of "The run script"[4].

My personal experience with concrete build tools

I am mainly developing in the Java world in Java and Groovy, using Maven and Gradle as build tools.

I personally used Maven a lot and very, very extensively and at some point I started to agree Kent R. Spillner in "Maven is a horrible implementation of bad ideas". But that's not completely true. I agree, that Maven is far away from being the best build tool in the world. It is horribly unflexible, creating custom tasks is only possible with workarounds, writing own plugins is unnecessarily complicated and your pom.xml files will get terribly long.

But the dependency management just works, it is fast, and it does it's job: compiling, testing, building and even running or deploying the software. There Maven's features end. If you want Maven to do something else, it often gets very complicated up to impossible, because you are bound tightly to Maven's defined project life cycle.

Gradle seemed to be the solution: flexible, enables you to create custom tasks and a relationship between those and using the well known dependency infrastructure of Maven. After using Gradle for a few years now, in some cases I am considering switching back to Maven again. I love the flexibility of Gradle. But it is slow (although Gradle tells us the opposite, but that's not the experience I made), and I found me fighting with configuration problems Maven solves out of the box.

Maven does a lot of things which need to be configured manually in Gradle. On the other hand: If you need some custom tasks not fitting in Maven's tight build life cycle of Java projects, prefer Gradle.

Lately I tried Kobalt, but had to discard it as I worked on a project depending on libraries provided in an old maven1 repository, which is not supported by Kobalt, and because of lacking IDE support[5]. I will try again in my next project, hoping that the IDE support will improve.

Using the Maven dependency ecosystem, there are still SBT from the Scala world, Ivy, Leiningen (which mainly targets Clojure projects) and Buildr to try.

I did not try Rake for Java projects as Spillner suggested in 2009, Thoughtwork's Technology Radar recommended in 2012 and Martin Flower wrote about, but it does not seem to be very common in the Java world, though.

For front end projects I used npm, Bower, Grunt and Gulp, but made positive experiences with npm as a single dependency management and "build coordination" tool, without any others, as Keith Circle recommended in 2014 already. I call it "Build coordination tool", because npm actually does not build my project, but manages and runs the scripts that will do so. Typically this includes minifying und bundling sources. As build tool (or "module bundler") I recommend webpack[6].

I recommend Addy Osmani's npm scripts boilerplate gist to pick the npm scripts you need for your project.

To bring the backend and front end worlds together, there is maven-frontend-plugin for Maven, and gradle-node-plugin for Gradle, which just execute your npm/Grunt/Gulp tasks.

  1. There are exceptions for some technologies, like the Microsoft .NET world, where the build configuration is bundled in the IDE project configuration. ↩︎

  2. Having fought hours or days with Maven and Gradle to achieve a clean build configuration, this may sound dodgy. But I am convinced, I would have spent much more time (maybe on other problems) without build tool. ↩︎

  3. I wish it was even more common. ↩︎

  4. There are two excellent articles about this concept from Thoughtworker Pete Hodgson in two parts, called "The praise of the ./go script, part I" and part II. I prefer to call it "run" to avoid confusion with the programming language or the Continuous Delivery Server. ↩︎

  5. The support of Kobalt in my favourite IDE (IntelliJ) is bad: if your Kobalt build script is not completely correct (including compile and runtime erros) -- which happens often when starting to learn a new tool -- the project tree disappears. ↩︎

  6. There is an excellent webpack introduction on Front End Center by Glenn Maddern. ↩︎