Scaling down continuous deployment

Before I started working on Mockito 1.10 release I set a goal for myself. I decided that the next release, and every subsequent release will be completely automatic and will happen on ‘git push’ without even any ‘release’ button pressing. Getting the release out of the door, without investing into automation, would be tons easier. However, I din’t want to take this shortcut _again_. I wanted to consume the lessons learned from the past: shameful 2-year release cadence, totally _not_ reflecting my attitude and experience in continuous delivery and software automation. I wanted the 1.10 version to start the new era in Mockito ecosystem. Now I need to scale this down a bit :)

Driven by the ‘completely automated release’ use case I’ve set up Travis CI, Bintray, GitHub and the master tool: Gradle to let the magic happen. At the time, I was already thinking that releasing on every push might be too eager. However, I couldn’t come up with convincing use cases and it was getting hard to decide on the ‘right’ release trigger. So I’ve applied my favorite approach: get stuff DONE and then watch REAL use cases and feedback showing up.

We learned pretty quickly that certain category of project events don’t make sense as release triggers. Consider all kinds of non-code changes that occur surprisingly often: updates to README.md, adding TODOs, minor changes to the automation machinery. We worked around this problem by including a “[ci skip]” incantation in commit messages. Travis CI does not trigger the build for changes annotated with “[ci skip]”. This approach helps a bit but removes the validation of the changes completely because the build does not run at all. Not to mention that it's easy to forget to include the magic string in the commit message which would lead to an awkward release. So, I’ve improved the automation by crafting a Gradle task that compares binaries between the current build and the previous release. The release is skipped if binaries are the same (the comparison method is simplistic).

The second category of events that I started observing is pretty interesting. Consider internal code refactorings, changes that do not affect any public API nor the software behavior. Those changes still update the code, so the byte code is different and the jars are different. Should such changes trigger a new publication of a library? What would be the content of the release notes?

I’m undecided on this matter yet but I’ll make the call soon. Interesting use cases are already showing up. For one of the recent PRs I’d love to get some refactorings done first, before the eager contributor picks up the main implementation. It feels that in this case, it’s more effective if I prepare a landing for a new feature, instead of continuing the pull request code-review cycle. This will push our current continuous delivery model to address the internal refactorings and find the right place for them in the process.