The Release Subway

Stress around software releases creates risk of human error and increases the risk of attrition due to burnout. So how do you reduce release stress? The key is to think of your process more like a release subway than a release train. What’s the difference?

The Release Train is a common analogy to explain and visualize release processes. Digging into that analogy, many teams operate a Release Train that’s more like buying a scheduled ticket on a particular train. If you miss that train, there’s at least a lot of inconvenience and at worst a lot of expense.

Having lived in New York City for 7 years, I always liked to think of the Release Train as the Release Subway where trains depart each station at frequent, regular(ish) intervals. If you come running down the stairs to the platform as the train’s leaving the station, it’s not that big of a deal because another train will come along shortly.

The Release Subway reduces the stress and pain around missing a release

The goal of the subway system is to run trains at frequent, regular, predictable intervals. If the system does this effectively, the impact of missing an individual train is minimal. You might be late to a meeting, but you don’t have to get a hotel to wait for the train the next day.

If we think about our release process as a subway system, it should be optimized to frequently release at regular, predictable intervals.

What Delays the Release Subway?

I love to take analogies way too deep so let’s dive into what causes delays in a subway system:

• Sick Passengers: From my experience living in NYC, I most often saw trains delayed by a sick passenger on the train. The train has to stop at the next station, everyone has to get off the train, the medics have to treat the person, the train needs to be cleaned, then it can resume.
• Mechanical Failure: Trains are mechanical things and they break down.
• Staffing Shortages: Subway systems need people to run them and become unreliable without good people

To correlate those ideas to actual releases:

• Sick Passengers: A feature branch that was merged before being fully tested and has a bug that blocks the release
• Mechanical Failure: Failures in the tooling and architecture that runs the release process
• Staffing Shortages: What happens if you don’t address release stress and burn out your team

For this post, we’ll stay focused on the Sick Passenger Problem, but you can see how Mechanical Failure and Staffing Shortages are related challenges.

The Integration Branch Dilemma

Let’s jump out of the subway analogy for a minute to discuss branching, everyone’s favorite subject right? It’s quite common in the Salesforce world for teams to utilize integration branches as a place to merge branches once development is complete so they can be deployed to and tested in a real environment.

Consider this basic scenario:

• During a 2 week sprint, 10 different feature branches are merged into the integration branch for testing
• During regression testing as part of the release cycle, 2 features are found to have blocking bugs

What do you do? None of the options are good and all are sure to increase release stress

1. Delay or cancel the release
2. Figure out how to unmerge 2 of 10 features from the integration branch
3. Have your developers and testers work overtime to rush a fix

Protecting from the Sick Passenger problem

Imagine if every subway stop could have a fully automated health check before each passenger steps onto the train. Sure, there are things your checks might miss here and there or things a check could never know such as a passenger who gets motion sick while on the train. But, there are lots of things you know how to check and could check with automation to drastically reduce the chance of a sick passenger on the train.

Think of your Continuous Integration and QA process as these health checks. If the bulk of your testing happens after you’ve merged a feature, you’ve potentially just put a sick passenger onto the train and increased the chances of encountering the Integration Branch Dilemma. The key is to move this fully automated health check to run pre-merge on every Pull Request targeting the main branch.

What we’re talking about here is the Shift Left concept that’s quite common in software engineering in general.

Conclusion

Reducing the risk surface area of releases by introducing robust checking of features before they are merged reduces the stress around releases and the risks that stress introduces to quality and staff attrition. Shifting left allows each feature to be evaluated not for being dev complete but for being release ready.

Doing this takes investment in automation to make it easy to create on-demand test environments that mimic real world customer environments.

Interested in discussing the concept of the Release Subway and how it would work for your company or organization? I love talking about these things with other Salesforce partners and customers. Grab an hour time slot at https://calendly.com/muselab