The Long and Winding Road to Short and Smooth Releases

Fábio Oliveira, Senior iOS Engineer, unveils GetYourGuide’s journey from biweekly to weekly iOS app releases, highlighting automation and teamwork wins along the way.

{{divider}}

In the dynamic realm of app development, the release process is a pivotal element that determines how swiftly and efficiently new features and fixes are delivered to users. At GetYourGuide, we embarked on an 18-month journey to transform our iOS app release process from lengthy and complex to short and streamlined, with similar changes mirrored on the Android side. 

Here's a deep dive into our transformative journey.

The Starting Point: A Biweekly Burden

Back in August 2021, our release process was very manual and involved most of the apps’ engineers (note: we don’t have a QA team). Every two weeks, Monday 2pm, a rotating release captain would manually initiate a code freeze by forking from our main branch and creating a release candidate build. This was followed by a series of manual release tests (46 in total) conducted by seven rotating engineers. 

If the green light was given, which was usually on Tuesday or Wednesday, the release captain would then shepherd the app through the submission and App Store review process. This would usually happen around Monday the next week, to avoid releasing on a Friday, which would translate to a three-week lead time, in the worst-case, from code submission to user access.

Bi-weekly releases

Release team: 8 engineers
Code freeze to release: 7 days
Development lead time: 3 to 1 weeks

For context, we used Fastlane and Ruby scripts built around it for most of the automation and Bitrise as our CI platform.

The First Leap: Automation and Efficiency

Automating release tests

Having recognized the inefficiencies, our first commitment was to automate the most expensive part of the process: the manual release tests.

These tests simulate real user scenarios, ensuring that the core flows of our applications are performing as designed from start to finish for our ready-to-ship versions. They cover features like search, navigation, payments and various methods of loading and checking the content of the bookings made.

We decided we’d start by automating a set of release tests and integrate them into our CI and release process so other teams could quickly follow and automate their own tests and get immediate value from it.

Choosing the Right Framework

After evaluating a couple testing frameworks, we settled on a combination of the XCUITest and KIF Test frameworks to power our release tests. 

XCUITest, being Apple's own UI testing framework, provided native support and seamless integration with our existing iOS development environment. That proved ideal for writing the most critical E2E tests against our production environment. While these are black box tests, we configured custom launch arguments to disable animations, skip onboarding, disable crash reporting, among other customisations.

On the other hand, KIF (Keep It Functional) allowed for more detailed functional tests, complementing the broader UI tests from XCUITest. We used these for UI tests against mocked responses, covering a wider variety of simulated scenarios.

These were great additions to the top of our test pyramid to complement our unit and snapshot tests.

Our test pyramid 

‍

CI Integration

To maximize the benefits of E2E tests, they were integrated into our Continuous Integration (CI) pipeline. This meant that with every code push, tests would run automatically, flagging any potential issues before they reached the production stage. This not only reduced the manual testing burden but also ensured a stable main branch.

To deal with any potential flakiness we implemented a retrial mechanism for the E2E tests along with a summarised test report.

Rolling it out

Once we had one suite of tests implemented, we brought this to the rest of the team in the form of a workshop and by early 2022, we had automated ~50% of our release tests. Doing this allowed us to reduce the number of engineers needed for manual release tests from seven to four. As a great byproduct of this work, we got tests we could run throughout the day to tell us in advance if we broke something crucial, keeping the feedback loop short.

Simplifying the Release Captain role

Automating steps

Simultaneously, we streamlined the release captain's duties. From an initial 15-16 step process, between scripts they needed to run, actions in AppStoreConnect they needed to do and messages in Slack they needed to send, we whittled it all down to just 5. 

We looked for ways to chain the various automations and leave the human to tend to the human part of the process. For instance, after all the testers gave their green light, the release captain could now just push a tag into the git repository which, thanks to Bitrise using tagging as a possible workflow trigger, would set off a series of automations. We used that to submit the build for review with an automatic release date and upload release notes on GitHub.

An all seeing aide

To further aid the release captain, we introduced Polly, a Slack companion. Polly's role is to notify the release captain of the success or failure of these automations, ensuring timely interventions when necessary. It has funny gifs too!

For failed release tests, we also tagged the teams owning the failed tests on Slack.

The Second Leap: Optimistic Submissions

With a more efficient process in place, we shifted the code freeze to Sundays. This meant that by Monday morning, the builds were ready for manual testing, allowing us to release within the same week. By late 2022, we had further reduced the number of engineers involved in the release process to just three, including the release captain.

Code freeze on weekends

Release team: 3 engineers
Code freeze to release: 3.5 days
Development lead time: 2.5 weeks to 3.5 days

By the start of 2023, we had automated almost all tests, leaving just four manual ones. This allowed us to reduce the release team to just the release captain, who could easily handle these tests. We also adopted an optimistic submission approach, submitting for review on Sundays when the automated tests passed and releasing on Tuesdays, leaving Monday for the release captain to do the final manual tests. This slashed our time from code freeze to release from three/four days to just around two.

Schedule with optimistic submission

Release team: 1 engineer
Code freeze to release: 2 days
Development lead time: ~2 weeks to 2 days

P.S. Since then we also removed our manual release tests 🎉

The Outcome: A Weekly Win

Emboldened by our streamlined process, we experimented with weekly releases in the first quarter of 2023. The results were overwhelmingly positive. Not only did we reduce the wait time for our code to reach users, we also fostered a faster learning and feedback loop for the product. Engineers reported they no longer had to plan around releases, making the process almost second nature.

To aid with the experiment, we changed to Calendar Versioning which allowed us to decouple the iOS releases from the Android ones and helped answer a frequently asked question: when was a certain feature released to users?

Our current release process

Release team: 1 engineer
Code freeze to release: 2 days
Development lead time: ~1 week to 2 days

The Bigger Picture: A Vision and Organic Growth

What's most noteworthy about our journey is its organic nature. There was no designated owner or official project status. It was a collective effort by engineers to optimize a process for their benefit, agreed in our guild meetings and executed in a series of focus days. And by consistently seeking the highest return on investment at each step, we transformed a once cumbersome process into a lean, reliable, and repeatable one.

In the middle of this journey, in mid-2022, our native apps guild crafted a 3-5 year vision for GetYourGuide's apps engineering. A key component of it was a streamlined release process, with an emphasis on automated, frequent releases. That helped cement the direction and focus on the last remaining steps left, which we reached shortly after.

In Conclusion

Our journey from a weighty biweekly release cycle to a nimble weekly one is a testament to the power of continuous improvement, collaboration and a clear vision. While our focus here was on the iOS side, similar strides were made on the Android front. 

Today, at GetYourGuide, we pride ourselves on a release process that's not just efficient but also seamlessly integrated into our workflow.

Where to go from here

After all this automation, the release captain job can be summarized to less than 30 minutes running manual tests and then checking daily for the stability of the new release. 

We are also finding ways to automate that last part. But that is a great topic for a new article.