Applying the Divide and Conquer Pattern in Software Engineering

Rome, 50 B.C.

Julius Caesar published one of the most famous military essays in history, the "De Bello Gallico", in eight books. This is the story of how Rome managed, under Caesar's command, to conquer Gaul in an eight-year military campaign, mainly based on one strategy as simple as it is effective: "Divide et Impera”, literally translated to divide and conquer.

{{divider}}

Zurich ...present day

Eleonora Borghi, senior fullstack engineer on the FinTech team, talks about how the divide and conquer pattern can improve the efficiency and productivity of teams and developers when dealing with complex projects.

Strategy is a key factor for winning military battles, but it’s also crucial when seeking success in a fast-paced and competitive business environment, such as software engineering. Good strategies, and the correct implementation of them, determine the success or failure of projects.

Divide and conquer, in computer science, is an algorithm design pattern that can be summarized as splitting a complex task into smaller sub-tasks and addressing those one by one.

The goal of the pattern is to minimize both the time and complexity of non-trivial tasks. The divide and conquer pattern can be successfully applied when writing incredibly fast algorithms, QuickSort blinking sarcastically at InsertionSort here, and can be a guiding strategy that engineers, the team, and entire project can benefit from.

Any experienced software developer is aware that software projects don’t scale linearly. Allocating N developers to the project won’t automatically shorten development time by factor N, and the global complexity of the project won’t decrease simply by providing more engineering power.

Hence the question we want to answer is:  

How can we minimize time and effort to accomplish big projects?

I’ll share three main lessons learned while working on a project that the FinTech team completed last year: the migration of the entire payment flow to a new stack. The project spanned several months, involved several developers, and tens of thousands of lines of code.

The three lessons learned focus on how to apply the divide and conquer pattern recursively to:

• The project
• The team
• The tasks

Payment project introduction

The project can be summarized as the migration of the payment page from the legacy PHP stack to a VueJs application. The scope of the project involved not only building a new user interface but also rebuilding all of the REST APIs endpoints the new page would consume to complete a payment on the GetYourGuide website. On top of that, the APIs would need to be universal to also serve native apps.

The acceptance criteria for the finished project was, equal or improved functionality and user experience to the legacy stack, and the new flow should allow customers to pay seamlessly using their preferred payment method.

Learning 1: Divide and conquer for the project

Dividing the entire project scope into smaller incremental milestones provided a less risky estimation of time to completion. The team focusing on smaller milestones of the project isn’t distracted by features out of scope, and milestones constitute a subset of the acceptance criteria of the entire project. They’re not independent, since each milestone completion depends on the previous milestone status, and the work on different milestones can’t always be parallelized.

Given the overall complexity of the payment migration project, we started to implement a divide and conquer pattern by dividing the project into four milestones that would incrementally accept traffic matching specific criteria. We defined a milestone as a subgroup of features that would enable a specific subset of payment flows.

An example is the implementation of payments using a credit card. As soon as the milestone was achieved and tested, it could be shipped to customers that previously chose that specific payment method, while the team moved on to the next milestone.

Thanks to GetYourGuide’s experimentation platform, rolling out milestones was easy and efficient, as we could control the percentage of traffic sent to the new flows.

Learning 2: Divide and conquer for the team

Choosing the size of the team working on the project wisely is crucial when seeking fast-paced progress and a high-quality solution. Having too many engineers working on the same milestone makes communication difficult and fragmented, while not having enough eyes on it can negatively impact the quality of the final product.

When working on the payment migration, the average number of engineers allocated to it was three. The main reason for this number comes from an analysis of how many tasks inside the milestone could be parallelized. The projects spanned three main layers:

• The presentation layer, a VueJs application customers interact with
• A RESTful API, responsible for aggregating backend microservices into a single entry point for the consumer
• The core payment business logic that would integrate with a third party payment provider

Given the breadth of the technologies involved, from frontend to backend, including TypeScript, VueJs, Sass, Java and PHP, having T-shaped engineers in the team was a critical asset. We focused on small iterations with many cycles of manual integration tests in our testing environment, frequent pair programming sessions, and cross code reviews in the form of pull requests to increase velocity without compromising on quality.

Learning 3: Divide and conquer the tasks

Code reviews are industry standard, and meant to increase code quality and robustness. Code reviews done within the same project also allow for a better understanding of the global progress of the project among the engineers working on different tasks in parallel.

When trying to maximize the velocity and quality of code reviews (i.e. the number of spotted bugs or potential improvements), the number of lines of code submitted for review will play a central role. Too much code to review not only requires non-linear time and effort, but can easily distract the reviewer and cause them to miss bugs.

During the implementation of the payment migration, we would often parallelize tasks. For example, once the team agreed on the specification of the API, frontend work could easily be done in parallel with the implementation of the endpoints in the backend.

In order to avoid too much time for pull request reviews, we tried to split the single tasks into multiple branches that could be reviewed and merged individually. These branches would generally follow this pattern:

1. Foundation
2. Core implementation
3. Eventual refactoring or improvements

For example, in the implementation of any new API endpoint, a code-splitting pattern would become:

1. OpenApi specifications of the endpoint
2. Implementation and unit tests
3. Refactoring

The above approach allowed us to be fast and efficient during code reviews, and increase the number of interactions between engineers collaborating on the project.

Conclusion

Divide and conquer is a key pattern in software engineering. It helps individual developers become more efficient and productive, and when applied to bigger projects, it enables teams to bond and unleash their true potential, thanks to closer and more frequent collaboration. Seamless and close collaboration is the key for delivering successful projects, keeping high velocity while not compromising on quality.

Less than three months after creating the payment migration project proposal, our monitoring system registered the first credit card payment in production on the new flow.

The entire project has been an outstanding example of team effort that allowed me to grow tremendously as an engineer and a team member, and I’m looking forward to the new challenges ahead.

If you’re interested in joining our Engineering team, check out our open roles.

‍