We realised thathaving PRs be raised by the engineer who is actually responsible for them would be better: clearer ownership, easier feedback, and no need for a gatekeeper team. One subtle issue with Codelift was that all of its PRs came from a bot user: this created a social expectation upon the owners of the Codelift system to thoroughly vet every change, and became a major bottleneck.Keeping a record of what we did is important, but it doesn’t have to be in the form of a reusable script. But in many cases we can be confident that changes will be fixed once. Having change scripts at all is only really useful if you plan to do the same mass-refactoring activity again. And sometimes the easiest thing to do is an automated change that works for 95% of repositories, hand-tweaked in the few repos which need it. Sometimes, firing up an editor or IDE is a heavyweight but easy option. Writing change scripts in Python was constraining: sometimes the easiest way to express a change is just a simple shell command or invocation of a more specialised refacotring tool like codemod or comby.If engineers are going to locally clone repos anyway, why not just make this part of the process? Previously, to write a reliable Codelift change script, engineers would frequently have to clone many or all of the affected repos locally, just to test that the change would work.Turbolift is a re-imagining of the mass change process: Codelift gradually fell out of use, but the need for it remained. However, it’s pretty difficult to write a script that will reliably work across diverse repositories just the expertise needed to review the change scripts was a bottleneck, and scripts frequently needed multiple rounds of tweaking to deal with the inevitable failures. We needed to be able to make non-trivial changes in tens or hundreds of repositories at a time.įor a long time we nurtured an in-house system named Codelift: essentially a batch system which would apply a Python change script against hundreds of repositories overnight, raising Pull Requests (PRs) with any changes. And while we’re taming our repository count where we can (including combining repositories where it makes sense), we still have a lot of repositories. However, not every change we want to make is in a library - despite our best efforts, we still have boilerplate config/code that needs to be improved from time to time. Each is in its own GitHub repository, which has some upsides in terms of separation of concerns, but clearly has some costs: when the same change needs to be made to all of these repositories, how can it be done? Most of our microservices use common shared libraries, so updating to receive a new security patch, resilience improvement or observability feature (for example) is often just a Dependabot bump away. All-in, there are several hundred microservices and microsites (webapps that support a specific portion of the site), supported by hundreds more lambdas and internal libraries. With millions of travellers using our site and app every month, we handle dizzying volumes of requests across a microservice architecture that, itself, is pretty huge. Skyscanner’s systems are anything but small-scale.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |