Highlights

The first scaling crisis happened in 1996, when Linus wrote that he was "buried alive in emails". It was addressed by adopting a more modular architecture, with the introduction of loadable kernel modules, and the creation of the maintainers role, who support the contributors in ensuring that they implement the high standards of quality needed to merge their contributions.

The second scaling crisis lasted from 1998 to 2002, and was finally addressed by the adoption of BitKeeper, later replaced by Git. This distributed the job of merging contributions across the network of maintainers and contributors.

In both cases, technology was used to reduce the amount of dependencies between teams, help contributors keep a high level of autonomy, and make it easy to merge all those contributions back into the main repository, Bernhard said.

Technology can help reduce the need to communicate between teams whenever they have a dependency on another team to get their work done. Typical organizational dependencies, such as when a team relies on another team’s data, can be replaced by self-service APIs using the right technologies and architecture, Bernhard mentioned. This can be extended to more complicated dependencies, such as infrastructure provisioning, as AWS pioneered when they invented EC2, offering self-service APIs to spin up virtual servers, he added.

Another type of dependency is dealing with the challenge of merging contributions made to a similar document, whether it’s an illustration, a text, or source code, Bernhard mentioned. This has been transformed in the last 15 years by real-time collaboration software such as Google Docs and distributed versioning systems such as Git, he said.

Anyone trying to scale an agile organization should study lean thinking to benefit from decades of experience on how to lead large organizations while staying true to the spirit of the agile manifesto, he concluded.