Note: This is a blog post about a github project I am working on – to skip all the words and check out the code, go here.
Ever since Charles Babbage’s Difference Engine, programmers have been drawing an analogy from computers to brains. Fundamental terms like “memory” and “compute” are borrowed from the animal kingdom, and hot trends like deep neural networks continue the tradition today. Perhaps we draw the analogy as part of a fantasy of making a machine in our own image, or perhaps it is part of a hope that, by creating a thinking machine, we can better understand ourselves. Whatever the motivation, one thing is clear: the computer-brain analogy is not is a useful way to think about how computers work.
Computers are rigid, impractical, and if there is even the tiniest thing wrong they completely grind to a halt. They are not creative, they are not struggling to survive, they don’t have compassion, and they won’t tell you what’s really going on with them unless you really wrench it out of them. In other words,
Computers aren’t like brains, they’re like bureaucracies.
All this time, we were drawing the wrong analogy! The structure of code much more closely resemble social patterns than thought patterns. This is ingrained in an idea in computer science called Conway’s Law. It states that the architecture of a piece of software (i.e., the pattern of function calls) will mirror the communication pattern of the organization that produces it. Or, in my paraphrasing,
Our software works like how we work with one another.
So, if your organization is a little monarchy, with people at the top deciding on fundamental technology choices which persist throughout your applications, you will have a monolithic architecture. Dictatorships are great for getting things done quickly, but they have a hard time evolving because decisions made a long time ago are difficult to undo. For example, you may have decided in the 1980s that your banking system should run on Cobol because it is the hottest new thing. A hundred million lines of code later, you have an entire industry written in a dead language. Monarchies are so bad at adapting, in fact, that they often just get replaced completely by new regimes- more nimble competitors who aren’t burdened by legacy code.
One solution to avoiding that collapse is by delegating more power to small groups, and asking that they all work together to make the organization function. Each group gets a tiny plot of code to manage, but they get full reign for that tiny slice of the business – managing all technology choices, hiring, and deployment of their code as they see fit. This is service-oriented architecture (or microservices, if they are really small), and it’s the architecture that runs Amazon, Netflix, and Google. These architectures are a constellation of fiefdoms: many lords ruling over little plots of the business, only collaborating through well defined interfaces. The problem with fiefdoms is that, while they can evolve, they can only evolve. It is difficult to pull off big infrastructure projects in fiefdoms because you have to get all the little lords to agree to make it work. This is why medieval England didn’t have a great interstate system!
What I am proposing today is a middle ground, inspired by one of the enlightenment’s hottest trends. If software architecture is like social structure, we should choose an architecture that has been proven to be fantastically scalable, and strikes a balance between the rigid monolithic monarchies and hopelessly quibbling fiefdoms.
The software architecture that strikes the best balance between sustainability and competitiveness at a large scale will be a democracy: independent microservices governed by en elected body.
- A microservice applies for citizenship with the government web service. The government service manages a library of API interfaces, and checks the applicant service url to verify that it passes some simple tests. These could be, for example, “is alive” checks, logging protocols, etc.. If the url passes the tests, it becomes a citizen.
- Developers who work on a codebase that is a citizen can run for office and / or vote in an upcoming election. An “office” is a role in maintaining the government codebase in github – ie, a collaborator who manages pull requests for a specific portion of the code.
- At some point, election day comes and new representatives are elected. These reps are automatically made collaborators for the github project, and anyone who didn’t get elected (except for the owner) gets booted. These elected reps are the ones who set the vision for the government, define the interfaces, etc.
This way, everyone is free to make virtually any technology choices they want for their slice of the system, but they still get regulated, and in return they have basic guarantees on what other citizens are up to and how they can interact. To prevent stagnation in leadership and resentment of that regulation, they have a say in who is at the top.