Recently, I had a very intriguing conversation around reusability, team independence and self-contained services. Since I’ve been reading about this in the context of microservices and modern immutable infrastructure, figured out it is an interesting topic for a post.
There are a number of practices and technologies claiming to solve the problem with dependencies by preferring self-contained services or no-code-sharing policy. Although this is a useful concept, we (software engineers) sometimes underestimate the complexity of composite systems regardless if they are monolithic or service-oriented. We should analyze the dependencies in our system carefully, track them and monitor them instead of trying to ignore and hide their very existence. Service-reuse is a powerful tool, but it is transforming the problem of handling dependencies and not solving it.
First of all, why do we have dependencies in our code? Following the logic of Conway’s law, the challenges of managing dependencies are closely related to the broader concept of reuse and are a reflection of the people developing the software. If you are not a lone wolf working on some small (pet) project, you have to deal with dependencies, regardless if you are aware of that or no and no fancy hype like containers or serverless can save you from that; you have to sit down and engineer you way around this.
Classic Dependency Management
The standard form of reusing code is through dependency management tooling. We describe the libraries and components we are using with their versions in some declarative form and use the corresponding tool (gem, npm, Maven, nuget, CMake) to resolve them and bring the artifacts locally. After that we link them to our program either at compile time (statically) or at runtime (dynamically) and we are good to go.
As software is becoming more complex and more libraries are added, we eventually face the dependency-hell problem. Different components of our solution require different version of the same library while in a single runtime you can have only one. And there are even more challenges: we need to update those dependencies regularly, we scan them for vulnerabilities, exploits and bugs.
Microservices to the rescue! If we split our software into hundreds of small self-contained programs, each running with its own view of the shared libraries and filesystem, we no longer have the problem of dependencies, right?
Let’s take a look on a higher level. Those services communicate with some form of RPC, so how is the mechanism implemented? In every service? What if we need internal trust mechanism? Then we can have some common code (note we are not tackling the challenges of polyglot system) to handle this. But what happens if we update the mechanism, e.g. a key length, algorithm, protocol? Would two services be able to communicate with each other if one of them has the new version of the trust library, but not the other? If we are not touching this part, we are OK, but if we change something we have to roll-out the entire system (or subsystem) with all microservices. Note that this particular challenge can be handled differently, but that’s a topic for another post.
No-code reuse! Only service reuse will be allowed in our awesome microservice architecture.
Let’s explore that for a minute and for simplicity we will continue with the internal trust problem. If we are self-contained, we have dependencies to other services on API and protocol level, so if we change any of those we are pretty much in the same situation. And while in a monolithic system we can easily detect the dependency-hell, on API level in a distributed system it is much harder.
I am not saying that service-reuse is not a valuable tool for solving big composite systems problems, but we cannot solve one complexity with another complexity. We need to think about dependencies both on technical and on organizational level (e.g. how do we require a feature/bugfix for another team’s service), establish our aggregates carefully, have backward compatible APIs, use versioned APIs etc. Microservices give us a lot of agility, but from time to time we still need to think holistically about our system.
I will follow-up with another post that drills in the service-reuse concept..
- Tracking and Controlling Microservice Dependencies (Silvia Esparrachiari, Tanya Reilly, and Ashleigh Rentz), acmqueue, July-August 2018)
- Microservices – a definition of this new architectural term (Martin Fowler)