Microservice Orchestration, Part 1
The complexities introduced by the move towards microservices
Over the last 10–15 years the software industry has been moving away from monolithic architectures towards microservices.
A monolith is one in which everything required to power an end-user product experience is jammed into a single indivisible deployable unit. In a monolithic application there is no need for service-to-service calls or for multiple databases — all the logic and data is encapsulated into one monolithic deployable artifact.
By contrast microservices compose an end-user product experience by organizing well defined components (i.e microservices) into a graph structure connected together through RPCs. In a microservices oriented architecture, SOA, each microservice exposes a well defined interface for other services to call and each microservice has its own isolated persistence store.
Despite the industry all running towards microservices, there are very significant complexities that are introduced with this move. Next we will talk about the two gnarliest complexities associated with this move.
Complexity 1: No Single Transactional Datastore
In a monolithic application there is a single datastore. This datastore acts as the singular source of truth and provides transactional atomicity and consistency for user operations. By contrast in an SOA based approach there is no singular datastore. Various datastores, managed by loosely connected microservices, need to all agree in order to accurately record a user operation. Furthermore there is no guarantee that the datastores used across the microservices support two phase commits — in fact with high likely different microservices will use totally different types of datastores.
In order to demonstrate the complexity this introduces, consider a banking application which enables transferring money between customers and generating monthly reports for the SEC. Consider what the operation of Bob transferring $100 to Alice would look like in a monolithic architecture and in a microservices architecture.
The key observation to make here is with the monolithic based approach a single transaction can be committed using ACID transactions to represent the operation. But in an SOA based approach there is no single transactional datastore that can represent the full operation. This means all the nice properties of ACID (atomicity, consistency, isolation and durability) are harder to achieve and reason about in an SOA based approach.
Complexity 2: Transient / Timeout Failures
When computer_a makes a request to computer_b, there are three types of responses that computer_a can get back:
- Success: Indicating that
computer_bwas successful - Known Failure:
computer_bsuccessfully responds tocomputer_aindicating some failure occurred and the operation was not successful. - Unknown Failure:
computer_agets some response that makes it unable to determine whatcomputer_bdid. This could be due to a failure on either computer, a failure on the network or a simple timeout.
Whenever two computers need to talk to each other these three outcomes are possible.
It is great when computers get back either success or known failures because the those state machines are easy to reason about. But getting back some unknown failure is pretty rough. It is unclear if the call was received, partially handled or fully handled — the state is totally unknown. For operations that are just reading data this is fine, the caller just continues to retry. But for endpoints which write data (e.g posting a credit card payment or filing a report with the SEC) it is much less clear if simply retrying is safe. The downstream endpoint needs to be made idempotent. I won’t get into idempotency in this post, but if you are interested you can check out my former post on it.
The bottom line is getting unknown failures during computer-to-computer communication is tough to deal with. This problem exists in both monolithic architectures and microservice architectures. But in microservices this problem has to be addressed all over the place. With SOA, services are calling each other everywhere. In contrast in a monolithic architecture the vast majority of the logic runs on the same binary, on one computer, and therefore network hops are rarely needed.
When moving from a monolith to microservices, unknown failures go from being a rare part of software occupying a small portion of the total code to being a central design consideration that needs to be solved nearly everywhere in code.
This post covered the complexities associated with moving from a monolithic architecture to an SOA based approach. The next post in this series will talk about the how microservice orchestration can be done in order to address these complexities.
