This post is the first from an on-going series from our engineering blog, Shift6, which features developer-focused content.
Tech teams move to microservices for many reasons. The flexibility that microservices afford hits on a lot of the “keys to success” agreed to in the industry:
- Scale independently
- Fail independently
- Things that change together stay together
- The right tool for the right job
- Loose coupling
- API-driven design for services
- Etc, etc, etc
These are all well and good, and we certainly care about all of these. However, at Redox, we’re moving to Microservices for one primary reason:
Help developers make awesome stuff as fast as they possibly can.
In other words, our developers have a “get out of the way and let me make awesome stuff” attitude, and we believe microservices is the best way to allow that attitude to actually become awesome stuff.
Throughout this series, we’d like to document our journey from monolith to microservices in hopes that we can share our learnings—and others can learn from our mistakes without making them themselves.
A brief history of how we got here...
The Redox platform is about three years old now. We’ve evolved in the same way many platforms and teams have evolved before us (and how many will evolve after us, too). When we started, we had one service that did everything. We didn’t really know what we were building, so that one service had a little bit of web front end stuff in it, a little bit of backend processing, a little bit of database definition, a little bit of monitoring, and a little bit of everything else. For a long time, that worked great because it was just a little bit to manage and we could all easily see what was going on.
After a year or so, we were clear on what we were building, and though there was a lot that we wanted to build, it was easy for us to see how it would fit into our existing architecture, so we just kept adding on...and adding on…and adding on.
All of this added up to a single service with about 120K lines of node code, among lots of other stuff. Not the biggest monolith you’ve ever seen, but 120K lines of code is a lot to hold in your head while you’re working.
As the platform grew, so did our team. We went from one small team, to a number of small teams, all still trying to work on the same service. Despite doing our best to have each team responsible for a different part of the service, conflicts and confusion abounded, and the simplest changes started taking days (or even weeks!) to complete due to the crazy amount of testing that was required. Even with a full automated testing suite!
It didn’t take long for the team to figure out that smaller apps that communicate over APIs are a lot more fun and a lot easier to work on. So sort of by accident, we started moving into the microservice world, slowly and without a clear understanding of what it means to do it right.
That brings us to now
That’s basically where we are now. We’ve got one giant service that does almost everything, and a small-but-growing number of tiny services that all interact in different ways, have different style requirements built in, and run their tests in different ways, etc.Time to standardize all the things.
However, microservices aren’t a silver bullet.
As we’ve learned from our first accidental foray into microservices, they aren’t necessarily going to make things better. If we’re not careful, we’ll end up with a whole set of other problems that are much worse and harder to deal with. With microservices, you are trading one set of problems (big monolith that’s hard to work on) for another set of problems (lots of little services that are hard to orchestrate and manage).
Our bet is that we will be significantly faster overall as an engineering team by trading the problems we have today for the problems that come with microservices—as long as we can standardize the way we create and maintain these services.
So that’s the why. Over the course of this blog series, we’ll share our approach to breaking apart a monolith. We’ll try to write as often as something interesting happens, which will be relatively frequently in the beginning (every couple of weeks), and most likely trail off as we settle in.
Oh, and if you’d like to help us in our microservices journey, we’re hiring!