CREST organisation: A pre-requisite to achieving Microservices success…
“Organisations which design systems … are constrained to produce designs which are copies of the communication structures of these organisations.” — M. Conway
As more and more organisations are moving towards the ‘Microservices’ world (whether rightly or not is a topic for another post), the quote above by Conway (given in 1967!) holds true today more than ever. Most companies approach the task of creating Microservices as a technical challenge, focussing on the technology stack/frameworks/databases/tools etc, but rarely does one fail due to wrong technology choices (many do fail though).
After having experienced both in my career, colossal failure and sweet success, in creating a Microservices based product offering, I have come to realise that the biggest factor determining an organisation’s success in moving towards Microservices is, “How effectively is the organisation structured according to the CREST (Cross-functional Responsible Empowered Small Teams) model”.
Cross-functional Teams
Cross Functional Teams
Even before Microservices became the trend, agile development practices had made cross-functional, self-sufficient teams the norm in modern development organisations.. Gone are the days when teams used to be competence silos (Java team, UI team, test team, performance team etc), and delivery/project managers had to negotiate resource availability across all these silos to create a virtual delivery team. This created conflicting priorities, extra communication boundaries, and lack of accountability, leading to lack of agility and quality.
Now, contrast that to a vertical team for each service, that owns the API, Business, UI and the Data layer of the service. There are no dependencies on other teams (to deliver the service) as long as each service exposes a robust and documented API. Such a structure enables the teams to run independently (thereby faster), and if (when) they fail, the impact is limited.
Responsible (end-to-end) Teams
Team Ownership
Once you have created cross-functional, next step is to have ‘complete ownership’ of the service, from development to deployment, within the team. Giving this end-to-end responsibility of the service to the team, is what is driving the DevOps culture, or as Amazon calls it ‘you build it, you own it‘ culture.
Imagine that your development organisation is creating hundreds of micro-services, delivering them to production every two weeks… there’s no way a traditional Ops organisation can keep up with this pace, and so its imperative that the teams own the delivery themselves. Whether it’s done by specialists within the team, or by developers themselves, doesn’t really matter… What is important is that the responsibility lies with the team.
Empowered Teams
Empowered Teams
Now that you have teams that are self-sufficient, and have the end to end responsibility of the service they are building, next step is to trust them, and empower them to make their own decisions. As the number of services making your product grows (from dozens to hundreds), the ‘one size fits all’ kind of guidelines (process/technical) don’t work any longer. Each team must be allowed to chose the process they feel works for them and the tools and technologies they feel are appropriate for their service. Of-course there need to be broad boundaries set at the organisation level, such as the infrastructure choices, operating systems, platforms, open-source vs enterprise, etc, but those should be few (and flexible). When it comes to choosing languages, libraries, patterns or even processes (scrum vs kanban) teams must be given the liberty and encouragement to choose the right tools for the job.
Empowerment must also mean that the teams are allowed to fail (but fail fast). Only in an environment where failure is not discouraged, do teams dare to experiment and achieve great results.
Small Teams
Small Teams
A Microservice team’s ideal size is 6-8 members, and certainly not greater than 10… This is the basis of Jeff Bezos’ famous quote “if a team couldn’t be fed with two pizzas, it was too big”.
As team size starts to grow beyond 10, the increase in coordination and communication costs outweigh the advantages of extra heads. Also, various studies have indicated that individuals in larger teams are less motivated and perform sub-optimally, compared to same individuals working in smaller focus groups. In a true Microservices architecture, each service is small and has a single responsibility. A team of 6-8 members is sufficient for such a service (or even a couple of them), and if not, most-likely your service is not ‘micro’ enough.
CREST is necessary but is it sufficient?
So, the question to be asked is, if we create CREST based organisation, are we guaranteed to succeed in the Microservices journey. Unfortunately, while CREST is a necessary ingredient, it’s hardly sufficient. Various other factors, such as alignment with the product organisation, dev organisation’s skillset, maturity of continuous delivery pipelines etc play a role in the journey.
Conclusion: While CREST might not guarantee success in building Microservices, without it failure is guaranteed. So, evaluate your organisation’s CREST readiness, before you embark on your Microservices’ journey… 🙂
