In physics, the law of conservation of energy states that the total energy of an isolated system remains constant—it is said to be conserved over time. Energy can neither be created nor destroyed; rather, it transforms from one form to another.
In IT I postulate that there is another similar law that we all should get familiar with
The conservation of complexity states that the total complexity of an isolated business / IT system remains constant – it is said to be conserved over the lifetime of the solution. Complexity can neither be created nor destroyed; rather, it transforms from one form to another.
If energy is the ability to do work or the ability to move or elicit change then complexity in this context, is the resistance to change in a system. This resistance causes the effort and time to implement a change to increase which in turn increases the cost of sustaining and enhancing a system.
How do I come to that conclusion?
The follow diagrams model the way in which system architecture has developed over the last few decades. Each diagram represents the same set of business problems solved through differing architectural styles. Monolithic systems have given way to client/server or tiered designs which have in turn fallen out of favour to be replaced by service architectures whether the SOA or the Microservices kind.
Each progression above can be classed as simpler than its predecessor. Components and then services have been introduced as a means to compartmentalise business logic in order for it be reused or replaced altogether. The drive to Microservices takes this further. It is the manifestation of the Single Responsibility Principle at architectural level. Building a service that does exactly one thing well, is much easier that trying to weave that code into a monolith. There are no distractions and it is straight forward to articulate the acceptance criteria or what done looks like.
However, one service does not make a solution, so what is the impact to the overall system complexity.
Layering on inter-component or inter-service interaction on to the client/server and Microservices models above highlights that the complexity has shifted to the networks and communication channels between the components that make up the system. It seems that the client and server components are complex to build but simple to connect, whereas Microservices are simpl(er) to build but more complex to interconnect.
Building communication networks between components adds another layer of complexity. Nodes in the network need wiring together and managing. Security becomes more of a concern as the traffic travelling on the network needs protecting. More network hardware is introduced and someone has to manage it. It may be easier to test each component individually but how do you know that the system in its entirety is working? When things go wrong how do you pinpoint the cause?
The complexity has simply been relocated from the software and code into the network and solution management.
People also have roles to play in complexity, after all they are the users of the system. In less sophisticated IT systems, the software plays the part of a glorified filing cabinet. Records are accessed in the system, they are reviewed, changed if necessary and then pushed back. The users and the software are working together to achieve some business activity, often with the users holding relatively complex processes in their heads.
As activities are automated and the burden on people is reduced the complexity moves into the system. New user interface styles are designed and built so users can be more efficient. Workflow systems are introduced that allow humans and systems to communicate more effectively. Monitoring systems are built that enable the proactive detection of issues and provide a means to track and resolve them either systematically or by referring to system operators.
System architecture should not be thought of as something that will simplify a problem. It generally doesn’t work that way. Instead you are choosing an architecture style that arranges the complexity in a form that can be best dealt with. Through system design you are moving not removing complexity.