Why should you dream big about your architecture?

There are a lot of articles about how important the product/project vision is, when you are working in an Agile way, but almost none of them mentions architecture vision, however I think that it is at least as important as the former ones, and I will show, why you should think that too. (Besides that, I promise that I'll keep the bullshit level of this post as low as possible.)

Whining about the constantly evolving and changing tech scene and business needs as a software engineer might be a cliche, nevertheless there is some truth in that: it's a real challenge to keep your software up to date in terms of maintainability and used technologies, while you have to keep (or even increase) your team's pace. Practically, it means that you should implement new features and fix bugs as quickly as possible, while you also have to make sure you deal with technical debt in a forward-looking manner. Yeah, sometimes it's cumbersome, but that's the way how things are going nowadays, so you're better off figuring out something that can be your "guide" on this journey.

The two types of technical debt

When we are talking about technical debt, most of the times it's about some mess in the codebase: an outdated dependency, a large class, which needs to be refactored, or something similar to these. In my opinion, there is also an other type of technical debt that is about the way how you distribute (or don't) your business logic, responsibility and data between your higher-level components (different services, gateways, databases, etc.), let's call this architecture debt. While eliminating the former one is more about "How to do it?", the latter is about "Where to do it?".

Erasing technical debt is a hard job, so focusing on the prevention is a critical part, if you want to build an efficient engineering team. To be honest, there are cases when you can do anything, technical debt will grow anyway (e.g. the time itself is a very important, but uncontrollable factor here, you can read more about it in this great article), however, when it comes to architecture debt, you can decrease its build-up with a little attention, and that's where architecture vision comes into play!

What is "architecture vision"?

You can call it whatever you want, the important part is that it's the ideal architecture of your system, like a perfect plan in an ideal world. Imagine it as a blueprint that your team would draw - based on the current state of your system - if there wouldn't be any limitation of time, money or resources. In terms of format, it shouldn't differ from a classic architecture plan, like this:

Why is it important?

Even if a commonly agreed plan or vision doesn't exist, there are always some long-term goals for an engineering team, for example:

• We want to go to the cloud!
• We want to kill/tear down application X, because it's unmaintainable!
• We want to speed up something!

These goals are noble, however they are at a very high level, which means that it's hard to use them as reference points in decisions. The good news however is that these goals can be specified in an architecture vision:

• You can plan with resources available in the cloud
• You can split up monolithic applications into smaller ones
• You can choose technologies/solutions that are performing better than others

The whole thing is like dreaming about the perfect architecture you could ever imagine, which would satisfy all your current needs as well.

Let's imagine a game where there are two sides: you start on the left (your current architecture), and you have to reach the patch on the right (the "perfect" architecture that would be the best fit for your current use cases). The size of the patch symbolises how clear your architecture vision is: the smaller it is, the more clear your goal is about what you want to achieve regarding architecture. There are some obstacles (business needs = bugs to fix, new features to implement) that you have to tackle by getting around them. The more you move, the more resources you spend: the shortest route = the fastest and most efficient path to your ideal architecture.

(I beg you not to judge me by my drawing skills 🙏)

You can see on the animation above how the count of possibly ideal (purple) routes changes, depending on the clarity of your architectural goals.

The point here is that you make different - and probably also better - decisions, if you exactly know the goal (a.k.a. vision), and you are aware of the consequences of your decisions regarding this goal. For example, if there are two, totally perfect, but different solutions for a single problem, and one of them will take you closer to your goal, it's hard to guess which one is that, unless you know your goal. This way a complete architecture vision can be your best guide when you are dealing with technical debt, or trying to implement something new.

How to do it?

Defining your ideal architecture shouldn't be more complicated than any other architectural planning, however there are a few things you have to keep in mind while you are working on it:

0. Know your current architecture and use cases - I think it's fairly self-explanatory. If you don't know what your system is currently doing, neither can you imagine its perfect version.
1. Start from the top, iterate a lot - Start your vision with the most abstract goal, e.g. "This should be in the cloud" or "It should be able to run on a low-end VPS". Add more and more detail in every iteration, refine your ideas again and again, don't try to figure out everything in the first round.
2. Stop before you start adding implementation details - Implementation details are totally out of scope for architecture vision, so don't try to dig into them. They should be specified later, when you work on an actual initiative.
3. Write down the "hidden" consequences - If there are non-trivial consequences in your vision that are coming from certain decisions, you should explicitly specify them for future reference. (e.g. if we want to deploy our software as a service (SaaS), then we will need twelve-factor apps)
4. Review and update the vision regularly - Consider the process as a 2-way flow: not just the vision should affect our future decisions, but the constantly changing business needs could also induce changes in it. An outdated vision is much worse than a non-existent one.

When it comes to point number 1 and 2 you should take a look at the C4 architecture model, because it has a very similar concept when it comes to abstraction. It differentiates four levels of architecture, as follows:

• System - Shows the software system you are building and how it fits into the world in terms of the people who use it and the other software systems it interacts with
• Container - Zooms into the software system, and shows the containers (applications, data stores, microservices, etc.) that make up that software system
• Component - Zooms into an individual container to show the components inside it. These components should map to real abstractions (e.g., a grouping of code) in your codebase
• Code - You can zoom into an individual component to show how that component is implemented

I’ve highlighted the first two, because they are the relevant ones when you want to create a proper architecture vision: you’ll start at the system level, and stop before you get into components, since the latter could contain implementation details as well.

What should the team be aware of? (a.k.a. "the pitfalls")

• It’s just a vision, therefore it might have never been realised completely.
• Business needs still have a higher priority - Constantly thinking about and iterating on the vision as a daily routine is definitely not a sustainable thing, and it shouldn’t block you from doing your “regular” tasks.
• It’s not a sacred guide, don’t push it too hard - Having an architecture vision doesn't mean that you should never make compromises again. It's about the benefit that you'll be aware of the consequences, if you have to make a well-founded compromise.
• Don’t try to solve non-existent problems in advance - Focus on your current pain points and use cases, and don't try to "over engineer" your vision (YAGNI also applies here)
• Mutual consent is very important - If you are planning to make a committee of a chosen few, who will then create an architecture vision for the whole organisation, they will probably fail, because no one will identify with it. (See TAGRI here.) Let the whole team to contribute!