Why Software Architects Are Essential and Why You Need One?
part 1 of 2 — The Challenge
There’re so many names that “people who can code” like to call themselves, right? Programmer, developer, coder, analyst, engineer, architect. Why don’t they make up their minds and pick just one? Is it a vanity struggle to distinguish themselves among their peers? Or a strategy to sound important to non-IT people from other departments? Maybe they want to copycat other professions because of low self-esteem? None of that, obviously. I just brought it up because I’ve heard such absurdities. Although everything starts with Computer Sciences, if one doesn’t work in research & development, there’s no point claiming the title of scientist. There are genuinely distinct paths of specialization in Information Technology. Some overlap with others, some don’t, and one tries to encompass them all. Albeit hard to explain in lay terms exactly how, different types of specialists are necessary to perform different roles in a software development team.
But I won’t just have you take my word for it. Whether you’re a Project Owner, a Project Manager, a Managing Director, or a client, I’ll refer to you as “manager” for the sake of simplicity, and this is for you. In this two-parts article, I’ll do my best to concretize the role of a Software Architect in a way I hope you’ll not only learn exactly what ubiquitous problem they solve, but also how to identify the gap left by their absence and how it could lead to disaster.
Software evolution
Have you ever thought about how software evolves? Did you know software evolved? I’m not addressing here why software evolves, but how it does. I mean, request an app to be developed today and the first prototype presented already brings some cool impressive features whatever those might be. Go back a decade and this first prototype would basically be a screen where no button or link would do anything. Go back yet another decade and an app wouldn’t exist, it’d be an ugly website or a desktop application where almost anything that we have today would be either too expensive or impractical. It isn’t a hardware thing, hardware has to do with speed, size, and peripherals. It is a software thing; the way software behaves. And software evolves by code reuse.
I’ve written a full article about code reuse which I invite you to read at another time. But being brief: Code reuse is when an engineer does something valuable that could be useful in other projects, and thus chooses to do it in layers. The first layer or base block does the valuable important thing; the blocks above that base make it usable by any requirement, and a third floor of blocks do the things specific only to the project on which the engineer is working. If it were a car, the first floor would be the engine, wheels, and chassis; the second floor would be the pedals, shifter, and steering wheel; and the third floor would be the driver.
That valuable base block (core component) together with its interfaces can then be packaged separately and uploaded to a repository to be referenced by the project as a dependency. From that point on, every time the engineer needs to do that first-floor valuable thing, he/she will reference it in other projects, worrying about only the specific blocks (changing that car’s driver), never having to do the whole thing again. That dependency can even be maintained in collaboration with other engineers, communities, or organizations to make it broadly useful and can also be published for anyone to use. Give this ecosystem of shared dependencies some years for natural selection and a new generation of engineers will never have to figure out how to come up with things that were tricky in the past, focusing their efforts on new things that make the difference today. That’s how software evolves, that’s why IT changes so much and so fast: After something is done, it doesn’t have to be redone. But only if the assigned engineer took the proper care.
In an organization, software evolution happens toward the business model. When a financial institution specializes in financial products, for instance, it must purchase or develop every single software to continuously cover all aspects of its financial processes. A telephone company on the other hand will evolve its systems towards another set of ever-changing necessities. A software company will focus on some specific technical requirements which it’s committed to deliver. It’s safe to assume that the development teams of these companies will get better and better in their own industry. When the team reaches a level of fluidity and predictability where estimates are accurate most of the time and little code has to be refactored, we can say the development team has reached its maturity concerning the software development life cycle.
Code Decay
Believe me, software is like a living thing. It can evolve, it can grow, and it can also decay. If you have kids, I imagine you make them eat their vegetables, go to school, engage in educational activities, and make them sleep on time. It’s undoubtfully a daunting endeavor because kids just want to eat sweets, play all day long, engage in dangerous activities, and stay up late. Fail to do good parenting and you’re buying your future self a problematic teen that is way more complicated to deal with.
Software is not a kid; you can just kill the “teenager” and make yourself another “baby”. Software is also not a person; it doesn’t want anything. But software needs quite good care if you don’t wish to spend all of that money all over again. Someone must make sure that best practices are put into place for it to grow healthy and capable of performing its function. And it’s likewise a daunting endeavor because whoever gets to demand projects and features usually wants everything cheap and in production by the end of the month. The main argument is that competitors are already doing it, and developers can’t throw the “if all your competitors jumped off a cliff, would you do it too?” reply.
Consenting to bold aggressive estimates is what forces good practices to go south, and that’s precisely when code decays, rotting the system even before it achieves full maturity. I’ll cover how to solve this dilemma, but first let’s talk about bad smells.
If you’ve spent enough time demanding a team that maintains the same system for a while, odds are you can relate to this message. More often than not, a supposedly simple feature triggers sprint(s) long refactory. You can question the team’s skills, time management, or even their honesty, but it’s an underlying characteristic of software development that small features might trigger extensive changes. It is one of the symptoms of a rotting design and you should know it!
The subject is extensive and there’s no point covering it in technical details in this article. But if you’re a manager and wants to sniff out when the code being developed is rotting, you should get familiar with these scents:
- Code Rigidity: When simple changes tend to cause a cascade of unpredictable subsequent changes in dependent modules. You smell it when the reason for a delayed task is “it was more complicated than I thought”. In this case, the engineers know why, it just wasn’t clear before starting the task.
- Code Fragility: Similar to rigidity, but in this case, the breakage occurs for apparently no reason in areas that have no conceptual relationship with the area that was changed. You smell it when the reason for a delayed task is “I don’t know what happened, I didn’t even touch the part that broke. We’ll have to investigate”. It’s followed by the dreadful feeling that the team has no control over the code, which is a correct assumption.
- Code Immobility/Inseparability: It’s when changes were written so pragmatically, so fast, so straight to the point, that no visible layer of separation exists between the piece and the whole. You smell it when that expensive time-consuming work invested in that module from another project can’t be reused in this new one, no matter how similar both systems are. When the time it’d take to separate the code outweighs the time it’ll take to write the part from scratch.
- Code/Environment Viscosity: When the software, the module, or the environment is so corrupt, that the new code must abide by the corruption. The engineers will deliberately engage in hacks and anti-patterns because doing the right thing is impractical, thus adding to the blob of filth. For a manager, it’s harder to detect this stink because engineers seldom disclose technical decisions to non-technical listeners, but keep an ear out for words like “hack”, “workaround”, “find a way”, “deal with it”, or any other euphemistic expressions to a course of action that isn’t the usual.
- Code Opacity: I’ll have to provide some context for this one: Not so long ago, good software ought to have a well-documented code. It was required if you wanted to rotate engineers through different projects, in order to assure small learning curves. That proved unrealistic for most organizations because engineers are usually running tight schedules and the first tasks to be ruled out are documentation tasks. And nobody misses documentation until it’s too late. Neglect documenting for one or two sprints and the whole thing is outdated, and outdated documentation is garbage. But this is a lesson long learned, engineers don’t even try to document their code anymore. What they do instead is coding in a way that tells a story, written not just for the machine to interpret, but for the convenience of the next engineer. The code should now be its own documentation. In this regard, software is said to be opaque when someone new can’t deduce what the code is supposed to do just by reading it. You smell it when you assign a new engineer to an existing project and she or he makes little or no progress after a considerable time. Morale may plummet because of unjustifiable imposter syndrome. If the engineer who wrote the code is still with the team, the novice may express an inappropriate subordination towards the previous engineer sure to compromise overall team synergy.
- Needless repetition: Repeated code happens when the engineer, having to do something twice or more, resorts to copy & paste instead of reusing the same code through abstraction. You smell it when a task must be done twice or more, usually because the engineer forgot that another occurrence of the same algorithm existed elsewhere.
- Needless complexity: That’s a tricky one to sniff out by a manager who doesn’t have a technical background in the technologies used in the project. You can smell it when you, being familiar with how things should be done, can’t understand what a certain structure is doing. It usually happens when the technical decision maker is following a community trend, using arguments like “everyone’s doing this way” or “this is better”, but other than that can’t explain exactly why this is better and everybody is doing it, and the direct benefit of maintaining the structure. A good rule of thumb is that every technical decision-maker must be able to explain any aspect of the system to a six-year-old, and you should qualify.
If you sense bad smells in your system, wait no further: It is a malignant issue and should be addressed before metastasis brings the system to a point of no return.
So, basically, are we doomed?
Well, today we have access to exceptionally good software, don’t we? Our society is surrounded by them. So of course, there’s a way. Computer science has already pushed software development techniques to a rather sweet spot, it’s just that most organizations don’t invest on keeping their mindsets as updated as their antivirus. It’s dumbfounding how many IT areas are managed by individuals who doesn’t know better. They aren’t experts, don’t read technical publications often and keep insisting on yesterday’s strategies.
Up until now I introduced you to the concept of software evolution, explaining how software can grow strong and healthy in an ideal environment. Then I listed a set of degenerative tendencies that usually take place in real-world organizations. I bet you related to most of them, and I gracefully dared to say that it was your doing more than the team’s for pushing deadlines so tight that forced them to cut corners. At the end of the day, we can all agree that the market demand things to be hasty like this, there’s no point arguing about it. Then how do we deal with this conundrum?
For the past three decades at least, very smart minds have invested the better part of their career devising, applying, measuring, and improving the way software goes from kick-off to release. Computer Science pioneers Edsger Dijkstra and David Parnas were already emphasizing, back in the late 60s, that the structure of a software system matters and getting the structure right is critical, but it wasn’t until the 90s when O.O. finally became popular that the term “Software Architect” took hold. From 1995 to 2000 Robert C. Martin a.k.a. Uncle Bob has been publishing a set of principles that would later be dubbed S.O.L.I.D. and popularized by his 3-books collection: Clean Code, The Clean Coder and Clean Architecture. David Hansson in 2004 introduced the “Convention over Configuration” paradigm by releasing Ruby on Rails framework, in my opinion one of the most important values to uphold. In 2005, at the height of the SOA architecture, Peter Rodgers came up with the minimalistic concept of microservices that is so much debated to this day and has yet to reach consensus. Because of the increased infrastructure workload cause by microservices, Allana Brown conceived and launched the State of DevOps report in 2012, vital discipline in a modern development team.
In my effort to keep the previous paragraph short, I can’t help but feeling guilty for omitting so many prominent names like Martin Fowler or Chris Richardson. But the message I’m trying to convey here is that much thought has already been invested, facts measured, conclusions reached, and a very solid body of reliable best practices is readily available. It’s always an error appealing to tradition when the subject is software development. As a rather young science, it still experiences yearly breakthroughs.
So, armed with years of expertise in software engineering, continuously updated best practices skills and aimed at pursuing the ideal balance between minimum code quality and maximum delivery rate, there’s the role of software architect. There’ll always be forces pressing for speed, so there must be an equal opposing force pushing for quality.
The Software Architect
Now that you’ve felt the weight of the challenge, follow the link to the next part which covers how they do what must be done.
