Would you be surprised to hear software developers do test their software? It might not feel like it with all the outages this year, but the average developer spends 42% of their work week on maintenance. Then what’s with all the outages?
Crowdstrike might have made the most thunderous noise, but 2024 has seen outages stop fast food deliveries, take WhatsApp and Instagram offline, strand passengers at Heathrow Airport, and halt fresh food at the Brexit Border.
These outages didn’t happen because developers didn’t test software. They happened due to hypercomplex software configurations – too many changes to the products by too many people for too many reasons. Many companies are testing code within a software architecture nobody understands anymore. It’s a hideously large Jenga tower. Just one more ‘feature block’ might collapse the whole thing. No one wants to touch it. No one even remembers how it was built. But sooner or later, an executive decrees “this product needs AI.”
That’s the eternal software engineering battle: innovation versus maintenance, and it’s eroding the very fabric of the world’s software.
Why is software eroding?
Like rocks and mountains, our software is slowly eroding over time. If software is a rock, then developers are the wind and water breaking it up. Thank the dependency hell we’ve put ourselves in.
There’s so much new code being added to codebases which affects, or is affected by, other moving parts of software that it has exploded the complexity of products. Some of this is owed to pressures from senior managers to out-compete rivals, but sometimes it’s developers just trying to shortcut workflows.
A developer gets asked to add a new feature. Said feature bloats the codebase. The developer adds a shortcut to work faster, which adds complexity. A manager asks the developer to expand the product. The shortcut from earlier? It’s incompatible with the update. Things break. The developer starts patching. It takes a long time. The developer adds another shortcut and….round and round it goes.
Software erosion eventually becomes a self-fulfilling prophecy – a destabilizing chain reaction, where the tiniest quality-of-life patch is both a headache to implement and a risk to functionality for teams in other silos.
The human cost of software erosion
If over 40% of development time is just keeping code alive, that’s not adding value to the product. When you factor in meetings, time for comments and feedback, etc, you surely end up with less than half the week left for value-add development.
That’s mind-blowing for a development manager, who suddenly can only use half their developers’ time for innovating. It’s even more miserable for developers: what kind of satisfaction and pride could they derive from constantly fixing code that keeps breaking? The eventual outage disaster and ensuing PR blunder is just an extra sledgehammer to morale.
The next thing that happens is attrition from engineers who have had enough. Onboarding new engineers then lengthens time to market, frustrating everyone involved. Old mistakes resurface, more people leave, except now it’s also seasoned veterans. Where does it end?
Fixing ‘shift left’ itself
So much talk over the years about ‘shift left’, and yet some businesses have not taken onboard the reason we yapped on about that philosophy. The cyclical misery many developers face won’t end until we fix ‘shift left’ itself.
Fixing ‘shift left’ means not just dumping more testing time into developers’ busy schedules. Some companies hire external QA testers to lessen the burden, but that’s an expensive mitigation. It’s slapping scotch tape on the leaking holes of a deck that’s being battered by warships.
A late fix is a costly fix. The least costly fix is getting your architecture right when you’re still designing it. Weave QA tightly into your software development before writing all the code, not after. If you’re early into prototyping, then sure, QA might be unnecessary overhead. But the moment you build a viable business that attracts customers, you need quality code. You don’t want to lose customers early in your product’s lifespan. If your product ships next week, and you have to re-engineer your architecture to avoid a product recall, that’s a catastrophe of epic proportions. It will push your time to market out, balloon development cost, and burn everyone out.
How do you get quality code? You need multiple sources of intel. Don’t skimp on static code analysis and functional tests, which should be run as new code is written. You need to know how much code you’re cloning, where your hidden dependencies sit, how your components communicate with each other, etc. When you know these things and run your architecture verification, it’s easier to identify problems. If you can’t do every type of testing under the sun right away, that’s fine, but start building out these processes over time.
More importantly, does your architecture even let you achieve your objectives? If not, time to re-architect. A company with decades of legacy code might not be able to, but an SME with five years of code? The litigiousness of unhappy customers dwarfs any headache involved in re-architecting.
Understand, too, that different roles have different incentives. Developers sometimes resist static code analysis because it’s ‘additional work’ and adds time to projects. Whose responsibility should this be? Figure that out early.
With outages and crippling bugs starting to feel like a dime a dozen, it’s never been more important to understand how the Jenga tower was built. Too few people know their architecture inside out. With a little discipline, that can change. It has to, because that tower is collapsing soon.
We've featured the best laptops for programming.
This article was produced as part of TechRadarPro's Expert Insights channel where we feature the best and brightest minds in the technology industry today. The views expressed here are those of the author and are not necessarily those of TechRadarPro or Future plc. If you are interested in contributing find out more here: https://www.techradar.com/news/submit-your-story-to-techradar-pro
