Five reasons big software rewrites fail

This article is part of a larger series that will lay out the truth behind technology migration and decoupling projects. I’ve taken part in a couple of BIG REWRITE™️ and DECOUPLING™️ and SCALABILITY™️ projects in the past 3 years, so I decided to share the whole truth behind why many such projects fail. I hope this article and the other five sitting in the editing pipeline will help you spot bad trends and do course correction early on. If you haven’t yet decided how you want to migrate your application to that new framework, here’s an article I wrote about the two strategies I’ve seen at work, for migrating to a new technology. Talk is cheap, so here are the reasons.

Hiring only after the house caught fire

Increasing capacity half-way through the project, looking for an immediate productivity boost is crazy. Just as crazy as purchasing fire insurance while your house is burning, hoping to save the furniture already destroyed by the fire.

Remember, for every member you add to the team, an existing member’s productivity is reduced by half. That happens for as long as the on-boarding process takes. There’s also the Tuckman’s stages of group development — Forming-Storming-Norming-Performing the team goes through, every time a new addition to the team is made. EVERY TIME!

Not involving the whole team

When only a small group — read: “the architects” — decides what tech stack to use, your project is headed for trouble. This is one of the reasons management ends up complaining that the team lacks commitment. They might have chosen the best tech stack, if the whole team wasn’t involved, they will reject it like the body refuses an incompatible organ after a transplant.

And it can only get worse. If the tech stack is decided upon by the business, based on buzzwords and rumours they heard while dining with their other business-friends who are better off, they should say hello to “it’s not my job” and “I told you that won’t work” and “X is way better” and “It’s John’s fault”.

My advice:

If it’s a team effort, then it should be a team decision. PERIOD!

Lack of planning

Failing to plan is planning to fail

— Someone a lot more wiser than me

I see this behaviour usually with software developers who believe writing code is more of an art than a craft. It’s not limited to developers, though. In 2018 some business people still believe that if they have an idea and they have developers, they can just tell developers what to do and they will do it.

You Don’t Understand!®

A software developer is not like a blacksmith. A blacksmith may face uncertainty in terms of the quality of the material they work with, the quality of the coal or the quality of their tools. But the level of uncertainty in software development is much higher. The software developer first needs to figure out what he/she is implementing, then outline a solution, then validate it, then implement it. They might have to go through this process multiple times, with a single piece of work.

Here’s a great analogy I found a while back, on Quora. It perfectly depicts the software development process — how to make a cup of tea

Not sticking to decisions

Failure to FOCUS (Follow One Course Until Successful)

— Robert Kiyosaki (I guess)

I remember reading about Henry Ford’s decision making principles in a book, last year. The gist was that decision making should not be delayed, you should be able to make quick decisions, but once decisions are made, people should be slow in changing their mind. Constantly changing your mind about decisions, will lead a team to build half-a-features and end up throwing away work. Make up your mind!

Build half a product, not a half-assed product.

Jason Fried, Rework

Sticking too long to poor decisions

This is a manifestation of the sunk cost fallacy. It’s the inability to let go of poor decisions just because “We already invested a lot of _______” — fill in the blanks — or “We’ve always done it this way”. This also highlights a dangerous behaviour triggered by the fear of taking responsibility for poor decisions and the insistence to continue on the wrong path.

This can, and will lead a team into a death-march.

Time for some cliche talk.

Remember, no matter how far you walked on the wrong road, turn back. Or to put it differently, even if you are on the right path, if you just stand there, eventually you’ll get run over.

Lack of capacity planning

If there’s no capacity planning, critical people will end up being taken off their tasks to work on some unplanned, unforeseen and most importantly unrelated work. If the organisation already has the headcount, please, be realistic about the number of people that are supposed to work on the project. If the plan is to migrate a full-blown production application, magic numbers don’t work. Just by picking a number — three, four or seven — won’t get you closer to your goal. At least not while respecting the roadmap milestones. Believe me, I know a lot about magic numbers. I studied philology in high school 😎.

Here’s a link to a paper on magic numbers in fairy tales, folklore and mythology I found on the web but never read.

The avoidance mentality

You avoid the obvious even when it’s trying to stare you back in the face. It’s walking around with your eyes shut, acting like everything is fine. It’s ignoring that the next milestone is in two weeks and you’re two months worth of work behind schedule. Take the bull by the horns early on. Software development should not feel like the Pamplona bull run. Stop running from the bull because it will ultimately catch up to you and pierce your behind.

If you’re really scared, you can have your developers signal problems. We developers are pretty good at complaining at even the slightest sign of change. If you foster a culture of sharing, openness and courage, where people can speak up, no matter how “sensitive” the problem is or who is the person causing the problem, then you will have your developers screaming at the top of their lungs, tell you that the sky is falling.

Plans that stretch too far out

Planning one year worth of work when there are only two months worth of roadmap clarity.

One-year long roadmaps are just fantasy, to me. You may think you have everything on lock, at the beginning of the year, but wait until Valentine’s Day. After that, you’ll want your team to be more “agile”. You will want them to adapt. Why not save yourself the trouble and plan for a quarter, instead? A quarter, in my opinion, is the best stretch of time to be planned for. Also, setting quarterly goals means you’ll be more likely to achieve them, as you will deal with less uncertainty, and your goals will be more realistic.

Trying to dump a whole year worth of work in a project management tool, and planning software like you plan the construction of a house, is a waste.

What if a hurricane hits?

What if Elon Musk decides to take over the Internet and creates an army of zombie robots?

What if your only option to deploy to production would be to take your app on an external hard drive, to Amazon’s datacenters?

What if in order to do that you’ll have to battle with Elon’s zombie robot army?

These are all valid questions. Just as valid as “How long do you think it’s going to take?” or “How many story points should we add to this user story?”.

You are not building a house. Houses are easy to plan for, because everything is “done” you just have to dig a hole, add some “glue”, put the pieces together and you’re done. With software, you have to even produce the spot where you want to dig the hole, and then dig it.

The only similarities between building a house and building software are in terminology, not in how the actual concepts are applied. In construction you don’t iterate over the architecture. If you do, you build a new house. And to build a new house, you have to tear the old one down. There is little material reuse. Well, if the software is architected “well enough” you’re able to change the architecture without rebuilding the whole project.

Stop building monolithic, tangled, shitty, poorly-architected software just because “you need to deliver” and “you will refactor after you get investments”. This will eat you up with your shoes on! I mean it!

I’m not saying monoliths are bad. You can build a modular monolith that will work way better in the short term than any microservices based fluff you might find out there. But think before you build. Use a decent front-end framework, choose a UI library that fits your needs and learn that thoroughly.

Measuring the wrong things

I’m not going to talk about specific measurements, you have to decide those for yourself. You should objectively evaluate your current situation and figure out what you want to measure, whether it’s velocity or time-to-market.

But let me give you a personal example of wrong measurements.

Let’s assume my goal is to increase my email subscriber list (which it is, so subscribe). It would be wrong for me to focus on the website visitor count. If I start watching that like a lunatic, month after month, I could start thinking I’m pretty popular and live like I already have more email subscribers — most fantasy-world story-point-adoring people live like this. Or I could look at look at my subscribers list and notice there’s no improvement. Then I would look at the visitor count and see that I have plenty of people on the site and figure out why they’re not subscribing.

The process is too rigid — or you think it is

From my experience this happens only because people tend to get too religious about the process when their only focus should be delivery. Many teams use SCRUM and complain that it is too rigid. In my opinion it is, but I’m biased. Objectively speaking, any agile-ish process is capable of adaptation and has a lot of flexibility built in. It is only our nature to apply everything by the book — especially when we are not the ones impacted by the process — that makes everything seem rigid.

Hiring me

You hire me to put out the fire and do software development instead of letting me help you with your technical strategy.

Big mistake! Although I love to do implementation work, and I somehow end up solving the hardest problems the client is facing, I’m not the most talented software developer. I know people who can put me to shame just by breathing in the same room. But one of my biggest assets is that I’m able to zoom-out quickly and spot trends and opportunities you can ride to success. And I never lose sight of the big picture. Now more than ever, you need to do big picture thinking on the technical side, as well. It’s not enough to have a great product vision and good software developers. You need a technical vision that’s just as quick to adapt to the world and the industry as your product vision is.

Post Scriptum

I know, there are more than five reasons rewrites fail. This post embodies the core nature of software development. You set up to solve one problem and you find three other problems you need to solve, along the way. Or as Vasco Duarte puts it, in his book — #NoEstimates — “building software is like when you try to unclog the pipes in your bathroom and end up having to build three more houses just to get the toilet working again.”

Help!

Did you enjoy the article? Please share it with your peers, managers, bosses or whoever you think will get the same value out of it.

Also, subscribe to the newsletter to be notified when the next article of the series is available.

Cheers!