Why Software Projects Take Longer Than We Think (Pareto Principle)

Photo by Jo Szczepanska on Unsplash

Software projects take longer than expected.

There are many reasons for this. The one I suffer most from is to underestimate amount of work left to do. Especially at the beginning of a task I used to think: "this is simple, I only need to do X and Y and then I'm finished!" Nope.

I can tell you this gets better with experience. But even senior developers sometimes fall into this trap!

Where does this error come from? Let's dive in.

What is the happy path?

When you start developing, you do the happy path first. The happy path is the user journey where everything goes to plan. The user is logged in, makes only valid input, has a modern browser, all lists already contain items, every system you rely on works, the sun is shining, and the planets are aligned. This is the easiest path for you to implement.

According to the Pareto principle, 80% of outcomes come from 20% of the effort. For our software task this means: the easiest 80% take only 20% of the time. The happy path takes only 20% of your time and produces 80% of the result.

So when you think you're 80% done (everything works!), you have 80% of work left to do.

This is true in each pull request. When the happy path is done, you still have error handling, edge cases, testing, code style fixes, adjustments to the deployment pipeline and infrastructure ahead of you. Also, your feature will need ongoing maintenance: dependency version upgrades, bugs you discover later, ... .

This is also true for software projects in general. First the happy path with things the user wants to do. This makes for a great showcase. Then you need other things like billing, email templates, error handling, error reporting, input validation, proper authentication and authorization, access control policies, database backups and a recovery plan, edge case handling, bugs will pop up, and not to mention keeping everything up to date.

Because of this, you are not even close to done when it works.

But: you can use Pareto to your advantage. Identify the 20% most useful features for your user, the ones that make up 80% of the product’s value, and build them to sufficient quality. With this you can provide the maximum user value in minimum time.

How can I avoid the Pareto trap?

When you gain experience in developing and maintaining software, your respect of the amount of work remaining increases.

To reduce the impact of the Pareto trap, I take my most optimistic time estimate (that is also realistic) and double it. This works out pretty well. I try to think about those 80%, but it is difficult to see them. They lie in The Fog.

The Fog covers every aspect in a project that you don't yet see clearly. This is mostly two things: the exact features your product will have, and the technologies you will use for this. Agile methods tackle the fog by only planning the next iteration in detail, with ideas and implementation details being less and less concrete the further they lie in the future.