How to Avoid Big Design Up Front

Photo by Charles Deluvio on Unsplash

1'500 tickets in your client's project management tool. A fixed deadline. And not enough resources. You know this will suck. But you create a project proposal, anyway. Your client knows that you will push the deadline back. But they agree anyway, because they don't trust any other process.

Why do clients operate this way? Why do they want a Big Design Up Front (BDUF)? And how do you get out of this treadmill?

TLDR: Use an iterative approach, hold your client's hand, and step by step, build up trust.

Big Design Up Front = Waterfall

BDUF means to gather all requirements upfront and then model and design everything according to those requirements. We call these phases:

• Big Modelling Up Front (BMUF)
• Big Requirements Up Front (BRUF)
• Big Design Up Front (BDUF)

BDUF usually comes with fixed price projects. Fixed Price means: fixed time + fixed scope + fixed budget (the "iron triangle"). BDUF is a relic of waterfall project management

Excursion
Waterfall relies on a sequence of pre-defined steps, where you must complete every previous step before starting the next phase. Typical phases are: Requirement Gathering and Documentation System Design, Implementation, Testing, Delivery/Deployment, and Maintenance.

Waterfall can work for small teams with predictable projects. But for larger teams and more complex projects waterfall results in failed projects, higher costs, and unused functionality.

Why people use BDUF

BDUF builds on the assumption that we can gather all requirements up front. By this reasoning, we can create a plan that provides the most efficient way to reach the project goals. Another assumption of BDUF is that the gathered requirements form an effective product that solves the user's problems.

The reasons some customers demand BDUF are:

• Inexperience with large software projects
  91% of large software projects with waterfall fail (source). So the client was very lucky, is unwilling to learn from past failures, or has no experience with large software projects
• Purchasing department demands it
  The purchasing department within your client's organisation has control over the money and demands a full project proposal and plan. They usually need a list of features for a fixed price by a fixed deadline where they can check off the things you delivered. Also, the purchasing department often needs to choose between several solution providers and wants to compare their offerings. (In my experience, looking for cultural fit and competency with their solution provider would be much better. Both properties increase the project's success rate massively.)
• Project management wants project with highest return on invest (ROI)
  And to select that, they need cost estimates for each proposed project.
• Client wants to feel safe with the project
  BDUF aims to find all errors and flaws before implementation, when design and change of design are cheap. People and organisations see high value in this safety.

When BDUF is a terrible idea

In complex projects, it is impossible (or at least incredibly expensive) to gather all requirements up front.
Note: when it is a small project, and you have ideally done similar projects before, waterfall (with BDUF) can be a pleasant process.

Without thorough user testing, you can't know what users need. You also won't know if the proposed solution covers all user needs optimally.

Humans are awful at estimating. 21% of waterfall projects fail, and 53% of projects go either over time, over budget or miss planned functionality (Chaos Study 2019). This shows that most estimates are wrong somewhere.

Correct estimates heavily depend on the team. Doing something with a team of experienced engineers is faster than with two junior developers right out of college. Teams also constantly change. Your team learns all the time. Some people leave, and additional people join.

Gathering all requirements and estimating precisely is a tremendous waste of time. A rough estimate about the price range of a project will take a few hours or days. Making the estimate more precise takes much longer: weeks to months (see: law of diminishing returns).

Stakeholders are most often not interested in a project 100% to spec but in the ROI of the project. The ROI of a project is a balance of engineering cost, time to market, and perceived value to the user.

Most of the time clients don't know what they need. And they can't articulate what they want.

Just like it is costly to gather all requirements up front, it is also incredibly expensive to fix all errors and flaws during design. "If the cost of planning is greater than the cost of fixing then time spent planning is wasted" - Wikipedia on BDUF. We also call this phenomenon analysis paralysis.

How to avoid BDUF

When you want to turn a waterfall project into an iterative project, these are your goals:

• You want your client to understand the benefits of agile.
• You want your direct contact partners within the clients organisation to trust you and fight for your project.
• Your client only comes to you with the problem. You develop the solution (together with them).

Start by finding out why your customer wants a BDUF project.

When you know the why, highlight the dangers of waterfall for their project:

• If they want safety, explain that scope changes always happen, therefore the estimates won't be accurate.
• If they want to maximise ROI, explain that their project idea likely contains a lot of waste work that the users won't use.

Then, explain the advantages of iterative projects (you don't even need to call them "agile"):

• An iterative approach can cope with scope changes fast, and without a huge change management process. You can address any scope changes after the current iteration (usually less than two weeks away).
• An iterative approach always focuses on what brings the most value to the user, therefore minimises waste.
• With regular working software increments, you can demo, deploy, sell, or use that solution earlier.
• Therefore, they can gather feedback earlier and make the product much more valuable in a shorter amount of time by getting user feedback earlier and prioritising features or changes based on that feedback.

Finally, propose them an alternative:

Smaller Project Proposal

Offer them a smaller Proof of Concept (answering: "is it feasible?") or Minimum Loveable Product (answering: "is it valuable?") version based on what you know. Make it small enough that you can estimate it. In this smaller project, work iteratively and step-by-step show your client how agile software development is beneficial to them.

Large Proposal Focused on High-Level Goals

Your other alternative is to offer them a longer project with a defined vision and some high-level goals. Write the iterative process that you want to follow and how they can always prioritise what they find important and can directly influence the development through the iterative process. Write that they will get a predictable piece of working software after each iteration that they buy. Define what happens when you don't meet the iteration's goals. Help them find out what is valuable to prioritise. Contradict your client when you know a better way.

And over time, through good and working software, predictable results and instant feedback, the client will learn to trust you.

Hold them by the hand and step by step let them build trust with you and your iterative approach.

You will first build trust with your direct contact persons in the client's organisation. Then the light of your project will shine over to other departments. Depending on your client, this takes time. Sometimes only a few iterations, sometimes months or years. But it pays off. It is so incredibly rewarding to hear from your client that others in the organisation notice how well your project is going.

Finishing Words

A colleague of mine said the best project she ever had was when the manager that hired her as a freelancer was not interested in the project itself. He just said: "this is our budget and here are the people who need a solution". She had time to understand the users and build a solution that optimally supported their workflow. After she finished the project, the manager in question got praise for such an outstanding success.

The answer to BDUF is "Just Enough Design Up Front" - Enough design to do your next few backlog items.