TL;DR

I recently re-read “How Big Things Get Done” by Bent Flyvbjerg and Dan Gardner and there were a number of topics that aligned well with my professional experience. Many of the case studies and lessons learned from the success and failure of major engineering projects are good material for both engineers and project managers to consider. The authors present 11 heuristics for better project leadership:

  • Hire a master-builder
  • Get your team right
  • Ask “why?”
  • Build with LEGO
  • Think slow, act fast
  • Take the outside view
  • Watch your downside
  • Say no and walk away
  • Make friends and keep them friendly
  • Build climate mitigation into your project
  • Know that your biggest risk is you

Unfortunately, most people do not have the opportunity to drive and determine high level project specifications and goals such as building a dam or a nuclear power plant. Based on my experience and my take working in a role at the intersection of software and hardware, . The actionable takeaways I drew from the book based on my professional experience were:

  • Planning is hard and has a long-tailed distribution
  • Iterating in the prototype phase is cheaper than in production
  • Lost time compounds
  • Say no to bloat

Although this book is not about hardware/software engineering projects, I think that the lessons learned can be readily applied to every day project planning and the authors give a good framework for decision making when trying to keep a project on track. I would recommend this book to both early career engineers looking to better understand the anatomy of a project and why things somethings work out they way they do, as well as for mid-level/senior engineers looking for affirmation for their lived experiences with good and bad projects. I lay out my main takeaways below.

1. Planning is hard and has a long-tailed distribution

Most engineers know this already, but most methods used for planning in the workplace (waterfall, scrum, Kanban, Agile, story points) each have their strengths and weaknesses and sometimes generate undesirable consequences. Ask two engineers their estimate for a given task and you will get two different answers. Scale this up to a whole team project and these errors quickly compound. Part of the difficulty with planning is that most teams have dedicated technical and project management (PM) leads for a given project;. At best, this artificially separates the timeline from the tasks and drives the need for clear communication; at worst, it can totally derail the project as the technical/PM teams have different objectives and incentives. Everyone wants to get the project done, but the key performance metrics for engineers and PM conflict. Resolving such conflict and ideally have technically inclined PMs go a long way to benefit both teams.

For all but the simplest, most concrete tasks, estimates only can serve as a guideline. A “weeklong” task could take 3 days or 7 days depending on the difficulty of the implementation and any complexity arising from interfaces with other existing or ongoing work. Commonly, exploratory work may also uncover new tasks that act as blockers that need to be done before the original task can be completed, or new tasks that need to be pushed down the road to address issues found. A common planning rule of thumb is to double or triple the initial rather pessimistic outcome; even though it may feel unrealistic, the buffer attributed by doing this comes in handy when planning meets reality and invariably hits a speed bump.

Where teams get into trouble are when project deadlines are not correlated with the estimates provided by the technical team, either due to poor planning or political pressure (internally or externally). Such planning is doomed from the start, and drives burnout when the technical team is required to “push harder” to reach an unrealistic pace to close out a project. It is not surprise that project outcomes based on poor planning or political pressure can drive the extended tail of the distribution… everyone knows someone who knows someone who worked on a multi year project that was “only supposed to take a year” which is either still ongoing many years down the road or killed off suddenly.

2. Iterating in the prototype phase is cheaper than in production

My experience with planning in CAD has taught me a lot about the old saying “measure twice, cut once”. My background planning lab instrumentation, optics and cryogenic/UHV systems in CAD has been invaluable to be able to iterate frequently before committing time and money to any particular configuration. While prototyping systems typically afford some level of tweaking, there are some systems where minor issues can quickly derail the whole assembly and require machine new components to address problems found. The beauty of CAD and other computer-aided prototyping software is in the simplicity to not only determine key dimensions, but also provide feedback to end-users to make sure that the design is functional and also will meet the needs of day to day use. I am sure that any who has had to change an oil filter on their car where reasonable access to the filter for routine replacement was not considered in the planning phase would have a thing or two to say to the automotive engineers in charge of designing it.

Another benefit of iterating in the prototype phase is that the feedback loops are much tighter, so end-to-end changes can be tested, validated and feedback can be obtained to drive future design. Before committing significant time to a project, proof of concept work plays a significant role in burning down risk and highlighting the necessary milestones for a project.

On the software side, systems level design and black box abstraction is important to determine key points of failure or bugs arising at interfaces. This can help reduce unnecessary complexity and streamline the process to reach the core features and necessary infrastructure.

3. Lost time compounds

I like to think of lost time as the opportunity cost of compounding growth and development. For hardware prototyping, time lost in manufacturing, procurement and shipping can add up quickly. This is one of the main reasons why A/B type testing can be helpful to explore the design phase space with multiple concepts at once to help nail down the final design in parallel rather than serially.

The other side of the coin is that delays compound much as interest on a loan might. Time is a finite resource, and time spent idle is time that cannot be put towards engineering and improvement of the core product or processes. The downstream effect is that delayed features means delayed revenue, and delayed revenue eats into profitability and cash flow. Doing things quickly also comes with its own costs, but for fast moving engineering companies a careful consideration of the trades should be made with an eye on the future. While well funded VC backed firms can easily buy time back with hiring talent and throwing money at the problem, I do not think that the unicorn startup playbook is the right one for most companies. Proper planning can mitigate the worst effects of lost time, but my main experience is that outside of an established workflow or manufacturing process, spending money to avoid lost time usually works out to be the better solution.

4. Say no to bloat

The last topic I want to touch on is bloat. This can happen in both hardware and software where eventually new proposed features begin to reach that critical point of diminishing marginal utility. The best thing from a project execution and engineering sanity point of view is to say no to bloat. I want to distinguish here between normal features and less well-defined and sometimes more spontaneous “bloat”. A normal feature goes through the planning feature and should have a reasonable time estimate with some preliminary work done to scope out what exactly is required; “bloat” occurs when someone thinks they have a good idea on paper, but quickly becomes a tarpit of time and resources to disentangle once the task begins.

There are many times on good teams where discussions can develop key new features or make more out of existing work; however, I believe that even the best ideas should be fleshed out and get red teamed to ensure that they really are good ideas and do not fall apart upon closer inspection.