Informative little book. Suffers from a massive problem: data.

What if I told you that I could fix global warming by just finding and releasing unobtanium into the atmosphere? Easy, right? Except, where do I find unobtanium? Where do I find the valuable data that will guide my project towards the light?

This is the book’s biggest weakness. You see, among all of the heuristics the author shares to make project planning better, the one that is supposed to make the biggest difference is “Reference-Class” estimation. This entails seeing your project not as a special snowflake, but as “one of those”. This, the author explains, is the “outside view”. By employing this technique you are more likely to put your project in a reference class where, presumably, you will be able to find data regarding the many aspects of your project and how long they usually take, on average. From here, you can adjust your estimates accordingly based on your particular knowledge of the details, i.e. using your “inside view”. This is supposed to improve estimates dramatically.

But as I said before (and as the author confesses to), data is not easy to come by. Even in the chapter where he is discussing kitchen renovations, we discover that such data does not in fact exist. So you’d have to resort to calling people that have done kitchen renovations and ask them, if you can find them and if they will talk to you.

But some of his other advice is a little idealistic or great in theory, but ultimately downright impractical, too. For instance, you are supposed to “Think slow and Move fast” and use “Pixar planning”. This translates to: spend lots of time planning, experiment and simulate as much as you can, and deliver with speed to reduce the size of the risk window.

The problem with the above is that you won’t always be able to do this. Some projects, like those common in the SaaS and Enterprise software sector, do not follow this lifecycle and instead “waterfall” down from the stakeholders to the project/product managers, who will have to execute on a moving-target of a vision, and on predefined timelines.

So what should you do, according to the author? Commit to not commit or just walk away. Eh…I have a mortgage and a two-year old at home, I can’t just drop everything and pack my bags, and I am willing to bet the case is the same for most of you reading this.

Oh, I forgot, he also suggests you “hire a master builder” (i.e. an expert - and have him bring his team) and get your team right! Well, what if you can’t afford an expert, can’t find one, or if he is already booked? And the team? Yeah, I’d love to start fresh and rebuild my team, but then I’d have to re-hire, re-train, etc. Sometimes you have to use the tools that you have. Again, impractical.

There’s one more issue that stems from data that I think this book suffers from, and that is Survivorship bias. For all the talk about all the projects the author was involved in that used his methods of “thinking slow, moving fast”, using reference-class estimates, hiring experts, thinking from right to left, etc., there is little talk about all of those projects that did but still managed to go over time, over budget, and under delivered. Or worse, just failed miserably. For all the projects he cites that live in the fat-tail of the time/budget distribution (i.e. the extreme cases), there’s no mention of the proportion of those that got there even though they used the methods outlined in this book. The data is not offered, or is strategically omitted.

But that’s enough complaining. Here’s what I found useful and relatable about the book as per my experience working on multi-million dollar tech projects. For starters, it’s the use of “wrong anchors” in conjunction with the “rush to commit” to derive bad estimates. I have literally sat in meetings where someone trying to look good in front of the C-Suite overpromised on timelines and planted a bad anchor in the head of the powers-that-be ("one week!"…), which ultimately caused everyone to look like the delivery was botched. The delivery was good, the estimate was crap. Once a nice, round time estimate is uttered, everything else will be modeled from there. I will make sure to nip this in the bud going forward and openly question any and all anchors provided without care.

I can also relate to the modularity argument and “finding your LEGO”, that is, seek to build small and scale up, rather than building one massive thing in a long, giant effort. I will be seeking to do a better job implementing this in future projects with my team. As the author explains, custom/bespoke is expensive and does not build expertise in the way that modular building does, since you get to repeat the same process time after time and leads to “positive learning” (i.e. things get easier as you learn more).

Another area my projects can improve on is making sure we don’t marginalize experience, intentionally or otherwise. While we have very experienced people in my company, we have not done a great job availing ourselves of their experience, either because we thought we could do it without them, or because they were swamped with other work. This chapter was done very well.

Overall, I think the author has spent too much time researching and working on mega projects, and his perspective has suffered a bit because of it. Mega projects have massive budgets and can afford to hire the best and brightest, among other useful things. I wouldn’t be so critical if the book was meant as a study of only big projects; it’s supposed to be all-encompassing. But it’s still a very informative book and worth the time to read it if you can adapt some of the lessons, even if just partially so.

Recommended.