The James Webb Space Telescope is the largest optical space telescope ever built. It is designed to see back more than 13 billion years to the dawn of the universe. While the telescope’s functionality is meeting if not exceeding expectations, the project cost more than 10 times what it was originally expected to cost and came in 14 years late. The Webb team learned what IT has long known—the bane of project management is estimating.
Flip a coin a hundred times and half the time it will come up heads and half the time it will come up tails. Project estimates (effort, time, and cost)?—well that’s a different story. Antidotal information is that project managers are more than five times as likely to underestimate effort, time, or cost than overestimate them. Why do we underestimate so often? Well, perhaps it is in our genes. If we ignore for the moment evolutionary biology, genetics, neuroscience, neuroepigenetics, phylogeny, phrenology, not to mention the Hardy–Weinberg principle, we see how underestimation is an evolutionary advantage for our species. How many times have you said that if you knew how hard it was going to be to do something you never would have undertaken it? If Thomas Edison knew how hard it would be to invent the light bulb would he have tried? Had the U.S. Congress known what the Webb telescope would eventually cost, would they have funded it? Underestimating is a tremendous advantage for our species, for it allows the creation/discovery of things that rational minds avoid.
However, while some evolutionary traits are an advantage for the species, they can be a disaster for the individuals of that species. For example, salmon swimming upstream to spawn is good for the salmon species, but the individual fish does not survive the journey. And the praying mantis who eats her mate—well let’s not even go there. Similarly, while underestimating might be essential for the human species to advance, it can play havoc for the individual, such as a project manager.
The reality of the situation is that whatever you do, you might be destined, perhaps by some aberrant gene, to underestimate the effort required to build your system. This is the estimation conundrum. The systems development Kobayashi Maru. The no-win scenario.
There is only one practical workaround for this genetic peculiarity—feedback. The project manager needs frequent, timely, and accurate feedback as to the quantity and quality of all functionality produced along with the time and cost it took to produce it (See “Planning for the Perfect,” SD Times, March 2020). With constant feedback the project manager can finally overcome his or her genetic destiny. The number one source for understanding the scope of actual project deliverables and their costs is the post-project review (PPR).
The project’s PPR is a chapter in the project manager’s and IT’s history book. It provides necessary feedback so that the project manager can continually learn and improve project management skills. It can also be a valuable learning tool for other project managers, or would-be project managers, as to what to do, what not to do, and what to avoid like the plague.
The Internet is awash with PPR templates and sample reports free for downloading. You need only pick one and follow it. They all look a bit different, but the differences are largely unimportant if they include a traditional mathematical look at schedules, staff effort, costs, deliverables, etc. as well as two additional critical components. The first is lessons learned, a review of the project’s successes and failures. This section is the lynchpin for any hope for future project managers to learn from the experiences of those who went before them. The project manager should detail what worked, what didn’t work, and why. All subjects are fair game including tools, techniques, staff (user and IT), productivity, user availability, vendors, testing issues, working conditions (office space, technical resources, pizza delivery, etc.), and even those moments of brilliance as well as the mistakes made by the project manager.
The second critical component is recommendations for future systems development projects, where the project manager speaks to future project managers (or his or her future self), detailing what future project managers should look for and what they should do differently. Think of Dear Abby giving lovelorn advice, only here the advice is for the managers of future projects.
However, this is only possible if a robust and truthful PPR exists. Many PPR tasks are included in initial project plans but they are eaten up by the two PPR enemies: poor project planning and scope creep. Once the project manager discovers that schedules or costs will likely overrun the plan, then the hunt is on for the tasks that can be sacrificed. The PPR is often the first. The 20-person day PPR is cut to 10 days, then five days, and then completely eliminated in an IT sleight-of-hand.
There are a few critical success factors for a good PPR.
- Bundle the PPR into the Project Plan. The best way a project manager can increase the chances of a decent PPR is to bundle the review (including its schedules and costs) into the project plan. Some user managers might balk at paying for a PPR. Telling one user organization that they are being billed for a task that might only benefit another user organization is often a non-starter. If the PPR is a seamless part of IT’s development methodology, then it might very well pass user scrutiny. If it becomes an issue with user management, then the project champion might be useful in convincing the user of the value and importance of a robust PPR (See “Projects, Politics, and Champions,” SD Times, March 2022).
If user management refuses to fund the PPR then the project manager should encourage IT to foot the bill. It might take some selling (See “Half of Managing is Selling,” SD Times, November 2020) but if the project manager focuses on the benefits to IT, all might turn out well. If IT doesn’t have the funds to pay for the PPR directly, then IT might consider the cost of the PPR a project overhead item and bundle it into IT’s daily billing rate.
- Positives and Negatives. This is not summer camp where every kid gets a trophy. Lay out what went well but also point out what could have been done better. Do not defend what is not defensible. If IT management failed to have needed developers or development tools available on time, say it. If user management never provided the promised space the team needed, say it. If the project manager miscalculated testing time, ‘fess up. (Don’t worry about retaliation. When was the last time a user or IT management voluntarily read a project team deliverable?)
- Be Honest and Objective. There is no sense in going through the effort of a PPR if it becomes a puff piece (just the good things) or thin soup (a two-page Hallmark card congratulating the team). Remember all those times you went home frustrated and kicked the dog? This is the opportunity to explain those vet bills, cleanse the soul of the disappointments with being a systems developer, and help the next project’s team members with better canine relationships.
Honesty is particularly needed in understanding project productivity. Accounting can give you an accurate “dollars spent” (cost) on the project, and HR the staff hours consumed (effort), both of which might be the only numbers that senior management cares about. However, for understanding productivity, both of these numbers are meaningless without also considering the work completed (the fully developed and tested functionality of the system). If functions were eliminated or their usefulness reduced, if testing was abbreviated, or if documentation was shortchanged, and these changes are not taken into account when calculating productivity, then a false number will emerge, the cycle of poor and inaccurate feedback will continue, and any hope of project managers learning from their experiences will evaporate. Ensure that the actual functionality delivered is the basis of any PPR.
- Include Many Inputs and Many Comments From Many People. The PPR is not the project manager’s opportunity to settle scores. Every team member, IT and user, IT management and user management, should have the opportunity to add his or her comments and rants to the PPR. However, the PPR is not a social media blog. There should be an “official” opinion penned by the project manager. However, just as the U.S. Supreme Court might have a minority opinion accompanying the opinion of the majority, there should be an opportunity and place in the PPR for those who disagree with the project manager’s conclusions to post their opinion.
There are four important PPR takeaways:
- Unless you are retiring after the current project, past projects can help you perfect your skills for future projects.
- The PPR is the best vehicle for learning from past projects.
- Use the PPR to build a picture of yourself as a project manager. (a) Where do you habitually underestimate or overestimate? (b) What should you have done differently? (c) What processes, data, or personal skills would help you be a better project manager?
- Tell the truth—lying, shading, or coloring what really happened is a waste of precious project time.
Done properly the post-project review can be one of the most useful and most cost-effective tools in IT’s systems development arsenal. It will also please your dog.
George Tillmann is a retired programmer/analyst, project manager, and CIO. This article is adapted from his book, Project Management Scholia: Recognizing and Avoiding Project Management’s Biggest Mistakes (Stockbridge Press, 2019). He can be reached at georgetillmann@gmx.com.
