Consistency of the PMO

I read a very interesting article today from the 2018 Edition of the Harvard Business Reviews’ “10 Must Reads” book on Noise.

There are some SERIOUSLY talented people in as writers on this article, and the book in general. Daniel Kahnemann is a Nobel Prize Winner.

The book itself is a series of “short stories” that take real life dilemmas and use analytical approaches to a potential solution. You can apply the stories and solution to your industry as you see fit.

The reason that this article in particular hit home was that it dealt with a topic that we all know exists, yet choose, irrationally, to ignore.

In a previous article I discussed “Early Warning Signs in your Project Plan” and “Project Risks and Issues – Properly quantifying across the program“, yet I didn’t deal with the cause. I ignored the need that exists that makes those articles necessary.

I felt totally shocked that an analytical person like myself dealt only with the symptoms, yet not at all with the root cause. And I believe that the book deals with 3 specific aspects of an IT PMO that we are especially susceptible to, yet do nothing to mitigate.

Noise, Bias and Drift.

Let’s say one of your team members comes to you on Monday morning and asks for a decision from you, then imagine that the same person came to you with the same decision on Friday afternoon. We would all have to admit that it is possible that our decision may vary based on factors that have nothing to do with the facts involved.
This is called Noise.

Let’s say that you are doing a presentation to a group of people about a portfolio of projects. Some of them you are directly involved in. If you are asked which of the projects you believe will be successful and provide benefit, it is most likely that you will choose a project that you are directly involved in.
This is called Bias.

Let’s say that you setup your PMO, you give training on how Project Managers should fill in their weekly reports. Life is good. After time 3 of the 10 Project Managers are new and did not receive the training, they begin coping with the process of the weekly reports, they begin to interpret the requirements differently and insert bias rather than logic. They are not following the method, the tools and the process.
This is called Drift.

The book proposes, and I completely agree, with algorithm based approaches to making decisions.

Not all decisions can be moved to algorithm based methods, however in order to remove Cognitive Bias, and reduce the effect of Noise, certainly in the area of the IT PMO we can put in place a much more checklist, analytically oriented approach to the Program Office.

The PMO’s job is not only to collect status and simply pass it up the chain, yet many PMOs lack the technical ability to do more than this, the PMO is also responsible for ensuring that the information collected is obtained under the same set of assumptions, methods and analysis and is relatively free of Noise and Bias.

The PMO is also responsible for keeping those methods up to date and consistent out into the Project Management Community such that the effects of Drift do not influence the collection and assessment of Project Delivery information. This can be achieved via training on tools and methods.

The article outlines that the effects of Noise can drive inconsistency in decision making to an average of 50%! – As an executive, are you SURE that the information you are reading and the decisions your people are making are consistent?!

In pre-audit interviews, executives stated that the inconsistency rate they believed would come out of the assessment would be between 5%-10%. In addition, the level of Noise did not reduce with more years experience on the job. In fact, with more experience comes a greater degree of Confirmation Bias and a greater risk of Drift.

I won’t spoil the book for you however, mitigating actions such as

  1. using check-lists and workflow documentation to facilitate decision making
  2. using supporting algorithms
  3. training on tools and methods by the PMO
  4. training to detect bias in yourself and others
  5. eliminating false positive behaviours (e.g. cheaper is always better)
  6. enabling external decision making monitoring
  7. welcoming the review process
  8. understanding how to recognize when a decision impacts you personally

… should be put in place to ensure that the PMO continues to provide a valuable service to the Organization.