Resulting and Credibility Scores

Last year, I read Annie Duke’s latest book, “Thinking in Bets“, and liked it so much I gave it to all my close friends as a holiday gift. It’s a life book as much as it is a biz book.

If you are not familiar with Annie, she was for years the winningest female in the World Series of Poker and, since retiring from playing in 2012, now works with UPENN doing research around Decision Science.

The book isn’t a poker book and doesn’t even over-index on poker stories. It’s a collection of research and anecdotes surrounding common mistakes in how we all make decisions and evaluate their outcomes.

I took a handful of things from the read, but two really stuck with me. The first one being the ability to clearly recognize and hopefully avoid “resulting” when I can.

Resulting is the common trap in which we erroneously offer credit or blame based on the outcomes of the decisions that were made. People tend to falsely believe that all necessary information was known and/or static when decisions were being made. When stated that plainly, it is easy to see the inherent issue.

Most if not all complex decisions rely on incomplete or uncertain information as well as some elements that are changing. All needed information typically can’t be known or may evolve, for any number of reasons, over time.

Annie encourages us to ignore outcomes and instead evaluate the decision-making process based on what we know and can control. That could include our process, research, thoroughness or the evaluation of multiple orders of consequences leading to an outcome. It might also include speculation about what is unknown or the assignment of probabilistic values to the elements which could change.

Excellent decision makers can be unlucky and people who wing it can get it right here and there. The point is to ensure credit or blame is placed appropriately based on the quality of the process for deciding and evaluating the expected outcome. In other words, we should put the greatest value (and potential incentive) toward consistent and excellent forecasting.

My second major takeaway is the “Credibility Score”, which I’m not sure is her words or if I’m paraphrasing at this point. If “Resulting” is a macro framework which can be used often, I consider Credibility Score the micro framework which can be used daily.

Tons of research (internet) shows that people will overvalue information which is shared by another person (or the media). Another bucket of research (internet) shows that people want to be helpful because 1) many are inherently good and 2) because it makes them feel good. Unfortunately, a third bucket of research (internet) shows that in an effort to be helpful and feel good lots of people will just make crap up on the fly instead appropriately saying “I don’t know” when they are out of their depth.

Most of us are guilty of listening to others more than we should. It’s easy at times. It’s even easier when it’s an opinion from your boss, spouse, parent, friend, board member, investor, co-worker, etc. who you respect.

We’re also all guilty of confidently suggesting things in a friend, family or business setting when we should really be saying, “I’m not sure”, “I haven’t actually done that”, “I don’t know”, “This is just my hunch” or “Let me find out more and get back to you”.

Being on either side of the ball can create real problems for you, the people you care about or that you are accountable to.

Taking all of that into account, these days I try to affix a credibility score to the information I share or receive from others. When on the receiving end, I ask “On a scale of 1-10, how certain are you?” Followed typically by a “Why did you score it a #?” I’ve even seen people back off their original number when I ask them to back it up. Very useful.

On the giving end, I try to proactively offer a, “Remember I’m an N out of 10 confidence level on this because I have/haven’t done X amount of it.” I try to be religious about this when speaking from the adviser, board independent or CEO coach seat.

I don’t want people I care about to undervalue or much more importantly, overvalue my advice in their decision-making process. Instead of just a rough “Remember, it’s just my opinion”, I want to offer a more useful numeric or contextual score.

I have to believe that the skeptics out there have a built-in credibility scoring mechanism. For those like me who tend to be more optimistic in their approach, the credibility score is a great check and balance on a potential blind-spot.

These two people patterns are so simple and easily understood and yet they cause issues in so many places well beyond the office. Once you can spot them, however, they are not that tough to work around via a little extra diligence and/or some checking of the ego.

If you want more of Mrs. Duke and are being lazy, she popped up on the a16z podcast this spring and frolicked further in these weeds for a solid hour. It’s a great listen.

Thanks for retraining my brain a bit, Annie. Useful to say the least.