Travels around my prostate

If you are over 50 and male this could be the most important post you read this year.

In October 2013 I had just finished reading The Signal and the Noise: The Art and Science of Prediction by Nate Silver. The book discusses a how a diverse set of forecasts ranging from politics, baseball and the weather are prepared, the errors that are often made and how in many cases 'expert predictions' should be treated with many grains of salt.

However a real strength of the book is the description of Bayesian reasoning in Chapter 8 which is a technique every manager should learn. A lot of management effort and time is spent altering forecasts as new information is received. Unfortunately most of us just apply such information intuitively. Bayes allows to you make better predictions. This book made me really learn about Bayes Theorem. I am the first to admit that although I had listened to lectures about Bayes at both Cambridge and London Business School I have never really sat down and learned it.

Simply put one starts out with a prior probability. One then takes the probabilities of a new event being either true or false and then calculates a posterior probability. Say x equals the prior probability, y equals a new event probability that is true, and z equals a new event probability that is false. The posterior probability is xy over xy plus (1-x)z. The secret to Bayes is calculating both the true and false positives. An example will make this much clearer.

Say for example there has been an accident in a city involving a taxi cab.

* 85% of the cabs in the city are white, and 15% are silver.

* A man identified the cab involved in a hit and run as silver.

* The court tested the witness' reliability, and the witness was able to correctly identify the correct color 80% of the time, and failed 20% of the time.

What is the probability the taxi cab was silver?

Here's how we figure it out using Bayes theorem.

If the cab was silver, a 15% chance, and correctly identified, an 80% chance, the combined probability is .15 * .8 = .12, a 12% chance. These are true positives.

If the cab was white, an 85% chance, and incorrectly identified, a 20% chance, the combined probability is .85 * .2 = .17, a 17% chance. These are false positives