## MEASURING PROBABILITIES USING A CALIBRATION EXPERIMENT

Probabilities are generally hard to measure. It is easy to measure probabilities of events that are extremely rare or events that are extremely likely to occur. For example, your probability that the moon is made of green cheese (a rare event) is probably close to 0 and your probability that the sun will rise tomorrow (a sure event) is likely 1. But consider your probability for the event "There will be a white Christmas this year". You can remember years in the past where there was snow on the ground on Christmas. Also you can recall past years with no snow on the ground. So the probability of this event is greater than 0 and less than 1. But how do you obtain the exact probability?

To measure someone's height we need a measuring instrument such as a ruler. Similarly, we need a measuring device for probabilities. This measuring device that we use is called a calibration experiment . This is an experiment which is simple enough so that probabilities of outcomes are easy to specify. In addition, these stated probabilities are objective; you and I would assign the same probabilities to outcomes of this experiment.

The calibration experiment that we use is called a chips-in-bowl experiment. Suppose we have a bowl with a certain number of red chips and white chips. We draw one chip from the bowl at random and we're interested in

Probability(red chip is drawn)

This probability depends on the number of chips in the bowl. If, for example, the bowl contains 1 red chip and 9 white chips, then the probability of choosing a red is 1 out of 10 or 1/10 = .1. If the bowl contains 3 red and 7 chips, then the probability of red is 3/10 = .3. If the bowl contains only red chips (say 10 red and 0 white), then the probability of red is 1. At the other extreme, the probability of red in a bowl with 0 red and 5 white is 0/5 = 0.

Let's return to our event "There will be a white Christmas this year". To help assess its probability, we compare two bets -- one with our event and the second with the event "draw a red chip" from the calibration experiment. This is best illustrated by example. Consider the following two bets:

• BET 1: You get \$100 if there is a white Christmas and nothing if there is not a white Christmas.

• BET 2: You get \$100 if you draw red in a bowl of 5 red and 5 white and nothing otherwise.

Which bet do you prefer? If you prefer BET 1, then you think that your event of a white Christmas is more likely than the event of drawing red in a bowl with 5 red, 5 white. Since the probability of a red is 5/10 = .5, this means that your probability of a white Christmas exceeds .5. If you prefer BET 2, then by similar logic, your probability of a white Christmas is smaller than .5.

Say you prefer BET 1 and you know that your probability is larger than .5, or between .5 and 1. To get a better estimate at your probability, you make another comparison of bets, where the second bet has a different number of red and white chips.

Next you compare the two bets

• BET 1: You get \$100 if there is a white Christmas and nothing if there is not a white Christmas.

• BET 2: You get \$100 if you draw red in a bowl of 7 red and 3 white and nothing otherwise.

Suppose that you prefer BET 2. Since the probability of red in a bowl of 7 red and 3 white is 7/10 = .7, this means that your probability of a white Christmas must be smaller than .7.

We've now made two judgements between bets. The first judgement told us that our probability of white Christmas was greater than .5 and the second judgement told us that our probability was smaller than .7. What is our probability? We don't know the exact value yet, but we know that it must fall between .5 and.7. We can represent our probability by an interval of values on a number line.

```     OUR PROBABILITY OF WHITE CHRISTMAS LIES IN HERE:

*********
+----+----+----+----+----+----+----+----+----+----+
0   .1   .2   .3   .4   .5   .6   .7   .8   .9    1
```
What if we wanted to get a more accurate estimate at our probability? We need to make more comparison between bets. For example, we could compare two bets, where the first bet used our event and the second used the event "draw red" from a bowl of chips with 6 red and 4 white. After a number of these comparisons, we can get a pretty accurate estimate at our probability.
Page Author: Jim Albert (© 1996)
albert@bayes.bgsu.edu
Document: http://www-math.bgsu.edu/~albert/m115/probability/calibration.html