prior prior pants on fior (sp.?)

This post will be about the application of prior knowledge in decision making. I'll start with a very simple example involving coin flips, then move on to discussing the results of a very cool cognitive science experiment.

So, here's our example. Let's imagine that you and I are betting on coin flips. We each wager $1, and you get to flip the coin. If it lands on heads, I win your dollar (and get mine back), while if it lands on tails, you get my dollar (and you get yours back). Simple, right. Good. Now, I pull out from my pocket a US 25-cent coin, and we start flipping coins. It lands on heads on each of the first 3 flips, giving me a profit of $3.

Now, before we go any further, you want to estimate the probability that any given flip will land on heads versus tails. From the data available in our (3-flip) experiment, it looks like the coin always lands on heads. So, based purely on that information, you should stop betting, and you should call me a cheat! Is that really the best course of action?

Intuitively, you know that it's not so simple. We all know the exact sequence of flips we saw would happen 12.5% of the time, if we were using a fair coin. And, we've seen enough coins before in our lives to expect that they are fair (land heads or tails roughly equally often). And that's the crux of the issue: you have some prior knowledge about coins that tells you not to rush to hasty decisions.

Now, I promised you a very cool cognitive science experiment, and I'm going to deliver just that. Here's the experiment. Tom Griffiths (now a Berkeley prof) asked a bunch of human subjects (randomly selected undergrads) questions, such as (quoted from Griffiths and Tenebaum's paper)

"Imagine you hear about a movie that has taken in 10 million dollars at the box office, but don’t know how long it has been running. What would you predict for the total amount of box office intake for that movie?"

or

"If your friend read you her favorite line of poetry, and told you it was line 5 of a poem, what would you predict for the total length of the poem?"

In statistics, if you know the distributions of, say, lengths of poems, it's a fairly straightforward (Bayesian inference) problem to calculate the answers to these questions. But Griffith's subjects were not stats wizards, and they didn't have time to calculate, and they were not provided with the distributions. Furthermore, they were explicitly instructed to make intuitive guesses, not calculations.

Shockingly (to me, anyway), the subject's answers (on average) match the statistically optimal predictions!

So, somehow, your brain automatically "knows" all these statistics distributions from your everyday experience. And, when you make seemingly random intuitive guesses about stuff, your brain draws on that information to make (statistically) the best possible decision.

Not too bad for a giant lump of fat.