of mice and men

Apologies to fellow northern Californian (Steinbeck) for the title of this post, which is about motivation and reward structures.

For the uninitiated, let me first give you a quick run down of a typical day at the office for a grad student:

8-9 am: check email, scan the contents of my favorite journals for any new papers of interest
9am-noon: look over the results from the previous day's experiments or simulations (often, these run overnight). Usually, this is when you realize that your experiment failed (or your simulation crashed, or whatever).
12-1: Lunch! Read some of the papers that I found in my quick morning scan. Be amazed by how smart the paper-writers seem to be.
1-4 pm: set up more experiments (or simulations). Most of this time is spent de-bugging, figuring out why the thing isn't working
4-5 pm: go to a lecture by a visiting scientist. Be impressed by how smart (s)he is.
5-7 pm: commute home, make dinner, eat dinner, make conversation with housemate(s)
7pm-midnight: think about science, either actively or passively (maybe brainstorming in a quiet room, or watching TV).

Now, you'll notice that nowhere in this typical day is there "Eureka! I understand the brain now!" You'll also notice that the typical day also doesn't contain "win a prize for being awesome" or "get compliments on how smart you are" or anything resembling a "reward" that would motivate getting out of bed and putting forth your best scientific efforts.

To understand why myself (and my colleagues!) keep getting up to go to work in the morning, let's consider an old experiment by a guy named B. F. Skinner. In his experiments, he put a mouse in a box with a lever. When the mouse pushed the lever, he (let's assume it's a male mouse for now) may or may not get a food pellet as a reward.

If you give him a pellet with every lever press (consistent reward), he learns that the food is there waiting for him, and he presses the lever sometimes. No surprises here.

If you never give him a pellet, he learns to not bother pressing the lever. Also unsurprising.

So, what happens if you sometimes give him pellets for lever presses? You might guess that the result would be somehwere in the middle: he presses it less often than when the reward is consistent, but still sometimes. If you did make that guess, you would be wrong. Very, very wrong!

Here's the interesting part: if you give the mouse pellets for some, but not all lever presses, he learns that pressing the lever is good, but that he can't just rely on the lever giving him food. The result? The mouse frantically presses the lever, over and over again.

These experiments give a lot of insight into motivation. For the scientist, even though most days are pretty frustrating, the rare day (maybe one in 100 if you're really successful) when you win a grant (or fellowship, or whatever), or discover something new and exciting, are just frequent enough to make you keep doing it in the interim. To complete the analogy, scientists are mice, their labs are Skinner's boxes, and their lab equipment is the lever.