Dead salmon sensing human emotions...(?)

As anyone who know me will testify, I am a strong proponent of the scientific method. It is quite simply the best available method we have to gain knowledge about the universe, and it has a fantastic track-record.

Theoretically, science is I think, flawless. However, science is done (mostly) by scientists, i.e. people, and we all know that people are, in general, not quite perfect. This is why scientists are so obsessed with writing out the method. Doing so means that other scientists can replicate the experiment and see if they get the same results.

One excellent replication of a scientific method was done by Bennet et. al. Bennet and his colleagues were concerned that a popular statistic method used for fMRI data actually produced statistically significant results that did not represent any real activity. fMRI, in short, is a method that that measures blood flow in the brain, which in turn, is a measure of neural activity (because brain regions with active neurons will consume more oxygen which is delivered by the blood). A statistically significant result is a result that is very unlikely to happen by chance alone.

As mentioned Bennet et al were concerned that a popular method for analyzing fMRI lead to unwarranted conclusions. To prove this he took a trip to the local market were he bought a Salmon, a dead, frozen salmon. He took this fish back to the laboratory and put it in the fMRI machine. The salmon was then shown pictures of situations depicting different emotions (e.g. anger). Bennet then used the fMRI data to see if there were differences in the dead and frozen salmon brain, depending on what type of situation it had just seen, and guess what, there was! Does this prove that dead and frozen salmon have the ability to see what type of emotion a particular situation depicts? No, of course not. Rather, the experiment is an elegant way of showing that the statistical method used lead to invalid conclusions.

See the poster, which is actually quite funny, here.

In a second example appeared in the excellent journal "Psychological Science". This article was perhaps more worrying since the the author Simmons and co-workers reached absurd conclusions using perfectly valid mathematical/statistical methods. Specifically the authors showed that when you listen to music about old age, you become younger - you don't feel younger, you actually get younger. They reached this conclusion simply by varying some decisions about the analysis, after the experiment was done, which is something I believe many scientists do. Ideally you should decide which exact tests you will use before the experiment and then stick with that analysis - if you find unexpected results these have to be checked in another experiment.

The good news is that science is self-correcting i.e. science, unlike say religion, eventually detects its own error, and it is also worth noting that in both examples above, scientists, following the scientific method, found the problems and reported them.