Here's What's Wrong with Science...

[Clodfobble]

...Most studies are designed by stupid people.

I've often thought that my ideal job, the one where I could contribute the most to humanity, would be to sit on a committee that reviewed proposed scientific studies before they were done, and had the opportunity to point out the logical flaws or complicating factors in their design, and helped them redesign them to provide more accurate or meaningful results.

Case in point: this recent study, that claims to have found a trend whereby people with a "negative gut reaction" to a picture of their new spouse are more likely to be unhappy/divorced in the coming years than those with a "positive gut reaction."

Big design flaw: they showed these people a picture of their spouse, and then measured whether they could pick out positive or negative words more rapidly. But they didn't find out whether these same people tended to pick out positive/negative words without being shown a picture of their spouse. My suspicion is that those people with supposed "negative gut reactions" to their spouses were just more negative overall, and a negative person is less likely to have a successful relationship. This would have been an easy thing to control for in the experiment, either by testing the participants in multiple rounds, or by splitting them into a test group and a control group that was not shown a picture of their spouse. As it is, the study is a waste.

Awhile back I had a long email exchange with a graduate student, who asked me point-blank why I had participated in her study, and my opinion on why most people were refusing. I told her the honest truth, that I only finished it because I have a very ingrained need to finish what I've committed to starting, and that the whole time I was thinking what a stupid study it was. It was sort of trying to determine the relationship between poor sleep and autism, but again had many major design flaws, which I detailed for her. Meanwhile I've participated in literally dozens of other autism studies that I can see have smaller but no less fatal design flaws.

I don't really have a point here, I'm just venting a little. A good study is a thing of beauty, but they are so rare.