Alex Reinhart has written a fantastic article entitled “Statistics Done Wrong” which goes into a lot of the mistakes that scientists made when conducting experiments and performing data analysis. Reinhart is a 20 year old physics major who says that while science has embraced the use of statistics it hasn’t embraced statistics education, often allowing undergraduate students to complete a science degree without taking any statistics classes (it was possible to take a Bachelor of Applied Science at QUT without even taking the first year data analysis unit, you just had to be prepared to do first year chemistry, life sciences and introductory physical science).
The article is very good and goes into problems of insufficient statistical power, misunderstandings as to what p values are and what significance is, pseudoreplication when you think you’re doing independent replication, and how big a sample to use.
I am not as well versed in statistical design as I would like to be, particularly when working in a lab which does experimental science. One of my supervisors is very passionate about good design, and for good reason; if you design your experiment or observation campaign well you will have a more efficient data collection system, an easier job doing the data analysis and the power of your tests means that you can be more confident in your analysis.
As dull as statistics seems to be for first year science students, I could see it being very valuable to run a course on how to set up an experiment, both from a statistical design point of view and what is required of them in their field, and perform data analysis. I don’t think we can expect experimentalists to become great statistical teachers, so it may require an amount of service teaching by statisticians within the science units, as an optional class on basic data analysis techniques doesn’t seem to cut it.
I really enjoyed the article and think it’s a great follow up to what I was talking about at Healthy Buildings. The next step is to identify how we teach these things to science students in a way that they don’t switch off.