The American Statistical Association Says [Mostly] No to p-values

Norman Matloff has published a new post after receiving criticism and comments about stating “The ASA says No to p-values” in his post I wrote about yesterday. He defends his interpretation in this new post. However, I think, the interpretation of  the statement in a context different from the “Big data” field to which he is used to does not need to always be “Says No to p-values” but instead in many cases could be “Use p-values to assess the strength of the evidence and nothing else”.  However, “tests” with binary outcomes on probabilities that are essentially continuous, will always be based on an arbitrary threshold and discard a great deal of information. Consequently to me using as suggested by Norman Matloff  “assess” in place of “test” makes a lot of sense.

The new post is at https://matloff.wordpress.com/2016/03/09/further-comments-on-the-asa-manifesto/

The American Statistical Association Says No to p-values

Norman Matloff (2016) writes in his post:

Sadly, the concept of p-values and significance testing forms the very core of statistics. A number of us have been pointing out for decades that p-values are at best underinformative and often misleading…

Source: After 150 Years, the ASA Says No to p-values | Mad (Data) Scientist


Yesterday, the statement by the American Statistics Association was published on-line in the journal “The American Statistician”. Many statisticians have been aware of the problems of significance tests for a long time, but general practice, teaching and journal instructions and editors’  requirements had not changed. Let’s hope the statement will start real changes in everyday practice.

John W. Tukey (1991) has earlier written quite boldly about the problem:

Statisticians classically asked the wrong question—and were willing to answer with a lie, one that was often a downright lie. They asked “Are the effects of A and B different?” and they were willing to answer “no.”

All we know about the world teaches us that the effects of A and B are always different—in some decimal place—for any A and B. Thus asking the effects different?” is foolish.

What we should be answering first is ”Can we tell the direction in which the effects of A differ from the effects of B?” In other words, can we be confident about the direction from A to B? Is it “up,” “down” or “uncertain”?

The third answer to this first question is that we are “uncertain
about the direction”—it is not, and never should be, that we
“accept the null hypothesis.”

Disrupting the subscription journals’ business model for the necessary large-scale transformation to open access 

Author: Schimmer, Ralf et al.; Genre: Paper; Published online: 2015-04-28; Open Access; Title: Disrupting the subscription journals’ business model for the necessary large-scale transformation to open access

Source: Disrupting the subscription journals’ business model for the necessary large-scale transformation to open access :: MPG.PuRe

An interesting viewpoint on the future of scientific publication, and assessment of its cost.

Lovelacen kreivitär kirjoitti tietokoneohjelman, ja häntä opasti Mary Somerville (Hessari 7.11.2014)

Today there was an interesting article in Helsingin Sannomat.

http://www.hs.fi/ulkomaat/a1415168967244

The book by Mary Somerville, that is mentioned in the article is in the public domain and available, as many other old and interesting books, through the Internet Archive.

https://archive.org/details/onconnexionphys00somegoog

I was particularly impressed by the preface written in the 1830’s:

Mary_SomervilleIn one of the comments in HS a reader writes that programming is boring… I disagree, just coding may be boring, but designing software and algorithms is anything but boring!

Another reader correctly says that Ada is not a super-computer langauge. In a way it was ment to be when it was designed, but in the sense of being a tool for creating reliable and bug-free software. However, as some other languages designed by a commitee it ended being too complex and inconsistent, and because of this difficult to use as a general purpose language.