Acting Pro-Vice-Chancellor of the University of Queensland Alastair McEwan has released a brief statement on the Cook et al paper.
McEwan states “all data” of ‘any scientific value’ has been released by the authors. This is not true. Rater ids and their time-stamps have not been released.
McEwan also states some material was not released: information that could “identify the individual research participants was withheld.”
This information was not withheld. Volunteers are listed by name as authors and in the acknowledgements in the paper.
Furthermore, volunteers who rated abstracts are not ‘research participants’ in the paper. If they were, Cook et al 2013 would not be a survey of the literature. It would become a survey of volunteers.
A commenter at Hiizuru writes:
So are they admitting that this “research” was actually a study about the raters skills to classify scientific papers according to a set of parameters and not about the actual content of the papers at all?
Richard Tol wrote to Vice-Chancellor Peter Høj a year ago. Declaring volunteer identities confidential would mean the paper was a ‘survey of the raters’ beliefs and skills, rather than a survey of the abstracts they rated’.
If, on the other hand, this was a survey of the raters’ beliefs and skills, rather than a survey of the abstracts they rated, then Mr Cook is correct that their identity should remain confidential. But this undermines the entire paper: It is no longer a survey of the literature, but rather a survey of Mr Cook and his friends.
Rud Istvan notes the contradiction: if volunteers were research participants according to the institutional ethics review, the authors themselves were in breach of it. They wrote about their own work as participants!
In reality, these absurdities are easily resolved. Cook et al 2013 is not a survey of ‘Mr Cook and his friends’. It is a survey of the literature, as the title declares.
What the authors did was not survey themselves as participants but apply a uniform rating scale on abstracts. Is this process reproducible and valid? The question carries scientific value. The answers may potentially overturn the paper.
Answering this question requires not the identities of the volunteers but of the abstracts they each rated.