The IPCC and grey literature
But that is how many references in the IPCC report, an ‘open letter from scientists’ told us, were from ‘peer-reviewed scientific journals’.
The reference list of the AR4 contains about 18,000 citations, the vast majority of which were published in peer-reviewed scientific journals.
This letter was signed by such climate luminaries as Michael Oppenheimer, Kevin Trenberth, Jonathan Overpeck, Gerald Meehl and Stephen Schneider.
The mind-boggling propaganda website realclimate.org reproduced the claim:
The IPCC cites 18,000 references in the AR4; the vast majority of these are peer-reviewed scientific journal papers.
Any non-amnesiac would remember vividly that these are not the only occasions, the club of ‘peer-review’ was brandished at anyone asking questions about the IPCC. You can look here, for example, to refresh your memories.
Even post-Climatagate, the ‘peer-review’ fetish-mongering hadn’t stopped. Strain your eyes and watch the video below, you might see what Mark Steyn thought was surely there:
Nothing to worry about, folks. “We’ll go down the path and see what happens in peer-reviewed studies,” said Ed airily. “Those are the key words here, Stuart. ‘Peer-reviewed studies.'”
Hang on. Could you say that again more slowly so I can write it down? Not to worry. Ed said it every 12 seconds, as if it were the magic charm that could make all the bad publicity go away. He wore an open-necked shirt, and, although I don’t have a 76″ inch HDTV, I wouldn’t have been surprised to find a talismanic peer-reviewed amulet nestling in his chest hair for additional protection.
Such tirades however, were praised by a famous climate scientist.
The problem with ‘peer-review’ and the IPCC
The problem is that we now know — the overbearing drivel about peer-reviewed scientific journals forming the basis for the IPCC — is simply not true. A basic quality assurance of scientific claims provided by peer review—just did not exist for a significant chunk of the IPCC’s material.
Bibliometric analysis of the third assessment report by Andreas Bjustrom showed the following:
IPCC TAR consists of 14,000 references, of which 62 % are journal references (see Table 1).
Which means 38% of the references are from the non-peer-reviewed ‘grey literature’ category. The situation is the exact opposite of the claims
Things are no different with the version released in 2007. A project led by Donna Laframboise, analysing of the IPCC fourth assessement report showed the following:
5,587 references in the IPCC report were not peer-reviewed. Among these documents are press releases, newspaper and magazine articles, discussion papers, MA and PhD theses, working papers, and advocacy literature published by environmental groups.
What was more—in 21 out of 44 chapters in the IPCC report, 40% or more of the references cited were not peer-reviewed.
The picture gets worse the closer one looks. Only 39% of the references cited in the entire chapter on Latin America which contained the notorious Amazongate WWF quote, are from the scientific peer-reviewed literature.
What do we make of this?
The inference is quite clear – the IPCC, working as it were, in areas with sketchy and threadbare primary research, has served a vehicle and a trojan horse for non-substantive, potentially exaggerated claims originating in scientifically dubious publications to queue-jump and attain institutional status. This of course is nothing more than judgement one reaches, applying the same standard the IPCC boasted it had attained, on its own material.
A focused examination reveals a bit more about the specific nature of material employed by the IPCC which contained most of its ‘erroneous’ exaggerated conclusions – Glaciergate, Amazongate, Africagate and so forth.
All these errors arose from material sourced from one specific type of grey literature – environmental pressure group literature.
One can perhaps look the other way as the IPCC employs government report-derived data or quotes newspaper articles to refer to specific events.
But what excuse can the IPCC offer to sourcing scientific material from reports of organizations whose stated missions is to enhance public perception of risk and danger from specific environmental issues?
A bizarre situation
There can be none, but yet there is a great reluctance from the IPCC to give up its addiction to such sources.
For instance, after taking quite the beating over the IPCC sourcing material from World Wildlife Fund (WWF) reports, RK Pachauri said in May,
“The media and several other people have completely misunderstood the need for using non-peer-reviewed literature. The loose term that is used is ‘grey literature’ as though it was some form of grey muddied water flowing down the drains,”
“But I’d like to highlight what non-peer-reviewed literature constitutes: reports from the reports from the International Energy Agency, the OECD, the UN Food and Agriculture Organization, the World Bank, the Asian Development Bank… and some NGOs – there are some highly prestigious NGOS that are doing detailed academic work, and you cannot ignore this.”
Reports by non-governmental organizations like the WWF can be used (as in the Himalaya glacier and Amazon forest cases) but any information from them needs to be carefully checked (this guideline was not followed in the former case).