(As an anti-democrat, I cannot and do not claim much significance for the majority status of climate scepticism. I merely make the point that it is very strange that the majority view is not even respectable)
The leak of the CRU data has tempted me to stick my head over the parapet once more. The first thought when they appeared was that it was too good to be true, and that the emails must have been faked or planted. That fear faded, but what eventually emerged was that the emails, at least, are less compromising than they appeared at first glance.
To take the most quoted example, the "Nature trick", it is at the very worst an attempt to spin the data in such a way that the headlines of articles point more in the direction of unprecedented modern warming. It might not even be that, but it doesn't matter, because we already had proof that the data was being spun in that way, which I presented in April 2008.
The more I look at the documents the less bad they seem. I had set store by the "Rules of the game" presentation as evidence of excessive politicisation, but when I looked at it I saw it was produced by an advertising agency for the government. It is a shocking disgrace that it was produced and is being used, but one can hardly blame the CRU just for having a copy of it. It is nothing to do with them, and in fact the advice it gives is that the scientists' work should be ignored or glossed over - even suggesting there might be scientific questions is something the politicians want to avoid.
Next, we had the "harry_read_me" file, and the contortions that had to be done to turn a heterogeneous heap of instrument data and adjustments into a presentable, usable gridded global temperature history. People who've never had to anything of the sort were shocked by the problems described - incompatible data sets, inconsistent data sets, code written by departed programmers doing things they don't understand, corruption introduced by format conversions, ad-hoc fixes to cope with missing or corrupted data, mysterious factor-of-ten discrepancies, struggling with inappropriate out-of-date programming languages, success defined as getting data out at the end that "looks right" after nights and weekends of failure. I've worked on software producing summary reports of data from multiple sources, and I've seen all those things, and done many of them myself. It's pretty much inevitable.
After that came the famous
2.6,2.6,2.6]*0.75 ; fudge factor
None of the blogs I've seen that seized on that actually traced the output of the code through to the papers where it was presented, to see if the adjustments were explained there, assuming the output was ever published at all, which has been denied
I don't now expect the leaked documents to show deliberate fraud, but I never believed there was any in the first place. At least, since there is no reason for Phil Jones to resign and be replaced by someone else, one eternal truth can be upheld - George Monbiot is always wrong.
What the emails do show are two things:
Fact 1. That the researchers - Jones, Briffa, Mann, and others - see themselves as having a responsibility not just to do the science but to persuade the public of the seriousness of the problem and of the need for political action
Fact 2. That as part of this, they want to prevent sceptical research from being published or believed, and at least believe they have some power to do so
Neither of these this are actually serious accusations against the individual scientists. Today it would be thought very strange to argue that a scientist finding what he believed was a serious threat to humanity should not act on that belief by seeking to influence politics. When the government funds research, it wants to fund research that is relevant, and all that the scientists' activism amounts to is arguing that their research is, in fact, relevant. And of course someone has to review and edit papers and decide which are good science and which aren't - that's what peer review means.
And that is the real point here. Because although both of these facts are just the result of Jones et al doing the jobs they're employed to do, the result of the combination is that science is broken. It's not their fault.
You can have partisan presentations of the evidence, provided there is opportunity to compare competing partisan presentations. And you need to have assessments of the value of scientific claims, but those should not be made or controlled by partisans of either side. What has gone wrong is that one side has been allowed to silence the other.
The analogy I like is to agency ratings - it worked well until it was made official. When the only asset a rating agency had was the trust that the market had in their judgement, they were extremely conservative. Nobody would pay the agency to give an instrument a rating that nobody else would believe. Once there were laws that many of the largest investors were required to invest in instruments that had AAA rating, that gave the ratings a value in excess of their credibility.
Peer review is the same - publication in the more prestigious scientific journals was valued because it was understood by other scientists as a recommendation that the work was of a high quality. The editors of journals were motivated not only to maintain but to improve the reputation of their journals. Now that review has gone from being solely a recommendation of quality directed at the scientist's peers, to a stamp of worth directed at politicians and the public, the incentives in the system have been distorted.