Discovery upheld, paper nearly unchanged
Lots of sourballs and jealous experimenters (and theorists) have been trying to sling mud on the March 2014 discovery of the primordial gravitational waves by BICEP2. Some of them have been suggesting that it wasn't even kosher to write or talk about the discovery before it gets through the "peer review".
Well, those people associating the process of "peer review" with supernatural abilities have yet another reason to shut their mouth because the work was just published in the appropriately prestigious Physical Review Letters:
Unless you have memorized individual sentences in the original draft (see arXiv) really carefully, you won't really find a difference. The paper claims the discovery of these waves. I can't even safely say whether it lists more methods to be confident that the discovery is real or fewer methods to do so.
The abstract claims that the null hypothesis is excluded at "more than five sigma" confidence level (and, later in the abstract, "seven sigma" confidence level) using the first method, that the dust is 5-10 times smaller than the observed signal if various models of the dust available in the literature are being used, and that cross-correlation arguments and the right spectral index exclude the dust at "three sigma" or "one point seven sigma" even without any models. It also says that if all these things are ignored, it's plausible that some completely new model of the dust could change the conclusion or at least the confidence level. What a surprise. Any development in any part of science may change things.
To summarize, it's a robust collection of evidence that the signal is real even though the certainty isn't perfect.
If one compares the papers carefully, he may see that some parts of the previous draft were omitted. For example, the analysis based on the reverse-engineered PDF file from the Planck Collaboration is no longer included in the paper. The two anonymous referees may have had various reasons to force the BICEP2 Collaboration to omit these pieces.
What I find very important to emphasize is that the referees are not an infallible omnipresent Gods who are superior relatively to the BICEP2 folks in some metaphysical way. Chances are 50%-50% that every individual change that the referees forced upon the BICEP2 authors has made the paper better; or it has made it worse. The idea that the authors are submitting a highly imperfect paper and the referees – or the process of the "peer review" itself – guarantees that the resulting paper is perfect is a downright silly idea.
It's likely that the referees (or one of them) are someone who is close enough to the Planck Collaboration if not a member of it. At least, if I were the editor, I would be very likely to choose a referee who is close enough to Planck. We shouldn't forget about this likely proposition when we read e.g. the last sentence of the abstract of the published paper:
At the very end of the paper, we learn that these "upcoming data sets" should be from the 353 GHz Planck maps and from the (BICEP2-superseding) Keck Array. If a Planck person were the relevant referee, this "mandatory insertion" of the comment about the upcoming Planck data may be viewed as a self-advertisement. The BICEP2 folks will add a new-frequency, 100 GHz data to their existing data. The latter are (and this paper is) exclusively based on the frequency 150 GHz. And it seems totally plausible to me that the new data from the BICEP2-related team will actually teach us much more than anything that the Planck will be able to publish.
As far as I am concerned, the publication of the discovery in PRL isn't really a big event. What it means is that one external expert or a few external experts were asked to add their voices and communicate with the BICEP2 folks about their work. No clear problem with the discovery was found so the modifications were restricted to minor changes – the usual minor changes that are supposed to prove that the referees have been at work.
Of course that the question whether the changes were improvements or not is a subtle one. It has no clear answer. If the referees had been members of the BICEP2 from the beginning, their opinions would have already affected the draft – the draft would have been different by similar minor changes that the referees have imposed on the paper in the PRL. The referees aren't too different from "just two other potential co-authors who could have joined the research but he has not". The idea that an "extra person who looks at it" makes things vastly more accurate is heavily overrated, much like the supposed "independence" of the external reviewers. They're external and independent but they may still be wrong – they may be wrong about the same things as the authors and they may also be wrong in a different way or more wrong than anyone in the BICEP2 Collaboration.
If the uncertainty ultimately evaporates and it's clear that the BICEP2 discovery is legitimate, I think it will be important to look at their original draft, and not the edited (or partly censored?) version in the PRL, to quantify their contribution to physics. There are cases in which peer review is likely to bring more confusion to the business, and I think that the mandatory editing or censorship of papers from large enough and competent enough collaborations is always an example of that. Any softening of claims in such papers is potentially due to jealousy of the folks who were not lucky to participate in the discovery.
There are reasons to think that the peer review increases the average quality of the published papers in a wide array of situations. However, we shouldn't forget about the other side of the coin. Peer review involves some kind of censorship and if correct claims are being censored, it's clearly not helping the progress in science. Some claims in published papers may be wrong but it may often be better to allow the authors to publish their results and views in an uncensored form.
Lots of sourballs and jealous experimenters (and theorists) have been trying to sling mud on the March 2014 discovery of the primordial gravitational waves by BICEP2. Some of them have been suggesting that it wasn't even kosher to write or talk about the discovery before it gets through the "peer review".
Well, those people associating the process of "peer review" with supernatural abilities have yet another reason to shut their mouth because the work was just published in the appropriately prestigious Physical Review Letters:
Detection of B-Mode Polarization at Degree Angular Scales by BICEP2 by Ade et al. (BICEP2)Not just the abstract above but the whole paper is available for free. We should have entered the era in which almost all cosmo/astro/particle/theoretical physics papers should be available for free due to a contract.
Unless you have memorized individual sentences in the original draft (see arXiv) really carefully, you won't really find a difference. The paper claims the discovery of these waves. I can't even safely say whether it lists more methods to be confident that the discovery is real or fewer methods to do so.
The abstract claims that the null hypothesis is excluded at "more than five sigma" confidence level (and, later in the abstract, "seven sigma" confidence level) using the first method, that the dust is 5-10 times smaller than the observed signal if various models of the dust available in the literature are being used, and that cross-correlation arguments and the right spectral index exclude the dust at "three sigma" or "one point seven sigma" even without any models. It also says that if all these things are ignored, it's plausible that some completely new model of the dust could change the conclusion or at least the confidence level. What a surprise. Any development in any part of science may change things.
To summarize, it's a robust collection of evidence that the signal is real even though the certainty isn't perfect.
If one compares the papers carefully, he may see that some parts of the previous draft were omitted. For example, the analysis based on the reverse-engineered PDF file from the Planck Collaboration is no longer included in the paper. The two anonymous referees may have had various reasons to force the BICEP2 Collaboration to omit these pieces.
What I find very important to emphasize is that the referees are not an infallible omnipresent Gods who are superior relatively to the BICEP2 folks in some metaphysical way. Chances are 50%-50% that every individual change that the referees forced upon the BICEP2 authors has made the paper better; or it has made it worse. The idea that the authors are submitting a highly imperfect paper and the referees – or the process of the "peer review" itself – guarantees that the resulting paper is perfect is a downright silly idea.
It's likely that the referees (or one of them) are someone who is close enough to the Planck Collaboration if not a member of it. At least, if I were the editor, I would be very likely to choose a referee who is close enough to Planck. We shouldn't forget about this likely proposition when we read e.g. the last sentence of the abstract of the published paper:
...with \(r=0\) disfavored at \(7.0\sigma\). Accounting for the contribution of foreground, dust will shift this value downward by an amount which will be better constrained with upcoming data sets.The original draft was more specific in this sentence. It said that the confidence level was reduced to \(5.9\sigma\) if the best foreground model is assumed. But you see that qualitatively, even this sentence is the same.
At the very end of the paper, we learn that these "upcoming data sets" should be from the 353 GHz Planck maps and from the (BICEP2-superseding) Keck Array. If a Planck person were the relevant referee, this "mandatory insertion" of the comment about the upcoming Planck data may be viewed as a self-advertisement. The BICEP2 folks will add a new-frequency, 100 GHz data to their existing data. The latter are (and this paper is) exclusively based on the frequency 150 GHz. And it seems totally plausible to me that the new data from the BICEP2-related team will actually teach us much more than anything that the Planck will be able to publish.
As far as I am concerned, the publication of the discovery in PRL isn't really a big event. What it means is that one external expert or a few external experts were asked to add their voices and communicate with the BICEP2 folks about their work. No clear problem with the discovery was found so the modifications were restricted to minor changes – the usual minor changes that are supposed to prove that the referees have been at work.
Of course that the question whether the changes were improvements or not is a subtle one. It has no clear answer. If the referees had been members of the BICEP2 from the beginning, their opinions would have already affected the draft – the draft would have been different by similar minor changes that the referees have imposed on the paper in the PRL. The referees aren't too different from "just two other potential co-authors who could have joined the research but he has not". The idea that an "extra person who looks at it" makes things vastly more accurate is heavily overrated, much like the supposed "independence" of the external reviewers. They're external and independent but they may still be wrong – they may be wrong about the same things as the authors and they may also be wrong in a different way or more wrong than anyone in the BICEP2 Collaboration.
If the uncertainty ultimately evaporates and it's clear that the BICEP2 discovery is legitimate, I think it will be important to look at their original draft, and not the edited (or partly censored?) version in the PRL, to quantify their contribution to physics. There are cases in which peer review is likely to bring more confusion to the business, and I think that the mandatory editing or censorship of papers from large enough and competent enough collaborations is always an example of that. Any softening of claims in such papers is potentially due to jealousy of the folks who were not lucky to participate in the discovery.
There are reasons to think that the peer review increases the average quality of the published papers in a wide array of situations. However, we shouldn't forget about the other side of the coin. Peer review involves some kind of censorship and if correct claims are being censored, it's clearly not helping the progress in science. Some claims in published papers may be wrong but it may often be better to allow the authors to publish their results and views in an uncensored form.
BICEP2 gets published in PRL
![BICEP2 gets published in PRL]()
Reviewed by MCH
on
June 20, 2014
Rating:
No comments: