In any discussion about scholarly communication today two thorny issues quickly emerge: the so-called access problem, and the problem of declining peer review standards. Kamal Mahawar, co-founder and CEO of a new web platform for publishing biomedical research called WebmedCentral, believes he has a solution to both problems. WebmedCentral, however, is not without its critics.
The access problem afflicting the research community is essentially an issue of affordability. Researchers submit a paper to a publisher, who then invites other researchers to assess it for quality and value. Assuming it is deemed to be adequate the paper is then published in a journal. To fund this process publishers sell subscriptions to their journals. Research institutions buy these subscriptions to ensure that their researchers have access to all relevant research being done around the world.
As library budgets have declined, however, research institutions have struggled to find the necessary money to pay for all the subscriptions their faculty require, depriving researchers of access to more and more research — a phenomenon known as the serials crisis.
Advocates of open-access publishing (as distinct from self-archiving) believe that the answer to this access/affordability problem is for researchers to abandon publishing in subscription journals and publish in OA journals instead. Rather than imposing charges for access (subscriptions), OA journals levy a publication fee, or “article-processing charge” on authors, or more usually their institutions. This enables publishers to make the papers they publish freely available on the Web, and so provide unfettered access to them.
The problem with OA publishing, however, is that the publication fees are far from cheap — around $1,350 to $3,000 per paper. Consequently, not all researchers have access to the necessary funds. While some OA publishers offer waivers for those without the wherewithal, this is generally done on a case-by-case basis, and the practice could very well be discontinued at some point in the future. If it was, indigent authors could find themselves unable to get their research published, particularly authors based in the developing world.
Raghavendra Gadagkar, a researcher at the Centre for Ecological Sciences at the Indian Institute of Science in Bangalore, made this point eloquently in a letter to Nature in 2008. He concluded that author-pays OA, “does more harm than good in the developing world”.
Showing posts with label Select Committee. Show all posts
Showing posts with label Select Committee. Show all posts
Wednesday, August 17, 2011
Wednesday, June 22, 2011
Peer review: Still no practical alternative?
The UK House of Commons Science & Technology Select Committee is currently conducting an inquiry into peer review. It held its fourth oral evidence session on June 8th, taking evidence from both funders of scientific research and from Government.
UPDATE: THE COMMITTEE’S REPORT HAS NOW BEEN PUBLISHED. THE DETAILS ARE AVAILABLE HERE.
While the Committee gave no specific reason for launching the current inquiry it seems evident from the questions MPs have been asking that two particular incidents have been exercising their minds: the long-running saga over Andrew Wakefield and the MMR vaccine scare, and the so-called Climategate incident.
In the June 8th session the Committee asked questions not just about the efficacy of peer review, but about scientific fraud, bias, the willingness of universities to investigate allegations of misconduct, and the need to make research data freely available so that others can access, examine and test them.
MPs seemed particularly concerned that universities may be unwilling to investigate claims of misconduct. As MP Graham Stringer put it in one of the questions he asked the witnesses, “There is a certain amount of evidence that very little fraud is detected in universities and major research institutions in this country. Do you think we should be doing more to try and detect that, because in one sense there is an interest within those bodies not to discover or expose the problems they have, to sweep it under the carpet, isn’t there? If you are running a university and you find you have a researcher who just writes down his figures without doing the work, which has happened in one or two cases, the university doesn’t want to say that it has been employing a fraudster for 10 years, does it?”
Politicians also probed the witnesses about the use of journal impact factors as a “proxy measure for research quality” when assessing the performance of academics, and whether “the growth of online repository journals” like PLoS ONE is a “technically sound” development.
Robust defence
For their part the witnesses put up a robust defence of current practices. They denied that universities would cover up fraud; they dismissed suggestions that the impact factor is used as a proxy measure of quality; and they insisted that, while it might not be perfect, there is no practical alternative to traditional peer review.
In support of the latter claim they repeated the oft-made analogy with Winston Churchill’s description of democracy. Churchill famously described democracy as “the worst form of government, except for all those other forms that have been tried from time to time.” Thus it is with peer review, averred the witnesses: no one has come up with anything better.
Those with any experience or knowledge of how peer review works in practice might have been tempted to conclude that analogising peer review with democracy is to obfuscate the issue. At the very least, it appears oxymoronic.
Such a conclusion was all the more likely in light of the opening question and answer. The Chair suggested that it might be helpful to conduct some research into the efficacy of the current system — on the grounds that “evaluation of peer review is poor”. To this Wellcome Trust director Sir Mark Walport replied: “Peer review is no more and no less than review by experts. I am not sure that we would want to do a comparison of a review by experts with a review by ignoramuses.”
Sir Mark’s statement can only have served to remind the audience that peer review is more oligarchic than democratic in effect. Rather than encouraging egalitarianism, it promotes elitism, and all the privileges one might associate with an old boy’s club (appositely perhaps, there was not a single female witness called to give evidence on June 8th).
Of course the Churchillian analogy is not really meant to suggest that peer review is a democratic process. Nevertheless the witnesses’ repeated claims that the current peer review system is “good enough” would surely be challenged by many junior researchers, who frequently complain that scholarly journals tend to be controlled by small elite groups of insiders, invariably senior researchers.
As one researcher pointed out to me recently, this is particularly problematic for those working outside North American and Europe. As he put it, “Peer-reviewed journals with a high impact factor are either dominated by certain gangs, or groups, or the editors rely on the opinion of reviewers too much.”
The upshot, he added, is that “a small guy from Russia, Brazil or Thailand will never get published, even with excellent results, unless he or she has a prominent Western colleague as a co-author.”
In fact, it is not just researchers in less privileged parts of the world who can struggle to get published in scientific journals today. Nor is it only junior researchers who complain about the peer review system. In 2009, for instance, 14 leading stem cell researchers wrote an open letter to journal editors highlighting their disquiet at the way in which the system operates.
Speaking to the BBC about the letter Professor Lovell-Badge commented: "It's turning things into a clique where only papers that satisfy this select group of a few reviewers who think of themselves as very important people in the field is published.”
Responding in a (separate) BBC interview, Sir Mark downplayed the criticism. Scientists, he said, “are always a bit paranoid” about peer review. And to make his point Sir Mark again used the analogy with democracy — peer review is not perfect, but it is the best system that the research community has been able to come up with.
UPDATE: THE COMMITTEE’S REPORT HAS NOW BEEN PUBLISHED. THE DETAILS ARE AVAILABLE HERE.
While the Committee gave no specific reason for launching the current inquiry it seems evident from the questions MPs have been asking that two particular incidents have been exercising their minds: the long-running saga over Andrew Wakefield and the MMR vaccine scare, and the so-called Climategate incident.
In the June 8th session the Committee asked questions not just about the efficacy of peer review, but about scientific fraud, bias, the willingness of universities to investigate allegations of misconduct, and the need to make research data freely available so that others can access, examine and test them.
MPs seemed particularly concerned that universities may be unwilling to investigate claims of misconduct. As MP Graham Stringer put it in one of the questions he asked the witnesses, “There is a certain amount of evidence that very little fraud is detected in universities and major research institutions in this country. Do you think we should be doing more to try and detect that, because in one sense there is an interest within those bodies not to discover or expose the problems they have, to sweep it under the carpet, isn’t there? If you are running a university and you find you have a researcher who just writes down his figures without doing the work, which has happened in one or two cases, the university doesn’t want to say that it has been employing a fraudster for 10 years, does it?”
Politicians also probed the witnesses about the use of journal impact factors as a “proxy measure for research quality” when assessing the performance of academics, and whether “the growth of online repository journals” like PLoS ONE is a “technically sound” development.
Robust defence
For their part the witnesses put up a robust defence of current practices. They denied that universities would cover up fraud; they dismissed suggestions that the impact factor is used as a proxy measure of quality; and they insisted that, while it might not be perfect, there is no practical alternative to traditional peer review.
In support of the latter claim they repeated the oft-made analogy with Winston Churchill’s description of democracy. Churchill famously described democracy as “the worst form of government, except for all those other forms that have been tried from time to time.” Thus it is with peer review, averred the witnesses: no one has come up with anything better.
Those with any experience or knowledge of how peer review works in practice might have been tempted to conclude that analogising peer review with democracy is to obfuscate the issue. At the very least, it appears oxymoronic.
Such a conclusion was all the more likely in light of the opening question and answer. The Chair suggested that it might be helpful to conduct some research into the efficacy of the current system — on the grounds that “evaluation of peer review is poor”. To this Wellcome Trust director Sir Mark Walport replied: “Peer review is no more and no less than review by experts. I am not sure that we would want to do a comparison of a review by experts with a review by ignoramuses.”
Sir Mark’s statement can only have served to remind the audience that peer review is more oligarchic than democratic in effect. Rather than encouraging egalitarianism, it promotes elitism, and all the privileges one might associate with an old boy’s club (appositely perhaps, there was not a single female witness called to give evidence on June 8th).
Of course the Churchillian analogy is not really meant to suggest that peer review is a democratic process. Nevertheless the witnesses’ repeated claims that the current peer review system is “good enough” would surely be challenged by many junior researchers, who frequently complain that scholarly journals tend to be controlled by small elite groups of insiders, invariably senior researchers.
As one researcher pointed out to me recently, this is particularly problematic for those working outside North American and Europe. As he put it, “Peer-reviewed journals with a high impact factor are either dominated by certain gangs, or groups, or the editors rely on the opinion of reviewers too much.”
The upshot, he added, is that “a small guy from Russia, Brazil or Thailand will never get published, even with excellent results, unless he or she has a prominent Western colleague as a co-author.”
In fact, it is not just researchers in less privileged parts of the world who can struggle to get published in scientific journals today. Nor is it only junior researchers who complain about the peer review system. In 2009, for instance, 14 leading stem cell researchers wrote an open letter to journal editors highlighting their disquiet at the way in which the system operates.
Speaking to the BBC about the letter Professor Lovell-Badge commented: "It's turning things into a clique where only papers that satisfy this select group of a few reviewers who think of themselves as very important people in the field is published.”
Responding in a (separate) BBC interview, Sir Mark downplayed the criticism. Scientists, he said, “are always a bit paranoid” about peer review. And to make his point Sir Mark again used the analogy with democracy — peer review is not perfect, but it is the best system that the research community has been able to come up with.
Sunday, June 12, 2011
UK politicians puzzle over peer review in an open access environment
The UK House of Commons Science & Technology Committee is currently conducting an inquiry into peer review. The third public event of the inquiry was held on Monday 23rd May, when the Committee heard evidence from experts on open access publishing and post-publication review, and from representatives of the research community.
SOME OF THE ISSUES EXPLORED BY THE COMMITTEE ARE HIGHLIGHTED IN THE EDITED QUESTIONS AND ANSWERS LISTED BELOW. THE ASTERISKED HEADINGS ARE MINE.
I COMMENT ON THE SESSION AT THE BOTTOM OF THIS POST.
UPDATE: THE COMMITTEE’S REPORT HAS NOW BEEN PUBLISHED. THE DETAILS ARE AVAILABLE HERE.
The Chair of the Science & Technology Committee is Andrew Miller, Labour MP for Ellesmere Port and Neston. Other politicians to pose the questions below were Graham Stringer, Labour MP for Blackley and Broughton, Roger Williams, Liberal Democrat MP for Brecon and Radnorshire, and Stephen Metcalfe, Conservative MP for South Basildon and East Thurrock. (A full list of Committee members is available here).
The hearing was split into two sessions.
Those giving evidence in the first session were Dr Rebecca Lawrence, Director, New Product Development at Faculty of 1000 Ltd., Mark Patterson, Director of Publishing at the Public Library of Science, Dr Michaela Torkar, Editorial Director at Biomed Central, and Dr Malcolm Read OBE, Executive Secretary of JISC.
SOME OF THE ISSUES EXPLORED BY THE COMMITTEE ARE HIGHLIGHTED IN THE EDITED QUESTIONS AND ANSWERS LISTED BELOW. THE ASTERISKED HEADINGS ARE MINE.
I COMMENT ON THE SESSION AT THE BOTTOM OF THIS POST.
UPDATE: THE COMMITTEE’S REPORT HAS NOW BEEN PUBLISHED. THE DETAILS ARE AVAILABLE HERE.
The Chair of the Science & Technology Committee is Andrew Miller, Labour MP for Ellesmere Port and Neston. Other politicians to pose the questions below were Graham Stringer, Labour MP for Blackley and Broughton, Roger Williams, Liberal Democrat MP for Brecon and Radnorshire, and Stephen Metcalfe, Conservative MP for South Basildon and East Thurrock. (A full list of Committee members is available here).
The hearing was split into two sessions.
== FIRST SESSION ==
Friday, May 20, 2011
More on the UK Inquiry into Peer Review
The UK House of Commons Science & Technology Committee is currently conducting an inquiry into peer review. The second public event of the inquiry was held at Portcullis House, in London, on May 11th.
(NOTE: The third public event will be held next Monday, 23rd May, and will explore Open Access (OA) and post-publication review — details and a link are available at the bottom of this post; some subsequent abstracts and commentary are available here).
UPDATE: THE COMMITTEE’S REPORT HAS NOW BEEN PUBLISHED. THE DETAILS ARE AVAILABLE HERE.
The second public event was split into two sessions.
The first Session was concerned with publication ethics. For this the witnesses were Tracey Brown, Managing Director of Sense About Science, and Dr Liz Wager, Chair of the Committee on Publication Ethics (COPE) and Board Member of the UK Research Integrity Office (UKRIO).
The second session took evidence from publishers. Here the witnesses were Mayur Amin, Senior Vice President, Research and Academic Relations at Elsevier, Dr Philip Campbell, Editor-in-Chief of Nature Publishing Group, Robert Campbell, Senior Publisher at Wiley-Blackwell, Dr Fiona Godlee, Editor-in-Chief at the BMJ Group, and Dr Andrew Sugden, Deputy Editor and International Managing Director of Science.
The Chair of the Science & Technology Committee is Andrew Miller, Labour MP for Ellesmere Port and Neston.
Other politicians to pose the questions extracted from the transcript and listed under the video of the event inserted below were Graham Stringer, Labour MP for Blackley and Broughton, David Morris, Conservative MP for Morecambe and Lunesdale, Roger Williams, Liberal Democrat MP for Brecon and Radnorshire, and Gavin Barwell, Conservative MP for Croydon Central. (A full list of Committee members is available here).
The (uncorrected) transcript of the meeting is available here.
The video of the event can be accessed here.
SOME OF THE ISSUES EXPLORED BY THE COMMITTEE ARE HIGHLIGHTED IN THE QUESTIONS AND ANSWERS LISTED BELOW
(NOTE: The third public event will be held next Monday, 23rd May, and will explore Open Access (OA) and post-publication review — details and a link are available at the bottom of this post; some subsequent abstracts and commentary are available here).
UPDATE: THE COMMITTEE’S REPORT HAS NOW BEEN PUBLISHED. THE DETAILS ARE AVAILABLE HERE.
The second public event was split into two sessions.
The first Session was concerned with publication ethics. For this the witnesses were Tracey Brown, Managing Director of Sense About Science, and Dr Liz Wager, Chair of the Committee on Publication Ethics (COPE) and Board Member of the UK Research Integrity Office (UKRIO).
The second session took evidence from publishers. Here the witnesses were Mayur Amin, Senior Vice President, Research and Academic Relations at Elsevier, Dr Philip Campbell, Editor-in-Chief of Nature Publishing Group, Robert Campbell, Senior Publisher at Wiley-Blackwell, Dr Fiona Godlee, Editor-in-Chief at the BMJ Group, and Dr Andrew Sugden, Deputy Editor and International Managing Director of Science.
The Chair of the Science & Technology Committee is Andrew Miller, Labour MP for Ellesmere Port and Neston.
Other politicians to pose the questions extracted from the transcript and listed under the video of the event inserted below were Graham Stringer, Labour MP for Blackley and Broughton, David Morris, Conservative MP for Morecambe and Lunesdale, Roger Williams, Liberal Democrat MP for Brecon and Radnorshire, and Gavin Barwell, Conservative MP for Croydon Central. (A full list of Committee members is available here).
The (uncorrected) transcript of the meeting is available here.
The video of the event can be accessed here.
SOME OF THE ISSUES EXPLORED BY THE COMMITTEE ARE HIGHLIGHTED IN THE QUESTIONS AND ANSWERS LISTED BELOW
Thursday, May 19, 2011
UK Inquiry into Peer Review
On 27th January 2011 the UK House of Commons Science & Technology Committee announced that it planned to hold an inquiry into peer review. That inquiry is now underway. Below are some links to the first public event held to take oral evidence. (Click here for details of the second public event).
The terms of reference for the inquiry are the following:
The Committee welcomes submissions on all aspect of the process and among the issues it is likely to examine are the following:
The terms of reference for the inquiry are the following:
The Committee welcomes submissions on all aspect of the process and among the issues it is likely to examine are the following:
- the strengths and weaknesses of peer review as a quality control mechanism for scientists, publishers and the public;
- measures to strengthen peer review;
- the value and use of peer reviewed science on advancing and testing scientific knowledge;
- the value and use of peer reviewed science in informing public debate;
- the extent to which peer review varies between scientific disciplines and between countries across the world;
- the processes by which reviewers with the requisite skills and knowledge are identified, in particular as the volume of multi-disciplinary research increases;
- the impact of IT and greater use of online resources on the peer review process; and
- possible alternatives to peer review.
The first public event took place on May 4th, with oral evidence being given by a number of experts, including:
- Dr Nicola Gulley, Editorial Director, Institute of Physics Publishing Ltd
- Professor Ronald Laskey CBE FRS FMedSci, Vice-President, Academy of Medical Sciences
- Dr Robert Parker, Interim Chief Executive, Royal Society of Chemistry
- Professor John Pethica FRS, Physical Secretary and Vice-President, Royal Society
The opening question from the chair of the Committee, Andrew Miller, Labour MP for Ellesmere Port and Neston, was:
Q: Peer review is perceived to be "fundamental to scholarly communications". If it disappeared tomorrow, what would the consequences be?
The first reply, from Dr Robert Parker, was:
A: You would have to come up with something else with which to replace it. There isn’t anything very obvious to replace peer review with currently. The danger would be to the scientific record, really. The importance of it is laid out in the evidence that has been submitted with great clarity from most people who have submitted evidence in writing to this review. The value and quality of that scientific record is paramount, and peer review helps to keep that in place.
The written evidence Dr Parker refers to can be read here.
Q: Peer review is perceived to be "fundamental to scholarly communications". If it disappeared tomorrow, what would the consequences be?
The first reply, from Dr Robert Parker, was:
A: You would have to come up with something else with which to replace it. There isn’t anything very obvious to replace peer review with currently. The danger would be to the scientific record, really. The importance of it is laid out in the evidence that has been submitted with great clarity from most people who have submitted evidence in writing to this review. The value and quality of that scientific record is paramount, and peer review helps to keep that in place.
The written evidence Dr Parker refers to can be read here.
Subscribe to:
Posts (Atom)