The UK House of Commons Science & Technology Select Committee is currently conducting an inquiry into peer review. It held its fourth oral evidence session on June 8th, taking evidence from both funders of scientific research and from Government.
UPDATE: THE COMMITTEE’S REPORT HAS NOW BEEN PUBLISHED. THE DETAILS ARE AVAILABLE HERE.
While the Committee gave no specific reason for launching the current inquiry it seems evident from the questions MPs have been asking that two particular incidents have been exercising their minds: the long-running saga over Andrew Wakefield and the MMR vaccine scare, and the so-called Climategate incident.
In the June 8th session the Committee asked questions not just about the efficacy of peer review, but about scientific fraud, bias, the willingness of universities to investigate allegations of misconduct, and the need to make research data freely available so that others can access, examine and test them.
MPs seemed particularly concerned that universities may be unwilling to investigate claims of misconduct. As MP Graham Stringer put it in one of the questions he asked the witnesses, “There is a certain amount of evidence that very little fraud is detected in universities and major research institutions in this country. Do you think we should be doing more to try and detect that, because in one sense there is an interest within those bodies not to discover or expose the problems they have, to sweep it under the carpet, isn’t there? If you are running a university and you find you have a researcher who just writes down his figures without doing the work, which has happened in one or two cases, the university doesn’t want to say that it has been employing a fraudster for 10 years, does it?”
Politicians also probed the witnesses about the use of journal impact factors as a “proxy measure for research quality” when assessing the performance of academics, and whether “the growth of online repository journals” like PLoS ONE is a “technically sound” development.
Robust defence
For their part the witnesses put up a robust defence of current practices. They denied that universities would cover up fraud; they dismissed suggestions that the impact factor is used as a proxy measure of quality; and they insisted that, while it might not be perfect, there is no practical alternative to traditional peer review.
In support of the latter claim they repeated the oft-made analogy with Winston Churchill’s description of democracy. Churchill famously described democracy as “the worst form of government, except for all those other forms that have been tried from time to time.” Thus it is with peer review, averred the witnesses: no one has come up with anything better.
Those with any experience or knowledge of how peer review works in practice might have been tempted to conclude that analogising peer review with democracy is to obfuscate the issue. At the very least, it appears oxymoronic.
Such a conclusion was all the more likely in light of the opening question and answer. The Chair suggested that it might be helpful to conduct some research into the efficacy of the current system — on the grounds that “evaluation of peer review is poor”. To this Wellcome Trust director Sir Mark Walport replied: “Peer review is no more and no less than review by experts. I am not sure that we would want to do a comparison of a review by experts with a review by ignoramuses.”
Sir Mark’s statement can only have served to remind the audience that peer review is more oligarchic than democratic in effect. Rather than encouraging egalitarianism, it promotes elitism, and all the privileges one might associate with an old boy’s club (appositely perhaps, there was not a single female witness called to give evidence on June 8th).
Of course the Churchillian analogy is not really meant to suggest that peer review is a democratic process. Nevertheless the witnesses’ repeated claims that the current peer review system is “good enough” would surely be challenged by many junior researchers, who frequently complain that scholarly journals tend to be controlled by small elite groups of insiders, invariably senior researchers.
As one researcher pointed out to me recently, this is particularly problematic for those working outside North American and Europe. As he put it, “Peer-reviewed journals with a high impact factor are either dominated by certain gangs, or groups, or the editors rely on the opinion of reviewers too much.”
The upshot, he added, is that “a small guy from Russia, Brazil or Thailand will never get published, even with excellent results, unless he or she has a prominent Western colleague as a co-author.”
In fact, it is not just researchers in less privileged parts of the world who can struggle to get published in scientific journals today. Nor is it only junior researchers who complain about the peer review system. In 2009, for instance, 14 leading stem cell researchers wrote an open letter to journal editors highlighting their disquiet at the way in which the system operates.
Speaking to the BBC about the letter Professor Lovell-Badge commented: "It's turning things into a clique where only papers that satisfy this select group of a few reviewers who think of themselves as very important people in the field is published.”
Responding in a (separate) BBC interview, Sir Mark downplayed the criticism. Scientists, he said, “are always a bit paranoid” about peer review. And to make his point Sir Mark again used the analogy with democracy — peer review is not perfect, but it is the best system that the research community has been able to come up with.
Wednesday, June 22, 2011
Sunday, June 19, 2011
Open Access by Numbers
Few can now doubt that open access (OA) to scholarly research is set to become an important feature of the scholarly communication landscape. What is less certain is how much of the world’s research literature is currently available on an OA basis, how fast OA is growing, and what percentage of the world’s academic and scientific literature will be OA in the long-term.
Trying to crunch the numbers is complicated by the fact that research papers can be made OA in two ways: Researchers can continue to publish in subscription journals and then make them freely available by self-archiving them in an institutional repository (Green OA), or they can pay to publish their work in an OA journal (either a pure Gold journal or a Hybrid OA journal) so that the publisher will make it freely available for them.
OA enthusiasts like librarian Heather Morrison — who publishes a series called “Dramatic Growth of Open Access” — tend to estimate OA occurrence and growth primarily by the simple counting of things.
In March, for instance, Morrison reported that there are now over 6,000 OA journals listed in the directory of open access journals (DOAJ), and implied that the number of OA articles is now growing more quickly than the number of papers being published in subscription journals. As she put it: “Data is presented that strongly suggests that the success rate for open access journals is already higher than that of subscription journals.”
In the same post, Morrison argued that by counting the number of papers flagged as OA on the Mendeley research sharing service we could conclude that self-archiving had grown by 171% in the first quarter of 2011.
Counting in this way presents an upbeat picture, suggests that the world is in the process of being flooded with OA, and that universal OA is just around the corner.
Refining the counts
Critics, however, point out that simple counting is too crude when trying to measure OA. Counting Gold OA journals, for instance, is not helpful since many of them publish just a handful of papers a year, if that.
Likewise, counting items that have been self-archived can be deceptive: Many records in institutional repositories will consist of metadata alone, or non-target items like presentations and other non-reviewed material.
Certainly publishers describe the incidence and growth of OA in a less upbeat manner. When I spoke to Springer’s Derk Haank at the end of last year, for instance, he estimated that only around 2% to 2.5% of the world’s papers are being published in Gold or Hybrid journals today.
And since the total number of research papers is growing at around 6% to 7% a year, he said, OA remains “just a drop in the ocean”.
In fact, predicted Haank, OA publishing will never be more than a niche activity. “I expect it to remain between 5% and 10% at a maximum,” he said.
Haank did not provide an estimate of Green OA, but implied that it was relatively low. Pointing out that he would be anxious if it did become commonplace he added, “But we are such a long way from that situation today that we are very easy going about author archiving.”
A few researchers, meanwhile, have been busy trying to arrive at more precise figures. When I last wrote on this topic in 2010 I spoke to a number of researchers, including Bo-Christer Björk.
Based at the Hanken School of Economics in Helsinki, Björk has undertaken several studies aimed at sizing the growth of OA, primarily Gold OA.
For a variety of reasons, Björk explained, this is not an easy thing to do. Nevertheless, when I spoke to him in January 2010 Björk estimated that Gold OA was probably increasing its share of the market by 0.5% per annum.
He added, however: “I have no evidence to show any acceleration in growth. On the contrary it seems that growth has been relatively stable, after a short expansive period when BioMed Central and PLoS were founded”.
“Tremendous growth of Gold OA”
Since then, Björk has taken a closer look at the many new OA journals that have been launched from 1993 - 2009, as well as the many subscription journals that have been converted into Gold journals.
There has also been the rise of “mega journals” like PLoS ONE, now the largest peer-reviewed journal in the world, and which expects to publish 12,000 papers in 2011 alone. In the wake of PLoS ONE’s success a number of PLoS ONE clones have recently been launched.
On June 13th 2011 Björk and colleagues published a new paper reporting an average annual growth rate since 2000 of 18% for the number of OA journals and 30% for the number of articles.
This, the paper suggests, “can be contrasted to the reported 3.5% yearly volume increase in journal publishing in general. In 2009 the share of articles in OA journals, of all peer reviewed journal articles, reached 7.7%. Overall, the results document a rapid growth in OA journal publishing over the last fifteen years.”
And in a note he posted on the American Scientist Open Access Forum (AmSci) Björk said that the results, “show the tremendous growth of gold OA over the past decade”.
As we said, Björk’s primary focus is on Gold OA. What about Green OA?
This is an area that Yassine Gargouri, a postdoctoral researcher who works with OA advocate Stevan Harnad at the Université du Québec à Montréal (UQAM), has been working on for the past four years.
Gargouri’s numbers suggest that between 2005 and 2010 the percentage of Green OA rose from about 15% per year to about 21%, which amounts to an increase of about 1% per year.
His numbers also suggest that introducing a Green mandate (requiring all an institution’s researchers to self-archive their papers) triples the yearly percentage of OA papers from the mandating institution.
Taken together with Björk’s work, this would seem to suggest that around 30% of the academic and scientific literature published in 2011 worldwide may now be freely available on the Web, two thirds of it as Green OA and one third of it as Gold OA.
Can Gold alone buy OA?
Nevertheless, it remains difficult to be precise about OA numbers, and especially difficult to make accurate predictions about future growth.
Like all attempts to understand and predict the world by means of numbers and statistics, much depends on how one derives them in the first place, how one crunches them, and how one subsequently interprets the results. In the case of OA, a key question that emerges is whether Gold OA is able on its own to accelerate the growth of OA to the degree that the OA movement would wish.
Why is it necessary to fret over such things? It is necessary for a number of reasons, but above all because if OA advocates knew exactly what was happening, and why, they would be able to put their main effort into those activities most likely to achieve their goal.
Vitally, they would be better able to answer a question that has plagued the movement for many years: Should the priority be given to Green or to Gold OA?
In the PDF file attached below I am publishing a Q&A interview with Gargouri. With a PhD in cognitive informatics, Gargouri has also participated in projects dealing with knowledge management, semantic web applications and ontologies. He has also taught in the computer science department at UQAM.
The interview includes contributions from Harnad — a leading OA advocate and self-styled archivangelist.
####
If you wish to read the interview please click on the link below.
I am publishing it under a Creative Commons licence, so you are free to copy and distribute it as you wish, so long as you credit me as the author, do not alter or transform the text, and do not use it for any commercial purpose.
To read the interview (as a PDF file) click here.
See also: John Whitfield Open access comes of age Nature, 21st June 2011
Trying to crunch the numbers is complicated by the fact that research papers can be made OA in two ways: Researchers can continue to publish in subscription journals and then make them freely available by self-archiving them in an institutional repository (Green OA), or they can pay to publish their work in an OA journal (either a pure Gold journal or a Hybrid OA journal) so that the publisher will make it freely available for them.
OA enthusiasts like librarian Heather Morrison — who publishes a series called “Dramatic Growth of Open Access” — tend to estimate OA occurrence and growth primarily by the simple counting of things.
In March, for instance, Morrison reported that there are now over 6,000 OA journals listed in the directory of open access journals (DOAJ), and implied that the number of OA articles is now growing more quickly than the number of papers being published in subscription journals. As she put it: “Data is presented that strongly suggests that the success rate for open access journals is already higher than that of subscription journals.”In the same post, Morrison argued that by counting the number of papers flagged as OA on the Mendeley research sharing service we could conclude that self-archiving had grown by 171% in the first quarter of 2011.
Counting in this way presents an upbeat picture, suggests that the world is in the process of being flooded with OA, and that universal OA is just around the corner.
Refining the counts
Critics, however, point out that simple counting is too crude when trying to measure OA. Counting Gold OA journals, for instance, is not helpful since many of them publish just a handful of papers a year, if that.
Likewise, counting items that have been self-archived can be deceptive: Many records in institutional repositories will consist of metadata alone, or non-target items like presentations and other non-reviewed material.
Certainly publishers describe the incidence and growth of OA in a less upbeat manner. When I spoke to Springer’s Derk Haank at the end of last year, for instance, he estimated that only around 2% to 2.5% of the world’s papers are being published in Gold or Hybrid journals today.
And since the total number of research papers is growing at around 6% to 7% a year, he said, OA remains “just a drop in the ocean”.
In fact, predicted Haank, OA publishing will never be more than a niche activity. “I expect it to remain between 5% and 10% at a maximum,” he said.
Haank did not provide an estimate of Green OA, but implied that it was relatively low. Pointing out that he would be anxious if it did become commonplace he added, “But we are such a long way from that situation today that we are very easy going about author archiving.”
A few researchers, meanwhile, have been busy trying to arrive at more precise figures. When I last wrote on this topic in 2010 I spoke to a number of researchers, including Bo-Christer Björk.
Based at the Hanken School of Economics in Helsinki, Björk has undertaken several studies aimed at sizing the growth of OA, primarily Gold OA.
For a variety of reasons, Björk explained, this is not an easy thing to do. Nevertheless, when I spoke to him in January 2010 Björk estimated that Gold OA was probably increasing its share of the market by 0.5% per annum.
He added, however: “I have no evidence to show any acceleration in growth. On the contrary it seems that growth has been relatively stable, after a short expansive period when BioMed Central and PLoS were founded”.
“Tremendous growth of Gold OA”
Since then, Björk has taken a closer look at the many new OA journals that have been launched from 1993 - 2009, as well as the many subscription journals that have been converted into Gold journals.
There has also been the rise of “mega journals” like PLoS ONE, now the largest peer-reviewed journal in the world, and which expects to publish 12,000 papers in 2011 alone. In the wake of PLoS ONE’s success a number of PLoS ONE clones have recently been launched.
On June 13th 2011 Björk and colleagues published a new paper reporting an average annual growth rate since 2000 of 18% for the number of OA journals and 30% for the number of articles.
This, the paper suggests, “can be contrasted to the reported 3.5% yearly volume increase in journal publishing in general. In 2009 the share of articles in OA journals, of all peer reviewed journal articles, reached 7.7%. Overall, the results document a rapid growth in OA journal publishing over the last fifteen years.”
And in a note he posted on the American Scientist Open Access Forum (AmSci) Björk said that the results, “show the tremendous growth of gold OA over the past decade”.
As we said, Björk’s primary focus is on Gold OA. What about Green OA?
This is an area that Yassine Gargouri, a postdoctoral researcher who works with OA advocate Stevan Harnad at the Université du Québec à Montréal (UQAM), has been working on for the past four years.
Gargouri’s numbers suggest that between 2005 and 2010 the percentage of Green OA rose from about 15% per year to about 21%, which amounts to an increase of about 1% per year.
His numbers also suggest that introducing a Green mandate (requiring all an institution’s researchers to self-archive their papers) triples the yearly percentage of OA papers from the mandating institution.
Taken together with Björk’s work, this would seem to suggest that around 30% of the academic and scientific literature published in 2011 worldwide may now be freely available on the Web, two thirds of it as Green OA and one third of it as Gold OA.
Can Gold alone buy OA?
Nevertheless, it remains difficult to be precise about OA numbers, and especially difficult to make accurate predictions about future growth.
Like all attempts to understand and predict the world by means of numbers and statistics, much depends on how one derives them in the first place, how one crunches them, and how one subsequently interprets the results. In the case of OA, a key question that emerges is whether Gold OA is able on its own to accelerate the growth of OA to the degree that the OA movement would wish.
Why is it necessary to fret over such things? It is necessary for a number of reasons, but above all because if OA advocates knew exactly what was happening, and why, they would be able to put their main effort into those activities most likely to achieve their goal.
Vitally, they would be better able to answer a question that has plagued the movement for many years: Should the priority be given to Green or to Gold OA?
In the PDF file attached below I am publishing a Q&A interview with Gargouri. With a PhD in cognitive informatics, Gargouri has also participated in projects dealing with knowledge management, semantic web applications and ontologies. He has also taught in the computer science department at UQAM.
The interview includes contributions from Harnad — a leading OA advocate and self-styled archivangelist.
####
If you wish to read the interview please click on the link below.
I am publishing it under a Creative Commons licence, so you are free to copy and distribute it as you wish, so long as you credit me as the author, do not alter or transform the text, and do not use it for any commercial purpose.
To read the interview (as a PDF file) click here.
See also: John Whitfield Open access comes of age Nature, 21st June 2011
Tuesday, June 14, 2011
Bernard Rentier Interview in Portuguese
Brazilian Open Access advocate Dr Hélio Kuramoto is currently translating the recent interview I did with Bernard Rentier, Rector of the University of Liège, into Portuguese.
Dr Kuramoto plans to undertake the translation in stages. The first part was posted today, and can be accessed here.
Thank you Hélio!
Dr Kuramoto plans to undertake the translation in stages. The first part was posted today, and can be accessed here.
Thank you Hélio!
Sunday, June 12, 2011
UK politicians puzzle over peer review in an open access environment
The UK House of Commons Science & Technology Committee is currently conducting an inquiry into peer review. The third public event of the inquiry was held on Monday 23rd May, when the Committee heard evidence from experts on open access publishing and post-publication review, and from representatives of the research community.
SOME OF THE ISSUES EXPLORED BY THE COMMITTEE ARE HIGHLIGHTED IN THE EDITED QUESTIONS AND ANSWERS LISTED BELOW. THE ASTERISKED HEADINGS ARE MINE.
I COMMENT ON THE SESSION AT THE BOTTOM OF THIS POST.
UPDATE: THE COMMITTEE’S REPORT HAS NOW BEEN PUBLISHED. THE DETAILS ARE AVAILABLE HERE.
The Chair of the Science & Technology Committee is Andrew Miller, Labour MP for Ellesmere Port and Neston. Other politicians to pose the questions below were Graham Stringer, Labour MP for Blackley and Broughton, Roger Williams, Liberal Democrat MP for Brecon and Radnorshire, and Stephen Metcalfe, Conservative MP for South Basildon and East Thurrock. (A full list of Committee members is available here).
The hearing was split into two sessions.
Those giving evidence in the first session were Dr Rebecca Lawrence, Director, New Product Development at Faculty of 1000 Ltd., Mark Patterson, Director of Publishing at the Public Library of Science, Dr Michaela Torkar, Editorial Director at Biomed Central, and Dr Malcolm Read OBE, Executive Secretary of JISC.
SOME OF THE ISSUES EXPLORED BY THE COMMITTEE ARE HIGHLIGHTED IN THE EDITED QUESTIONS AND ANSWERS LISTED BELOW. THE ASTERISKED HEADINGS ARE MINE.
I COMMENT ON THE SESSION AT THE BOTTOM OF THIS POST.
UPDATE: THE COMMITTEE’S REPORT HAS NOW BEEN PUBLISHED. THE DETAILS ARE AVAILABLE HERE.
The Chair of the Science & Technology Committee is Andrew Miller, Labour MP for Ellesmere Port and Neston. Other politicians to pose the questions below were Graham Stringer, Labour MP for Blackley and Broughton, Roger Williams, Liberal Democrat MP for Brecon and Radnorshire, and Stephen Metcalfe, Conservative MP for South Basildon and East Thurrock. (A full list of Committee members is available here).
The hearing was split into two sessions.
== FIRST SESSION ==
Tuesday, June 07, 2011
The OA Interviews: Bernard Rentier, Rector of the University of Liège
What is striking about Open Access (OA) is that it so obviously the right and rational way for the research community to respond to the networked world. Indeed, one could aptly describe it as a no-brainer — or, as OA advocate Stevan Harnad likes to put it, OA is “raincoat science” (“It's raining, kids, and you're getting wet: Time to put on your raincoats!”).
What is odd about OA is that so few in the research community yet appear to have understood (or at least accepted) its inevitability. Such myopia is doubtless no accident — many have a vested interest in the status quo, while others instinctively fear change, and have an irrational abhorrence of the new.
As a consequence, what is surely “inevitable and optimal” has been delayed now for over a decade and a half: While universal OA could in theory be realised practically overnight, it is estimated that still only around 30% of the world's academic and scientific literature is freely available on the Web.
Fortunately some do “get it” the moment the concept is explained to them — as did Bernard Rentier, a professor of virology and immunology at the University of Liège (ULg), in 1998.
Rentier had been appointed vice-rector in charge of research policy and libraries at ULg the year before — a position that inevitably focused his professorial mind on a number of problems for which OA turns out to be the solution. This became apparent to him during a conversation he had with a science librarian, who introduced him to the concept behind the then incipient OA movement.
“[I]t was immediately obvious to me that this was the logical approach to take,” says Rentier, “especially as the new technology that was then emerging made it entirely possible.”
What is odd about OA is that so few in the research community yet appear to have understood (or at least accepted) its inevitability. Such myopia is doubtless no accident — many have a vested interest in the status quo, while others instinctively fear change, and have an irrational abhorrence of the new.
As a consequence, what is surely “inevitable and optimal” has been delayed now for over a decade and a half: While universal OA could in theory be realised practically overnight, it is estimated that still only around 30% of the world's academic and scientific literature is freely available on the Web.
Fortunately some do “get it” the moment the concept is explained to them — as did Bernard Rentier, a professor of virology and immunology at the University of Liège (ULg), in 1998.
Rentier had been appointed vice-rector in charge of research policy and libraries at ULg the year before — a position that inevitably focused his professorial mind on a number of problems for which OA turns out to be the solution. This became apparent to him during a conversation he had with a science librarian, who introduced him to the concept behind the then incipient OA movement.
“[I]t was immediately obvious to me that this was the logical approach to take,” says Rentier, “especially as the new technology that was then emerging made it entirely possible.”
Friday, May 20, 2011
More on the UK Inquiry into Peer Review
The UK House of Commons Science & Technology Committee is currently conducting an inquiry into peer review. The second public event of the inquiry was held at Portcullis House, in London, on May 11th.
(NOTE: The third public event will be held next Monday, 23rd May, and will explore Open Access (OA) and post-publication review — details and a link are available at the bottom of this post; some subsequent abstracts and commentary are available here).
UPDATE: THE COMMITTEE’S REPORT HAS NOW BEEN PUBLISHED. THE DETAILS ARE AVAILABLE HERE.
The second public event was split into two sessions.
The first Session was concerned with publication ethics. For this the witnesses were Tracey Brown, Managing Director of Sense About Science, and Dr Liz Wager, Chair of the Committee on Publication Ethics (COPE) and Board Member of the UK Research Integrity Office (UKRIO).
The second session took evidence from publishers. Here the witnesses were Mayur Amin, Senior Vice President, Research and Academic Relations at Elsevier, Dr Philip Campbell, Editor-in-Chief of Nature Publishing Group, Robert Campbell, Senior Publisher at Wiley-Blackwell, Dr Fiona Godlee, Editor-in-Chief at the BMJ Group, and Dr Andrew Sugden, Deputy Editor and International Managing Director of Science.
The Chair of the Science & Technology Committee is Andrew Miller, Labour MP for Ellesmere Port and Neston.
Other politicians to pose the questions extracted from the transcript and listed under the video of the event inserted below were Graham Stringer, Labour MP for Blackley and Broughton, David Morris, Conservative MP for Morecambe and Lunesdale, Roger Williams, Liberal Democrat MP for Brecon and Radnorshire, and Gavin Barwell, Conservative MP for Croydon Central. (A full list of Committee members is available here).
The (uncorrected) transcript of the meeting is available here.
The video of the event can be accessed here.
SOME OF THE ISSUES EXPLORED BY THE COMMITTEE ARE HIGHLIGHTED IN THE QUESTIONS AND ANSWERS LISTED BELOW
(NOTE: The third public event will be held next Monday, 23rd May, and will explore Open Access (OA) and post-publication review — details and a link are available at the bottom of this post; some subsequent abstracts and commentary are available here).
UPDATE: THE COMMITTEE’S REPORT HAS NOW BEEN PUBLISHED. THE DETAILS ARE AVAILABLE HERE.
The second public event was split into two sessions.
The first Session was concerned with publication ethics. For this the witnesses were Tracey Brown, Managing Director of Sense About Science, and Dr Liz Wager, Chair of the Committee on Publication Ethics (COPE) and Board Member of the UK Research Integrity Office (UKRIO).
The second session took evidence from publishers. Here the witnesses were Mayur Amin, Senior Vice President, Research and Academic Relations at Elsevier, Dr Philip Campbell, Editor-in-Chief of Nature Publishing Group, Robert Campbell, Senior Publisher at Wiley-Blackwell, Dr Fiona Godlee, Editor-in-Chief at the BMJ Group, and Dr Andrew Sugden, Deputy Editor and International Managing Director of Science.
The Chair of the Science & Technology Committee is Andrew Miller, Labour MP for Ellesmere Port and Neston.
Other politicians to pose the questions extracted from the transcript and listed under the video of the event inserted below were Graham Stringer, Labour MP for Blackley and Broughton, David Morris, Conservative MP for Morecambe and Lunesdale, Roger Williams, Liberal Democrat MP for Brecon and Radnorshire, and Gavin Barwell, Conservative MP for Croydon Central. (A full list of Committee members is available here).
The (uncorrected) transcript of the meeting is available here.
The video of the event can be accessed here.
SOME OF THE ISSUES EXPLORED BY THE COMMITTEE ARE HIGHLIGHTED IN THE QUESTIONS AND ANSWERS LISTED BELOW
Thursday, May 19, 2011
UK Inquiry into Peer Review
On 27th January 2011 the UK House of Commons Science & Technology Committee announced that it planned to hold an inquiry into peer review. That inquiry is now underway. Below are some links to the first public event held to take oral evidence. (Click here for details of the second public event).
The terms of reference for the inquiry are the following:
The Committee welcomes submissions on all aspect of the process and among the issues it is likely to examine are the following:
The terms of reference for the inquiry are the following:
The Committee welcomes submissions on all aspect of the process and among the issues it is likely to examine are the following:
- the strengths and weaknesses of peer review as a quality control mechanism for scientists, publishers and the public;
- measures to strengthen peer review;
- the value and use of peer reviewed science on advancing and testing scientific knowledge;
- the value and use of peer reviewed science in informing public debate;
- the extent to which peer review varies between scientific disciplines and between countries across the world;
- the processes by which reviewers with the requisite skills and knowledge are identified, in particular as the volume of multi-disciplinary research increases;
- the impact of IT and greater use of online resources on the peer review process; and
- possible alternatives to peer review.
The first public event took place on May 4th, with oral evidence being given by a number of experts, including:
- Dr Nicola Gulley, Editorial Director, Institute of Physics Publishing Ltd
- Professor Ronald Laskey CBE FRS FMedSci, Vice-President, Academy of Medical Sciences
- Dr Robert Parker, Interim Chief Executive, Royal Society of Chemistry
- Professor John Pethica FRS, Physical Secretary and Vice-President, Royal Society
The opening question from the chair of the Committee, Andrew Miller, Labour MP for Ellesmere Port and Neston, was:
Q: Peer review is perceived to be "fundamental to scholarly communications". If it disappeared tomorrow, what would the consequences be?
The first reply, from Dr Robert Parker, was:
A: You would have to come up with something else with which to replace it. There isn’t anything very obvious to replace peer review with currently. The danger would be to the scientific record, really. The importance of it is laid out in the evidence that has been submitted with great clarity from most people who have submitted evidence in writing to this review. The value and quality of that scientific record is paramount, and peer review helps to keep that in place.
The written evidence Dr Parker refers to can be read here.
Q: Peer review is perceived to be "fundamental to scholarly communications". If it disappeared tomorrow, what would the consequences be?
The first reply, from Dr Robert Parker, was:
A: You would have to come up with something else with which to replace it. There isn’t anything very obvious to replace peer review with currently. The danger would be to the scientific record, really. The importance of it is laid out in the evidence that has been submitted with great clarity from most people who have submitted evidence in writing to this review. The value and quality of that scientific record is paramount, and peer review helps to keep that in place.
The written evidence Dr Parker refers to can be read here.
Tuesday, March 22, 2011
Of Citizenship and Software
What is the role, purpose and point of software? Most people might assume that question to have an obvious answer: as the Berkman Center for Internet & Society puts it, software is “the programs or other ‘instructions’ that a computer needs to perform specific tasks. Examples of software include word processors, e-mail clients, web browsers, video games, spread sheets, accounting tools and operating systems.”
What this description omits of course is human agency, which ultimately determines what software does, how it does it, the degree to which it supports or undermines the rules and laws of society, and how it encourages or discourages ordinary citizens to participate in the process of defining those rules and laws.
Lawrence Lessig came to understand the power of software to construct and shape our world when he was (briefly) “special master” during the Microsoft antitrust case. As he later put it to me, “[Y]ou can code software however you want, to produce whatever kind of product you want. And that capability is unique with software: you can't, for instance, say that an automobile will be something that is a transmission and a radio wrapped in one. But you can do exactly that with software, because software is so plastic.”
As such, he added, the Microsoft case was just “a particular example of a more general point about how you need to understand the way in which technology and policy interact.”
Yet, as more and more of our lives are organised and controlled by computers, and the role that software plays in society becomes increasingly central, most people still assume that the virtual world that opens up before them when they switch on the computer, and the choices they are offered onscreen, is how things are and ought to be — not a consequence of the way in which the underlying software has been coded.
Most of us now realise that there are bad guys in cyberspace — people who will try to steal your identity, or harass you in some way — but we too often fail to understand that the computer-generated world we enter, and what it does and does not allow us to do, has been specifically constructed to behave in that way. It is not the way things inevitably have to be, but the way someone has decided to code the underlying software. Importantly, how software is written also has a direct impact on our lives, and the world we inhabit off-screen.
For that reason the software choices that individuals, companies, organisations and governments make have important political, economic and social consequences for us all.
Yet when we attend a basic computer course we are instructed how to point and click, how to send and receive email and, perhaps, how to create a simple database — but we are not told why our choice of software is important, why it is imperative to insist that our governments and political administrators use open data formats, and why software raises important ethical issues.
Importantly, it is not made clear to us that we can challenge the way in which software is written and used.
Frustrated by the limits and inadequacies of most computer courses Marco Fioretti, a former telecom engineer based in Rome who teaches about digital rights issues, has added to his repertoire a basic online course on digital citizenship.
The course, which is open to everybody, will not teach students how to code, or send email, says Fioretti, but how computers impact on and determine what happens in the real world and how, by means of active citizenship, we can help shape and influence that process, and create a more democratic world as a result. As he puts it, “Understanding digital issues is no longer optional for citizens. It's necessary whether one likes computer or not.”
Below Fioretti explains the background to his course, and the thinking that lies behind it. Unsurprisingly, we learn that Fioretti has been influenced by Richard Stallman and the Free Software Movement, although he is not an uncritical fan.
What this description omits of course is human agency, which ultimately determines what software does, how it does it, the degree to which it supports or undermines the rules and laws of society, and how it encourages or discourages ordinary citizens to participate in the process of defining those rules and laws.
Lawrence Lessig came to understand the power of software to construct and shape our world when he was (briefly) “special master” during the Microsoft antitrust case. As he later put it to me, “[Y]ou can code software however you want, to produce whatever kind of product you want. And that capability is unique with software: you can't, for instance, say that an automobile will be something that is a transmission and a radio wrapped in one. But you can do exactly that with software, because software is so plastic.”
As such, he added, the Microsoft case was just “a particular example of a more general point about how you need to understand the way in which technology and policy interact.”
Yet, as more and more of our lives are organised and controlled by computers, and the role that software plays in society becomes increasingly central, most people still assume that the virtual world that opens up before them when they switch on the computer, and the choices they are offered onscreen, is how things are and ought to be — not a consequence of the way in which the underlying software has been coded.
Most of us now realise that there are bad guys in cyberspace — people who will try to steal your identity, or harass you in some way — but we too often fail to understand that the computer-generated world we enter, and what it does and does not allow us to do, has been specifically constructed to behave in that way. It is not the way things inevitably have to be, but the way someone has decided to code the underlying software. Importantly, how software is written also has a direct impact on our lives, and the world we inhabit off-screen.
For that reason the software choices that individuals, companies, organisations and governments make have important political, economic and social consequences for us all.
Yet when we attend a basic computer course we are instructed how to point and click, how to send and receive email and, perhaps, how to create a simple database — but we are not told why our choice of software is important, why it is imperative to insist that our governments and political administrators use open data formats, and why software raises important ethical issues.
Importantly, it is not made clear to us that we can challenge the way in which software is written and used.
Frustrated by the limits and inadequacies of most computer courses Marco Fioretti, a former telecom engineer based in Rome who teaches about digital rights issues, has added to his repertoire a basic online course on digital citizenship.
The course, which is open to everybody, will not teach students how to code, or send email, says Fioretti, but how computers impact on and determine what happens in the real world and how, by means of active citizenship, we can help shape and influence that process, and create a more democratic world as a result. As he puts it, “Understanding digital issues is no longer optional for citizens. It's necessary whether one likes computer or not.”
Below Fioretti explains the background to his course, and the thinking that lies behind it. Unsurprisingly, we learn that Fioretti has been influenced by Richard Stallman and the Free Software Movement, although he is not an uncritical fan.
 |
Marco Fioretti |
Monday, March 14, 2011
The Demise of the Big Deal?
Claudio Aspesi — an analyst based at the sell-side research firm Sanford Bernstein — predicts a difficult future for Reed Elsevier, particularly for its scholarly journal business. He also predicts the demise of the Big Deal, the business model in which scholarly publishers sell access to multiple journals by means of a single electronic subscription.
In a report published last year Aspesi warned that a combination of the global financial crisis and the rise of the Open Access (OA) movement would impact negatively on the revenues of scholarly publishers. Yet, he said, Reed Elsevier appeared to be "in denial on the magnitude of the issue potentially affecting scientific publishing".
A year later Aspesi appears even more gloomy. In his most recent report he has downgraded Reed Elsevier to “underperform”, and warns that the widely-used “Big Deal” arrangement, is becoming “unsustainable in the current funding environment.”
While the Big Deal may have worked well as a solution for over a decade, he says, we can expect to see research libraries start cancelling their contracts — a development that will “lead to revenue and earnings decline”.
Speaking to me last week Aspesi repeated his belief that Reed Elsevier is in denial. “[I]f management has a Plan B, they have certainly kept it under wraps, and everything they have said supports my current view that they are in denial”, he told me.
Of course Reed Elsevier is not the only publisher threatened by the current climate, and clinging in desperation to the Big Deal. When I spoke to the CEO of Springer Derk Haank last year, for instance, he told me that it was “in the interests of everyone — publishers and librarians — to keep the Big Deal going.”
Unlike Elsevier, however, Springer embraced OA seven years ago.
Cleary these are difficult times. And the nub of the matter is money: With the number of research papers produced annually constantly rising publishers expect to increase their prices each year to reflect that rise. And the Big Deal, they believe, is the best way of ensuring they can do this, while providing research institutions with access to the greatest number of journals.
Librarians, however, are adamant that prices must fall. To that end library organisations like Research Libraries UK (RLUK) — which represents the libraries of Russell Group universities — have begun organising public campaigns designed to pressure big publishers to end up-front payments, to allow them to pay in sterling, and to reduce their subscription fees by 15%.
The RLUK campaign is being led by Deborah Shorley, the director of Imperial College London Library. “[T]he fact is”, she said recently, “we don’t have money in the sector and we can’t afford to go on spending as we have.”
Meanwhile OA is both a threat and an opportunity for scholarly publishers, and Reed Elsevier should have embraced it by now, says Aspesi.
OA advocates want to see all papers arising from publicly-funded research made freely available on the Web. Green OA, or self-archiving, would see researchers doing this themselves, and so pose a significant threat to publishers’ revenues.
For that reason, suggests Aspesi, Elsevier should have pre-emptively experimented with Gold OA — or OA publishing — as Springer did. By levying a fee for publishing papers, rather than a subscription to read them, publishers can hope to reduce costs, and perhaps maintain their current profit levels as a result.
But there are no certainties in scholarly publishing today. And while many in the research community also believe that OA publishing offers the best long-term solution, it is far from evident that it will resolve the affordability problem. It could also lead to a decline in the quality of published research.
Either way, prices look set to fall over time. This would inevitably mean a decline in the revenues of scholarly publishers.
In short, both scholarly publishers and the research community are caught between a rock and a hard place. For researchers, suggests Aspesi, it may mean having to accept that they will be able to publish fewer papers in the future.
Aspesi explains why in the email interview below, conducted at the end of last week.
Claudio Aspesi
RP: Bloomberg reported in February that Elsevier’s most recent profits are up more than 60%, to $1.03bn. In a report published last week, however, you downgraded Elsevier to “underperform”. Why?
CA: There are several reasons, but it ultimately boils down to two. The stock market is a powerful discounting mechanism for future events: a company can be performing very well but have a share price that overvalues the future (or perform very badly but have a share price that undervalues the future).
In the case of Reed Elsevier, I believe that market expectations about the future of Elsevier are too optimistic. In addition, I have concerns about the future performance of other parts of the portfolio (like their US legal business) that also lead me to think the stock is overvalued.
RP: In your report you say that the days of The Big Deal are coming to an end. The death of the Big Deal has been long predicted. Why do you think we have finally reached the “crunch point” as you call it?
CA: I think there are three trends overlapping: a long term unsustainable trend, a cyclical funding crisis and a more tough minded and analytical community of librarians.
Revenues for STM publishers have been rising faster than library budgets for many years, and librarians have had to cope with this discrepancy by cutting back their spending in other areas.
We can all speculate whether this could have continued indefinitely or not, but it does not really matter: the financial crisis has led to widespread cuts in library budgets, forcing research libraries to take a harder look at what they spend on serials.
Overlay to the funding crisis the realisation that Big Deals forced librarians to take journals that nobody (or almost nobody) really accessed and you set up a perfect storm.
RP: The consensus is that Elsevier will see 4% growth in 2013. You predict only 2%, possibly lower. Is that correct?
CA: Yes. Please bear in mind that Elsevier also sells books and databases and electronic retrieval services, and I expect these businesses to prove more resilient over time. It is the Big Deal itself that looks increasingly under pressure.
The funding issue
RP: When I spoke to Springer CEO Derk Haank at the end of last year he described The Big Deal as the “best thing since sliced bread”. He added: “The truth is that it is in the interests of everyone — publishers and librarians — to keep the Big Deal going.” What is he not seeing that you see?
CA: The funding issue. Of course — from a publisher’s perspective — to be able to gain some revenues, any revenues, from journals which were not really read is terrific. We have seen the usage data of some universities and found that as many as two thirds of the titles in some Big Deals were accessed once a month or less.
I suspect that many librarians, if they run the numbers, will find that one third or less of the titles they acquire through Big Deals really matter to their user community.
But it was nice for librarians to be able to offer “everything” to their users. They did not consider that the Big Deals were depriving libraries of the funding needed for other activities.
Professor Darnton of Harvard University published in December 2010 an article in the New York Review of Books (The Library: Three Jeremiads) which captures perfectly the impact of publishers revenue growth on academic libraries and on the academic press.
RP: Haank pointed out to me that the number of research papers is growing at 6% to 7% per year. “Librarians need to accept that if they want access to a continually growing database, then costs will need to go up a little bit,” he said. “We try to accommodate our customers, but at a certain point, we will hit a wall.”
This seems to be the nub of the problem: Publishers are processing more and more papers each year, and so they expect to raise their prices. But the research community insists that it cannot afford to pay any more. Nevertheless, Haank assumes that eventually more money will be found. As he put it, “[O]ver time, I expect people will realise that scientists have to have sufficient funding to keep abreast of new developments.”
You, however, believe that publishers will simply have to accept that their revenues are going to fall, because there really is no more money?
CA: I have no doubt that — over time — adjustments would be made. But it remains to be seen if they need all the 2,200/2,400 journals that the each of the largest publishers maintain today.
You know, my job is not to pass judgement on how people run their business or to decry capitalism, only to advise investors whether they should buy or sell stocks.
I can observe, however, that there is something unhealthy about an industry which has managed to alienate its customers to the point their membership associations increasingly focus time and attention on how to overturn the industry structure. It is not a good thing to have your customers spend their time trying to put you out of business.
No Plan B?
RP: In a report you published last June you said that Elsevier was in denial about the situation. In this report you argue that even though the company was aware of the current problem as early as 2005, it ignored it. Is that correct? Why do you think they have no Plan B — as you put it?
CA: I dug out a presentation that Reed Elsevier organised for investors back in 2005 on Elsevier. At the bottom of a slide titled “Attractive Growth Markets”, and dedicated to show how global university and R&D funding were growing between 5 and 8% a year, was the sentence “Although library budgets have not kept pace: 1-3%”. If they thought they needed to address this issue, it did not show in the following years.
To be fair, the financial community also ignored that line. In my recent report, I quoted one of my own reports from December 2007 in which I wrote “Elsevier has still enjoyed a healthy underlying growth rate of 5%...We expect these levels of revenue growth to be sustained going forward”. I am just as guilty of ignoring it as everyone else.
RP: If you too were wrong in 2007 it is presumably possible that your assumptions are wrong today?
CA: Of course. I wish I could always be right.
But if management has a Plan B, they have certainly kept it under wraps, and everything they have said supports my current view that they are in denial. My report opens by quoting David Prosser, the Executive Director of Research Libraries UK, who told the Wall Street Journal back in November 2010 “We do not wish to cancel big deals, but we shall have no alternative unless the largest publishers substantially reduce their prices”.
Somehow, this issue has to be addressed; Elsevier may be in a better position than me to answer why they are not.
RP: I wonder if Elsevier might be stuck on the horns of a dilemma: They have to keep delivering profits for their shareholders, and this may be restricting their ability to do the right thing for their long-term future.
CA: Again, I think it would be interesting to know what the top management of Elsevier thinks is the right thing for their long term future. More of the same? Hope that funding returns to libraries by the end of 2011 so that the storm blows over? Go back to the 1-3% libraries funding growth that was already putting publisher revenue increases on a collision course with library budgets?
In Haank’s own words “We will hit a wall”. Does Elsevier think that walls stand only on the path of other publishers?
Open Access
RP: In September last year Elsevier launched its first Gold OA journal, The International Journal of Surgery Case Reports. Was that a good move on their part? Should they be doing more of that?
CA: Excellent companies experiment and try new concepts all the time.
RP: Does Open Access publishing offer a solution to the current difficulties confronting scholarly publishers? Some argue, for instance, that OA could provide them with the same revenues as they currently get from the Big Deal. Moreover, since some/much of the costs would be met by research funders, rather than libraries, new money would flow into the system?
CA: I have asked myself that question several times. Publishers talk about tapping the larger revenue streams that come from the funding of science, and Open Access would seem a logical way to do that.
I think what holds them back is the fear that the transition will be long and messy, with many researchers unable to secure the funding to pay for publication fees. Moreover, some journals with very high impact factors probably reject so many submissions that the publication fees would have to be absurdly high.
RP: In your earlier report you said that the real threat comes from self-archiving. Has that threat grown or eased in the last year would you say?
CA: It is remarkable how slowly the world of academic publishing changes from year to year. Just to show examples from a business we are all familiar with, at least to some extent, 12 months ago the iPad did not exist, Nokia was a well-liked mobile handset manufacturer, and so on; a year later, everyone talks about tablets and apps and Android.
Somehow, the academic community moves at a glacial pace. It is no wonder that investors decided that Open Access will never happen: they are used to seeing most activities and businesses change constantly.
RP: Can I push you on whether you see self-archiving as a greater or lesser threat than a year ago. Essentially we are talking about the growth of self-archiving mandates.
CA: Even on mandates we have seen little progress. There have been several new mandates from both public and private institutions, but there is such a patchwork of mandates (and little transparency on the compliance) that I doubt there is a single research library that has changed its buying patterns to reflect the new or the cumulated mandates.
An aggressive US Federal Government mandate could change that, both because it would affect a significant amount of research and because the rest of the world would likely feel pressured to follow suit. Even so, if the embargo of copyrighted material was long enough, the impact would be negligible.
RP: What are the implications of the current situation for the research community?
CA: If the Big Deal goes away altogether, fewer journals will be sustainable, which means that less research will be published. This headline sounds threatening for the research community, until you ask yourself how much of the research which is being published today is actually read. My guess is that if fewer subscription journals are published, something else will take their place, probably a combination of Open Access journals and self-archiving repositories.
If, on the other hand, only a meaningful portion of the Big Deals are discontinued, the publishers will be faced with the dilemma of whether to cull marginal journals or not, since they support their pricing for the portion of the market which would still buy Big Deals. My guess is that truly marginal titles would be discontinued, but it would be difficult to cut most.
RP: As you say, most articles are not read, and so librarians are currently paying for products that their patrons don’t use. One could perhaps argue that in an OA environment this would not matter since the customer becomes the author, not the reader, and authors have to publish to get promotion and tenure. As such, reading is not as important as publishing, and given the publish-or-perish pressures they are under one might assume that researchers would inevitably find the money to publish, even if it meant taking it from their own pockets. If correct, this suggests that publishers could preserve their revenues by embracing Gold OA?
CA: I am always puzzled by the hostility of the leading publishers towards Open Access. As you point out, there is no reason why revenues, over time, could not be roughly equal for the two models. I suspect the hostility derives from the need to operate in a period of transition that is difficult to manage. The fog of change can be daunting.
RP: Some in the research community warn of the danger of a catastrophic collapse of the scholarly communication system. Might they have a point?
CA: Yes and no. It is clear that if 15 to 30% of the academic publishing industry revenues were to evaporate over a three to five year period, the industry would have to slim down substantially.
Lower revenues would lead to fewer articles being published, and probably lower profits. Lower profits and dimmer prospects for profit growth may in turn lead to less investment in innovation as investors would demand that less capital is allocated to an industry with less future profit growth. Does this mean that there would be no innovation? Not necessarily.
Again, I suspect that other models could emerge to fill the space: Open Access journals, repositories, and perhaps some we cannot even imagine today.
A bubble
RP: So how would you characterise the current situation with regard to scholarly journal publishing?
CA: I will use an analogy that may be imperfect, but which contains some lessons. For a very long time, the music industry sold to consumers more music than consumers really wanted. It did so by forcing consumers to buy albums even when they may have just wanted one or two songs; in fact, it killed the single because it was not profitable enough, in spite of the fact it had always been very popular with consumers because it was cheap and it contained the music they wanted.
Then consumers figured out they could rip music from CDs, put it on a hard drive, and burn it on a blank CD. If their friends did not have the tracks they wanted, they could even go to peer to peer sites and get all the music they wanted, and — even better — it was free.
Physical piracy and peer to peer distribution of music did lead to the catastrophic collapse of the music industry in many countries because the revenue base shrank to the point that it became impossible to sign up artists and produce albums. But the music industry is still in existence, albeit on a much smaller scale, and an endless stream of innovators is trying to find ways to revive it with all kinds of new consumer propositions.
Perhaps there is too much research being published today, and the value of the long tail of research is probably modest. This industry benefitted — just like many others — from a bubble, and that bubble has now burst. But the basic demand for dissemination and retrieval of important scientific and medical research is still there, and I am confident it will continue to be fulfilled.
RP: Do you still think that Reed Elsevier should go for a “progressive divestiture”, as you suggested last year?
CA: It all depends on the value of the sum of its parts at any given time. Last time we calculated it, back in January, we decided there was a 15/20% upside to the share price — meaningful but not staggering. I can think of companies in my coverage where the upside would be as much as 50 to 60%. In any case, Reed Elsevier management seems to show no inclination to pursue this path.
RP: Thank you for your time.
Claudio Aspesi’s latest report can be accessed here.
Tuesday, March 08, 2011
PLoS ONE, Open Access, and the Future of Scholarly Publishing: Response from PLoS
Last year a researcher drew my attention to a row that had erupted over an article published in the Open Access journal PLoS ONE. Believing that the row raised some broader questions about PLoS ONE, and about Open Access, I sent a list of questions to PLoS. The publisher eventually declined to answer them, sending over a statement instead. I nevertheless decided to write something, but sent a copy of what I had written to PLoS before publishing it. And I invited PLoS to comment on what I had written. Below is the publisher’s response:
We were disappointed to see Richard Poynder’s current article about PLoS, but grateful that Richard sent us a draft, so that we could prepare a brief response.
Richard is absolutely correct about one thing – we at PLoS are really committed to open access, and we are doing our absolute best to inspire a broader transformation in scientific communication. We make no apology for that. We expect to be watched and scrutinized and indeed have been the subject of some criticism over the years, but not at the length or with the amount of negativity that we see in Richard’s essay.
Much of the article focuses on PLoS ONE and Richard uses some selected examples (about 5 of them) from the more than 17,000 peer-reviewed articles that we’ve published in PLoS ONE to draw much broader conclusions about the quality of its content. Although we would be the first to agree that PLoS ONE isn’t perfect, neither is any journal, as Richard points out – although not until around 30 pages into the article. But, just to quote one statistic, is it not more striking that of the 4400 articles published in PLoS ONE in 2009 around 55% of them have been cited 3 or more times (Scopus data)? The evidence points to the fact that PLoS ONE is attracting a vast amount of high-quality content.
PLoS ONE is attempting to challenge the conventional model of a journal. The peer review criteria for PLoS ONE are focused on rigour, ethical conduct and proper reporting. Reviewers and editors are not asked to judge work on the basis of its potential impact. Our argument is that judgments about impact and relevance can be left (and might be best left) until after publication, and this argument is clearly resonating with the tens of thousands of researchers who work with and support PLoS ONE as authors, reviewers or editors. It’s now also resonating with many other non-profit and for-profit publishers who are exploring the same model.
We do not argue that the PLoS ONE approach is the only way to publish research, and indeed we view PLoS ONE as just one aspect of a much more fundamental transformation of scholarly communication.
Another aspect of that transformation is in the assessment and organization of research findings, which is currently done using conventional journals. That’s why we have launched article-level metrics and PLoS Hubs as new and alternative approaches to post-publication evaluation. There will be much more to come from PLoS and many other innovators.
At several points, Richard’s article uses quotes from staff, press releases and so on that are now several years old and misses the point that much has changed even in the short few years since PLoS ONE launched. We are learning all the time from PLoS ONE. His frequent quotes from PLoS staff also show that we’ve answered many of his questions (including some less than friendly ones) over the years.
Nevertheless, he places great emphasis on the fact that we declined to answer a set of more than 20 detailed and complex questions about general aspects of PLoS ONE, as a follow up to a series of exchanges about the peer review process on a particular PLoS ONE article about which there was some disagreement. Indeed we posted a comment to try and clarify the issues in light of Richard’s questions, and comments from researchers. We were surprised by the number and wide-ranging nature of Richard’s subsequent questions about PLoS ONE, and chose not to answer them because we felt that the issues surrounding the PLoS ONE article were closed. If Richard had signaled his intention to write a lengthy article about the history and status of PLoS at the outset of the exchange, our response might have been rather different.
But the more significant point is that PLoS ONE has evolved since its launch. We did originally place a lot of emphasis on ‘commenting’ and ‘rating’ as tools for post-publication assessment, but we rapidly realized that much commentary and other activity happens elsewhere. If we could capture this activity and add it to the PLoS articles (in all our journals), that could be a powerful approach to post-publication assessment, and could also be used to filter and organize content. Thus, the article-level metrics project was born. The PLoS ONE editorial and publishing processes are also under constant review and revision as the journal’s size and complexity has grown, and we post some updated general information about these processes on the PLoS ONE web site.
Another theme in Richard’s article is whether PLoS ONE represents value for money. PLoS Journals are an ecosystem and they all contribute to the whole, both financially and to PLoS’ reputation and brand. Looked at in isolation, PLoS ONE, as well as the Community Journals (PLoS Genetics, PLoS Computational Biology, PLoS Pathogens and PLoS Neglected Tropical Diseases), all make a positive financial contribution to PLoS. They help to support PLoS Biology and PLoS Medicine, as well as the development of the journal websites and other important initiatives such as article-level metrics, PLoS Hubs, PLoS Currents and our work on advocacy. From the PLoS authors’ perspective, wherever they publish in PLoS their work will reach anyone with an interest in it, and their work will be stamped with a brand that is associated with social change, innovation and quality. There’s much more to value than the direct costs of publishing a single article.
And a final point on value. Publishing in the conventional system is estimated to cost the academy around $4500 per article. What PLoS (and for that matter BioMed Central, Hindawi, Co-Action, Copernicus and other successful open-access publishers) is showing is that high-quality publishing can be supported by publication fees that are substantially less than the costs of the conventional system.
There is a revolution in the making, and despite the wealth of support that we are seeing, it won’t be comfortable for everyone. We need constructive criticism, but also some optimism and creativity to make it work. There are grounds for hope that we’ve moved beyond the antagonism that has characterized many discussions around open access. There really is a lot to celebrate.
The article can be accessed here.
**** UPDATE FEBRUARY 2012: AN INTERVIEW WITH PUBLIC LIBRARY OF SCIENCE CO-FOUNDER MICHAEL EISEN IS NOW AVAILABLE HERE ****
We were disappointed to see Richard Poynder’s current article about PLoS, but grateful that Richard sent us a draft, so that we could prepare a brief response.
Richard is absolutely correct about one thing – we at PLoS are really committed to open access, and we are doing our absolute best to inspire a broader transformation in scientific communication. We make no apology for that. We expect to be watched and scrutinized and indeed have been the subject of some criticism over the years, but not at the length or with the amount of negativity that we see in Richard’s essay.
Much of the article focuses on PLoS ONE and Richard uses some selected examples (about 5 of them) from the more than 17,000 peer-reviewed articles that we’ve published in PLoS ONE to draw much broader conclusions about the quality of its content. Although we would be the first to agree that PLoS ONE isn’t perfect, neither is any journal, as Richard points out – although not until around 30 pages into the article. But, just to quote one statistic, is it not more striking that of the 4400 articles published in PLoS ONE in 2009 around 55% of them have been cited 3 or more times (Scopus data)? The evidence points to the fact that PLoS ONE is attracting a vast amount of high-quality content.
PLoS ONE is attempting to challenge the conventional model of a journal. The peer review criteria for PLoS ONE are focused on rigour, ethical conduct and proper reporting. Reviewers and editors are not asked to judge work on the basis of its potential impact. Our argument is that judgments about impact and relevance can be left (and might be best left) until after publication, and this argument is clearly resonating with the tens of thousands of researchers who work with and support PLoS ONE as authors, reviewers or editors. It’s now also resonating with many other non-profit and for-profit publishers who are exploring the same model.
We do not argue that the PLoS ONE approach is the only way to publish research, and indeed we view PLoS ONE as just one aspect of a much more fundamental transformation of scholarly communication.
Another aspect of that transformation is in the assessment and organization of research findings, which is currently done using conventional journals. That’s why we have launched article-level metrics and PLoS Hubs as new and alternative approaches to post-publication evaluation. There will be much more to come from PLoS and many other innovators.
At several points, Richard’s article uses quotes from staff, press releases and so on that are now several years old and misses the point that much has changed even in the short few years since PLoS ONE launched. We are learning all the time from PLoS ONE. His frequent quotes from PLoS staff also show that we’ve answered many of his questions (including some less than friendly ones) over the years.
Nevertheless, he places great emphasis on the fact that we declined to answer a set of more than 20 detailed and complex questions about general aspects of PLoS ONE, as a follow up to a series of exchanges about the peer review process on a particular PLoS ONE article about which there was some disagreement. Indeed we posted a comment to try and clarify the issues in light of Richard’s questions, and comments from researchers. We were surprised by the number and wide-ranging nature of Richard’s subsequent questions about PLoS ONE, and chose not to answer them because we felt that the issues surrounding the PLoS ONE article were closed. If Richard had signaled his intention to write a lengthy article about the history and status of PLoS at the outset of the exchange, our response might have been rather different.
But the more significant point is that PLoS ONE has evolved since its launch. We did originally place a lot of emphasis on ‘commenting’ and ‘rating’ as tools for post-publication assessment, but we rapidly realized that much commentary and other activity happens elsewhere. If we could capture this activity and add it to the PLoS articles (in all our journals), that could be a powerful approach to post-publication assessment, and could also be used to filter and organize content. Thus, the article-level metrics project was born. The PLoS ONE editorial and publishing processes are also under constant review and revision as the journal’s size and complexity has grown, and we post some updated general information about these processes on the PLoS ONE web site.
Another theme in Richard’s article is whether PLoS ONE represents value for money. PLoS Journals are an ecosystem and they all contribute to the whole, both financially and to PLoS’ reputation and brand. Looked at in isolation, PLoS ONE, as well as the Community Journals (PLoS Genetics, PLoS Computational Biology, PLoS Pathogens and PLoS Neglected Tropical Diseases), all make a positive financial contribution to PLoS. They help to support PLoS Biology and PLoS Medicine, as well as the development of the journal websites and other important initiatives such as article-level metrics, PLoS Hubs, PLoS Currents and our work on advocacy. From the PLoS authors’ perspective, wherever they publish in PLoS their work will reach anyone with an interest in it, and their work will be stamped with a brand that is associated with social change, innovation and quality. There’s much more to value than the direct costs of publishing a single article.
And a final point on value. Publishing in the conventional system is estimated to cost the academy around $4500 per article. What PLoS (and for that matter BioMed Central, Hindawi, Co-Action, Copernicus and other successful open-access publishers) is showing is that high-quality publishing can be supported by publication fees that are substantially less than the costs of the conventional system.
There is a revolution in the making, and despite the wealth of support that we are seeing, it won’t be comfortable for everyone. We need constructive criticism, but also some optimism and creativity to make it work. There are grounds for hope that we’ve moved beyond the antagonism that has characterized many discussions around open access. There really is a lot to celebrate.
The article can be accessed here.
**** UPDATE FEBRUARY 2012: AN INTERVIEW WITH PUBLIC LIBRARY OF SCIENCE CO-FOUNDER MICHAEL EISEN IS NOW AVAILABLE HERE ****
Monday, March 07, 2011
PLoS ONE, Open Access, and the Future of Scholarly Publishing
Open Access (OA) advocates argue that PLoS ONE is now the largest scholarly journal in the world. Its parent organisation — Public Library of Science (PLoS) — was co-founded in 2001 by Nobel Laureate Harold Varmus. What does the history of PLoS tell us about the development of PLoS ONE? What does the success of PLoS ONE tell us about OA? And what does the current rush by other publishers to clone PLoS ONE tell us about the future of scholarly communication?
Our story begins in 1998, in a coffee shop located on the corner of Cole and Parnassus in San Francisco. It was here, Harold Varmus reports, that the seeds of PLoS were sown, during a seminal conversation he had with colleague Patrick Brown. Only at that point did Varmus realise what a mess scholarly communication was in. Until then, he says, he had been “an innocent person who went along with the system as it existed”.
Enlightenment began when Brown pointed out to Varmus that when scientists publish their papers they routinely (and without payment) assign ownership in them to the publisher. Publishers then lock the papers behind a paywall and charge other researchers a toll (subscription) to read them, thereby restricting the number of potential readers.
Since scientists crave readers (and the consequent “impact”) above all else, Brown reminded Varmus, the current system is illogical, counterproductive, and unfair to the research community. While it may have been necessary to enter into this Faustian bargain with publishers in a print environment (since it was the only way to get published, and print inevitably restricts readership), Brown added, it is no longer necessary in an online world — where the only barriers to the free-flow of information are artificial ones.
Physicists, Brown said, have overcome this “access” problem by posting preprints of all their papers on a web-based server called arXiv. Created by Paul Ginsparg in 1991, arXiv allows physical scientists to ensure that their work is freely available to all. “Should not the biomedical sciences be doing something similar?” Brown asked Varmus.
It was doubtless no accident that Brown — who had previously worked with the Nobel Laureate — chose Varmus as his audience for a lecture on scholarly publishing: at the time Varmus was director of the National Institutes of Health (NIH) — the largest source of funding for medical research in the world. He was, therefore, ideally placed to spearhead the revolution that Brown believed was necessary.
Fortunately for the open access movement (as it later became known) Varmus immediately grasped the nature of the problem — aided perhaps by some residual Zen wisdom emanating from the walls of the coffee shop they were sitting in, which had once been the Tassajara Bakery. Varmus emerged from the café persuaded that it would be a good thing if publicly-funded research could be freed from the publishers’ digital padlocks. And he went straight back to the NIH to consult with colleagues to that end.
Again fortuitously, one of the first people Varmus broached the topic with was David Lipman — director of the NIH-based National Center for Biotechnology Information (NCBI). NCBI was home to the OA sequence database GenBank, and Lipman was an enthusiastic supporter of the notion that research should be freely available on the Web. By now Varmus’ conversion was complete.
This conversion was to see Varmus embark on a journey that would lead to the founding of a new publisher called Public Library of Science, the launch of two prestigious OA journals (PLoS Biology and PLoS Medicine), and subsequently to the creation of what OA advocates maintain is now the largest scholarly journal in the world — PLOS ONE.
As we shall see, Varmus’ journey was to prove no walk in the park, and some believe his project lost its bearings on the way. Rather than providing a solution, they argue, PLoS may have become part of the problem.
Certainly PLoS ONE has proved controversial. This became evident to me last year, when a researcher drew my attention to a row that had erupted over a paper the journal had published on “wind setdown”.
Even some of the journal’s own academic editors appeared to be of the view that the paper should not have been published (in its current form at least). As the row appeared to raise questions about PLoS ONE’s review process — and about PLoS ONE more broadly — I contacted PLoS ONE executive editor Damian Pattinson.
The response I got served only to pique my interest: While Pattinson invited me to send over a list of questions, I subsequently received an email from PLoS ONE publisher Peter Binfield informing me that it had been decided not to answer my questions after all.
To read on please click here (for a long PDF file).
The PDF includes a response from PLoS. I will also be publishing the response as a separate post. (Now available here).
This article was cited in a 2011 report on peer review in scientific publications by the British House of Commons Science and Technology Committee.
**** UPDATE FEBRUARY 2012: AN INTERVIEW WITH PUBLIC LIBRARY OF SCIENCE CO-FOUNDER MICHAEL EISEN IS NOW AVAILABLE HERE ****
**** UPDATE FEBRUARY 2012: AN INTERVIEW WITH PUBLIC LIBRARY OF SCIENCE CO-FOUNDER MICHAEL EISEN IS NOW AVAILABLE HERE ****
Friday, February 04, 2011
Audio of discussion on Open Data now available
Last August I sat down in a pub (The Panton Arms) in Cambridge to discuss Open Data with Peter Murray-Rust, Jordan Hatcher and others.
The event was the first of what Murray-Rust has dubbed the Panton Discussions. Murray-Rust is a Reader in Molecular Informatics at Cambridge University.
The Panton Arms regularly plays host to members of the University Chemistry Department, so it was an obvious place to meet. This special relationship between Cambridge chemists and The Panton Arms is doubtless a consequence of the pub being just down the road from the Chemistry Department.
However, it could be that The Panton Arms is an appropriate location for discussing things like copyright, Open Data, Open Access, Creative Commons and the Public Domain for another reason.
The Panton Arms, and Panton Street (in which the pub is located), are associated with the Panton family. And in 1806 “Polite” Tommy Panton succeeded in having Parliament pass the Barnwell Enclosures Act, leading to the enclosure of what was then farmland.
Today many argue that the frequent and increasingly maximalist changes made to copyright laws represent a new enclosure movement. And it is partly in response to that process that we have seen a proliferation of “free” and “open” movements like Open Data and Open Access – with the aim of preventing, or at least mitigating, the new enclosures.
The audio of our discussion has now been made available, and can be accessed here. For the transcript click on the tab marked transcript.
Subscribe to:
Posts (Atom)