Some Open Access (OA) advocates shocked by the shooting at the University of Alabama in Huntsville (UAH) may not be aware of the OA link to the story: The last paper authored by Amy Bishop — the researcher charged with killing three of her colleagues – was published in the International Journal of General Medicine, an OA journal published by New Zealand-based OA publisher Dove Medical Press. Since Bishop cited three of her teenage daughters as co-authors of the paper, and Dove charges an article-processing fee for publishing papers, commentators have been quick to conclude that OA is little more than vanity publishing. Dove is a member of the Open Access Publishers Association (OASPA). Is OASPA asleep at the wheel?
On 12th February 2010, Amy Bishop, a 45-year-old biology professor in the Department of Biological Sciences at the University of Alabama in Huntsville, allegedly shot dead three of her colleagues and wounded three others — in what was quickly characterised as a rampage shooting.
Reporting on the incident the following day the New York Daily News explained: “Bishop returned to the faculty meeting shortly after being denied tenure around 4 p.m. and opened fire in a third-floor meeting room at the Shelby Center for Science and Technology,”
The paper added that if she is convicted Bishop, a Harvard University-trained neuroscientist, could face the death penalty.
Not adequate
As news reporters and bloggers began sifting through the data in an effort to understand what had gone so tragically wrong, and why Bishop had been denied tenure (assumed to be the trigger for the shooting), they soon alighted on her web site.
“Most of Ms. Bishop's published papers concern nitric oxide, a molecule that cells use to communicate with other cells,” The Chronicle of Higher Education reported on 15th February. “But at high levels, nitric oxide is toxic. It is believed to play a role in the development of certain cancers — and also neurodegenerative disorders such as multiple sclerosis and Lou Gehrig's disease.
“If those mechanisms can be somehow turned into genetic therapies," continued The Chronicle, "they might eventually lead to new treatments for a variety of neurogenerative ailments. That was the premise of a $219,750 grant that Ms. Bishop received in 2008 from the National Institutes of Health.”
However, it seems that UAH had concluded that Bishop was not showing sufficient promise to justify being given tenure. “[A]ccording to several accounts — including a Chronicle interview with her husband on Sunday — Ms. Bishop's output was not regarded as adequate by the tenure committee at Alabama," explained The Chronicle. "She was notified last spring that she did not get tenure, and although she made several appeals of that decision, they were ultimately unsuccessful.”
All of which would seem to imply that UAH's decision was heavily influenced by Bishop's publication record. And that apparently was not so great: she had published only one peer-reviewed paper in each of the years 2004, 2005, and 2006, and she published no papers at all in 2007 and 2008, reported The Chronicle.
True, last year Bishop published three papers, but this was still not deemed sufficient — partly, The Chronicle seemed to suggest, because one of those papers "appeared in the little-known International Journal of General Medicine”. As we've stated, this is an OA journal published by Dove Medical Press.
The Chronicle added that other biologists who do similar work on neurodegenerative disorders “tend to publish at a much faster clip.”
However it was Bishop's OA paper — entitled ‘Effects of selective serotonin reuptake inhibitors on motor neuron survival — that journalists and bloggers began to home in on, and not without cause perhaps.
Vanity journal?
On 15th February, for instance, The Daily Kos pointed out that the International Journal of General Medicine charges authors to publish their papers. This fact alone, The Daily Kos seemed to imply, meant that the paper would have been inadequate. But there was more.
“Forget the fact that the Chronicle of Higher Education calls the journal ‘little-known’. Forget the fact that the it's a vanity journal, which only has an ‘honorary’ rather than a real editorial board and which rapidly accepts and publishes almost anything submitted, as long as the author pays the publication fee of many hundreds of dollars. Forget the fact that the journal is published by ‘Dove Medical Press,’ which is infamous for spamming people about its fraudulent journals. Here's the kicker — Bishop almost certainly listed her own minor children as the lead authors of the article!”
The Daily Kos' conclusion, it seems, was that any researcher who pays to publish their work must be engaged in some form of vanity publishing. If, in addition, the author cites her own children as co-authors the quality of that work must inevitably be low. That the publisher also promises to make the papers it publishes available rapidly seemed only to confirm such a conclusion.
Initially it was The Daily Kos' "kicker" that attracted most attention, and was invariably viewed as evidence not only that Bishop's work was inferior, but that Dove Medical Press and (by implication) OA at large was suspect.
On February 18th the Wall Street Journal noted : “On the university's Web site, Ms. Bishop listed 16 published academic articles that she co-authored between 1992 and 2009. The most recent listed article was published in May 2009 and names her husband and three of her four children, all of whom are under 19 years of age, as co-authors.”
Bishop’s husband, James Anderson Jr., (also 45), told the WSJ that the paper was “a family research effort that evolved out of a science-fair project. ‘Some families do soccer. We do science’.”
A week later the Boston Herald devoted an entire news story to the Dove paper, under the headline: “Amy Bishop, husband listed teens on research paper”.
When The Herald asked UAH spokesman Ray Garner about the paper, he insisted that the school was unaware that Bishop and Anderson had put their daughters’ names to a research project, “for which the paper also credits the school and the couple’s home-based science research company Cherokee Lab Systems. ‘It’s unusual,’ Garner said.”
For by now another blogger had discovered that “googling with street view the claimed address for Cherokee Labsystems — 2103 McDowling Dr. SE, Huntsville, AL — shows a residential home and not a laboratory allegedly involved in genetic research.”
By 20th February The New York Times also appeared to have concluded that having a research paper published in an OA journal was to engage in vanity publishing. Indeed it directly referred to the International Journal of General Medicine as a vanity journal or, as it put it, "essentially a scientific vanity press."
As interest in the Dove paper grew news reports began to use the term OA directly, and generally not in a flattering way. In a further NYT news report published on 22nd February, the paper commented. “One 2009 paper was published in The International Journal of General Medicine. Its publisher, Dovepress, says it specialises in ‘open access peer-reviewed journals.’ On its Web site, the company says, ‘Dove will publish your paper if it is deemed of interest to someone, therefore your chance of having your paper accepted is very high (it would be a very unusual paper that wasn’t of interest to someone).”
Peer review
But is it fair to characterise Dove, or OA publishing at large, as "essentially a scientific vanity press"? After all, this assumes that published papers are not peer reviewed. However, while it clearly it has not helped that Bishop cited three of her teenage children as co-authors of her paper, can we conclude from that that Dove does not have papers evaluated before publishing them?
Not necessarily: On the same page that The Daily Kos cites as evidence that Dove's journal is a vanity publication it is also stated, “Although peer review is rapid it is also very thorough — at least 2 peer reviewers comment on each paper. Many authors have found that our peer reviewer’s comments substantially add to their final papers.”
On the other hand, Dove's claim that it will publish anything "deemed of interest to someone" could imply something quite different.
Who then would have made the decision to publish the Bishop paper, and is it possible to confirm that Bishop's paper was peer reviewed? When I spoke to Dove Medical Press Publisher Tim Hill in 2008 he said: “The Editor-in-Chief and/or their Associate Editors make the decision as to what is published and what is rejected.”
So I emailed the Editor-in-Chief of the International Journal of General Medicine Professor Scott Fraser, a consultant ophthalmologist at Sunderland Eye Infirmary in the North East of England. My emails, however, have so far gone unanswered.
(In looking up Fraser's name on the Dove site I was surprised to note that he appears to be the Editor-in-Chief of no less than nine Dove journals).
I also emailed Hill. Could he confirm that Bishop's paper was peer reviewed? And was he aware that Bishop had cited three of her teenage daughters as co-authors of her paper?
“We do ask that all co-authors be cited in any paper sent to us,” Hill replied. “Dr Amy Bishop was the corresponding author of this paper. Her paper (‘Effects of selective serotonin reuptake inhibitors on motor neuron survival’) was peer-reviewed by 3 experts and revised by Dr Bishop prior to an editorial decision to accept the revised paper for publication.”
Hill concluded: “It appears, on the basis of media reports, that she [Bishop] was in breach of our authorship criteria.”
The problem is that as the number of controversial incidents connected with OA continues to grow (e.g. here, here and here) it is no longer enough for OA publishers simply to assure the public that the papers they publish are peer reviewed. It has got to the point where they really need to be able to demonstrate that it is so.
What would doubtless help, therefore, is if OA publishers embraced the practice of open peer review. After all, if reviewer reports were all freely available on the Web the world would be far better able to judge whether or not a paper had been peer reviewed, and how thoroughly.
In fact, some OA journals do already practice open peer review — including some of the journals published by BioMed Central (e.g. here) and OA journals like Atmospheric Chemistry and Physics (ACP) — e.g. here.
Dove, however, does not. As Hill explained to me in 2008, “We adhere to the traditional manner of conducting peer review."
But peer review is only one issue: There is also the issue of author-pays. The media response to the Bishop paper, and the assumptions made, has served once again to draw attention to the fact that the author-pays business model that most OA publishers (and a growing number of subscription publishers) now utilise is deeply problematic. In fact, it has become the Achilles Heel of the OA movement. As The Daily Kos post amply demonstrates, whatever its claimed merits or demerits, the public is deeply sceptical about it.
Nor is it just the public: It is becoming increasingly apparent to the research community that the APC rates now being levied by OA publishers are unjustifiably high, and that asking authors to pay to publish is prone to abuse (which would appear to confirm public scepticism).
Embarrassing
Whatever the details of the publication process the Bishop paper was subjected to, and the thoroughness with which it was evaluated, the incident has undoubtedly been an embarrassment for Dove. Since Dove is a member of the Open Access Scholarly Publishers Association, an organisation created to self-regulate the OA publishing market, it is also an embarrassment to the OA publishing community at large.
In its efforts at self-regulation OASPA has drawn up a code of conduct. This code of conduct does not require OA publishers to verify the identities of all the authors cited in papers they publish, but the publicity around the Bishop paper has underlined the continuing credibility problem that OA publishers face, and the difficulties of maintaining adequate quality control in an environment in which authors have to pay to be published.
What are OASPA's views on the incident? In the hope of finding out I emailed a list of questions to the organisation. Amongst other things, I asked whether OASPA was concerned that three teenage children had been cited as co-authors of a peer-reviewed paper published by one of its members, whether it felt that having one person act as Editor-in-Chief of nine Dove journals might not be excessive, and whether it was desirable that the International Journal of General Medicine should have only an honorary editorial board.
In reply I received a short statement: “The issue raised could affect any publisher, and we do not wish to be drawn into a discussion around this tragic incident.”
Many subscription publishers would doubtless disagree that the same thing could happen to any publisher. Be that as it may, the whole episode serves once again to cast OA in an unsavoury light, and must surely leave researchers, research institutions, as well as the tax-paying public, with the strong impression that OA publishing is a somewhat shoddy business. Unfortunately, OASPA's reply suggests that OA publishers remain dangerously oblivious to the credibility gap yawning in front of them.
Certainly some librarians appear to be baffled. On 23rd February Beth Transue, a librarian at Messiah College in Pennsylvania, posted a note to the MEDLIB-L mailing list. Providing a link to the NYT article of 20th February she commented: "Making this situation more complicated is that the publisher [Dove] is a member of the Open Access Scholarly Publishers Association, which also lists members such as BioMed Central, BMJ Group, Oxford University Press, and Sage. It isn't clear to me how Dove became a member of this association, given their review practices, especially in comparison with these other reputable publishers."
Transue then asked: "Has anyone on this list had any experience with this publisher or journal?."Do you have it listed in your holdings? Do you plan to remove it from your holdings lists? We are considering removing all journals from this publisher from our holdings, and I was wondering about what other libraries might be doing."
Transparency
The Bishop incident raises one further issue, or at least it ought to: and that is another issue of transparency — because Dove is one of a number of OA publishers strangely secretive about who owns and manages its business.
When I spoke to Hill in 2008 he would say only: “The company headquarters are in the UK. We have offices in the UK and here in Auckland. Dove is owned by six private individuals and we have some 20 employees.”
When I pressed Hill to name those six individuals, he declined. Not only is that odd in itself, but it is odd that OAPSA failed to ask the same question before welcoming Dove into its organisation. Even today OASPA has no idea who owns Dove. Since one of the reasons for founding OASPA was to help improve transparency in the OA publishing market this seems anomalous indeed.
So far as Hill is concerned the question of who owns a private company is nobody's business but its own. Many, however, would disagree, especially when that company makes its profits out of public funds. As I put it in 2008: “[I]t is primarily public money that researchers will be using to pay to have their scholarly papers published in OA journals. Is it not reasonable therefore for taxpayers to expect to know exactly what is being provided for that money, and who will benefit from any profits that are made as a result?”
And I am not alone in believing this. Commenting on my interview with Hill, Cornell's Phil Davis remarked at the time: “The interesting thread that Poynder explores is the lack of transparency (and thus accountability) of some of these start-up companies. While certainly not limited to OA publishers, the open access movement is very focused on transparency and public accountability as central framing issues. One may argue that access to public funds requires both access to the output of the research and access to how the money was spent. You can't have it both ways.”
Peter Suber, de facto leader of the OA movement, is also uncomfortable with the lack of transparency over the ownership of some OA publishers. In fact, he told me, he had been under the impression that transparency of ownership was a pre-condition for membership of OASPA. "At least I am deeply suspicious of publishers who are unwilling to disclose their owners," he told me. "We need that kind of transparency to be able to investigate whether the owners have financial interests, e.g. with pharma companies, that might compromise the integrity of their journals."
Time to wake up
In short, what many OA publishers seem to have failed to take on board is that OA raises transparency issues not just about the quality of the services provided, but about the public's right to know how tax-payers' money is being spent, and who is pocketing the proceeds.
Given the number of scholarly publishing scandals in recent years it is becoming increasingly difficult for the public to trust the peer review process; when the ownership of the companies in charge of that process is similarly non-transparent, public confidence in how its tax dollars are being spent can only be further undermined. What is needed is transparency both of process and of ownership.
All in all, it is hard not to conclude that OASPA has fallen asleep at the wheel. Now is the time to wake up. And the first thing it should do — after smelling the coffee — is to insist that Dove either immediately publish full details of its ownership, or leave the organisation. And is it not also time for OASPA to set about persuading members to embrace open peer review? Transparency aside, the latter would have the benefit of helping to assuage the growing concerns about possible abuse of author-pays publishing.
Whatever it does, OASPA is going sooner or later to have to address the poor public image associated with author-pays scholarly publishing.
I hope in the near future to return to the question of the ownership of Dove Medical Press — as well as the ownership of Libertas Academica, the OA publisher managed by Tim Hill’s son Tom Hill (likewise based in New Zealand).
PR: 
LD: 
I: 
Rank: 
C: 
A LITTLE CONTEXT: PART I (of 2)
ReplyDeleteIt's no secret that there's no love lost between me and what I consider to be a spate of premature new Gold OA pay-to-publish journals at a time when what is really needed is not more journals, nor payment to publish, but more OA (via Green OA self-archiving and Green OA self-archiving mandates from universities and funders).
But I do think I need to come to the defense of Gold OA publishing on this -- fortunately rare, indeed unique -- occasion, involving an obviously troubled author who has committed some horrible crimes.
So let me list what I think are red herrings in the public press:
Listing your children as co-authors is silly, but OK: In and of itself, it does not mean your published findings are invalid.
Using your house as a lab is rare (if we don't count the humanities), but OK. In and of itself, it does not mean your published findings are invalid.
Many journals (OA and non-OA) are relaxed about co-authors (apart from insisting that they all sign over their rights, if rights-transfer is the journal's policy) and that's OK. The corresponding author is the main responsible party. When journals make a fuss about authors signing that they have made "substantive contributions," that has more to do with boosting their journal's tenure-value and credibility than with ensuring that the published findings reported are valid.
Publishing in a pay-to-publish Gold OA journals is unnecessary (in order to provide OA) today, but OK (in fact I do it myself, sometimes).
Trying to charge high prices for pay-to-publish Gold OA today is understandable, hence OK, on the part of the service provider; and paying those high prices is foolish on the part of the author (or the author's institution or funder) today, but likewise OK. Many non-OA (subscription) publishers over-charge too. It does not mean their published findings are invalid.
Classical peer review has some (minor) remediable flaws on which it can and will be optimized in the online era, but converting to "open peer review" is certainly not one of those remedies, nor is is advisable or desirable. There are many very valid reasons to expect that reviews will be less forthright if the referee has to worry about the repurcussions of criticizing a colleague or superior openly. The buck stops with the Editor. It is the Editor to whom the referees are answerable, for the competence and credibility of their reviews, and to the Editor that the authors are answerable for revising in accordance with the (valid) reviews. And the Editor is in turn answerable to the journal's readers, authors and referees, for the quality of the journal and the competence of its referees and refereeing.
So what is needed if there are any doubts about a journal's peer-review standards or procedures is a confidential audit of the refereed reports by an externally appointed honest-broker -- not a conversion to open peer review (though experimental tests of open peer review are welcome). http://bit.ly/bxhmCO
Having on Editor in Chief for multiple journals is not ideal (it's a journal-fleet-publishing phenomenon), but not necessarily malpractice, hence OK, if there are answerable Action Editors for the particular journals, topics or articles. They are then, in effect, the Editors.
Doing low-standard or no-standard peer review is not OK -- but far from unique to (low-standard) Gold OA journals. There are plenty of non-OA borderline and junk journals too.
The popular press have had a hey-dey with this isolated tragic case, from which the only lessons to be learnt are probably psychiatric ones.
[please see part 2]
A LITTLE CONTEXT: Part 2 (of 2)
ReplyDeleteBut Richard is probably right about this particular publisher's peer-review standards. He's probably onto something with the undisclosed ownership too. (My own guess would be that it's Big Pharma. The only way to dispel doubts is full disclosure.)
And Richard is surely right about OASPA's needing to tighten both its standards and its vigilance regarding both peer review and ownership.
I had also criticized OASPA for accepting as full members in good standing as "Gold OA Publishers" publishers that publish a fleet of journals, only one or a few of which are OA. I think that, too, gives the wrong impression about OA. But it does have one virtue (if the mostly-Non-OA publishers are reputable publishers with high peer-review standards): that established journals, with experienced editorial boards and referees, and good practices of long-standing, with a track-record for high peer-review standards, if and when they actually convert to Gold OA, are far less likely to fall into the shoddy practices of some of the current spate of new dot-Gold OA journal start-ups.
That said, there are plenty of low-quality non-OA publishers too, and it is unlikely that they would improve if they converted to Gold OA!
The moral of the story is that authors should be publishing in the journal with the highest quality standards that their work can meet, and if they wish to make their published articles OA, they should simply self-archive them.
I think it is really at best ridiculous (at worst tasteless) to frame this whole story as an OA issue. There are good and bad journals, and they can be OA or subscription based. There is good and bad research, and every scientist knows that you can get everything (yes, even bad research) published in lower-tier journals, open access or not. You demand more transparent peer-review and transparency in ownership, and I agree, but if you look around you will find a higher proportion of OA journals doing open peer-review compared to subscription-based journals, so you are barking at the wrong tree. You write "the incident has undoubtedly been an embarrassment for Dove. Since Dove is a member of the Open Access Scholarly Publishers Association, an organisation created to self-regulate the OA publishing market, it is also an embarrassment to the OA publishing community at large.". Apart from the fact that I disagree that a professor who has been denied tenure has published in a OA journal is an embarrassment to the OA publishing community at large (boy, wouldn't have the throw-away subscription journal industry have plenty of reasons to be embarrassed), I wonder if you, Mr Poynder, don't feel a bit embarrassed yourself, after you investigated my previous explicit and public warnings of Dove being a somewhat shady vanity publisher and you concluded that "while it does appear that there are a number of unscrupulous OA publishers operating, I could find no reason to conclude that Dove is one of them."? . Let me get this straight: I publish a warning about Dove being a vanity publisher who appears to use unethical business practices on my blog 18 months ago. Then you publish a blog implying that these claims are inaccurate. OASPA (perhaps reassured by this) subsequently admits Dove as member. Then, in your latest blog post, you accuse OASPA of "sleeping at the wheel". I am just wondering - on what grounds should have OASPA turned down Dove's membership application, in your opinion, as somebody who has previously concluded that previous accusations of Dove violating said Code of Conduct were inaccurate? On what grounds should OASPA turn it down now, or be embarrassed? Even in your latest blog you say that one can not necessarily "conclude from that that Dove does not have papers evaluated before publishing them". OASPA will certainly keep a good eye on Dove and will act on any emerging evidence that there has been a violation of its Code of Conduct, but the credibility of OASPA has certainly not suffered from this - while the same cannot be said about the credibility of open access critics trying to link open access to the Alabama shootings.
ReplyDeleteDisclosure: I am a OASPA member, but speak for myself, not OASPA. I am editing a Open Access journal, I even publish is some, and I don't have tenure. While this combination probably makes me look moderately insane, I don't know how to use a gun.
Dear Professor Eysenbach,
ReplyDeleteThank you for this. To answer your question: No, I am not embarrassed. On both occasions when I have written about Dove Medical Press I have tried to report what I found in a fair and balanced way. I realise you disagree; which is fair enough.
Given what I knew when I interviewed Dove's Publisher, Tim Hill, I concluded, as you correctly point out, that there was no reason to think that Dove was an unscrupulous publisher; nor was there. As you also point out, I am *still* not claiming that Dove is an unscrupulous publisher, although I have drawn attention to the fact that some people have raised questions about the thoroughness with which it reviews the papers it publishes, and concerns about its use of honorary editorial boards.
Thus I have concluded that Dove must be embarrassed, not unscrupulous. And I have no evidence right now to conclude anything else. So I have not, like you, charged the company with being a vanity publisher. Rather, I have said that the media appear to have concluded as much — based, it appears, on the fact that in her paper Amy Bishop cited her three teenage children as co-authors, and on the assumption that if a researcher pays to publish his or her work that author is engaging in vanity publishing. (I understand Dove has now taken down the paper).
You, by contrast, were quite happy to conclude in 2008 that Dove was a vanity publisher — a conclusion, so far as I can tell, based solely on the fact that you had received some unsolicited email invitations from the company inviting you to submit papers. When I interviewed Hill he said that Dove had ceased sending out unsolicited invitations, and I am not aware that the company has sent out any since then (Perhaps you know differently?). Thus your main complaint against Dove appeared no longer to be an issue at the time I interviewed Hill.
My criticism of Dove is that it is not transparent about its ownership. Likewise, my criticism of OASPA is that it appears not to have thought to obtain any ownership details from Dove before admitting it to membership of the organisation. My criticism, therefore, is that Dove's membership should only have been accepted after it had supplied full ownership details. I also criticised the skimpy response I received from OASPA to the questions I sent over.
Let me end with a couple of questions for you: If you and your colleagues at OASPA eventually concluded that Dove was a serious OA publisher (which must be the case if the publisher is now a member of OASPA), why have you still not retracted the claims you made on your blog in 2008 that Dove is a vanity publisher (a post you headed " FRAUD ALERT + SPAM WARNING + FRAUD ALERT + SPAM WARNING")? Did you object to Dove being admitted as a member?
As I said previously, I am not speaking on behalf of OASPA, and personally I was not involved in any decisions related to any membership applications. OASPA does not have a system in place where every member has to agree to every new member. It is my personal opinion and experience that spamming on a *massive* scale as done by Bentham and by Dove is a good predictor/indicator for unprofessional behavior in editorial matters, including a tendency of being unselective in publishing [="vanity publishing"], and subsequent events seem to have confirmed my sentiment (it is exactly those two publishers I singled out due to their spamming practices which subsequently made negative headlines on not being selective enough - with Bentham even accepting a computer-generated manuscript). Put simply, reputable journals do not have to resort to spamming to solicit manuscripts as they are getting far more submissions than they can accept, hence they can be selective. This is true for both OA and subscription-based journals. And also, how good can editors be who were recruited by means of a spamming campaign? This observation is my personal experience (which is why I linked "spamming" with "vanity publishing" on my blog), and while there is a strong association it is not a 100% accurate predictor. In other words, this personal observation is different from the hard evidence required by OASPA to turn down or expel a publisher, in particular as any sanctions related to spamming or unselective publishing/inadequate peer-review will rely on ongoing monitoring of user complaints and can't necessarily be done at the time of membership application. I also believe in the fact that people and organizations can change and can be educated, so while there is evidence that Dove spammed before, it does not automatically render it ineligible for OASPA, in particular after the public cease & resist statement on your blog (and I have not received any spam from Dove after that). There is no reason to withdraw the statements on spam and vanity publishing related to Dove on my blog, because they pre-date Dove's pledge to resist from these practices, and reflect my personal impression on the credibility of this publisher at the time.
ReplyDeleteOASPA acted on the evidence available at the time (and part of the evidence is your interview), just as you concluded at around the same time that Dove is not an unscrupulous publisher (and you are still not claiming it). It is beyond me why you now claim that OASPA should be embarrassed or is sleeping at the wheel when in fact you arrived at the same conclusion as OASPA, namely that there was no sufficient evidence to turn down Dove. You now say you criticize OASPA primarily because Dove's ownership is not clear, and this is certainly a valid discussion point, but anybody in his right mind will see that the ownership question of Dove has absolutely nothing to do with the Alabama shooting and the events surrounding it (would anything in this whole tragic story have had a different outcome if OASPA would have published the names of the owners of Dove? ). While I respect your previous work and attempts to be fair and balanced, I perceive this piece as everything but fair or balanced. A fair assessment would not have accused OASPA as "sleeping on the wheel" but would have acknowledged that assessing the quality of a publisher is an ongoing process, and would have acknowledged the fact that your own assessment of Dove was not much different from OASPA's assessment at the time.
(I will not further comment on this and give you the last word)
I am happy to leave the last word to you. I will simply add that my criticism of OASPA is not based solely on its failure to obtain the ownership details of Dove before accepting it as a member, but on its failure to check on the practices and policies of Sciyo before accepting it as a member.
ReplyDeleteI do not think, by the way, that you ever made any predictions about Sciyo based on its unsolicited emailing activities.
Great piece.
ReplyDeleteFor me Open Access without transparency is worth nothing.
As I asked the State and university library of Göttingen for the details of the Springer OA deal - I only got the answer that these are secret.
http://archiv.twoday.net/stories/5702753/
Richard, I don't quite agree with the means to the end of provoking a useful discussion about the issue, but there it is. This is not really about the Alabama shooting so why drag that in except to draw attention to what you really want to discuss here? There are better ways.
ReplyDeleteThere are a few issues here that I could discuss further, but the most constructive for would be clarification on what is meant by 'open peer review'. If open peer review means revealing the identity of the reviewers, that is one thing that could lead to repercussions and less forthrightness (depending on the academic culture).
However, if open peer review means publishing the process from draft to publication, that doesn't necessitate giving up the anonymity of the reviewers which is not desirable. If that is the case I don't understand the concerns brought by Stevan.
Open peer-review, and open comments and discussion from readers, and even a 'journal of rejected papers' would serve academia well and is a natural use of web 2.0, whose dialectic properties are obstructed by closed access publishing but which go almost ignored by open access publishers. All of these would help check the potential errors of peer-review.
I hypothesize that it is possible to have an OA publishing company for journals that are open access and the front, middle and back ends. No fees to readers, no fees to authors, transparent peer-review standards and process, full publication record accessible. In any 'game', the public and players in general demand referees that are forthright, fair and accountable and where everyone knows the rules and can observe their application.
The idea that peer-review would be compromised by being publicly observable is contrasted here with the idea that it cannot improve without it.
Stevan, you cannot simply state "converting to "open peer review" is certainly not one of those remedies, nor is is advisable or desirable." It is as if to say there is no value in what we are doing here.
This is worth discussing more, I believe. Certainty is a style too frequently adopted by experts, but expertise has its errors. Peer-reviewers and editors are expert gatekeepers of what knowledge and ideas should and should not be published in a process that no one sees - this has been one of the critiques of peer review.
ON EMPRICAL RISK AND UNCERTAINTY Part 1 of 2
ReplyDeleteDear salaamarifat (I don't know who you are, but I deduce that you may perhaps be the publisher or editor of a journal that uses "open peer review"?):
To reply:
(1) Publishing every draft would require the consent of the author, and many authors, and many disciplines, are not keen on making their unrefereed drafts public. Besides, that's not open peer review, that's just preprint + postprint OA (Green).
(2) Open peer review presumably means doing the peer review out in the open (which means posting the drafts and at least the referee reports, if not the names of the referees, for the reasons you state), and again, authors may rightly not be too keen on this.
(3) What the more radical open-peer-review reformers have in mind (and I suspect you may mean this yourself) is not really peer review at all, but public vetting of papers, post-hoc: The paper is posted, anyone who wishes (or perhaps those chosen by the editor) posts referee reports, and the author revises as he wishes and chooses, and some version or other (or all of it) is eventually dubbed "published."
(4) The reason (3) is certainly no solution -- modulo the kind of uncertainty that also afflicts apples and the pull of gravity -- is a combination of (4a) the author and referee confidentiality problems you have already acknowledged, (4b) the reluctance (and undesirability, in some fields, e.g. biomedicine) of publicizing unrefereed, hence possibly invalid and dangerous results, (4c) the nonanswerability of such an open-ended process (including the arbitrariness of the draft dubbed "published"), (4d) the scarcity of competent referees, the difficulty in finding them, even in the best of times, and of getting them to referee and to do so in a timely way; but worst of all (4e) the double load this puts on the poor user, who had been using refereed-journal names and track-records as the means of filtering what papers were safe to invest their scarce and precious time in reading, using and trying to build upon (now the task is to sort through unrefereed papers in various uncertain stages of undress, and self-selected "referees" of various uncertain degrees of competence, etc.).
See part 2 (Richard's blog-software word-limit!)
ON EMPRICAL RISK AND UNCERTAINTY (Part 2 of 2)
ReplyDeleteI did say, though, that it would be fine to test out these options locally (and if you are one of those doing so, best of luck to you). My prediction is that the radical options (inasmuch as they are coherent and implementable at all) will be neither sustainable nor scaleable, globally, in generating a research corpus that is at least as reliable as the one we have now. And if such an experiment were done globally, rather than locally, straightaway, on a blanket basis, there would first be a period of chaos, as standards fell and the research literature become unnavigable and unuseable -- and then (classical) peer review would simply be re-invented. (It just means answerable vetting of results by selected, answerable experts, to a known and trustworthy quality level, all answerable to a competent editor, who is in turn answerable for the quality and track-record of the contents of the journal.)
I am perhaps particularly positioned to understand the stakes and dynamics, and to predict the outcome, since I edited a journal with rigorous (closed) peer review, as well as capaciously open peer commentary for a quarter century. Hence I know all too well both the difference between closed peer review and open peer commentary, and the difference between raw, unrefereed drafts and rigorously refereed ones, with all parties having been answerable to an answerable editor. And I would not wish those raw, unrefereed drafts that reached my desk daily, nor the various successive incarnations -- among the minority that ultimately got accepted and published, let alone the majority that never managed to meet the standards for which the journal was known and trusted and ended up in other journals -- would not have wished them onto the research community that was looking to the journal name to be providing the reliable mark of quality and reliability.
And certainly not if what they were looking for was OA, rather than a-priori experiments on quality-control systems based on a-priori speculation rather than careful empirical and pragmatic validation.
For details, google: (harnad "peer review" reform open)
Seeing the process of drafts, reviews, author replies, revision to publication would be educational for those learning to write. It would enable one to challenge a peer review. It would allow research into the validity and reliability of peer review. It would allow the possibility of catching errors of peer review. One possible error would be that of exclusion of an article or part of an article that was edited out, which may contain something of import or interest that was not understood by reviewers. Who knows, if one gets to a dead end, maybe the key to discovery lies in the draft, or in the conversation. That's all. Scholarship should be more dialectic and publicly observeable.
ReplyDelete