Saturday, August 15, 2015

When email marketing campaigns go awry: Q&A with Austin Jelcick of Cyagen Biosciences

Earlier this week I received an unsolicited email message from a company called Cyagen Biosciences inviting me to cite its “animal model services” in my scientific publications. By doing so, I was told, I could earn a financial reward of $100 or more. And since the amount would be based on the Impact Factor (IF) of the journal in question, the figure could be as high as $3,000 — were I, for instance, to cite Cyagen in Science (IF of 30). 
Austin Jelcick
The email surprised me for a number of reasons, not least because I am a journalist/blogger not a scientist. As such, I have never published a research paper in my life, and have no plans to do so. Moreover, I have only the vaguest idea of what an “animal model service” is, let alone how I would cite a company selling such a service in a scientific paper.

But mostly I was surprised that — at a time when thousands of researchers are calling for the abandonment of the Impact Factor — any company would want to tie its reputation to what is widely viewed as a sinking ship.

Curious as to why I had received such a message I searched on the Web for the company’s name, only to find that the link from Google to Cyagen’s home page delivered an error message.

Eventually locating an email address I contacted the company and asked if it could confirm that the message that I received had been sent on its behalf (It appeared to have come from a direct marketing company called Vertical Response).

The next day I received a reply from Cyagen product manager Austin Jelcick, who explained that I had received the message “as part of our marketing campaign which is currently seeking to raise awareness within the scientific community for our citation rewards program.”

As I was associated with “several blogs and articles related to open access journals and publishing” he added, it was assumed I would be interested in “our newly launched campaign to actively reward scientists for citing us in their materials and methods section while simultaneously encouraging them to submit into higher impact journals for increased awareness of both their study and our services offered.”

He added: “we felt that it would be beneficial to the researcher to receive a sort of ‘store credit’ for doing something they already must do as part of the publication process.”

Now intrigued, I invited Jelcick to do an email Q&A so that he could explain in more detail who the company was and why it had launched this campaign.

Very surprised by the offer

While I was swapping questions and answers with Jelcick by email the company’s campaign was starting to attract a good deal of commentary on the Web.

Yesterday, for instance, high profile physician and science writer Ben Goldacre published a blog post entitled, “So this company Cyagen is paying authors for citations in academic papers”.

Goldacre concluded, “Perhaps my gut reaction — that this feels dubious — is too puritanical. But I am certainly very surprised by the offer.”

Goldacre’s intervention also sparked a post over on Retraction Watch entitled, “Researchers, need $100? Just mention Cyagen in your paper!”

By now there was also a steady stream of comments from scientists on Twitter, expressing everything from puzzlement to outrage — see this for instance.

By late yesterday Cyagen clearly felt the need to make a public statement, which it did by means of a Q&A on Facebook, explaining: “Please find below some of the questions which were asked of us and our response which should help clear up the misunderstanding which has occurred about this promotion.”

The post went on to list seven questions and answers. What the company did not explain, however, is that these had been extracted from the interview I was still in the process of doing with Jelcick. That is, Cyagen did not cite me!

What has become clear is that the company believes that its email invitation has been misunderstood. Linking to the Facebook post from a comment on Goldacre’s blog, Jelcick went so far as to complain that Cyagen has become a victim of “some gross miscommunication”.

Richard Van Noorden appears to agree, saying on Twitter that the story has been “gleefully badly reported”. He explained: “you can’t get $100 by citing them. You get a discount voucher for their products”. He nevertheless suggests that Cyagen should withdraw the offer “pronto”.

It would seem that the mistake Cyagen made was to link its promotion to the much-maligned Impact Factor, which has become a red rag to many scientists. (See also the first comment below).

Anyway, below is the full list of 17 questions and answers that make up the interview I did with Jelcick. Some of the answers are a little repetitive, but given the confusion surrounding Cyagen’s email I have chosen not to edit them.

See what you think.

Wednesday, August 12, 2015

Open peer review at Collabra: Q&A with UC Press Director Alison Mudditt

Earlier this year University of California Press (UC Press) launched a new open access mega journal called Collabra. Initially focusing on three broad disciplinary areas (life and biomedical sciences, ecology and environmental science, and social and behavioural sciences), the journal will expand into other disciplines at a later date.

One of the distinctive features of Collabra is that its authors can choose to have the peer review reports signed by the reviewers and published alongside their papers, making them freely available for all to read — a process usually referred to as open peer review.

This contrasts with the traditional approach, where generally the reviewers names are not disclosed to the authors, the authors names are not disclosed to the reviewers, and the reviewers reports are not made public (commonly referred to as “double-blind” peer review).

Since Collabra is offering open peer review on a voluntary basis it remains unclear how many papers will be published in this way, but the signs are encouraging: the authors of the first paper published by Collabra opted for open peer review, as have the majority of authors whose papers are currently being processed by the publisher. Moreover, no one has yet refused to be involved because open peer review is an option, and no one has expressed a concern about it.

Collabra’s first paper—Implicit Preferences for Straight People over Lesbian Women and Gay Men Weakened from 2006 to 2013was published on 23rd July, and the reviewers’ reports can be found here.

So how does open peer review work in practice and what issues does it raise? To find out I emailed some questions to UC Press Director Alison Mudditt, whose answers are published below.
Alison Mudditt

RP: Presumably both the author and all the reviewers have to agree to open peer review before Collabra can publish the reviews? What percentage of the papers it publishes does Collabra expect will have the reviews published alongside?

AM: Authors choose open peer review as an option upon submission, so it is always their decision and as such they have already agreed in advance. Reviewers are made aware that authors have chosen this option and could opt to decline the review if they are unwilling to have their review comments made publicly available.

As a secondary option, whether or not open review has been chosen by the author, reviewers can sign their reviews. So it is possible to have reviewer comments be open, but the identity of the reviewer remain anonymous. Or, for that matter, have closed review, but reviewers sign their reviews. This is all described here.

With only one published article it is hard to project what the percentage will be, but at this point the majority of authors—for the papers currently being processed in our system—have opted for open review.

We are not targeting certain percentages, but rather want to put new options in front of people, especially given the numerous critiques of traditional closed peer review systems. This will not be for everyone, but we believe there’s much to be learned from experimentation with new models.

RP: Will Collabra make any effort to seek out reviewers who are comfortable with open peer review?

AM: The academic editors are selecting reviewers, and their top consideration will of course be the reviewer’s expertise for any given paper.

We make all the options and elements of Collabra clear when inviting external editors to be involved. Some editors are particularly interested in the open review option, and other editors have not commented on it.

No one has refused to be involved because it is an option or expressed a concern about this option.

RP: I assume that not all the correspondence is shared when Collabra publishes the reviews, and perhaps they might be edited in some way first (at least sometimes)? If so, what considerations/editorial rules are applied before making reviews public?

AM: Currently, the “open review file” is constituted by the reviewers’ comments on the reviewer form, the editor’s comments to the author based on the reviewers’ comments, and the author’s response—all as captured in our editorial system.

It is clear on the review form that there is an area for confidential comments to the editor that would not be shown to the author nor included in the openly available comments. But, for the remainder of the form, it is made clear that comments may be seen by the author and used without editing.

What is not currently shown is any earlier version of the paper and any comments or tracked changes on that. We will continue to monitor this policy and will consider other options, if it seems that useful or important elements are being omitted by not including earlier versions/changes.

And, obviously, if any discussion occurs outside of the editorial system between a reviewer and an editor, that will not be captured.

As regards editorial rules and considerations for any edits or omissions, we would discuss that with the editors as they came up. It is hard to say in advance what that might be (other than any information which is confidential and not even being revealed in the paper), so we’ll deal with that on a case by case basis.

Naturally, we would opt to be transparent about this happening should it occur beyond normal confidentiality considerations. For now we will see how it goes with it’s being clear on the form that comments may be used as written.

RP: Having started down this road (and so given concentrated thought to the matter), what would Collabra say were the pros and cons of open peer review?

AM: Speaking on behalf of UC Press (I’m not sure it’s appropriate to speak as “Collabra” in this context), we think that the inner workings of the peer review process are, purely and simply, interesting for any reader, but in particular for people who would like to see more transparency in this process.

There is clearly an argument to be made that making things open (rather than, for example, the double blind process) will help to reduce biases, problematic opinions, or hierarchical sensitivities that can affect the review process.

Equally importantly, open review starts to demonstrate the value added by the review process and to recognize the contributions of reviewers to scholarship

Finally, we all know that traditional peer review has not put a stop to whole disciplines being rocked by scandals of fabricated data and unquestioned results, and it’s possible that open peer review will actually help to improve the scholarly record.

On a related note, one of our other aims with Collabra is to get rid of the phrase “peer review lite” which has plagued the type of review that Collabra (and other OA titles) employs.

We characterize our review criterion as being “selective for credibility only”—checking for the scientific, methodological, and ethical rigor of a paper, and removing, as much as humanly possible, more subjective reviewing criteria for novelty or anticipated impact. Open reviews will support this mission—to show that there is nothing “lite” about this kind of review (and in fact, sometimes quite the opposite).

It’s too early for us to be able to identify specific problems with open peer review for Collabra, although we are aware of studies suggesting that it may be harder to get reviewers and it may lengthen the review time. Our limited experience so far does not support either of these concerns.

The other cons of open peer (as opposed to double blind) review are clearly to do with concerns about bias, the highly variable nature of peer review, and the additional costs it could impose on an already overtaxed system.

For example, a reviewer might be worried about openly and critically reviewing a more senior author and believe there could be a negative effect on her own career.

Our hope is that a more open system will improve the integrity of the peer review process, but the reality is that any system will be subject to the biases of human nature—we just think that this is more likely to be surfaced through greater transparency.

RP: Does Collabra think that there are occasions when open peer review is inappropriate? If so, when and why?

AM: Anything raised in peer review of a confidential nature which does not make it into the published article should be carefully removed from any open peer review comments that get published during open review.

That said, we (UC Press) are not really the drivers of how open peer review will evolve in Collabra or elsewhere. Since Collabra works only with external editors, editorial policies should emerge that are firmly based on the standards of each research community that publishes in Collabra.

If a community-driven majority standard emerged which stated that, in certain situations, open peer review was inappropriate, then we would respect such a decision.

RP: Are there any other learning points that have emerged as Collabra has sought to implement open peer review?

AM: It’s too early in the launch of Collabra to really be able to comment, although we have been pleasantly surprised at authors’ and reviewers’ willingness to consider the option of open peer review. That seems to be a great start for this concept.

An earlier Q&A with Alison Mudditt can be read here.