Open Access Mandates: Ensuring Compliance

Three years ago, I wondered aloud how we should judge the success of an Open Access (OA) mandate. In particular, I wondered what level of compliance it was reasonable to expect a mandate to attain, and pointed out that OA advocates believe it is possible to achieve a 70% compliance rate within two years, if the mandate is compulsory.

It is therefore striking that, although the Wellcome Trust has had a compulsory mandate in place since 2006, The Times Higher Education reported in March that the Trust is still only achieving a 55% compliance rate. As a result, The Times Higher added, Wellcome’s head of digital services Robert Kiley wants to get tough on those who do not comply. The current situation, said Kiley, is “simply not acceptable”.

Elsewhere, Nature reported that of the 55% of researchers who comply with Wellcome’s mandate, 85% do so by means of Gold OA. One benefit of Gold OA, presumably, is that the author can hope to pass responsibility for compliance over to the publisher. While this does require paying a Gold OA fee, the Trust will pay this fee for the researcher. However, that 85% figure might seem to suggest that authors are more reluctant to comply with a mandate than some assume.

In my 2009 blog post I also reported that when, in 2008, the US National Institutes of Health (NIH) upgraded its request that researchers deposit their published papers into PubMed Central (PMC) into a requirement, compliance accelerated to 49% by the end of the year (from 19%).

In order to establish what level of compliance the NIH is achieving today, I contacted its Office of Extramural Research (ORE). Below is the (slightly edited) transcript of the email conversation I had, which was split over a number of emails.

The upshot is that the NIH is now achieving 75% compliance. 

This invites a question: Why is NIH proving more successful in achieving compliance than the Wellcome Trust? Here are some quick thoughts:

• NIH is the largest public funder of medical research in the world, and so perhaps has greater authority than the Wellcome Trust

• NIH has been more proactive in ensuring compliance

• Where the Wellcome Trust insists that any embargo imposed by a publisher is no longer than six months, the NIH still permits embargoes to be up to 12 months (so perhaps publishers are more willing to post NIH-funded papers in PMC?)

As I say, these are just initial thoughts. I welcome comments from others.

One thing I find noteworthy is that, while the NIH clearly takes a close interest in compliance levels, it does not collect statistics on enforcement actions. With the growth in compliance now apparently levelling off at NIH, this would surely be useful information — and not just for NIH itself, but also for other funders like the Wellcome Trust and Research Councils UK  (RCUK), both of whom are currently looking to beef up their open access policies.

Q&A with NIH

RP: Can you give me some information on the current compliance rates for the NIH Public Access Policy, and explain how the NIH ensures compliance?

NIH: OSTP has submitted a report to Congress that included a current overview of the NIH policy, compliance level and impacts on Page 12.

Our current compliance efforts have been focused on outreach and engagement with institutions and publishers, as outlined on pages 3 and 4 of David Lipman's testimony to Congress in 2010.

RP: Ok, so after reading these documents I conclude the following:

1. The initial voluntary policy saw compliance rates of 19%.

2. The mandatory policy (introduced in April 2008) saw compliance grow to 49% by the end of 2008, 70% by the end of 2009 and currently stands at 75%.

3. In order to increase compliance the NIH has a) engaged in awareness programs; b) improved its ability to track papers resulting from NIH research awards; c) developed new systems to assist sponsored research offices at universities and medical research centres in tracking their compliance; d) improved the submission process for authors by i) co-opting publishers to deposit for authors and ii) introducing the NIH Manuscript Submission System.

NIH: To be clear, we introduced the NIH manuscript submission system in 2005. We have made modifications and improved our guidance, but it is not new.

We don't think it is accurate to say that we have 'co-opted' publishers, as the NIH policy does not apply to them.

RP:  Why do you think publishers agree to deposit papers for authors?

NIH: We generally hear three reasons why some publishers take an active role:

1) They want to be on PMC because of its high visibility, integration with other resources, and secure and free archiving — many journals do this for all their content, regardless of whether it falls under the NIH public access policy

2) They want to provide a service to their authors

3) They want greater control over the version posted to PMC (some want only the published version posted, and others want only the final peer-reviewed manuscript posted)

Green or Gold?

RP: Can you say what percentage of the papers deposited in PubMed Central are papers that have been published in a subscription journal and then self-archived (i.e. Green OA) and what percentage have been published in an OA journal for a fee (i.e. Gold OA)?

NIH: We do not characterize PubMed Central (PMC) deposits in terms of green and gold Open Access (OA). PMC is more than a repository for papers resulting from NIH-funded research.  See here.

All 2.4 million PMC articles are available without charge on PMC. This number includes about 225,000 author manuscripts, and the rest are final published versions that have been provided to PMC by publishers.

We do not have a simple way of generating counts of which papers are from subscription based journals and which are not. However, to put the above numbers in context, note also that close to half of the articles in PMC are from the digitized (scanned) issues of journals from the period between the mid-1800s and the mid to late 1990s — predating the birth of OA journals and the NIH public access policy.

About 450,000 articles make up the PMC OA subset. The term Gold OA generally refers to articles that are available under a very open license (such as CC-BY) and involve an open access fee charged to authors. The articles in the PMC OA subset come with a variety of licenses, not just CC-BY.

In addition, some of these OA articles are from journals that, to our knowledge, don't charge an OA fee — the Rockefeller Press journals, for example here. 

Enforceable?

RP:  I came across a message on the Liblicence mailing list recently from someone who appeared to have concluded that there is no real force behind the NIH policy. Would it be accurate to say that the NIH currently has no way of enforcing compliance?

NIH: The NIH Public Access policy is a term and condition of award.  As we state in our FAQ: A grantee's failure to comply with the terms and conditions of award may cause NIH to take one or more enforcement actions, depending on the severity and duration of the non-compliance. NIH will undertake any such action in accordance with applicable statutes, regulations, and policies.

NIH generally will afford the grantee an opportunity to correct the deficiencies before taking enforcement action unless public health or welfare concerns require immediate action.

However, even if a grantee is taking corrective action, NIH may take proactive action to protect the Federal government's interests, including placing special conditions on awards or precluding the grantee from obtaining future awards for a specified period, or may take action designed to prevent future non-compliance, such as closer monitoring. See Enforcement Actions in the NIH Grants Policy Statement (11/03).

RP: Can you tell me how many researchers have received research funds for the last five years, and then give me annual figures on enforcement actions that have been taken against researchers who have  failed to comply with the public access policy. E.g. how many enforcement letters have been sent out, how many researchers have had special conditions placed on their awards, and how many have been precluded from obtaining future awards for a specified period (and for how long a period)?

NIH: We funded over 228,000 grants in the past 5 years, which support hundreds of thousands of investigators. NIH public access policies have been developed centrally, and compliance activities are carried out by individual NIH institutes and centers.

There are currently terms and conditions which require institutions to comply with the public access policy. When an investigator is found out of compliance with the NIH public access policy, the NIH institute or center in which the award resides will send out letters to bring the investigator into compliance.

NIH does not currently collect statistics on compliance efforts.