Saturday, May 23, 2009

Open Access mandates: Judging success

As Alma Swan has graphically demonstrated (here and here), mandates have begun to propagate nicely.

It is worth noting that many of the new ones are being introduced by faculty themselves, rather than by administrators imposing mandates on them from above. It is also noteworthy that many of the mandates in the recent upsurge have been introduced by library faculties.

But what level of compliance can we expect from these mandates? After all, a mandate is only as good as the compliance rate it achieves. And how do we judge success so far as compliance is concerned anyway?

Arthur Sale’s analyses of the effect of mandates on Australian researchers suggest that a high level of compliance with a mandate can be achieved within two years. (Sale appears to have judged success as being a compliance rate of 70%)

Perhaps the most controversial and hard-won mandate was the one introduced at the US National Institutes of Health (NIH) in May 2005. Initially this was a request.

By November 2005 it was reported that fewer than 5% of NIH grantees were complying with the request. As a result, last year the mandate was upgraded to a requirement.

The new policy came into effect in April 2008. Since then, an NIH spokesperson tells me, “Compliance has increased almost 250% ... It has jumped from 19% of our target estimate 80,000 papers per year arising from NIH funds during the voluntary policy to almost half (49%) of the target estimate of papers arising from NIH funds at the end of 2008.”

And compliance, he added, continues to improve. “In January and February 2009 we collected over 3 times as many manuscripts as we did in January and February 2008, before the requirement took effect, and March and April numbers appear even higher.”

A similar increase in deposits has been experienced at the University of Stirling following the introduction of a mandatory self-archiving policy by its Academic Council last year (which came into force last September).

Last week it was announced that deposit rates in the University’s repository (STORRE) have grown from 20 a month to 120 a month, and STORRE now hosts 1,000 papers, reports and book chapters.

Commenting on the JISC-REPOSITORIES mailing list, the University of Stirling’s eLearning Developer Michael White said, “Whilst we are aware that we don't yet have 100% compliance with our mandate, the key point is that we only managed to get 63 journal articles over the 3 years prior to the announcement of the mandate, but have got 687 items (excluding eTheses) in the year since (with the vast majority coming in after the mandate came into force in September).”

A number of questions naturally arise:

— Can we expect the NIH compliance rate to match the levels reported by Sale in Australia by next April (i.e. two years after it became mandatory)?

— Can we expect the surge of new mandates to achieve the same levels of compliance reported by Sale?

—  Is there any significance in the fact that many of the new mandates are being introduced by library faculties, and can we expect that to affect compliance rates?

— Will the fact that many of the new mandates are self-imposed affect compliance rates? (Will it make them appear more voluntary than mandatory)?

— Will the fact that many of the new mandates include opt-outs affect compliance rates? (Will that make them appear more voluntary than mandatory?)

— What is full compliance so far as a self-archiving mandate is concerned? (Is Sale’s 70% level the objective, or should the research community be aiming higher?)

What other questions should we be asking, particularly when trying to judge the success of a mandate?

All comments welcome!

3 comments:

Stevan Harnad said...

You asked!(1) The latest and fastest-growing kinds of Green Open Access Self-Archiving Mandates are not only self-chosen by the researchers themselves, but they are department/faculty/school mandates, rather than full university-wide mandates. These are the "patchwork mandates" that Arthur Sale recommended adopting, rather than waiting for university-wide consensus to be reached.

(2) Another recent progress report for Institutional Repositories, following Stirling's, is Aberystwyth's, which reached 2000 deposits in May.

(3) Richard asks: "Will the fact that many of the new mandates include opt-outs affect compliance rates? (Will that make them appear more voluntary than mandatory?)"According to Alma Swan's international surveys, most authors report they would comply willingly with a self-archiving mandate. The problem is less with achieving compliance on adopted mandates than with achieving consensus on the adoption of the mandate. (Hence, again, Arthur Sale's sage advice to adopt "patchwork" department/faculty/school mandates, rather than waiting passively for consensus full university-wide mandates, is the right advice.)

And mandates themselves are to reinforce researchers' already-existing inclination to maximise access and usage for their give-away articles, not to force researchers to do something they don't already want to do.

(Researchers need to be reassured that their departments or institutions or funders are indeed fully behind self-archiving, and indeed expect it of them; otherwise researchers remain in a state of "Zeno's Paralysis" about self-archiving year upon year, because of countless groundless worries, such as copyright, journal choice, and even how much time self-archiving takes.)

(4) Richard also asks: "What is full compliance so far as a self-archiving mandate is concerned?"

Full compliance is of course 100% compliance, and the longer-standing mandates are climbing toward that, but their biggest boost will come not only from time, nor even from the increasingly palpable local benefits of OA self-archiving (in terms of enhanced research impact), but from the global growth of Green OA Self-Archiving Mandates that Alma has just graphically demonstrated.

(5) "What other questions should we be asking?"

We should be asking what university students and staff can do to accelerate and facilitate the adoption of mandates at their institution. (See "Waking OA’s “Slumbering Giant”: The University's Mandate To Mandate Open Access.")

And the right way to judge the success of a mandate is not just by reporting the growth in an institution's yearly deposit rates, but by plotting the growth in deposit rate as a percentage of the institution's yearly output of research articles, for the articles actually published in that same year.

Bill Hooker said...

Full compliance is of course 100% compliance, and the longer-standing mandates are climbing toward thatWhere could I find data to show this? Is there anything central, rather than making individual application to each repository for some version of their records?

Richard Poynder said...

Further comments on this post can be viewed here, here, and here.