December highlights from the world of scientific publishing

Some of what I learned last month from Twitter: takedowns, luxury journals, moves in peer review services and more.

‘Luxury journals’

A big talking point on my Twitter feed in December was the provocative comments about journal publishing made by Randy Sheckman as he received his Nobel Prize for Physiology and Medicine. Writing in the Guardian on 9 December, he criticised the culture of science that rewards publications in ‘luxury journals’ (which he identified as Nature, Cell and Science). He said “I have now committed my lab to avoiding luxury journals, and I encourage others to do likewise.” Although many applauded this, some pointed out that more junior researchers may not have the freedom to do likewise, and also mentioned Schekman’s potential conflict of interest as Editor-in-Chief of the new, highly selective journal eLife, which aims to compete for the best research with these journals (some responses are summarized by Stephen Curry). Schekman responded to the criticisms in a post on The Conversation, and suggested four ways in which the research community could improve the situation.

Elsevier steps up takedown notices

Subscription journals generally require the author to sign a copyright transfer agreement that, among other things, commits them not to share their paper widely before any embargo period has passed. It appears that in December Elsevier decided to increase their enforcement of this by sending takedown notices to sites where Elsevier papers were posted. Guy Leonard described what happened to him and the reaction on Twitter and elsewhere.

Various peer review developments

Jeremy Fox (@DynamicEcology), a population ecologist at the University of Calgary, explained why he likes the journal-independent peer review service Axios Review (and is joining their editorial board).

Publons, which gives researchers credit for post-publication peer review, announced that researchers can now also get credit for their pre-publication peer reviews for journals.

Wiley announced a pilot of transferable peer review for their neuroscience journals, in which reviews for papers rejected from one journal can be transferred to another journal in the scheme, thus saving time.

F1000Research announced that its peer reviewed articles are now visible in PubMed and PubMed Central, together with their peer reviews and datasets. Articles on F1000Research are published after a quick check and then peer reviewed, and indexing by PubMed and PubMed Central happens once an article has a sufficient number of positive reviews.

Jennifer Raff, an Anthropology Research Fellow at the University of Texas at Austin, published “How to become good at peer review: A guide for young scientists“, a very useful and comprehensive guide that she intends to keep updated as she receives comment on it.

Miscellaneous news

Jeffrey Beall, a librarian who has been curating a useful list of ‘predatory’ open access journals for several years, revealed his antagonism to open access as a whole in an article that surprised many with its misconceptions about the motivations of open access advocates. PLOS founder Mike Eisen has rebutted the article point by point. Although I feel that Beall’s list is still useful for checking out a new journal, it should be taken only as a starting point, together with a detailed look at the journal’s website, what it has already published, and its membership of the Open Access Scholarly Publishers Association (OASPA).

James Hayton (@JamesHaytonPhD) wrote a great post on his 3 Month Thesis blog on the seven deadly sins of thesis writing, which should also all be avoided in paper writing: lies, bullshit, plagiarism, misrepresentation, getting the basics wrong, ignorance and lack of insight.

Finally, I was alerted (by @sciencegoddess) to a new site called Lolmythesis, where students summarize their thesis in one (not always serious) sentence. Worth a look for a laugh – why not add your own?

SpotOn London session: The journal is dead, long live the journal

I’m co-hosting a workshop at SpotOn London next week on the future of journals.

It’s time to end a long blogging hiatus to tell you about an exciting event coming up on Sunday 11 and Monday 12 November. SpotOn London (formerly called Science Online London) is a community event hosted by Nature Publishing Group for the discussion of how science is carried out and communicated online. There will be workshops on three broad topic areas – science communication and outreach, online tools and digital publishing, and science policy – and I am involved in one of the ‘online tools and digital publishing’ ones. This has the title ‘The journal is dead, long live the journal‘ and it will focus on current and future innovations in journal publishing. If you’re interested in how journals could or should change to better meet the needs of science, this is for you!

In this one-hour session we will have very short introductions from four representatives from different parts of the journal publishing world:

  • Matias Piipari (@mz2), part of the team behind Papers software for finding an organising academic papers
  • Damian Pattinson (@damianpattinson), Executive Editor of PLOS ONE
  • Davina Quarterman, Web Publishing Manager at Wiley-Blackwell
  • Ethan Perlstein (@eperlste) of Princeton University

We will then open the floor to contributions from participants, both in the room and online. We hope to cover three themes:

  1. Megajournals; their impact on the journal and on how papers are going to be organised into journals. Will megajournals lead to a two tier marketplace of high end journals and a few megajournals, with mid-tier journals disappearing from the market altogether?
  2. How do we find the papers of interest, in a world where journal brand doesn’t help? In a world where issues disappear, and researchers’ main point of contact with the literature is through aggregation points such as Google Scholar and Pubmed, what are the signifiers that we can build or support that will enable researchers to find the content that they need?
  3. Once you get down to the paper, are there any innovations that we should be using now, at the individual paper level, and what are the barriers to us doing this?

Science Online events have a tradition of being more than just conferences – they aim to involve lots of people outside the room via the SpotOn website and Twitter as well as those in the room. So although the conference itself is sold out (though there is a waiting list for tickets), you can still follow along and get involved before, during and after the event itself. This session is at 4.30pm on Sunday 11 November, so look out on the Twitter hashtag #solo12journals around then. Beforehand, you can comment on co-host Ian Mulvany’s blog post introducing the session, look at the Google Doc that shows the thought processes the organisers went through in planning the session, check for tweets on the hashtag, and follow me (@sharmanedit), Ian (@ianmulvany) and co-host Bob O’Hara (@bobohara) and/or the speakers on Twitter for updates.

One the day, comments from Twitter will be moderated and introduced into the discussion in the room by Bob, who will be doing this remotely from Germany. The whole session (and all other SpotOn London session) will be live-streamed (probably here) and the video will be available afterwards; there will also be a Storify page collecting tweets using the #solo12journals hashtag.

This interaction with those outside the room is important because with only an hour there is a limit to the depth with which we will be able to cover the range of issues around journals. With online discussion as well we hope that more points can be discussed in more detail than would otherwise be possible. It might get a little confusing! I am new to this format, so I am slightly apprehensive but also excited about the possibilities.

Thoughts on megajournals

I am particularly interested in the aspect of the session on megajournals and how they are changing journal publishing. By megajournals we mean all the journals that have been set up to publish papers after peer review that assesses whether the research is sound but doesn’t attempt to second-guess the potential impact of the work. Some, like PLOS ONE, are truly mega – they published over 13,000 papers in 2011. Others, like the BMC series from BioMed Central, probably publish a similar number of papers but divided into many journals in different subject areas. Others have been set up to be sister journals to better known selective journals – for example, Scientific Reports from Nature Publishing Group and BioOpen from The Company of Biologists. All are open access and online only.

Some of these journals are now showing themselves not to be the dumping ground for boring, incremental research that they might have been expected. When PLOS ONE’s first impact factor was revealed to be over 4, there was surprise among many commentators. The question is now whether papers that are unlikely to be accepted by the top journals (roughly speaking, those with impact factors over about 10, though I know that impact factor is a flawed measure) will gradually be submitted not to specialist journals but to megajournals. The opportunity to get your paper seen by many people, which open access publishing provides, could often outweigh the benefits of publishing in a journal specific to your specialist community where your paper will be seen by only that community. I will be very interested to hear people’s thoughts on this issue raised by this session.

Get involved

So do comment using one of the channels mentioned above. Have you recently made a decision about where to send a paper that you knew wasn’t one for the top-flight journals, and did you decide on a specialist journal, a megajournal or some other route to publication? Regarding the other two themes of the session, how do you find papers in your field, and what do you want research papers to look like?

Journal metrics

Last week a new measure of the impact of a journal was launched: Google Scholar Metrics. So it seems like a good time to review the various metrics available for journals.

Below I summarise six measures of journal impact: the impact factor (IF), 5-year IF, Google Scholar Metrics, SCImago Journal Rank (SJR), Source Normalized Impact per Paper (SNIP) and Excellence in Research for Australia (ERA) ranking. As part of the research for this post I have found out the metrics (except 5-year IF) for a sample of 97 of the higher-impact biomedical journals and put them into a Google Docs spreadsheet, which can be viewed here (or on Google Docs directly here).

Most researchers get to know the IF fairly quickly when they start to read journals. So you probably know that an IF of 30 is high and that thousands of journals (the long tail) have IFs below 1. But fewer people have this kind of familiarity with the other metrics. So I have tried to estimate what range of numbers counts as ‘high impact’ for each metric. ‘High’ here means in the top 33% of my sample of 97 journals (already a high-impact sample).

To summarise the number that counts as high for each metric:
IF: 14
5-year IF: about 15
Google Scholar Metrics: 101
SJR: 0.53
SNIP: 3.85

Note that I am only talking about journal metrics, not metrics for assessing articles or researchers. As always, anyone using these figures should make sure they are using them only to judge journals, not individual papers or their authors (as emphasised by a European Association of Science Editors statement in 2007). Also remember that citations can be gamed by editors (see my previous post on the subject or a recent Scholarly Kitchen post on a citation cartel for more details).

Impact factor

The IF is provided by Thomson Reuters as part of their Journal Citation Reports, which covers ‘more than 10,100 journals from over 2,600 publishers in approximately 238 disciplines from 84 countries’.

It is calculated by dividing the number of citations in one year of articles published in that journal during the previous two years.

What counts as big: the highest-ranked journals have IFs over about 14; middle-ranking journals have numbers between 3 and 14; many low-ranked journals have numbers around 1.

Five-year impact factor

This is similar to the standard two-year IF except that citations and articles are calculated over the previous five years rather than two. It has been published only since 2007. This metric has advantages in slower-moving fields, where papers gather citations more slowly than a year or two after publication.

It is difficult to find lists of five-year IFs online, although some journals display them on their home pages. I did, however, find a study in the journal Cybermetrics that showed it is generally about 1.05 times the size of the two-year IF.

What counts as big: 15 using this figure.

Google Scholar Metrics

These were introduced on 1 April 2012 and are based on the Google Scholar database, which includes more journals and other publications than that used for the IFs. They are based on the h-index, which is defined on the Google Scholar Metrics page as follows:

The h-index of a publication is the largest number h such that at least h articles in that publication were cited at least h times each. For example, a publication with five articles cited by, respectively, 17, 9, 6, 3, and 2, has the h-index of 3.

This is a rather difficult concept to get your head around (at least it is for me). Basically the number cannot be bigger than the number of papers a journal has published, and it cannot be bigger than the highest number of times any one paper has been cited. So in the above example the h-index cannot be greater than 5 because there were only 5 articles, and the largest number of citations lower than 5 is 3, so the h-index is 3.

Google Scholar Metrics extends this as follows:

The h-core of a publication is a set of top cited h articles from the publication. These are the articles that the h-index is based on. For example, the publication above has the h-core with three articles, those cited by 17, 9, and 6.

The h-median of a publication be the median of the citation counts in its h-core. For example, the h-median of the publication above is 9. The h-median is a measure of the distribution of citations to the h-core articles.

Finally, the h5-index, h5-core, and h5-median of a publication are, respectively, the h-index, h-core, and h-median of only those of its articles that were published in the last five complete calendar years.

So the main metric is the h5-index, which is a measure of citations to a journal over 5 years to April 2012.

Note that this metric doesn’t involve any division by the number of papers published by the journal (unlike the other metrics discussed here). This means that journals that publish more papers will have proportionally larger values in Google Scholar Metrics than with other metrics.

What counts as big: the highest-ranked journals have h5-indexes over about 101; many journals seem to have numbers under 50.

SCImago Journal Rank (SJR)

The SCImago Journal Rank (SJR) is a metric produced by Scopus (part of Elsevier). It is calculated as follows:

It expresses the average number of weighted citations received in the selected year by the documents published in the selected journal in the three previous years, — i.e. weighted citations received in year X to documents published in the journal in years X-1, X-2 and X-3.

So it is also a measure of citations similar to a three-year impact factor, but the citations are weighted according to where the citation was. Further information is here (pdf). The weighting depends on how many citations each journal gets. So if journal A is cited a lot overall and journal B is not cited as much, and a paper in journal C is cited in journal A, that citation is given more weight in the calculation than a citation of journal C in journal B.

What counts as big: the highest-ranked journals have SJRs over about 3; many journals seem to have numbers under 0.5.

(Note that on the SCImago website the decimal point in the SJR is given as a comma in some places, so it looks as if the top journals have SJRs of over 1000 (1,000). On the spreadsheets that are freely downloadable from the same site or from the ‘Journal Metrics’ website (also from Elsevier) the metrics are given as 1.000 etc, so I think this is the correct version.)

Source-Normalized Impact per Paper (SNIP)

The Source-Normalized Impact per Paper (SNIP) is defined as the ratio of a journal’s citation count per paper and the citation potential in its subject field. It is designed to aid comparisons between journals in fields with different patterns of citations. It is calculated as follows:

Raw impact per paper (RIP)
Number of citations in year of analysis to a journal’s papers published in 3 preceding years, divided by the number of a journal’s papers in these three years

Database citation potential in a journal’s subject field
Mean number of 1-3 year old references per paper citing the journal and published in journals processed for the database

Relative database citation potential in a journal’s subject field (RDCP)
Database citation potential of a journal’s subject field divided by that for the median journal in the database

Source normalized impact per paper: (SNIP)
Ratio of a journal’ raw impact per paper (RIP) and the relative database citation potential (RDCP) in the subject field covered by the journal

So basically a three-year impact factor is weighted according how much papers in other journals in the same field are cited.

When I looked for lists of SNIPs for 2010 I encountered a problem: two different lists gave two different answers. The list downloaded from the Journal Metrics site gives the 2010 SNIP for Cell as 1.22, but when I searched on the CWTS Journal Indicators site (which is linked from the Journal Metrics site) it was given as 9.61. So there are two answers to the question of what counts as big: either anything over 0.7 from Journal Metrics or anything over 3 from CWTS Journal Indicators. If anyone can help me resolve this discrepancy I’d be grateful.

Australian Research Council Ranking

The Excellence in Research for Australia (ERA) evaluation exercise in 2010 included a system in which journals were ranked A*, A, B or C. Details of what the rankings mean is here. Top journals (in fact many that might elsewhere be called middle ranking) are ranked A*. It is not clear how these rankings were decided. These journal rankings were controversial and are not being used for the 2012 ERA.

Comparison of journals using these metrics

I have selected 97 high-impact journals in biology and medicine and compiled the metrics for them. I put the list of journals together by initially picking the top journals in the field by SJR, then removing all those that only publish reviews and adding a few that seemed important, or were in the MRC frequently used journal list, or were ranked highly using other metrics. The result is in a Google spreadsheet here. I have added colours to show the top, middle and bottom 33% (tertile) in this sample of each metric for ease of visualisation, and the mean, median and percentiles are at the bottom.

Sources of data:

  • IF: various websites including this for medical journals, this for Nature journals, this for Cell Press journals, this for general and evolutionary journals, this and this for a range of other journals, and individual journal websites. Please note that no data were obtained directly from Thomson Reuters, and they have asked me to state that they do not take responsibility for the accuracy of the data I am presenting.
  • Google Scholar Metrics: Google Scholar Citations Top publications in English and searches from that page.
  • SJR and SNIP: Journal Metrics.
  • ERA: ARC.

Notes on particular journals:

A few journals have anomalous patterns, unlike most that are high or lower in all the different metrics.

  • Ca: A Cancer Journal for Clinicians has a very high IF, SJR and SNIP, but comes out lower on Google Scholar metrics. A recent post in Psychology Today includes a suggestion of why this might be:

The impact factor reflects that the American Cancer Society publishes statistics in CA that are required citations for authoritative estimates of prevalence and other statistics as they vary by cancer site and this assures a high level of citation.

  • A few journals rank relatively low (out of this selection of journals) on all the metrics except the ERA rating, where they are rated A*: Development, The Journal of Immunology, Cellular Microbiology, Journal of Biological Chemistry, Molecular Microbiology, Developmental Biology, and Genetics. I don’t know why this might be, except that the ERA ratings appear to be subjective decisions by experts rather than being based on citations.
  • Proc Natl Acad Sci USA, Blood, Nucleic Acids Research, Cancer Research, Gastroenterology and most notably the BMJ come out high in Google Scholar Metrics but not so high in IF, SJR and SNIP. Perhaps they are journals that publish many papers, which is not accounted for by Google Scholar Metrics, or they could have more citations four or five years after papers are published, which would be picked up by Google Scholar Metrics but not the other metrics.
  • Finally, Systematic Biology has a high SNIP, a medium SJR and IF and a lower Google Scholar Metric. Perhaps it is in a field in which citations per paper are usually low, which is accounted for by the SNIP.

Your comments

Do you have experience of the lesser-known metrics being used by journals or by others to evaluate journals? Can you explain any of the anomalous patterns mentioned here or for the two different values for SNIPs?

Choosing a journal V: impact factor

This the fifth post in my series on choosing a journal, following posts on getting your paper published quickly, getting it noticed, practicalities, and peer review procedure.

It is all very well getting your paper seen by lots of people, but will that lead to an increase in your reputation? Will it lead to that all-important grant, promotion or university rating?

The impact factor of a journal is a measure of the average number of citations of papers published over the previous two years in the year being measured. A very common view among academics is that having your paper published in a journal with a high impact factor is the most important thing they can do to ensure tenure, funding, promotion and generally success. And in fact the impact factor of the journals your papers are in still has a big influence on many of those whose job it is to assess scientists (as discussed recently on Michael Eisen’s blog). It is also a factor in whether librarians choose to subscribe to a journal, which will affect how widely your paper is seen.

So even if the impact factor has flaws, it is still important. However, remember the following caveats:

  • Citations are only a proxy measure of the actual impact of a paper – your paper could have an enormous influence while not being cited in academic journals
  • Impact doesn’t only occur in the two years following the publication of the paper: in slow moving fields, in which seminal papers are cited five or ten years after publication, these late citations won’t get counted towards the impact factor so the journal’s impact factor will be smaller than justified
  • The impact factor measures the average impact of papers in the journal; some will be cited much more, some not at all
  • There are ways for journals to ‘game‘ impact factors, such as manipulating types of article so that the less cited ones won’t be counted in the calculation
  • The methods used for calculating the impact factor are proprietary and not published
  • Averages can be skewed by a single paper that is very highly cited (e.g. the 2009 impact factor of Acta Crystallographica A)
  • Although impact factors are calculated to three decimal places, I haven’t seen any analysis of the error in their estimation, so a difference in half a point may be completely insignificant
  • New journals don’t get an impact factor until they have been publishing for at least three years.

So although it is worth looking at the impact factor of a journal to which you are considering submitting your paper, don’t take it too seriously. Especially don’t take small differences between the impact factors of different journals as meaningful.

Other new metrics are being developed to measure average impact of journals, such as the Eigenfactor and Source Normalized Impact per Paper (SNIP) and SCImago Journal Rank (SJR). These might be worth looking at in combination with the impact factor when choosing a journal.

Your experience

How important is the impact factor of a journal in your decision to submit there? Have you taken other measures of impact into account? Do you think the impact factor of journals you have published in has affected the post-publication life of your papers?

And journal editors, how much difference does the impact factor of your journal make to how many papers are submitted to it, or to your marketing? Do you know the Eigenfactor, SNIP or SJR of your journal?

Choosing a journal II: getting your paper noticed

This is the second in a series of posts on factors to consider when choosing which journal to submit your paper to. Here, I will look at how your choice of journal can affect the extent to which your work is noticed.

Part one of the series, on getting your paper published quickly, is here.

How well known is the journal?

It goes without saying that papers in very well known journals like NatureScience, Proc Natl Acad Sci USA and Cell will be seen by more people than those in other journals. The more specialist journals may, however, be the ones that are seen by the people in your field, who you mainly want to reach.

How well they do they publicise their papers?

Most journals send out their tables of contents to interested readers by email. This is really a minimum, and any journal that does not doesn’t deserve your paper. But there are other ways that journals can publicise papers as well.

  • Press releases – do they have an active press office that writes attractive press releases and sends them to relevant media together with the embargoed article before publication? Do they also send press releases to bloggers (who are often more interested in science news than are the mainstream media)?
  • Do they have a blog highlighting recent papers?
  • Do they have a Twitter account with plenty of followers?
  • Do they have a Facebook page ‘liked’ by plenty of people?
  • Do they give awards for top papers of the year or similar?

Do they publish articles highlighting research papers?

Nature, for example, publishes News and Views articles, which are short pieces highlighting the most interesting research papers in the current issue. Many other journals publish similar ‘minireviews’ about recent papers.

Some journals have an editorial in each issue summarising the papers in that issue (such as in the current issue of Gut).

Some have news sections in print or online (e.g. Science) where more accessible pieces about recent papers from that journal or elsewhere can reach a general readership.

Open access versus subscription

There is some evidence that open access articles are cited more than those in subscription journals; there is also some evidence against a citation advantage. It is certainly worth considering whether you want your paper to be read just by potential readers in an institute that subscribes to the journal, or also by independent researchers, those in less well funded institutes, journalists and members of the public.

If you don’t go for a full open access journal (called ‘gold’ open access), will the journal allow you post a version on your own website or in a repository (‘green’ open access; definitions here)? Does the paper become free to read online 6 months or 12 months after publication, or never? Journals vary a lot in their restrictions on how an author can distribute their paper (see the SHERPA ROMEO directory for details).

Is the journal indexed by indexing services?

When a journal is set up it takes a while for it to be indexed by services such as PubMed, ISI Web of Science and AGRICOLA. New journals will therefore not yet be indexed, so articles won’t be findable in searches of these databases. This makes it slightly more risky to publish in very new journals.

Your experience

Have you had a published paper promoted in any of these ways, or others? Did you choose open access to get your paper seen by more people? What difference do you think these choices made?