We recently hosted a webinar introducing some concepts around research impact — what it is, why it’s important, how it can be achieved and how it can be measured. One of the questions asked was “how related are journal impact and research impact?” I gave my usual answer about the importance of separating academic impact from “real world” or broader impacts, and how they are essentially separate things. But, reflecting on the point made me wonder if they are not more intertwined than I have tended to allow for in my rhetoric.
Journal impact is not inherited by the research published by that journal
Journal impact is usually a synonym for Impact Factors and other scores based on the average number of citations received by a particular journal in a particular period. Journal impact is not an indicator of the quality of individual studies published in that journal. The journal’s impact “score” is retrospective — it cannot tell you the influence that an individual piece of research will have in the future, even in academic terms. It cannot be taken as an indicator of the likely number of citations a paper will receive, and it cannot indicate what sort of change a study might lead to in the “real world”.
It is fair to say that a good journal Impact Factor does likely mean that more people try to publish in that journal (because it can look good on an academic CV, though many countries are moving away from evaluating researchers on where they have published). It then follows, to an extent, that the more people are submitting to a particular journal, the more selective it can be. And this, in theory, should mean that the research being accepted for publication by that journal is “better”. But note all the caveats I have used here. What constitutes “better” is still subjective, and depends on different editors’ and reviewers’ opinions. You cannot say that “research published in a high impact journal is in itself high impact”. That has to be established paper by paper, using a number of criteria, and will take years (I think the latest estimates are that it takes on average a year to get one citation, and maybe ten years for a paper to achieve the lion’s share of all the citations it will ever receive).
Journal impact ≠ academic impact ≠ research impact
Even though a paper doesn’t inherit journal-level “impact”, that doesn’t mean its academic impact is not influenced by the journal in which it’s published. A more highly ranked journal might be more discoverable, and thus more widely read; more readership might lead to more citations. Possibly the journal brand might encourage a greater trust in the paper’s quality, or a greater exaggeration of the paper’s relevance, again broadening and increasing the citation potential. But citation counts don’t equal academic impact, anyway, whether or not they’re influenced by journal impact.
Academic impact means the extent to which a piece of research has advanced knowledge in the field. This is not actually defined by citation count, however often citation counts are used as a proxy — it might equally well be defined in terms of the re-use of the methods or data published alongside the paper.
And however you’re defining academic impact, and whatever you’re using to try and quantify it, it is only one part of research impact — some would argue quite a small part.
Research impact = real world impact
Research impact is made up of a much wider range of impacts with the lion’s share considered to be impact beyond academia. Indeed, the phrase “research impact” is defined by many in a way that specifically excludes academic impact — as “real change in the real world” (I’ve gathered a few definitions elsewhere). It breaks down into many different kinds of impact including changes in attitudes; awareness; economic or social status; policy, cultural or health outcomes (I like Mark Reed’s breakdown of types of impact). In the short term, such “broader impacts” can be quite independent from the academic impact, being achieved through dissemination and engagement directly with people like to benefit from or apply the research findings and recommendations. Broader research impact requires a purposeful approach to making sure that more people can find, understand and act upon your research.
But ultimately real-world impact potential is increased by academic and journal impact
The potential for broader impacts is arguably increased by the academic impact and even by the journal impact.
Broader / research impacts must derive from the relevance, reach and quality of the research itself. That is to say, they must improve something, solve something, change something (relevance); and the scale of this change is driven by quality (how well does the research solve the problem or bring about the change, and therefore how wide is the effect) – the latter also being a function of reach (how many people find it and understand it).
But each of relevance, reach, and quality can also be influenced by academic impact and journal impact. The more a piece of research advances knowledge in its field, the more follow up knowledge is generated — which in turn expands the scope and scale of broader impacts. So the academic impact increases the broader impacts. Meanwhile the higher the impact of the journal in which it is published, the greater the likelihood that the research will be found and read by those who can bring about that academic impact. So journal-level impact increases the likelihood of the academic impact of a specific paper, which in turn increases the broader impacts potential of the research the forms the basis of the paper.
