Our first Kudos pilot went live in September 2013. Since then, over 100,000 researchers have signed up to explain and share their work. What have we learned from these actions, so far?
This post is a companion to a piece on the LSE Impact blog and to today's press release on the Kudos blog.
At last year’s 2:AM conference, I met members of the Altmetrics Research Team at the Centre for HEalthy and Sustainable CitieS (CHESS), Wee Kim Wee School of Communication and Information, Nanyang Technological University (NTU, Singapore). We talked about the work Kudos had been doing and they suggested they’d be interested in analysing some of the data we’d built up1. As it happened, we had already been looking for a team of independent statistics experts to help answer key questions about the extent to which our toolkit was helping researchers to increase the reach and impact of their work. The NTU's Altmetrics team started designing a study in November 2015 and have recently delivered their results.
The key research questions were: do efforts to explain and share work increase downloads? Which platforms are most commonly used, and which are effective? And the simple answers are: yes (usage of Kudos leads to 23% higher downloads for a publication), and while researchers are most likely to use Facebook to share their work, links shared via LinkedIn are more likely to be clicked.
(The posting at LSE Impact gives more detail about these findings, and the talk I gave at this year’s UKSG conference covers that detail as well as looking at some of the challenges inherent in designing and executing a study of this nature - video | slides).
To complement the NTU's Altmetrics team’s work, we also undertook a survey during April this year, which received almost 3,000 responses from academics2, and carried out interviews with academics during that same period. I also covered some high-level findings from that work in my UKSG talk; this posting provides some new detail from both projects, partly in response to questions at my UKSG talk, based on segmenting our initial results by career level and subject area.
Figure 1 shows that Kudos users are about 58% STEM, 24% social sciences and 4% arts and humanities. However, if we look at the demographics of those who have shared, 8% is by people in the arts and humanities, 29% by social scientists, and 51% by those in STEM. This suggests that those in the humanities are disproportionately more likely to share their work.
How does discipline or career level affect the likelihood of researchers using social networks to create awareness of their work?
In the survey, we asked "In which of the following ways do you currently create awareness of or share materials relating to your work?". The results suggest that:
Charts show percentage of respondents in each category that chose option “social networking”
How does discipline or career level affect the tools or networks that researchers use?
When researchers sign up to use Kudos, they self-assign a subject area and give an indication of career level (professor, postdoc, etc). These fields were used by the Nanyang team to drill down into sharing activities on Kudos, and explore any variation by discipline or career level. The results (Figure 4) show that mid-career researchers use Facebook and Twitter to a broadly similar extent (57% of their shares were via Facebook; 49% via Twitter) whereas students are much more likely to use Facebook (61%) than Twitter (38%). This analysis picks up on a question at the UKSG conference about whether students / early-career researchers might be more comfortable sharing in the more closed environment of Facebook and suggests that indeed they are. There are also some interesting variations by subject area (Figure 5); for example, while Facebook and Twitter are fairly equal in popularity among natural scientists and social scientists, Facebook is markedly more popular than Twitter in Engineering & Technology and the Humanities.
Charts show percentage of Kudos users that shared via each channel.
Meanwhile, in the survey, we asked "Thinking about academic tools and networks more specifically, do you use any of the following?”. We presented a mixed range of services that researchers had commonly referred to in our interviews as playing a part in their approach to building visibility of their work. The results (figure 6) aligned with our expectations; for example, Academia.edu is much more commonly used by those in the humanities (and to a lesser extent by those in the social sciences) than by those in STEM subjects, who are more commonly users of ResearchGate. It’s interesting to see - again, as expected, perhaps - that identifier services ORCID and ResearcherID are more commonly used by those in STEM.
Note: while the survey wasn’t restricted to Kudos users, invitation emails to Kudos users achieved substantially higher response rates than our own social media outreach or that of our publisher partners. Hence the response in relation to usage of Kudos is skewed and should not be compared directly to the responses that relate to the other tools here.
How does discipline or career level affect the audiences that researchers are trying to research?
In the survey, we asked "How important is it to you that each of the following audiences read and understand your work?" Our survey findings do not deviate from what you might expect:
These studies are both being written up separately for publication and we will keep you posted when they have been published in full! If you would like to try Kudos for your own work in the meantime, you can sign up at www.growkudos.com/sessions/register.
1 The study started with a pool of 22,170 publications that had been explained and / or shared via Kudos. Publishers had provided us with full text download data for 4,867 of these. (Of our publisher partners, approximately 10 are currently able to supply us with daily counts of article-level usage). We created a control group of publications for which these publishers had supplied us with usage data, but for which the authors had not explained or shared the work via Kudos. We noted the download count on the day that the publications were added to Kudos and analysed the growth in downloads between those dates and the date of the analysis. In looking at which channels researchers most commonly use for sharing, we had to bear in mind that Kudos was not integrated with LinkedIn until July 2015 – for that question we therefore reduced the dataset further to 2,488 publications that had been explained / shared via Kudos after July 2015 and for which we had data for full-text downloads.
2 Approximately half would be considered early career (still completing their PhD or within 10 years of doing so); just under half were in Europe (about a fifth in each of Asia and North America); about three quarters indicated a main subject area of their research in STEM, around a quarter social sciences and (because we allowed people to choose multiple options) about 15% gave a subject in the arts and humanities.