AI Summaries by Kudos: authors' verdict (spoiler: it's good news)

January 31 2024 / By Charlie Rapple

As part of our work to introduce AI-generated summaries of research publications, we have worked with some of our publisher partners to test the quality of AI summaries, and explore how authors feel about them. We received such a positive response that we are now rolling out our AI summaries more widely.

First, let’s break down what an AI-generated Story page on Kudos looks like.

AI summary page optimized
Structure of an AI-generated Story page on Kudos (click for a PDF version you can zoom in on)

Simplified titles

Right at the top (1) we have the plain language title – this is the title of the summary, rather than of the original article. In our example, the original paper is called “Grounding perceptions: the value of otoendoscopy images in strengthening parents and children's understanding and decision making in a pediatric ENT clinic”. This is an important level of detail for specialists determining whether the article is likely to be relevant or useful to them. But it's hard to see that at a glance. In this age of information overload, summarized versions of titles are valuable even to specialists – when you are skimming through a long reading list or table of contents, it can help you "triage" which articles to look at more closely. Simplifying the language used in titles also helps them to rank more highly in search engines, and of course they are easier for people in adjacent fields, or practitioners, patients and policy makers, to understand. Our AI-generated version simplifies titles – in this case, to “Using real-time eardrum images to support decision-making”.  Easier for specialists to assess, easier for non-specialists to understand.

AI summary structure 1

What it is about and why it is important

The main text on the page (2) is broken into two paragraphs, answering the questions “what is this about?” and “why is this important?”. This is quite different to the original article abstract, which follows the scientific structure of outlining methods and results. Our focus is on drawing out the “key takeaways” (3) that need to be understood both by other researchers, but also by broader audiences including healthcare practitioners, patients, policy makers and educators. Authors can use Kudos tools to review and edit the summary that has been written by AI (4).

AI summary structure 2,3,4

Extra Storytelling features

Kudos is a leader in research storytelling, and part of this is bringing research to life with magazine-style illustrations and quotes. Authors can choose a featured image to illustrate their AI-generated summary (5), upload a visual abstract, infographic or an image from the research (6), and add a quote (7). They can also add links to related materials such as videos, code, datasets, press coverage and more (8). “Read article” links go to the version of record, typically on the publisher website.

AI summary structure 5,6,7,8

Authors’ feedback

Our test process generated a plain language title, text for “what’s it about” and “why is it important”, and key takeaways. We created basic story pages with this information and shared these with authors to review. At this point, pages did not have images, quotes from authors, or extra resources – these were all options for authors to add if they liked their page. We followed up with authors to share their page with them and ask them to rate how well the AI had done in summarizing their article. The results were really encouraging:

How would you rate the accuracy of your AI-generated article summary? Chart shows 40% of respondents saying "Excellent, no errors" and 60% saying "Good, it is mostly accurate with only minor mistakes"

How would you rate the readability of your AI-generated article summary? Chart shows 40% of respondents saying "Excellent, usable with no changes" and 60% saying "Good, it only requires small changes"

In conclusion

One of our favourite bits of feedback came from Professor Bernie Carter of Edge Hill University, who summed up one of the main benefits of using AI to generate research summaries:

Quote says: "The AI took a lot of effort out of having to start – much quicker to edit something than start from scratch. "

Enter your email address to receive notifications of new posts by email.