The Open Library of Humanities has just published a new policy on the use of AI in scholarly publishing that is well worth a read: Making sure you're not a bot!
(Sorry “making sure you’re not a bot” reflects the way OLH is defending itself and its journals against bots scraping OLH journals for content)
“authors planning on submitting research to an OLH journal must consult these guidelines first and must always provide a declaration of any substantial and known uses of generative AI to the editors of the journal they are submitting to for their consideration”
This helps editors in making quick decisions about papers if they find that the author has used AI without acknowledging it: you can always refer to the policy. At Glossa, we have found that several authors use AI to confabulate non-existing references, i.e. references that attribute non-existing papers to existing authors, and DOIs that do not lead anywhere. Desk reject!
At @uminhoeditora, Editors are also finding that fabricated or non-existent references are becoming an issue. At the Practical Community of Editors, we are addressing this topic in a meeting today, so this post could not have come at a better time.
Perfect timing indeed! I’m really curious to hear what comes out of your meeting and how editors are tackling this issue, including any feedback or tools they might be using.
The French Network Medici is also hosting a webinar at the beginning of February on the use of AI, showcasing tools that could be very useful, though unfortunately they are only available in French:
AI-generated abstracts check (detection of AI-generated summaries): aiAbstractCheck
Bibliographic integrity (detection of retracted or “hallucinated” references): bibCheck
I’m curious if there are any similar tools available in English/or others languages?
Hi Johan, that’s an interesting idea, it would be worth looking into it and seeing what might be possible, as well as how it could be managed in practice
Simon Everett of OLH just replied to me to say that iThenticate is also able to identify hallucinated references. But then again iThenticate is a service that may be too expensive for many publishers. Perhaps something EDCH shd look into
We have iThenticate at the @uminhoeditora . The problem is that it only works for English, Spanish, and Japanese.
One of the suggestions we gave, and it was proven by one of the journals, was doing a first triage of the article. The editorial assistant evaluates the article on those aspects more “formal”, such as the correct use of the template, identifying if the references are within the norms that they defined, and if they exist. Then they have another triage by a group of junior investigators of the investigation center, and they do a more in-depth work, such as seeing if the article is valuable or not.
Of course, some journals can’t do this job, and this journal is considering APCs because it cannot sustain the editorial assistant much longer without funding…
Research Libraries UK has bought together resources that examine the role and impact of AI in research and in the research library, including AI strategy and futures, skills development, open-source infrastructures, ethical and legal considerations, and the potential of AI to support more inclusive and trustworthy knowledge ecosystem. Some of the resources may be useful publishers as part of the same ecosystem.
Hi all, coming back to our discussion on AI policy, the Library Publishing Coalition (LPC) has just published an AI Editorial Policy Guide to help you develop an AI policy for your publishing program. This guide builds upon a previous LPC-hosted webinar on AI Policies.
Here is a quick overview of what you will find in this guide:
A step-by-step roadmap to create, use, and update your own AI policy.
The main elements that should be included in an AI editorial policy.
Real examples of AI policy created by other publishers (for example, the one by the Open Library of Humanities).
Triage is crucial and journals should pay greater attention to it, as good triage can save a lot of time. As someone working in a resource-limited environment, I can recommend a few simple, free strategies for checking references.
Use Crossref’s Simple Text Query to identify references that have a Crossref DOI. Paste references from a manuscript here: Simple Text Query. You’ll get a list where DOIs will be added to those that have a Crossref DOI, whereas the others will look the same as pasted. Please note that the latter may still be ok (e.g. print-only publications or publications from sources that don’t assign DOIs) and you should either check them manually or send the list from Simple Text Query to the authors (with a strict deadline) to verify (by providing links or records from reliable bibliographic sources and library catalogues) that those references exist. It they tried to cheat, it’s very likely that they won’t respond. This helps journals eliminate not only manuscripts with hallucinated references, but also references that are incorrectly cited or have been copied from the literature without verification.
Use Zotero to identify retracted references (this works only for references that have a Crossref DOI). Start the magic wand tool in Zotero (Add item(s) by identifier) and paste the list from Simple Text Query. Zotero will import metadata for references with a DOI and will clearly mark with a red warning cross the retracted ones. In the metadata panel in Zotero, a notice and details imported from the RetractionWatch Database (via Crossref) will be displayed above the metadata for each retracted item.
These strategies cannot solve all problems (nor can commercial tools), but they can help expedite triage and are freely available.
