Case 3: Enriching a List of Journals with Metadata

Last updated on 2026-10-23 | Edit this page

Estimated time: 45 minutes

Overview

Questions

  • How can metadata be retrieved automatically instead of being collected by hand?
  • What is an API, and when is it worth using one?
  • What can AI tools contribute, and what do they not replace?

Objectives

  • Explain in simple terms what an API is and how identifiers such as the ISSN are used to query it.
  • Compare ways to enrich a list with metadata with and without programming.
  • Assess the use of AI tools for collecting metadata, including the need for verification.
Callout

Scarytale: Babylonian confusion

Do you have a Japanese name? Fear for your reputation.

Play it as a yes/no guessing game, then read the solution: Babylonische Verwirrung (a change in the order of Japanese names leads to missed citations; persistent identifiers such as ORCID avoid the problem).

Source: Research Data Scarytales, Kompetenznetzwerk Forschungsdatenmanagement Thüringen, CC0.

The case

Researchers who want to publish a data paper ask which data journals exist in their discipline. A community-curated list of data journals by Maxi Kindling and Dorothea Strecker (2022, CC0) gives a good starting point: it contains the journals with their ISSNs. But to choose a journal, researchers need more: Is it open access? Does it charge publication fees? Under which licence are articles published? Which subjects does it cover? Is it still publishing?

Collecting this information by hand from every journal’s website would take days, and it would be outdated soon.

This case is based on the Data Journals Dashboard by Thomas Schmidt (UB Mannheim), presented as a project of the certificate course Research Data Management: Data Journals Dashboard: A web app for community-curated data journal metadata (CC BY 4.0).

Challenge

What would you recommend?

In small groups, discuss for 15 minutes:

  1. Which sources could provide the missing information for each journal?
  2. Which tools could retrieve it? What would one have to learn for each?
  3. What would you do for journals that are not covered by these sources?

What was done in the original case

  • Each ISSN of the list was queried against the API of the Directory of Open Access Journals (DOAJ). If the journal was found, its metadata was merged into the journal record.
  • For ISSNs not in the DOAJ, the metadata was collected with an AI coding assistant with web search, using the journals’ websites, the ISSN Portal and Wikidata. The results were checked and cleaned by hand, and all journals were checked manually whether they are still publishing.
  • For each journal, the source of the added metadata is recorded.
  • The result is a searchable dashboard and metadata exports (CSV, YAML, JSON), all under CC0. The code is on GitHub.

Tools in this case

Tool Good for What one needs to learn Installation
Web search of the DOAJ, ISSN Portal a few journals, checking nothing none
API in the browser understanding what an API returns reading JSON none
OpenRefine (add column by fetching URLs) lists of identifiers, no programming OpenRefine basics local installation
Python with requests many records, repeatable and documented workflow basic programming, JSON local installation or JupyterLite
AI assistant with web search filling gaps where no structured source exists prompting, verifying results depends on the tool
  1. Open the DOAJ API for one ISSN in the browser, e.g. Scientific Data: https://doaj.org/api/search/journals/issn%3A2052-4463. Show title, publisher, licence, publication fee (apc) and subjects in the JSON. Most browsers display JSON in a readable form.
  2. Show the same journal in the Data Journals Dashboard.
  3. Optional: show the field enrichment_source in the metadata collection to discuss documenting sources and AI use.

A small first step

Work with identifiers (ISSN, DOI, ORCID) instead of names and titles, and look up one identifier in an API in the browser. Seeing the result once makes it much easier to judge what can be automated.

Where to learn more

Key Points
  • Many metadata sources offer APIs, so manual collection is rarely necessary.
  • Identifiers such as the ISSN are the key to querying them reliably.
  • AI tools can help to fill gaps, but the results must be verified and their sources documented.