Get started

Describing a search as a plan

A plan separates describing a search from executing it. Plans are inspectable, saveable and version-controllable, and they can be partitioned, for example by year, so that a large retrieval stays under the API’s start < 5000 ceiling and can be cached and resumed.

The plan below describes the search the bundled records came from, so the rest of the article follows one worked example from description to export.

plan <- scopus_plan(
  "graphene supercapacitor",
  years     = 2015:2024,
  field     = "TITLE-ABS-KEY",
  partition = "year"
)
plan

Each row is one query cell. Field tags wrap the query and years become a date filter.

scopus_plan("language learning", field = "TITLE")$query

[1] "TITLE(language learning)"

scopus_plan("x", years = 2015:2020)$date

[1] "2015-2020"

# A plan is a classed object, and is_scopus_plan() confirms it.
is_scopus_plan(plan)

[1] TRUE

Sizing and fetching

scopus_has_key() reports whether a key is configured, without revealing it. It is the guard the package’s own examples use to skip the steps that need the API, so it is the natural switch for a reproducible script.

scopus_has_key()

[1] FALSE

With a key configured, you size a search cheaply and then execute the plan, optionally caching each cell so that an interrupted run resumes without re-spending quota. These contact the API, so they are not evaluated here.

scopus_count(
  "graphene supercapacitor",
  years = 2015:2024,
  field = "TITLE-ABS-KEY"
)

records <- scopus_fetch_plan(
  plan,
  cache_dir = scopus_cache_dir(),
  resume = TRUE
)

Without a key, the bundled corpus stands in for the result of that harvest, and the sections below run on it.

The record schema

Whether records come from the API or from the bundled corpus, they share one stable schema, so everything below would read the same on a harvest of your own. summary() takes stock of a set, and the first rows show the columns.

records <- example_records
summary(records)

<scopus_records> summary
138 records, from 2015 to 2024.
90 sources, 127 with a DOI.
Cited 7015 times in total, median 24 per record.
Most frequent source: ACS Applied Materials & Interfaces.
Most cited: Graphene for batteries, supercapacitors and beyond.

head(records)

# A record set is a classed tibble, and is_scopus_records() confirms the
# contract.
is_scopus_records(records)

[1] TRUE

scopus_records() produces this same shape from a raw API response, flattening the nested result into one row per record. The entry below carries the fields of a real article, one of those in the bundled corpus, in the form the API returns them.

raw <- list(entry = list(
  list(`prism:doi` = "10.1021/am509065d",
       `dc:title` =
         "Flexible and Stackable Laser-Induced Graphene Supercapacitors",
       `dc:creator` = "Zhiwei Peng",
       `prism:publicationName` = "ACS Applied Materials & Interfaces",
       `prism:coverDate` = "2015-01-13", `citedby-count` = "469")
))
scopus_records(raw, query = "TITLE-ABS-KEY(graphene supercapacitor)")

Most frequent sources and authors

A record set already answers the first descriptive questions. scopus_top() tallies the most frequent sources or authors, counting each contributor once per record. Across these 138 articles the tally is long-tailed, as a real literature is. They are spread over 90 distinct journals, and only one, ACS Applied Materials & Interfaces, appears more than five times.

scopus_top(records, by = "source")

vignette("analysing-a-literature") covers growth trends, top-source and top-author plots and abstract retrieval in depth.

DOIs and change tracking

Extract a clean, deduplicated DOI list for import into a reference manager, and compare two retrievals to see exactly what changed. Eleven of the 138 records arrived without a DOI, so 127 come back.

dois <- scopus_extract_dois(records)
length(dois)

[1] 127

head(dois, 4)

[1] "10.15541/jim20140527"            "10.1021/am509065d"              
[3] "10.1016/j.electacta.2015.02.019" "10.1002/smll.201403383"         

A search re-run later gains records and occasionally loses one to re-indexing. Here the baseline stops at 2023 and the second pull adds the 2024 articles while dropping the first record.

baseline <- records[records$year <= 2023, ]
later <- records[-1, ]
print(scopus_diff_dois(old = baseline, new = later))

<scopus_doi_diff> 14 added, 1 removed, 112 unchanged
# A tibble: 127 × 2
   doi                            status
   <chr>                          <fct> 
 1 10.1002/adfm.202315137         added 
 2 10.1002/asia.202400548         added 
 3 10.1002/slct.202302535         added 
 4 10.1016/j.cej.2024.148822      added 
 5 10.1016/j.diamond.2024.110842  added 
 6 10.1016/j.isci.2024.111696     added 
 7 10.1016/j.jallcom.2024.175000  added 
 8 10.1016/j.jallcom.2024.177248  added 
 9 10.1016/j.jpowsour.2024.234127 added 
10 10.1016/j.jpowsour.2024.236149 added 
# ℹ 117 more rows

You can write the DOIs to a path you specify, and read the file back to see exactly what lands on disk.

out <- file.path(tempdir(), "dois.csv")
scopus_extract_dois(records, file = out)
writeLines(head(readLines(out), 5))

"doi"
"10.15541/jim20140527"
"10.1021/am509065d"
"10.1016/j.electacta.2015.02.019"
"10.1002/smll.201403383"

Comparing topic trends

scopus_compare_topics() measures how the internal emphasis of a literature shifts, expressed as each comparison topic’s yearly share of the reference literature. It issues one count request per term per year, so it needs the API.

cmp <- scopus_compare_topics(
  reference_query  = "language learning",
  comparison_terms = c("effect size", "Bayesian"),
  years            = 2015:2020,
  field            = "TITLE-ABS-KEY"
)
plot_scopus_comparison(cmp)

The result is a tidy table with one row per topic and year, which plot_scopus_comparison() draws with direct line labels, a colour-blind-safe palette and shaded stability bands. vignette("comparing-topics") builds the object offline, shows the plot in its variations and explains how to read the bands.

Author keywords and references

A search only returns the fields the Search API carries. Author keywords and a document’s own reference list need view = "COMPLETE" and Abstract Retrieval respectively, both at a materially different quota cost from an ordinary search. vignette("keywords-and-references") walks through both, and scopus_corpus(), which combines them into a minimal id/title/year/keywords/references shape for downstream tools.

Export and interoperability

Hand results to bibliometrix-style workflows, or save and reload them.

m <- as_bibliometrix(records)
head(m[, c("AU", "TI", "PY", "SO", "TC")])

path <- file.path(tempdir(), "records.rds")
write_scopus_records(records, path)
identical(read_scopus_records(path), records)

[1] TRUE

Writing the search up

A search that is going into a paper has to be reported. scopus_search_report() assembles that record from the plan and the harvest, to the PRISMA-S standard, and formats it as a methods paragraph. It states only what the objects hold, so the record below is candid about the bundled corpus carrying no retrieval time of its own. The Search plans and quota-aware retrieval article works through it in full.

report <- scopus_search_report(records, plan = plan)
cat(format(report, style = "paragraph"))

The literature was searched in Scopus, on the Elsevier Scopus Search API, on a date this record does not carry. The search expression was TITLE-ABS-KEY(graphene supercapacitor), limited to publication years 2015 to 2024. It was partitioned into 10 cells, one per year, each retrieved through the STANDARD view in pages of 200 records, under a paging mode this record does not carry. The search retrieved 138 records. The API's own count of matching records was not recorded, so the harvest cannot be shown to be complete. Of the records retrieved, 127 carry a DOI. No de-duplication step was recorded for this set. The version of scopusflow that produced this set is unrecorded. The PRISMA-S items this record cannot supply, among them peer review of the strategy, grey literature and any other database searched, remain yours to report.

Handling failures

Network and API problems surface as typed conditions, all inheriting from scopus_error, so a workflow can respond to them in code.

tryCatch(
  scopus_fetch("..."),
  scopus_error_no_key     = function(e) message("No API key configured."),
  scopus_error_rate_limit = function(e)
    message("Rate limited, so backing off."),
  scopus_error            = function(e)
    message("Scopus error: ", conditionMessage(e))
)