Default pagination.paginate function doesn't check for empty string link headers

[joshua-leyshon-canva]

Describe the bug

• Node.js version: v12.22.1
• OS & version: MacOS Big Sur, Ubuntu 20

Sometimes (e.g. with Buildkite REST API), a response can contain a link header that is an empty string. The default paginate logic (see just below) doesn't handle this case and will throw when it tries to parse the string. This means we need to add our own paginate logic that is almost identical to the default.

Current default function:

got/source/core/options.ts

Line 677 in 06a2d3d

if (typeof response.headers.link !== 'string') {

A fix:

if (typeof response.headers.link !== 'string' || response.headers.link === '') {
  return false;
}

Actual behavior

got throws when parsing an empty string link header.

Expected behavior

The default pagination.paginate function should return false when the link header is an empty string.

Code to reproduce

const res = got.paginate.all('https://somewebsite.com/that-returns-empty-link-header');
// An example endpoint (requires Buildkite authentication):
// "https://api.buildkite.com/v2/organizations/{org.slug}/pipelines/{pipeline.slug}/builds/{build.number}/jobs/{job.id}/artifacts"

Checklist

• I have read the documentation.
• I have tried my code with the latest version of Got.
• I have tried my code with the latest version of Node.js.

[joshua-leyshon-canva]

Note that I have separately raised the issue of returning an empty link header with Buildkite as well, but I figured we could also improve the default paginate function here 🙂

[sindresorhus]

What does the spec say about empty link? If it's a spec violation, we should at least emit a warning, even if we decide to gracefully handle it.

[joshua-leyshon-canva]

Good question. This doc is the closest spec I found, which outlines what a link string should look like, but doesn't explicitly say anything about an empty link. Might be up to you how to handle this case 😄

[szmarczak]

Actually might be empty. The specification defines #link-value, which means from 0 to infinity occurrences.

https://datatracker.ietf.org/doc/html/rfc1945#section-2.1

[joshua-leyshon-canva]

Hey @sindresorhus @szmarczak I opened #1768 to fix this issue 🦆