test: add cache testing suite

[flakey5]

This adds the test suite from https://cache-tests.fyi to test the cache interceptor.

Note these aren't compliance tests, we don't need to pass 100% of these. Arguably we shouldn't since some of the tests aren't even applicable to Undici (i.e. cdn-specific, we can skip these though). However, I do think we should try to rely on these in CI.

Current issues:

• feat: support request cache control directives #3658 needs to be merged
• Lots of tests failing due to network errors (TypeError: fetch failed, mostly socket & http protocol issues)
• Some test failures

Closes #3852
Closes #3869

cc @mcollina @ronag

[KhafraDev]

Note these aren't compliance tests, we don't need to pass 100% of these. Arguably we shouldn't since some of the tests aren't even applicable to Undici.

If these tests are targeting fetch, and fetch is a browser spec, why wouldn't some of the tests be applicable?

[flakey5]

If these tests are targeting fetch

They aren't targeting fetch. It has tests for every part of the stack that can implement caching. They're written in js w/ fetch so that they can be ran in browsers, but browsers aren't the only thing the tests cover. A lot of the behaviors do overlap, however, there are still things specific to servers or cdns.

There are also tests for things that rfc9111 obsoletes (such as using the pragma header for caching directives)

[KhafraDev]

They aren't targeting fetch

What I mean is that the tests explicitly use fetch and it tests browser caching behavior (every http request made through browsers is done through fetch, at least in the specs). Anything that doesn't work would seem like a bug to me.

In the fetch spec, the http cache is built a layer above fetch, whereas in undici it's a layer below. For example, in multiple parts of the spec we are meant to query the cache directly (see http-network-or-cache-fetch). There are multiple situations where headers may get added to requests/responses, or the internal request is modified in some way that will cause bugs or deviate from browser behavior.

It's not a huge issue, as it's opt-in and not built specifically for fetch, but it does feel weird to not implement the caching steps in the spec, but still use fetch to test it.

A lot of the behaviors do overlap, however, there are still things specific to servers or cdns.

Is there anything specific?

There are also tests for things that rfc9111 obsoletes (such as using the pragma header for caching directives)

Is lack of pragma support intentional, because it's obsolete? Is there no plan on adding support for it?

[flakey5]

Is there anything specific?

They're mostly tests for CDN-Cache-Control

Is lack of pragma support intentional, because it's obsolete? Is there no plan on adding support for it?

It was intentionally left out, right now there aren't any plans to add it.

[flakey5]

So it does look like we can run tests that only apply to browsers, which should skip the cdn tests

[mcollina]

Why did you lift the entirety of the source code of that project? What's the long term maintainability strategy?

Either:

• we can use the npm module they publish, so just add a dependency
• we vendor all their repo, but you also add a way to automatically update it
• we only lift the test cases, essentially making future updates on these harder

Anyhow, I would recommend not to use fetch() for this, because it'd make things harder long term. Use undici.request().

[KhafraDev]

we can use the npm module they publish, so just add a dependency

We can't - it hasn't been updated in 4 years and there seem to be relevant tests added and fixed in that time.

Anyhow, I would recommend not to use fetch() for this, because it'd make things harder long term. Use undici.request().

afaict the tests are specifically vendored for use in browsers and the runTests function they export expects fetch. Using anything other than fetch would likely end up being harder to maintain.

[flakey5]

Why did you lift the entirety of the source code of that project

In addition to what @KhafraDev said, there's a fix I needed to make to the source (re http-tests/cache-tests#139)

afaict the tests are specifically vendored for use in browsers and the runTests function they export expects fetch. Using anything other than fetch would likely end up being harder to maintain.

+1

[flakey5]

Marking this ready for review now for the sake of getting the fixes so far in before the v7 release.

The current status is ~53-55% tests are passing. Roughly 20% of the tests are failing but marked as optional. Another roughly 20% of the tests are failing and aren't marked as optional, however, their behaviors either are optional according to RFC9111 or they're obsolete/deprecated behaviors. There are 3-4 that are failing that are an exception to this, however, any time I reproduce the tests outside of the test runner w/ the same parameters they pass 🤷 . A handful of other tests fail as well since we don't have support for status codes other than 200 and 307 which will come later.

cc @mcollina @ronag

[mcollina]

lgtm