Enable persistent cache in Faster + Rspack

[tats-u]

Have you read the Contributing Guidelines on issues?

• I have read the Contributing Guidelines on issues.

Motivation

Rspack 1.2.0-alpha.0 used in Faster 3.7 now supports the experimental persistent cache.

https://github.com/web-infra-dev/rspack/releases/tag/v1.2.0-alpha.0
https://rspack.dev/config/experiments#persistent-cache

We need to modify the following parts:

docusaurus/packages/docusaurus/src/webpack/base.ts

Line 111 in 97690ab

return undefined;

     if (props.currentBundler.name === 'rspack') {
       // TODO Rspack only supports memory cache (as of Sept 2024)
       // TODO re-enable file persistent cache one Rspack supports it
       //  See also https://rspack.dev/config/cache#cache
-      return undefined;
+      return true;
     }

docusaurus/packages/docusaurus/src/webpack/base.ts

Lines 144 to 154 in 97690ab

	return {
	// This is mostly useful in dev
	// See https://rspack.dev/config/experiments#experimentsincremental
	// Produces warnings in production builds
	// See https://github.com/web-infra-dev/rspack/pull/8311#issuecomment-2476014664
	// We use the same integration as Rspress, with ability to disable
	// See https://github.com/web-infra-dev/rspress/pull/1631
	// See https://github.com/facebook/docusaurus/issues/10646
	// @ts-expect-error: Rspack-only, not available in Webpack typedefs
	incremental: !isProd && !process.env.DISABLE_RSPACK_INCREMENTAL,
	};

         // @ts-expect-error: Rspack-only, not available in Webpack typedefs
         incremental: !isProd && !process.env.DISABLE_RSPACK_INCREMENTAL,
+        cache: {
+          type: 'persistent',
+          // Some extra options
+       },
      };

Self-service

• I'd be willing to do some initial work on this proposal myself.

[slorber]

I tried already but it didn't work yet. It did rebuild faster but then the SSG failed. At best, we could expose this through env variables but I suspect it will fail on all sites. I'll ping the Rspack team about this when I have time to investigate a bit more (not this week, taking a break)

[tats-u]

You have already tried it, and it's bad news that it fails in SSG (I only tried it in preview with single locale by directly overwriting the JS file in node_modules).
The current API lacks name: `${name}-${mode}-${props.i18n.currentLocale}` (existing in Webpack), but is it fine?

[slorber]

It's an initial version, afaik they plan to support the same options as Webpack.

This cache name feature is indeed useful to create one cache per env (client/server) and locale (for i18n sites)

See also my explanation here: web-infra-dev/rspack#5658 (comment)

---

BTW beta.0 is out right now: https://github.com/web-infra-dev/rspack/releases/tag/v1.2.0-beta.0

[tats-u]

The design of the API has not been confirmed. I see.

The type of buildDependencies is different too. Webpack is object but Rspack is array. Looks like it's not critical now because we have only to extract .config.

I noticed the release of beta from the subscribed direct notification from Rspack.
I don't think its persistent cache is compatible with the current Docusaurus.

[jerrykingxyz]

The type of buildDependencies is different too. Webpack is object but Rspack is array.

Yes, the rspack configuration is designed differently than webpack and will not be compatible with webpack.

There are some reasons for this.

1. The architecture of rspack cache is completely different from webpack, which makes some configurations meaningless on rspack, such as memoryCacheUnaffected.
2. rspack cache needs to support more features in the future, such as remote cache.

[tats-u]

https://rspack.dev/config/experiments#migrating-from-webpack-config

You prepared a migration guide that covers all options used by Docusaurus, so it seems to be in acceptable range and we can try it once.

module.exports = {
- cache: {
-   buildDependencies: {
-     config: [__filename, path.join(__dirname, "package.json")],
-     ts: [path.join(__dirname, "tsconfig.json")]
-   }
- },
  experiments: {
    cache: {
      type: "persistent",
+     buildDependencies: [
+       __filename,
+       path.join(__dirname, "package.json"),
+       path.join(__dirname, "tsconfig.json")
+     ]
    },
  },
};

module.exports = {
- cache: {
-   name: `${config.name}-${config.mode}-${otherFlags}`,
-   version: appVersion
- },
  experiments: {
    cache: {
      type: "persistent",
+     version: `${config.name}-${config.mode}-${otherFlags}-${appVersion}`
    },
  },
};

[tats-u]

/assets/images (imported images and images in Markdown) may not be created for builds after persistent cache is enabled and reused.

[tats-u]

Does the persistent cache in Rspack save files generated by loaderContext.emitFile?

[jerrykingxyz]

@tats-u Currently loaderContext.emitFile is not saved to persistent cache, we will support it in the next version. You can use loaderContext.cacheable = false to force rebuilding those module, of course this will cause some performance degradation.

[tats-u]

@jerrykingxyz Thank you for your quick response. That must be why. Does "next version" mean 1.3 as of now?

[jerrykingxyz]

I am actually working on fixing this right now, but it's a bit of work and will be included in version 1.2.3 on February 11th.

[tats-u]

Oh, you are going to release that feature in the next patch version. Much earlier than I thought. I appreciate your continuous quick work.

[tats-u]

Then we have to bump @rspack/core alpha to stable to accept that change in the near future.
Looks like 1.2.2 works well with Docusaurus 3.7.

[tats-u]

Created

• chore(deps): bump @rspack/core in Faster from 1.2.0-alpha.0 to 1.2.2 #10870

[moonwalk47]

the more understanding i get about this the better it becomes
thanks :-)

[tats-u]

1.2.3 has been released.

https://github.com/web-infra-dev/rspack/releases/tag/v1.2.3

This? I think the ball is in our court now.

web-infra-dev/rspack@7f11461#diff-4f349408e97f6b25e924bc599a06661d466fc7913ebed9fb6aa8ed05d7b9eb7aR329-R343

web-infra-dev/rspack@7f11461#diff-fd42ec611cd5acb13724f10d47d47067d31c35ae5a06062825c02fb81a1d8450R244

[slorber]

Yes, I noted to ad a new Docusaurus Faster option once this gets released. Will work on it soon.

[slorber]

I merged the persistent cache for v3.8

However there is a bug in Rspack cache that prevents many of our files to be cached properly (all the code-generated files, notably all those in .docusaurus).

I've documented this with benchmarks in #10931

I hope that they fix it soon so that we can announce better benchmarks when releasing v3.8

[tats-u]

Hooray!

The bug you mention is web-infra-dev/rspack#9413.
I wonder if it will have been fixed.