Automated Garbage Collection

[kurtextrem]

Hi, does the data-client implement any sort of automated garbage collection? I've seen that we can call clear (NetworkManager) or cleanup (SubscriptionManager), but I didn't find anything similar to e.g. "after 5min of the data being unused, remove it from cache".
I assume it is thus expected that this is done by userland, not the library, right?

[ntucker]

The quick answer, is yes, nothing is built-in for the store.

There is an action you can send that does GC for user-land implementations (https://github.com/reactive/data-client/blob/master/packages/core/src/state/reducer/createReducer.ts#L27)

Keep in mind the denormalization cache is garbage collected automatically by the javascript engine, as it is 100% WeakMaps.

We're open to design for good generalizable solutions as well as re-prioritization based on need. So far there hasn't been much demand for it, so we've focused on other things.

[ntucker]

0.14.21 The latest version comes with a configurable automatic garbage collection.

  // run GC sweep every 10min
  <DataProvider gcPolicy={new GCPolicy({ intervalMS: 60 * 1000 * 10 })}>
    {children}
  </DataProvider>

const { store, selector, controller } = prepareStore(
  initialState,
  managers,
  Controller,
  otherReducers,
  extraMiddlewares,
  gcPolicy: new GCPolicy({ intervalMS: 60 * 1000 * 10 }),
);

To maintain existing behavior, use ImmortalGCPolicy:

import { ImmortalGCPolicy, DataProvider } from '@data-client/react';

<DataProvider gcPolicy={new ImmortalGCPolicy()}>{children}</DataProvider>;

GCPolicy

This can be configured with constructor options, or custom GCPolicies implemented by extending
or simply building your own. Use ImmortalGCPolicy to never GC (to maintain existing behavior).

constructor

intervalMS = 60 * 1000 * 5

How long between low priority GC sweeps.

Longer values may result in more memory usage, but less performance impact.

expiryMultiplier = 2

Used in the default hasExpired policy.

Represents how many 'stale' lifetimes data should persist before being
garbage collected.

expiresAt

expiresAt({
    fetchedAt,
    expiresAt,
}: {
  expiresAt: number;
  date: number;
  fetchedAt: number;
}): number {
  return (
    Math.max(
      (expiresAt - fetchedAt) * this.options.expiryMultiplier,
      120000,
    ) + fetchedAt
  );
}

Indicates at what timestamp it is acceptable to remove unused data from the store.

Data not currently rendered in any components is considered unused. However, unused
data may be used again in the future (as a cache).

This results in a tradeoff between memory usage and cache hit rate (and thus performance).