Checksum cache

[rkistner]

Supersedes #12.

Checksum cache

Checksum calculation is currently the operation adding the most load on the storage MongoDB database.

This uses a LRU cache for bucket checksums (bucket, checkpoint). This value never changes per sync rules instance, so we use a cache per sync rules instance.

When a cached checksum is not available for a specific checkpoint, we look for any earlier checkpoints which does have a cached checksum available, and fetch a partial checksum from that point. For large buckets, this should be much faster than computing the entire checksum from scratch.

The cache should help for incremental updates within an active connection, as well as for re-using checksums over multiple connections.

Filtered data requests

When fetching data for a new checkpoint, we now only include buckets for which the checksum changed. Combined with the checksum cache, this should further reduce the load on the database by a small amount.

This has no effect on initial sync requests or the first request for a new connection yet. That is an optimization that could be included later.

Empty buckets

This now also returns zero checksums for empty buckets to the client, instead of omitting the bucket. This does not affect synced data on the client, but helps with diagnosing sync issues.

See powersync-ja/powersync-js#209 for the diagnostics app change to display these buckets.

Performance Impact

This specifically helps for cases where a large set of data is synced to clients (e.g. 100k+ rows), but the incremental updates are small. It reduces both the latency of getting these updates to the client, and the load on the MongoDB storage database.

Test case 1

Test case:

• 200k rows, 1kb each, all in a single global bucket
• 10 concurrent users

Before:

• Each incremental update would do 10x checksum queries, taking 300ms each.
• Around 300ms - 400ms latency for incremental updates.

After:

• When the the clients first connects, only a single checksum query taking 300m is performed.
• No slow queries for incremental updates.
• Incremental updates are "instant", instead of taking 300ms.

This is the "best case scenario" for the cache, since all users share a single large bucket.

Test case 2

Test case: Same as the above, but additionally add 400 small unique buckets per user.

Before:

• Each incremental update would do 10x checksum queries, taking 300ms each.
• Around 400ms latency for incremental updates.

Results (after):

• No slow queries for incremental updates.
• Around 150ms latency for incremental updates.

Test case 3

Test case: Same as the above, but reduce cache size to 3000 (less than the 4000 combined buckets for all connected users).

Before:

• Each incremental update would do 10x checksum queries, taking 300ms each.
• Around 400ms latency for incremental updates.

Results (after):

• Some slow queries are detected on the database for incremental updates, but much less than the "before" results.
• Around 150ms latency for incremental updates, sometimes more.

This is the "worst case scenario" for the cache - many small buckets, unique to each user. The cache does not help much for these buckets, and could evict the global bucket checksum cache in some cases.

We don't expect to hit this much in practice, since the default cache size (100k cached checksums) is enough to handle the limit of 1000 buckets for 100 concurrent users.

[stevensJourney]

LGTM