Skip to content

Add flexible transaction matching strategies to electric-db-collection #402

@KyleAMathews

Description

@KyleAMathews

The electric-db-collection currently uses PostgreSQL transaction ID (txid) matching to synchronize client mutations with server responses. This is the ideal approach - it provides instant, precise matching and optimal performance. However, it requires backend APIs to return txids, which isn't always feasible or desirable for existing systems.

We should expand the matching options to support three strategies, allowing developers to choose based on their constraints:

Current State (txid only):

  • Mutation handlers MUST return { txid: number }
  • Provides instant sync resumption when the transaction arrives
  • Requires modifying APIs to expose PostgreSQL transaction IDs

Why This Matters:
TanStack DB blocks incoming sync data to a collection while waiting for backend mutations to complete. This prevents race conditions but means the matching strategy directly impacts user experience:

  • With txid: Sync resumes immediately when the specific transaction arrives (~100ms)
  • Without precise matching: Must wait for timeout (3 seconds of blocked sync)

Proposed Enhancement:
Support three transaction matching strategies:

1. Transaction ID (Current behavior, remains recommended)

  • Returns: { txid: number | number[] }
  • Use when: Your API can return PostgreSQL transaction IDs
  • Performance: Instant sync resumption
  • Status: Already implemented, will remain the recommended approach

2. Custom Match Function (New)

  • Returns: { matchStream: (stream) => Promise<Message> }
  • Use when:
    • Can't get txid from backend
    • Multiple backend transactions involved
    • Need heuristic-based matching
  • Performance: Resumes when match found (with timeout fallback)
  • Example: Match by unique ID, timestamp, or combination of fields

3. Void/Timeout (New)

  • Returns: Nothing/void
  • Use when: Prototyping, testing, or accepting the delay
  • Performance: Always waits 3 seconds
  • Trade-off: Simple but blocks sync unnecessarily

Implementation Example:

// Option 1: Transaction ID (current, optimal)
onInsert: async ({ transaction }) => {
  const result = await api.todos.create(transaction.mutations[0].modified)
  return { txid: result.txid } // Instant sync resumption
}

// Option 2: Custom matching (new)
onInsert: async ({ transaction }) => {
  const item = transaction.mutations[0].modified
  await api.todos.create(item) // API doesn't return txid
  
  return {
    matchStream: (stream) => matchStream(
      stream,
      ['insert'],
      (msg) => msg.value.id === item.id,
      3000 // timeout fallback
    )
  }
}

// Option 3: Void with timeout (new)
onInsert: async ({ transaction }) => {
  await api.todos.create(transaction.mutations[0].modified)
  // No return - waits 3 seconds, good for prototyping
}

Benefits:

  • Backward compatible: Existing txid code continues working optimally
  • Flexible: Choose the right strategy for your architecture
  • Progressive enhancement: Start with void, upgrade to matchStream or txid as needed
  • No forced API changes: Use Electric without modifying backend

Documentation Updates:

  • Explain sync blocking behavior and performance implications
  • Provide migration path: void → matchStream → txid
  • Include performance comparison table
  • Add warnings about timeout impact on UX

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions