Self serve replication SQL API and server side support

[chenselena]

Summary

This PR builds off of @rohitkum2506's PR #185 to add support for the SQL API for self serve replication.

Self serve replication allows users to add their own replication config as part of the table policies to specify a destination cluster and cron schedule for cross cluster table replication. Using the replication config in table properties, we can trigger the replication job, removing the manual process of requiring users to open a ticket to specify a table replication.

The scope of this PR is to add the SQL API and server-side support for adding replication configs to table policies within table properties. SQL API that is supported:

ALTER TABLE db.testTable SET POLICY (REPLICATION=({cluster:'a', schedule:'b'}, {cluster:'aa', schedule:'bb'}))

Future Scope:
Add validations to support replication schedule within a scope and according to rules defined for data freshness and compliance.

Changes

• Client-facing API Changes
• Internal API Changes
• Bug Fixes
• New Features
• Performance Improvements
• Code Style
• Refactoring
• Documentation
• Tests

For all the boxes checked, please include additional details of the changes made in this pull request.

Testing Done

• Manually Tested on local docker setup. Please include commands ran, and their output.
• Added new tests for the changes made.
• Updated existing tests to reflect the changes made.
• No tests added or updated. Please explain why. If unsure, please feel free to ask for help.
• Some other form of testing like staging or soak time in production. Please explain.

For all the boxes checked, include a detailed description of the testing done for the changes made in this pull request.

Added unit tests.

Local docker testing:

ALTER TABLE db.testTable SET POLICY (REPLICATION=({cluster:'a', schedule:'0 0 0 9 0 ?'}))

This will store the replication config as part of table properties:

  "replication": {
    "schedules": [
      {
        "config": {
          "a": "0 0 0 9 0 ?"
        }
      }
    ]
  }

Re-running with:

ALTER TABLE db.testTable SET POLICY (REPLICATION=({cluster:'a', schedule:'0 0 0 9 0 ?'}, {cluster:'b', schedule:'0 0 0 9 0 ?'}))

  "replication": {
    "schedules": [
      {
        "config": {
          "a": "0 0 0 9 0 ?"
        }
      },
      {
        "config": {
          "b": "0 0 0 9 0 ?"
        }
      }
    ]
  }

Additional Information

• Breaking Changes
• Deprecations
• Large PR broken into smaller PRs, and PR plan linked in the description.

For all the boxes checked, include additional details of the changes made in this pull request.

[chenselena]

will need to validate cron schedule based on:

1. N days or less schedule (need to define N based on data)
2. Schedules less than 1 hour will not be allowed

will do in a separate PR

[autumnust]

Can we split API change, SQL extension and server side implementation into different PRs?

Let's get on the same page and identify future steps for the API change first.

[chenselena]

Can we split API change, SQL extension and server side implementation into different PRs?

Let's get on the same page and identify future steps for the API change first.

sure, closing this PR to split up 👍