From 3a48157fe3d8a6213bb20f278b776ce29b822d42 Mon Sep 17 00:00:00 2001 From: Jack Chen Date: Mon, 14 Oct 2024 10:30:21 +0800 Subject: [PATCH] feat: Add support-cron-scheduler retention docs (#1377) * feat: Add support-cron-scheduler retention docs refer to https://github.com/edgexfoundry/edgex-go/issues/4912 Signed-off-by: Jack Chen * fix: Address review comments Signed-off-by: Jack Chen --------- Signed-off-by: Jack Chen --- .../support/cronScheduler/Configuration.md | 7 +++++ .../cronScheduler/details/DataRetention.md | 28 +++++++++++++++++++ mkdocs.yml | 2 ++ 3 files changed, 37 insertions(+) create mode 100644 docs_src/microservices/support/cronScheduler/details/DataRetention.md diff --git a/docs_src/microservices/support/cronScheduler/Configuration.md b/docs_src/microservices/support/cronScheduler/Configuration.md index 256c59e1c..cc8c2fa09 100644 --- a/docs_src/microservices/support/cronScheduler/Configuration.md +++ b/docs_src/microservices/support/cronScheduler/Configuration.md @@ -53,6 +53,13 @@ Below are only the additional settings and sections that are specific to Support !!! edgey "EdgeX 3.2" For EdgeX 3.2 the Support Cron Scheduler service only supports `postgres` as persistence layer. +=== "Retention" + |Property|Default Value|Description| + |---|---|---| + |Enabled|true|Enable or disable data retention.| + |Interval|24h|Purging interval defines when the database should be rid of schedule action records above the MaxCap.| + |MaxCap|10000|The maximum capacity defines where the high watermark of schedule action records should be detected for purging the amount of the record to the minimum capacity.| + |MinCap|8000|The minimum capacity defines where the total count of schedule action records should be returned to during purging.| ## V3 Configuration Migration Guide No configuration updated diff --git a/docs_src/microservices/support/cronScheduler/details/DataRetention.md b/docs_src/microservices/support/cronScheduler/details/DataRetention.md new file mode 100644 index 000000000..b1b10369d --- /dev/null +++ b/docs_src/microservices/support/cronScheduler/details/DataRetention.md @@ -0,0 +1,28 @@ +--- +title: Support Cron Scheduler - Data Retention and Persistent Caps +--- + +# Support Cron Scheduler - Data Retention and Persistent Caps + +## Overview + +### Support Cron Scheduler service +In use cases, since support-cron-scheduler service persists data in the local database indefinitely, there is a need to persist the latest recent schedule action records (hereafter referred to as records) only and clean up the old ones, as keeping an infinite number of records is considered computationally expensive and can lead to a lack of machine storage. Thus, a retention mechanism is placed on support-cron-scheduler to keep a certain number of records. + +Under this mechanism, the maximum records capacity is called MaxCap and the minimum records capacity is called MinCap. Support Cron Scheduler will create an internal schedule according to the Interval configuration to check if the number of records are higher than the MaxCap. When the number of records reach the MaxCap, Support Cron Scheduler will purge the amount of the records to the MinCap. + +For example, the MaxCap is set to 10, the MinCap is set to 2 and the Interval is set to 3s. Now, support-cron-scheduler will check how many records are in the local database every 3 seconds. When the number of records reach 10, support-cron-scheduler will check the latest record to find the created timestamp and perform function DeleteScheduleActionRecordByAge to delete records by age. + +## Prerequisite Knowledge + +- For detailed information on the data retention see [Support Cron Scheduler Configuration](../Configuration.md) and browse to **retention** tab. + +## Disable Data Retention +The retention policy is enabled by defaut in support-cron-scheduler, and here is the way to disable data retention mechanism: + +- Using environment variables to override the default configuration +```yaml +RETENTION_ENABLED: false +``` + +For detailed information about environment variables override see [Service Configuration Overrides](../../../configuration/CommonEnvironmentVariables.md#service-configuration-overrides). diff --git a/mkdocs.yml b/mkdocs.yml index e15527223..94ed50cec 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -129,6 +129,8 @@ nav: - Configuration: microservices/support/cronScheduler/Configuration.md - API Reference: microservices/support/cronScheduler/ApiReference.md - Source Code: microservices/support/cronScheduler/SourceCode.md + - Additional Details: + - Data Retention: microservices/support/cronScheduler/details/DataRetention.md - microservices/support/eKuiper/Ch-eKuiper.md - Device: - Device Service: microservices/device/DeviceService.md