Skip to content

Commit

Permalink
feat: Add support-cron-scheduler retention docs (#1377)
Browse files Browse the repository at this point in the history
* feat: Add support-cron-scheduler retention docs

refer to edgexfoundry/edgex-go#4912

Signed-off-by: Jack Chen <jack@iotechsys.com>

* fix: Address review comments

Signed-off-by: Jack Chen <jack@iotechsys.com>

---------

Signed-off-by: Jack Chen <jack@iotechsys.com>
  • Loading branch information
jackchenjc authored Oct 14, 2024
1 parent ace9226 commit 3a48157
Show file tree
Hide file tree
Showing 3 changed files with 37 additions and 0 deletions.
7 changes: 7 additions & 0 deletions docs_src/microservices/support/cronScheduler/Configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,13 @@ Below are only the additional settings and sections that are specific to Support

!!! edgey "EdgeX 3.2"
For EdgeX 3.2 the Support Cron Scheduler service only supports `postgres` as persistence layer.
=== "Retention"
|Property|Default Value|Description|
|---|---|---|
|Enabled|true|Enable or disable data retention.|
|Interval|24h|Purging interval defines when the database should be rid of schedule action records above the MaxCap.|
|MaxCap|10000|The maximum capacity defines where the high watermark of schedule action records should be detected for purging the amount of the record to the minimum capacity.|
|MinCap|8000|The minimum capacity defines where the total count of schedule action records should be returned to during purging.|

## V3 Configuration Migration Guide
No configuration updated
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
---
title: Support Cron Scheduler - Data Retention and Persistent Caps
---

# Support Cron Scheduler - Data Retention and Persistent Caps

## Overview

### Support Cron Scheduler service
In use cases, since support-cron-scheduler service persists data in the local database indefinitely, there is a need to persist the latest recent schedule action records (hereafter referred to as records) only and clean up the old ones, as keeping an infinite number of records is considered computationally expensive and can lead to a lack of machine storage. Thus, a retention mechanism is placed on support-cron-scheduler to keep a certain number of records.

Under this mechanism, the maximum records capacity is called <code>MaxCap</code> and the minimum records capacity is called <code>MinCap</code>. Support Cron Scheduler will create an internal schedule according to the <code>Interval</code> configuration to check if the number of records are higher than the <code>MaxCap</code>. When the number of records reach the <code>MaxCap</code>, Support Cron Scheduler will purge the amount of the records to the <code>MinCap</code>.

For example, the <code>MaxCap</code> is set to 10, the <code>MinCap</code> is set to 2 and the <code>Interval</code> is set to 3s. Now, support-cron-scheduler will check how many records are in the local database every 3 seconds. When the number of records reach 10, support-cron-scheduler will check the latest record to find the created timestamp and perform function <code>DeleteScheduleActionRecordByAge</code> to delete records by age.

## Prerequisite Knowledge

- For detailed information on the data retention see [Support Cron Scheduler Configuration](../Configuration.md) and browse to **retention** tab.

## Disable Data Retention
The retention policy is enabled by defaut in support-cron-scheduler, and here is the way to disable data retention mechanism:

- Using environment variables to override the default configuration
```yaml
RETENTION_ENABLED: false
```
For detailed information about environment variables override see [Service Configuration Overrides](../../../configuration/CommonEnvironmentVariables.md#service-configuration-overrides).
2 changes: 2 additions & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,8 @@ nav:
- Configuration: microservices/support/cronScheduler/Configuration.md
- API Reference: microservices/support/cronScheduler/ApiReference.md
- Source Code: microservices/support/cronScheduler/SourceCode.md
- Additional Details:
- Data Retention: microservices/support/cronScheduler/details/DataRetention.md
- microservices/support/eKuiper/Ch-eKuiper.md
- Device:
- Device Service: microservices/device/DeviceService.md
Expand Down

0 comments on commit 3a48157

Please # to comment.