-
Notifications
You must be signed in to change notification settings - Fork 3.6k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[improve][pip] PIP-350: Allow to disable the managedLedgerOffloadDele…
…tionLagInMillis (#22688)
- Loading branch information
Showing
1 changed file
with
36 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,36 @@ | ||
# PIP-350: Allow to disable the managedLedgerOffloadDeletionLagInMillis | ||
|
||
# Background knowledge | ||
|
||
https://pulsar.apache.org/docs/3.2.x/tiered-storage-overview/ | ||
Pulsar provides the ability to offload the data from bookkeeper to the cloud storage with the tiered storage. | ||
Once the data is offloaded to the cloud storage, the data in the bookkeeper can be deleted after a certain period of time. | ||
We use the managedLedgerOffloadDeletionLagInMillis to control the deletion lag time for the offloaded data. | ||
The default value of managedLedgerOffloadDeletionLagInMillis is 4 hours. It means the offloaded data will be deleted after 4 hours by default. | ||
|
||
# Motivation | ||
|
||
In some test scenarios, we want to disable the deletionLag and never delete the data from the bookkeeper. | ||
Then when the tiered storage data is broken, we can still read the data from the bookkeeper. | ||
|
||
# Goals | ||
|
||
## In Scope | ||
|
||
Never deletes the bookkeeper data when the managedLedgerOffloadDeletionLagInMillis is set to -1. | ||
|
||
# Detailed Design | ||
|
||
## Design & Implementation Details | ||
|
||
Only need to check the value of managedLedgerOffloadDeletionLagInMillis in the ManagedLedgerImpl when it is going to delete the bookkeeper data. | ||
https://github.com/apache/pulsar/blob/774a5d42e8342ee50395cf3626b9e7af27da849e/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedLedgerImpl.java#L2579 | ||
|
||
# Backward & Forward Compatibility | ||
|
||
Fully compatible. | ||
|
||
# Links | ||
|
||
* Mailing List discussion thread: https://lists.apache.org/thread/7tlpkcm2933ddg95kgrb42943r4gq3v9 | ||
* Mailing List voting thread: https://lists.apache.org/thread/c3rh530dlwo6nhrdflpw0mjck85hhfbx |