Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

feat: add a garbage collector #56

Merged
merged 5 commits into from
May 20, 2024
Merged

feat: add a garbage collector #56

merged 5 commits into from
May 20, 2024

Conversation

ErinvanderVeen
Copy link

@ErinvanderVeen ErinvanderVeen commented May 16, 2024

Description

When a job is created and completed, but the result is not collected, the job would previously remain in the job map. With these jobs not being collected, they would remain in the job map forever. The garbage collector removes them after they have been stale for a while.

Motivation

Prevent a memory blow up when running the api for extended periods of time.

Checklist

Checklist before merging:

  • CHANGELOG.md updated
  • README.md up-to-date

@ErinvanderVeen ErinvanderVeen force-pushed the garbage-collection branch 2 times, most recently from f0d966f to bdd9c56 Compare May 16, 2024 14:04
Erin van der Veen added 2 commits May 17, 2024 11:27
When a job is created and completed, but the result is not collected,
the job would previously remain in the job map. With these jobs not
being collected, they would remain in the job map forever. The garbage
collector removes them after they have been stale for a while.
@ErinvanderVeen ErinvanderVeen force-pushed the garbage-collection branch 3 times, most recently from 291fe31 to 27c142c Compare May 17, 2024 13:57
@ErinvanderVeen ErinvanderVeen marked this pull request as ready for review May 17, 2024 14:01
@ErinvanderVeen ErinvanderVeen requested a review from balsoft May 17, 2024 14:49
Copy link
Collaborator

@balsoft balsoft left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The implementation LGTM; However, (1) I'd increase the default stale time (10 minutes seems short), and (2) perhaps add some sort of persistent job result storage to the roadmap, to enable e.g. sharing the result via a link of some sort.

@ErinvanderVeen
Copy link
Author

(1) Changed to one hour
(2) #57

@ErinvanderVeen ErinvanderVeen requested a review from balsoft May 20, 2024 07:47
@ErinvanderVeen ErinvanderVeen merged commit 3f70c66 into master May 20, 2024
9 checks passed
ErinvanderVeen pushed a commit that referenced this pull request May 27, 2024
feat: add a garbage collector
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants