Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Retry clear_locks! when ActiveRecord::Deadlocked is occurred #174

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

nishidayuya
Copy link

Sometimes, ActiveRecord::Deadlocked is occurred in Delayed::Backend::ActiveRecord::Job.clear_locks!. And some jobs are still locked.

F, [2019-06-20T11:30:53.459511 #123099] FATAL -- : Mysql2::Error: Deadlock found when trying to get lock; try restarting transaction: UPDATE `delayed_jobs` SET `delayed_jobs`.`locked_by` = NULL, `delayed_jobs`.`locked_at` = NULL WHERE `delayed_jobs`.`locked_by` = 'delayed_job.1 host:XXXXX pid:123099 (ActiveRecord::Deadlocked)

worker_name is generated with PID. So, Delayed::Backend::ActiveRecord::Job.clear_locks! is retryable method. This pull-request makes Delayed::Backend::ActiveRecord::Job.clear_locks! to retry when ActiveRecord::Deadlocked is occurred.

Before pull-request

  • When ActiveRecord::Deadlocked is occurred in Delayed::Backend::ActiveRecord::Job.clear_locks!, some jobs are still locked.

After pull-request

  • When ActiveRecord::Deadlocked is occurred in Delayed::Backend::ActiveRecord::Job.clear_locks!, retry it (max 10-times) and clear locked jobs.

@nishidayuya nishidayuya force-pushed the retryable-clear_locks branch from 28f70dc to 18affa0 Compare August 19, 2019 01:30
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant