Skip to content

Commit

Permalink
add jitter to the time when retries are scheduled. this is helpful in…
Browse files Browse the repository at this point in the history
… our particular use case
  • Loading branch information
woodhull committed May 3, 2017
1 parent ea6912b commit 210e629
Showing 1 changed file with 9 additions and 1 deletion.
10 changes: 9 additions & 1 deletion lib/sidekiq-rate-limiter/schedule_in_future_strategy.rb
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,16 @@ def call(work, klass, args, options)
Sidekiq.redis do |conn|
lim = Limit.new(conn, options)
if lim.exceeded?(klass)

# add a random amount of jitter that is proportional to the length of time the retry is in the future.
# this helps us spread out the jobs more evenly, as clumps of jobs on the queue can interfere with normal
# throughput of non-rate limited jobs. This jitter is additive. It's also useful in cases where we would like
# to dump thousands of jobs onto the queue and eventually have them delivered.
retry_in = lim.retry_in?(klass)
retry_in = retry_in + rand(retry_in/5) if retry_in > 10

# Schedule the job to be executed in the future, when we think the rate limit might be back to normal.
Sidekiq::Client.enqueue_to_in(work.queue_name, lim.retry_in?(klass), Object.const_get(klass), *args)
Sidekiq::Client.enqueue_to_in(work.queue_name, retry_in, Object.const_get(klass), *args)
nil
else
lim.add(klass)
Expand Down

0 comments on commit 210e629

Please # to comment.