job undefined in failed event after being stalled

[simonwindt]

When a job is stalled more then the allowable limit (maxStalledCount), it is failed and the failed event is emitted. The problem is that the job parameter in this failed event is always undefined. We can therefore not properly do cleanup.

notifyFailedJobs(failedJobs) { failedJobs.forEach((job) => this.emit('failed', job, new Error('job stalled more than allowable limit'), 'active')); }
here the failedJobs is an array of undefined.

[manast]

Thanks for reporting this issue, I will check why this is the case.

[manast]

I am wondering, which version are you using? and could you possible create a reproducible test case? I just checked and we have tests that covers this particular scenario:

  it('fail stalled jobs that stall more than allowable stalled limit', async function () {
    this.timeout(6000);

    const queueEvents = new QueueEvents(queueName, { connection, prefix });
    await queueEvents.waitUntilReady();

    const concurrency = 4;

    const worker = new Worker(
      queueName,
      async () => {
        return delay(10000);
      },
      {
        connection,
        prefix,
        lockDuration: 1000,
        stalledInterval: 100,
        maxStalledCount: 0,
        concurrency,
      },
    );

    const allActive = new Promise(resolve => {
      worker.on('active', after(concurrency, resolve));
    });

    await worker.waitUntilReady();

    await Promise.all([
      queue.add('test', { bar: 'baz' }),
      queue.add('test', { bar1: 'baz1' }),
      queue.add('test', { bar2: 'baz2' }),
      queue.add('test', { bar3: 'baz3' }),
    ]);

    await allActive;

    await worker.close(true);

    const worker2 = new Worker(queueName, async job => {}, {
      connection,
      prefix,
      stalledInterval: 100,
      maxStalledCount: 0,
      concurrency,
    });

    const errorMessage = 'job stalled more than allowable limit';
    const allFailed = new Promise<void>(resolve => {
      worker2.on(
        'failed',
        after(concurrency, async (job, failedReason, prev) => {
          expect(job.finishedOn).to.be.an('number');
          expect(prev).to.be.equal('active');
          expect(failedReason.message).to.be.equal(errorMessage);
          resolve();
        }),
      );
    });

    const globalAllFailed = new Promise<void>(resolve => {
      queueEvents.on('failed', ({ failedReason }) => {
        expect(failedReason).to.be.equal(errorMessage);
        resolve();
      });
    });

    await allFailed;
    await globalAllFailed;

    await queueEvents.close();
    await worker2.close();
  });

[simonwindt]

I tested with the latest version. I will create a test case to see if i can reproduce it.

[simonwindt]

I did some testing and the following options will cause job to be undefined. This is only on stalled jobs, jobs that throw an error are not undefined in the failed event handler.

{
  removeOnComplete: true,
  removeOnFail: true
}