Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

Fix DELETE /task/(task_id) #42

Closed
dn0 opened this issue Jan 6, 2017 · 2 comments
Closed

Fix DELETE /task/(task_id) #42

dn0 opened this issue Jan 6, 2017 · 2 comments
Assignees
@dn0 @secult
Labels
API bug QUE

Comments

@dn0
Copy link
Member

dn0 commented Jan 6, 2017 •
edited
Loading

The que.utils.delete_task() is broken (celery inspection of a task candidate for deletion). We should fix this and make the DELETE /task/(task_id) available for every PENDING task in the system.

The testing should also include use cases involving task cleanup (revoked tasks).

Target branch: v2.4
@dn0 dn0 added this to the 2.4.0 milestone Jan 6, 2017
@dn0 dn0 assigned dn0 and secult Jan 11, 2017
@secult
Copy link
Contributor

secult commented Jan 11, 2017 •
edited
Loading

Related to queue inspection done by the que.utils.delete_task() function:
In case the task is not in the queue, and the worker is not responding, there are at least two cases when the task is running even though the inspector doesn't know about it:

  1. Few seconds after the rabbitmq server is restarted, the active queue is not returned by the .active() call and therefore the inspector doesn't see any acknowledged running task there.
  2. When the worker's master process is shutting down (SIGTERM) and the task is running on a child process, the master process is detached from the queue already and it does not report about the task anymore. However, after the task is done, the child process reports about it's status correctly.

@dn0 dn0 changed the title Storage for pending tasks Fix DELETE /task/(task_id) Jan 14, 2017
@dn0 dn0 added bug and removed enhancement labels Jan 15, 2017
secult pushed a commit that referenced this issue Jan 24, 2017
Removed illusory safety checks from the task deletion. #42
36c9b39
secult pushed a commit that referenced this issue Jan 24, 2017
Adjusted style #42
5d085bd
@dn0 dn0 modified the milestones: 3.0.0, 2.4.0 Jan 24, 2017
secult added a commit that referenced this issue Jan 24, 2017
@secult
Removed illusory safety checks from the task deletion. #42 (PR#58)
88baf1f
secult pushed a commit that referenced this issue Jan 24, 2017
WIP- Rebuilding delete task to make it safe #42
8ea7115 
Under some conditions the deletion of the task should be safe. The uncertainty is greatly reduced by introducing some (mostly temporary) states in which we know it is unsafe to delete a task. When worker is not responding etc.
Also extracted queue_to_hostname to make it DRY.
@dn0 dn0 removed this from the 2.5.0 milestone Feb 24, 2017
@dn0 dn0 added the QUE label Mar 6, 2017
@dn0
Copy link
Member Author

dn0 commented Apr 2, 2017 •
edited
Loading

Although this work looks nice, I'm closing this issue for now. We can re-open this issue later (probably when upgrading celery).
The reason is that we currently cannot switch to CELERY_ACKS_LATE = True.
Oh, and let's keep the branch.

@dn0 dn0 closed this as completed Apr 2, 2017
# for free to join this conversation on GitHub. Already have an account? # to comment
Assignees

@dn0 dn0

@secult secult

Labels
API bug QUE
Milestone
No milestone
Development

No branches or pull requests
2 participants
@dn0 @secult