orphan cleanup task is not running periodically

[simenon]

Version
"versions": {
"deb": "3.3.1",
"gem": "0.6.1",
"rpm": "3.27.1",
"core": "3.62.0",
"file": "3.62.0",
"maven": "0.8.1",
"ostree": "2.4.3",
"python": "3.12.3",
"ansible": "0.22.1",
"certguard": "3.62.0",
"container": "2.21.0"
},

Describe the bug
In settings.pyt i have set the orphan cleanup time to ORPHAN_PROTECTION_TIME=1440. However i recently noticed that orphans aren't cleaned up.

relevant settings

ORPHAN_PROTECTION_TIME=1440
UPLOAD_PROTECTION_TIME=1440
TMPFILE_PROTECTION_TIME=1440
TASK_PROTECTION_TIME=1440

A call on the cmd-line pulp orphan cleanup --protection-time 2880 which is twice the timer in the settings should provide 0 orphans to cleanup, if there is a periodic task running to clean up.

It seems the orphan task is running based on this output, but it is restricted to uploads and temporary files ?

pulp task list --name-contains orphan
Not all 4271 entries were shown.
[
  {
    "pulp_href": "/pulp/api/v3/tasks/0193870a-0c41-748d-abaf-9ba662609380/",
    "pulp_created": "2024-12-02T11:03:22.946414Z",
    "pulp_last_updated": "2024-12-02T11:03:22.946432Z",
    "state": "completed",
    "name": "pulpcore.app.tasks.orphan.orphan_cleanup",
    "logging_cid": "706c3ef3c5a545248321811acfc6e1df",
    "created_by": "/pulp/api/v3/users/1/",
    "unblocked_at": "2024-12-02T11:03:23.322266Z",
    "started_at": "2024-12-02T11:03:23.374592Z",
    "finished_at": "2024-12-02T11:03:43.170788Z",
    "error": null,
    "worker": "/pulp/api/v3/workers/01926cd3-343e-7cf6-96f2-9720a48eaa7b/",
    "parent_task": null,
    "child_tasks": [],
    "task_group": null,
    "progress_reports": [
      {
        "message": "Clean up orphan Content",
        "code": "clean-up.content",
        "state": "completed",
        "total": 42,
        "done": 42,
        "suffix": null
      },
      {
        "message": "Clean up orphan Artifacts",
        "code": "clean-up.artifacts",
        "state": "completed",
        "total": 727,
        "done": 727,
        "suffix": null
      }
    ],
    "created_resources": [],
    "reserved_resources_record": [
      "pdrn:018fcee4-c5f6-769b-a46f-65417fe08e91:orphans",
      "shared:prn:core.domain:018fcee4-c5f6-769b-a46f-65417fe08e91"
    ]
  },
  {
    "pulp_href": "/pulp/api/v3/tasks/01938300-8a22-72cd-bf16-d22148d97896/",
    "pulp_created": "2024-12-01T16:14:30.946691Z",
    "pulp_last_updated": "2024-12-01T16:14:30.946703Z",
    "state": "completed",
    "name": "pulpcore.app.tasks.orphan.upload_cleanup",
    "logging_cid": "21f4c80800804d04bb3c802f4955081a",
    "created_by": null,
    "unblocked_at": "2024-12-01T16:14:30.964847Z",
    "started_at": "2024-12-01T16:14:31.006366Z",
    "finished_at": "2024-12-01T16:14:31.015431Z",
    "error": null,
    "worker": "/pulp/api/v3/workers/01926cd3-353c-781f-ae13-f748fe5e405e/",
    "parent_task": null,
    "child_tasks": [],
    "task_group": null,
    "progress_reports": [
      {
        "message": "Clean up uploads",
        "code": "clean-up.uploads",
        "state": "completed",
        "total": 0,
        "done": 0,
        "suffix": null
      }
    ],
    "created_resources": [],
    "reserved_resources_record": [
      "shared:prn:core.domain:018fcee4-c5f6-769b-a46f-65417fe08e91"
    ]
  },
  {
    "pulp_href": "/pulp/api/v3/tasks/019382fe-b447-700a-9b3c-145c4c9937c7/",
    "pulp_created": "2024-12-01T16:12:30.663696Z",
    "pulp_last_updated": "2024-12-01T16:12:30.663707Z",
    "state": "completed",
    "name": "pulpcore.app.tasks.orphan.tmpfile_cleanup",
    "logging_cid": "c61ab63a3b9640659a1c580468e335ac",
    "created_by": null,
    "unblocked_at": "2024-12-01T16:12:30.704833Z",
    "started_at": "2024-12-01T16:12:30.745235Z",
    "finished_at": "2024-12-01T16:12:30.754479Z",
    "error": null,
    "worker": "/pulp/api/v3/workers/01926cd3-343e-7cf6-96f2-9720a48eaa7b/",
    "parent_task": null,
    "child_tasks": [],
    "task_group": null,
    "progress_reports": [
      {
        "message": "Clean up shared temporary files",
        "code": "clean-up.tmpfiles",
        "state": "completed",
        "total": 0,
        "done": 0,
        "suffix": null
      }
    ],
    "created_resources": [],
    "reserved_resources_record": [
      "shared:prn:core.domain:018fcee4-c5f6-769b-a46f-65417fe08e91"
    ]
  },

if i limit it to orphan_cleanup i only see tasks that are manually triggerd

pulp task list --name "pulpcore.app.tasks.orphan.orphan_cleanup"
[
  {
    "pulp_href": "/pulp/api/v3/tasks/0193870a-0c41-748d-abaf-9ba662609380/",
    "pulp_created": "2024-12-02T11:03:22.946414Z",
    "pulp_last_updated": "2024-12-02T11:03:22.946432Z",
    "state": "completed",
    "name": "pulpcore.app.tasks.orphan.orphan_cleanup",
    "logging_cid": "706c3ef3c5a545248321811acfc6e1df",
    "created_by": "/pulp/api/v3/users/1/",
    "unblocked_at": "2024-12-02T11:03:23.322266Z",
    "started_at": "2024-12-02T11:03:23.374592Z",
    "finished_at": "2024-12-02T11:03:43.170788Z",
    "error": null,
    "worker": "/pulp/api/v3/workers/01926cd3-343e-7cf6-96f2-9720a48eaa7b/",
    "parent_task": null,
    "child_tasks": [],
    "task_group": null,
    "progress_reports": [
      {
        "message": "Clean up orphan Content",
        "code": "clean-up.content",
        "state": "completed",
        "total": 42,
        "done": 42,
        "suffix": null
      },
      {
        "message": "Clean up orphan Artifacts",
        "code": "clean-up.artifacts",
        "state": "completed",
        "total": 727,
        "done": 727,
        "suffix": null
      }
    ],
    "created_resources": [],
    "reserved_resources_record": [
      "pdrn:018fcee4-c5f6-769b-a46f-65417fe08e91:orphans",
      "shared:prn:core.domain:018fcee4-c5f6-769b-a46f-65417fe08e91"
    ]
  },
  {
    "pulp_href": "/pulp/api/v3/tasks/0193640e-533a-77a7-8d78-703ba20dd0ac/",
    "pulp_created": "2024-11-25T16:01:20.699187Z",
    "pulp_last_updated": "2024-11-25T16:01:20.699207Z",
    "state": "completed",
    "name": "pulpcore.app.tasks.orphan.orphan_cleanup",
    "logging_cid": "b2a0aba6ff714edc889e7f0b1cefd30c",
    "created_by": "/pulp/api/v3/users/1/",
    "unblocked_at": "2024-11-25T16:01:20.718776Z",
    "started_at": "2024-11-25T16:01:20.787101Z",
    "finished_at": "2024-11-25T16:01:21.156416Z",
    "error": null,
    "worker": "/pulp/api/v3/workers/01926cd3-343e-7cf6-96f2-9720a48eaa7b/",
    "parent_task": null,
    "child_tasks": [],
    "task_group": null,
    "progress_reports": [
      {
        "message": "Clean up orphan Content",
        "code": "clean-up.content",
        "state": "completed",
        "total": 4,
        "done": 4,
        "suffix": null
      },
      {
        "message": "Clean up orphan Artifacts",
        "code": "clean-up.artifacts",
        "state": "completed",
        "total": 12,
        "done": 12,
        "suffix": null
      }
    ],
    "created_resources": [],
    "reserved_resources_record": [
      "pdrn:018fcee4-c5f6-769b-a46f-65417fe08e91:orphans",
      "shared:prn:core.domain:018fcee4-c5f6-769b-a46f-65417fe08e91"
    ]
  },

To Reproduce
no idea

Expected behavior
orphan cleanup should run periodically, with regards to the setting ORPHAN_PROTECTION_TIME

Additional context
none

[simenon]

I just re-read the config and maybe i am expecting something which is just not there by design. Given the below docs it implies that it is not when there is a task running to clean up orphans, but what the criteria are to clean up orphans after X time. If so, is it still a possibility to schedule an orphan clean up task, or would that be a feature request ?

ORPHAN_PROTECTION_TIME¶

The time specified in minutes for how long Pulp will hold orphan Content and Artifacts before they become candidates for deletion by an orphan cleanup task. This should ideally be longer than your longest running task otherwise any content created during that task could be cleaned up before the task finishes. Default is 1440 minutes (24 hours).
UPLOAD_PROTECTION_TIME, TMPFILE_PROTECTION_TIME and TASK_PROTECTION_TIME¶

Pulp uses tasks, uploads and pulp temporary files to pass data from the api to worker tasks. These options allow to specify a timeinterval in minutes used for cleaning up stale entries. If set to 0, automatic cleanup is disabled, which is the default.

[mdellweg]

You are reading correctly. The ORPHAN_PROTECTION_TIME behaves different from the others. It is that way, because we weren't confident enough yet to change existing behaviour with respect to orphan cleanup.
So yes, this sounds pretty much like a feature request.

[simenon]

made a feature request instead.