Fix smoother removing tracking_id from detections

[LinasKo]

Addresses issue #928 matching implementation in ByteTrack, but sidestepping the underlying cause outlined in #943.

Description

Smoother takes as input Detections with a defined tracker_id, but when no detections are left after smoothing, it would return an object where tracker_id is None.

When accessing such detections, for example, using zip(detections.class_id, detections.tracker_id), a NoneType is not iterable error would be raised.

Similar to implementation of ByteTrack.update_with_detections() does it, the tracker_id is added explicitly in cases where it's not present already (in fact - only when no detections are left).

Fixes issue: #928
Related Issue: #943

List any dependencies that are required for this change.

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)

How has this change been tested, please provide a testcase or example of how you tested the change?

Input Video:
https://github.com/roboflow/supervision/assets/6500785/d61b7e9d-2992-46d0-9cd1-6a02388dbe72

Test Code:

Test code

import supervision as sv
from pathlib import Path
from ultralytics import YOLO

VIDEO_FILE = Path('data', 'test-traffic.mp4')
VIDEO_OUT = Path('data', 'test-traffic-annotated.mp4')


# Only a regression test. PIL will never be used to open video files.
trace_annotator = sv.LabelAnnotator()

model_det = YOLO("yolov8n")
video_info = sv.VideoInfo.from_video_path(video_path=str(VIDEO_FILE))
frames_generator = sv.get_video_frames_generator(source_path=str(VIDEO_FILE))
tracker = sv.ByteTrack()
smoother = sv.DetectionsSmoother()

with sv.VideoSink(target_path=str(VIDEO_OUT), video_info=video_info) as sink:
    for frame in frames_generator:
        result = model_det(frame)[0]
        detections = sv.Detections.from_ultralytics(result)
        detections = tracker.update_with_detections(detections)
        smoother.update_with_detections(detections)
        detections = smoother.get_smoothed_detections()

        # print(detections.tracker_id, detections.class_id)
        labels = [
            f"#{tracker_id} {class_id}"
            for tracker_id, class_id
            in zip(detections.tracker_id, detections.class_id)
        ]

        annotated_frame = trace_annotator.annotate(
            scene=frame.copy(),
            detections=detections,
            labels=labels
        )
        sink.write_frame(frame=annotated_frame)

Any specific deployment considerations

None.

Docs

No updates.

[LinasKo]

Ready for review.

[SkalskiP]

Suggested change

	if len(tracked_detections) == 0:
	if len(detections) == 0:

[SkalskiP]

Hi @LinasKo 👋🏻 ! Great find! That's probably the only change I'd make.

[LinasKo]

Done 👍

Also verified the __len__ implementation just in case. Seems to be alright - can't imagine changes to Detections.empty that would break this.

[SkalskiP]

I'm considering modifying the Detections.empty API to include arguments such as with_confidence, with_tracker_id, etc. So that in the future, in a similar situation, we would only need to do Detections.empty(with_tracker_id = True).

[LinasKo]

The problem with that (#943, Option 4) is that intermediary functions would also need with_confidence and with_tracker_id, or you need to forbid them from ever accepting zero-length iterables.

We're actually at the best place to show this.

If nothing is detected, detections = Detections.merge(tracked_detections) takes []. It then tries to call Detections.empty - but what params should it set?..