Coco-wholebody conversion not working

[Gnoot01]

Hi, I'm using the coco-wholebody 133 keypoints images + annotations (in coco format) from here, however it seems using the general_json2yolo.py doesn't leave the 0s (unannotated / undetected points) as it is in the .json, when converted to the .txt?

Eg. using this val_json file, this image, i get the following .txt file only has 137 kp, despite the format requiring to be 5 (class_id + bbox) + (133x3) = 404 kp, and if kp are not visible, they should be 0 0 0

The example is using "image_id" : 785 or 000000000785

json

txt

Please assist, thanks

[pderrenger]

@Gnoot01 thanks for reaching out. Please ensure you're using the latest version of the conversion script. If the issue persists, you might need to modify the script to handle unannotated keypoints correctly. Check if the script accounts for missing keypoints by setting them to 0 0 0 as required. For more details, you can refer to the documentation at https://docs.ultralytics.com/datasets/detect/voc/.

[Gnoot01]

Thanks for your reply. is my dataset considered voc? and also just curious, how does the yolo in the .txt file display key points that are not detected or annotated in the .json file?

[pderrenger]

Your dataset isn't VOC; it's COCO format with additional keypoints. In the YOLO format, undetected keypoints should be represented as 0 0 0 in the .txt file. Make sure your conversion script handles this correctly.

[Gnoot01]

Hi, the conversion script I'm using is the general_json2yolo.py from your JSON2YOLO library, does it mean this script doesn't account for COCO files with more than 17 keypoints yet?

[Gnoot01]

@pderrenger @glenn-jocher may I ask if this general_json2yolo.py from your JSON2YOLO library, is it applicable to pose as well? As the converter code seems to be missing "keypoints" from the json file and uses only "segmentation"

def convert_coco_json(json_dir="../coco/annotations/", use_segments=False, cls91to80=False):
...
# Segments
                if use_segments:
                    if len(ann["segmentation"]) > 1:
                        s = merge_multi_segment(ann["segmentation"])
                        s = (np.concatenate(s, axis=0) / np.array([w, h])).reshape(-1).tolist()
                    else:
                        s = [j for i in ann["segmentation"] for j in i]  # all segments concatenated
                        s = (np.array(s).reshape(-1, 2) / np.array([w, h])).reshape(-1).tolist()
                    s = [cls] + s
                    if s not in segments:
                        segments.append(s)

[pderrenger]

The general_json2yolo.py script currently focuses on segmentation and may not handle keypoints directly. You might need to modify the script to include keypoint conversion for pose data. If you're interested in contributing, feel free to submit a pull request.

[Gnoot01]

Ah I see, that makes sense. Might be good to specify that, unless I did not see it haha. Yup, got it working and I intend to share the code :)

1 last question - does reading from COCO .json file work this way for YoloV11? or is the .txt files the only way? If this works, is this path relative to ./data?

path: ./data # dataset root dir
train: images/train # train images (relative to 'path')
val: images/val # val images (relative to 'path')
test: # test images (optional)

train_ann: /annotations/coco_wholebody_train_v1.0.json
val_ann: /annotations/coco_wholebody_val_v1.0.0.json

Source

[pderrenger]

YOLOv5 primarily uses .txt files for annotations. The COCO .json format isn't directly supported for training. Ensure paths in your dataset configuration are relative to the dataset root directory.

[Gnoot01]

YOLOv5 primarily uses .txt files for annotations. The COCO .json format isn't directly supported for training. Ensure paths in your dataset configuration are relative to the dataset root directory.

YoloV5? what about v11? As well as the source mentions it's applicable for v8?

[pderrenger]

YOLOv8 supports COCO .json format directly, but for YOLOv5, you'll need to use .txt files. Ensure your paths are correctly set relative to the dataset root.

[Gnoot01]

I'm using Yolov11

[pderrenger]

Thanks for reaching out. YOLOv11 isn't officially released by Ultralytics. If you meant YOLOv8, please ensure you're using the latest version and refer to the documentation for guidance.

[Gnoot01]

Thanks, my code is here

[pderrenger]

Thank you for sharing your code. Please ensure you're using the latest version of the package, and let us know if the issue persists.