" KeyError: 'annotations' " on evaluating the model using evaluator.update(predictions)- DETR

[eloisacorti]

Search before asking

• I have searched the Roboflow Notebooks issues and found no similar bug report.

Notebook name

train-huggingface-detr-on-custom-dataset.ipynb

Bug

When training on custom dataset, it seems like I'm not getting any predictions.
During trainning I get loss resultos, so it seems like the trainning is doing fine, but then I can't get inferences.
If I try to run the CocoEvaluator I get this bug:

`KeyError Traceback (most recent call last)
in <cell line: 10>()
21 predictions = {target['image_id'].item(): output for target, output in zip(labels, results)}
22 predictions = prepare_for_coco_detection(predictions)
---> 23 evaluator.update(predictions)
24
25 evaluator.synchronize_between_processes()

3 frames
/usr/local/lib/python3.10/dist-packages/pycocotools/coco.py in getAnnIds(self, imgIds, catIds, areaRng, iscrowd)
144 anns = list(itertools.chain.from_iterable(lists))
145 else:
--> 146 anns = self.dataset['annotations']
147 anns = anns if len(catIds) == 0 else [ann for ann in anns if ann['category_id'] in catIds]
148 anns = anns if len(areaRng) == 0 else [ann for ann in anns if ann['area'] > areaRng[0] and ann['area'] < areaRng[1]]

KeyError: 'annotations'`

Environment

-Google colab

Minimal Reproducible Example

No response

Additional

No response

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

[github-actions]

👋 Hello @eloisacorti, thank you for leaving an issue on Roboflow Notebooks.

🐞 Bug reports

If you are filing a bug report, please be as detailed as possible. This will help us more easily diagnose and resolve the problem you are facing. To learn more about contributing, check out our Contributing Guidelines.

If you require support with custom code that is not part of Roboflow Notebooks, please reach out on the Roboflow Forum or on the GitHub Discussions page associated with this repository.

💬 Get in touch

Do you have more questions about Roboflow that we haven't responded to yet? Feel free to ask them on the Roboflow Discuss forum. Our developer advocates and community team actively respond to questions there.

To ask questions about Notebooks, head over to the GitHub Discussions section of this repository.

[SkalskiP]

Hi @eloisacorti 👋🏻! Does that happen with football-players-detection-3zvbc dataset? Or your custom one?

[eloisacorti]

Hi @SkalskiP It happen with custom dataset.

[SkalskiP]

@eloisacorti, is that dataset coming from Roboflow? If so, could you share a link?

[eloisacorti]

No, I have it my drive.

The directory structure is:

maize10----|__valid #images + .json
|__train #images + .json
|__test #images + .json

https://drive.google.com/drive/folders/1d0LUp_SPpcbPdRVYvO3lHZjTTlJJaPWj?usp=sharing

[SkalskiP]

Could you share you colab where you try and fail to load that dataset?

[eloisacorti]

Yes. I only changed the location of the dataset
https://colab.research.google.com/drive/1qIxmCAueQxkYd2b5Y6DAA-pWWVGuhskF?usp=sharing

[SkalskiP]

Okey. I'll take a look at it as soon as possible. But I'm afraid I won't be able to do it this week.

[eloisacorti]

Thanks!! @SkalskiP

[Heni-Loukil]

@eloisacorti have you find a solution ? I got the same problem, for some datasets it throws the exact error as you encountered but for other datasets it works fine !

[poyingHAHA]

@eloisacorti @eloisacorti
change this line in your colab in the section causing the error:
results = image_processor.post_process_object_detection(outputs, target_sizes=orig_target_sizes)
to:
results = image_processor.post_process_object_detection(outputs, target_sizes=orig_target_sizes, threshold=0)

I think this is because the default threshold is too high that causes the results to be empty in most values.

[SkalskiP]

@poyingHAHA so that happens because during evaluation, there are no detections produced by the model?

[poyingHAHA]

Yes, training step is fine, but the you get nothing during evaluation because of the default threshold and model not training well.

[Kalyugsasur]

how do i ensure that the model is training well? like i am using this notebook as a reference to train for the object detection on soccernet dataset, and i am facing the same issue.

[NielsRogge]

Hi folks,

Thanks for reporting, I was fine-tuning DETR and the new DETA model on a custom dataset and faced the same issue. Glad I found this 😅 will update my demo notebooks.

Regarding making sure the model is training well, fine-tuning is working well for me on the balloon dataset: https://github.com/NielsRogge/Transformers-Tutorials/blob/master/DETA/Fine_tuning_DETA_on_a_custom_dataset_(balloon).ipynb.

Also pushed an update for my DETR notebook.