Grad-CAM for YOLO (version 8)

This repository contains a modified version of the 8th version of You Only Look Once (YOLO), created by Ultralytics, for which the Grad-CAMs can be computed.

This implementation is similar to that of xiaowk5516 whom implemented Grad-CAM for YOLOv5.

This project was done as part of the Norwegian Artificial Intelligence Research Consortium (NORA) 2023 summer school interpretability in Deep Learning.

Installation

To get started, clone this repository. Then, from the root folder load the modified ultralytics package as follows.

sys.path.append('ultralytics')
from ultralytics import YOLO

Make sure that no other ultralytics packages are downloaded in the environment.

If you want to use the pre-trained weights used in the examples in this repository they can be downloaded from Dropbox using this link.

Applications

First load the (pre-trained) YOLOv8 model and an image you want to apply Grad-CAM to, for example:

model = YOLO("yolov8x.pt")
catimg = Image.open("cats.jpg")

Then, you have to tell the model where the label file, here cats.txt, is located. I have not yet implemented a way to pass this information into the model call so for now you have to enter this into the source code of the package. The file you should change is:

"./ultralytics/ultralytics/yolo/engine/predictor.py" At line 382, where I have left a comment, you should place the path to the label file:

labpath = "./cats.txt"

Then, run the prediction with visualize = True:

results = model.predict(source=catimg, save=False, visualize = True, imgsz=1280)

The resulting heat maps are stored in runs/detect/ as .npy files.

A demonstration is available in yolo8-gradcam.ipynb.