Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Fully convolutional version of all vgg models via constructor argument #184

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

warmspringwinds
Copy link

Added a feature to use any existing vgg model in a fully convolutional manner.
Backwards compatible, works as an argument fully_conv to any vgg model creator function.
Also the weights are loaded correctly (conversion between Linear to Conv2d. which can be seen in the example images below).

fcn = vgg16(pretrained=True, fully_conv=True)
fcn = fcn.eval()
from torch.autograd import Variable
import torch.nn as nn

import skimage.io as io
import skimage.transform
import numpy as np
import torch

img = skimage.img_as_float(io.imread('bus.jpg'))
img = img - [0.485, 0.456, 0.406]
img = img / [0.229, 0.224, 0.225]
img = img.transpose((2, 0, 1))
img = np.expand_dims(img, 0)
img = img.astype(np.float32)
final_img = torch.from_numpy(img)

fcn.cuda()
final_img = Variable(final_img.cuda())
res = fcn(final_img)

boo = res.cpu().data.numpy()
final = boo.squeeze().argmax(axis=0)
# 779 -- is a class for a bus from imagenet
io.imshow(final == 779)

The following image was evaluated and respective prediction for a bus region was acquired,
which proves that weights from imagenet vgg model were correctly loaded after converting Linear layers to Conv2d.

pytorch_bus
pytorch_bus_pred

Feel free to leave feedback. I can wrap it up with tests if you like it.

@christian-rauch
Copy link

christian-rauch commented Apr 14, 2020

Is there still interest in cleaning the commits and getting this merged? Or is this partially superseded by #643 or other recent changes (like https://pytorch.org/hub/pytorch_vision_fcn_resnet101/)?

@lijiaqi
Copy link

lijiaqi commented May 12, 2020

@christian-rauch Hi. I am also considering adding more flexible ways to instantiate a model. See #2200.

For example, support backbone mode (remove all last fc layers, make pooling layer after last conv layer as optional, etc).

I think keras style is a good implementation. Any thoughts?

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants