Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Sampling operation prevents gradients from the back-propagation #4

Open
hu-my opened this issue Apr 11, 2023 · 0 comments
Open

Sampling operation prevents gradients from the back-propagation #4

hu-my opened this issue Apr 11, 2023 · 0 comments

Comments

@hu-my
Copy link

hu-my commented Apr 11, 2023

Thanks for your implementation of Slot Attention module. However, I found that the sampling operation (in Line 40 at model.py) prevents gradients from the back-propagation. During training, the gradients of slot_mu and slot_sigma will be zero, which means the two variable will not change. I think the reparameterization trick is needed to make the sampling operation differentiable.

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant