Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

recycle/forget offset unused #397

Open
achalddave opened this issue Mar 1, 2017 · 1 comment
Open

recycle/forget offset unused #397

achalddave opened this issue Mar 1, 2017 · 1 comment

Comments

@achalddave
Copy link
Contributor

nn.Recurrent passes an offset variable to :recycle and :forget, which doesn't seem to be used anywhere. Am I missing something, or is the offset parameter unused?

Call to :recycle:

function Recurrent:recycle()

Call to :forget:

function Recurrent:forget()

Defn of :recycle:

function AbstractRecurrent:recycle(offset)

Defn of :forget:

function AbstractRecurrent:forget()

@achalddave
Copy link
Contributor Author

This causes the Recurrent class to store an extra copy of the RecurrentModule, which can be extremely expensive if the module requires significant memory:

require 'rnn'
local _ = require 'moses'

local model = nn.Recurrent(nn.MulConstant(2), nn.MulConstant(3))
print('#sharedClones after init:', _.count(model.sharedClones))

local input = torch.rand(1)
model:forward(input)
model:forward(input)
print('#sharedClones after forward:', _.count(model.sharedClones)) -- 1

model:forget()
print('#sharedClones after forget:', _.count(model.sharedClones)) -- 1
model:forward(input)
model:forward(input)
print('#sharedClones after 2nd forward:', _.count(model.sharedClones)) -- 2

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant