Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Empty index_list #2

Open
gold-Soul opened this issue Dec 2, 2024 · 3 comments
Open

Empty index_list #2

gold-Soul opened this issue Dec 2, 2024 · 3 comments

Comments

@gold-Soul
Copy link

I choice the selection scheme “HighestKGradients” and run the code as follows:
net_parser.register_hooks()
target_loss = (criterion(outputs, labels))
target_loss.backward(retain_graph=True)
index_list = net_parser.get_parameters()

But I found that the index_list is none and the function "hook" in function "generated_hook" from the file "HighestKGradients.py" didn't run.

Figure1

Figure2

@hslyu
Copy link
Owner

hslyu commented Dec 2, 2024

I wonder how you configure the parameter ratio and the input tensors.
It returns the empty array if hooks are not registered or the parameter ratio is zero.

Can you test whether the hook functions are called during the feed-forward and back-propagation?
It looks like you have customized the code. I wonder the original code returns the same empty array either.

@gold-Soul
Copy link
Author

-1. I have test it that the register hook function 'hook_fn = self.generate_hook(start_index)' has been run.
-2. The parameter ratio was set as 0.2.
-3. The input tensors include the loss of a subset of a target class and the loss of a data point.
-4. The hook function didn't run during the feed-forward and back-propagation.
-5. The customized code about GIF compution and hook is highly similar to the original code in GIF-main/scripts/table1-parameter_selection_comparison_cifar10_0.ipynb. But I didn't run this ipynb file

@gold-Soul gold-Soul reopened this Dec 2, 2024
@hslyu
Copy link
Owner

hslyu commented Dec 3, 2024

The hook functions are designed as callback functions that collect parameter indices when layerwise inputs/outputs (or gradient inputs/outputs) goes through their own layer.
Thus, if the hook functions are not running during the feed-forward and back-propagation, the get_parameters method should return the empty list.

It is hard to figure out the exact reason as I do not see the result output and your code snippet.

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants