Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Fixing errors with _obtain_steps function #27

Open
alexofficial opened this issue Feb 16, 2024 · 1 comment
Open

Fixing errors with _obtain_steps function #27

alexofficial opened this issue Feb 16, 2024 · 1 comment

Comments

@alexofficial
Copy link

Hi,

I tried to collect the offline dataset of an online trained model using the _obtain_steps function. I found an error there, in lines 694-697, where two parenthesis are missing inside the file: scope-rl/scope-rl/dataset/synthetic.py
image

The solution is:

actions = np.zeros((n_trajectories * step_per_trajectory, self.action_dim))
action_probs = np.zeros((n_trajectories * step_per_trajectory,self.action_dim))

Secondly, there is a variable rollout_lengths (line 704), which causes an error in line 723 within the for loop. However, since there are no examples with _obtain_steps, I could not compare this. A quick solution was to simply change it from:
for rollout_step in rollout_lengths[i]:to forrollout_step in range(rollout_lengths[i]):`

Note: I have now successfully trained an offline RL model with the _obtain_steps function. Let me know if I can contribute in any way :)

@aiueola
Copy link
Collaborator

aiueola commented Feb 27, 2024

@alexofficial

Thank you for reaching out with the issues and solutions! The codes (both lines 694-697 and line 723) should definitely be fixed as you mentioned. Could you make a PR with the suggested update? We appreciate your help in improving the software.

alexofficial pushed a commit to alexofficial/scope-rl that referenced this issue Mar 6, 2024
The error occurred when the for loop was used with `rollout_lengths[i]`,
where `rollout_lengths` was created with `self.random_.choice()`,
resulting in a `numpy.int32` object.

The problem has been fixed by iterating directly over the range of `rollout_lengths[i]` instead.

Fixing errors with _obtain_steps function hakuhodo-technologies#27
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants