Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

task 615-619 labels have "locator" in format that raises error #53

Closed
nicholaschenai opened this issue Oct 20, 2023 · 1 comment
Closed

Comments

@nicholaschenai
Copy link
Contributor

task 615-619 labels: all of them have

"locator": "[...document.querySelector('.submission__inner').querySelectorAll('[href],[src]')].map(elem => elem.getAttribute('href') || elem.getAttribute('src')).join(' ')",

however, in webarena/evaluation_harness/evaluators.py class HTMLContentExactEvaluator(Evaluator) line 236 on, locators are expected to be empty or startswith "document." or "func.". Hence, ValueError is raised for 615-619

@shuyanzhou
Copy link
Contributor

Good catch! #54 has fixed it!

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants