Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

TypeError: Object of type Element is not JSON serializable - Error when scraping a website #203

Open
vvkishere opened this issue Nov 8, 2024 · 0 comments

Comments

@vvkishere
Copy link

My code running in a windows 10 instance:

from botasaurus.browser import browser, Driver

@browser (reuse_driver=True)
def scrape_data(driver: Driver, link):
driver.get(link)
return driver.get_all_elements_with_exact_text("Equipment Install/Dismantle History")

url = "https://help.sap.com/docs/SAP_S4HANA_ON-PREMISE/ee6ff9b281d8448f96b4fe6c89f2bdc8/a18e976c8a244478a79e3b4177d525dc.html"
output = scrape_data(url)
print(output)

I get a json error:

output = scrape_data(url)
         ^^^^^^^^^^^^^^^^

File "C:\Users\XXX\AppData\Roaming\Python\Python312\site-packages\botasaurus\browser_decorator.py", line 295, in wrapper_browser
write_output(
File "C:\Users\XXX\AppData\Roaming\Python\Python312\site-packages\botasaurus\decorators_common.py", line 202, in write_output
write_json(result, filename, False)
File "C:\Users\XXX\AppData\Roaming\Python\Python312\site-packages\botasaurus\output.py", line 90, in write_json
write_json(data, filename)
File "C:\Users\XXX\AppData\Roaming\Python\Python312\site-packages\botasaurus\utils.py", line 52, in write_json
json.dump(data, fp, indent=indent)
File "C:\Python312\Lib\json_init
.py", line 179, in dump
for chunk in iterable:
^^^^^^^^
File "C:\Python312\Lib\json\encoder.py", line 430, in _iterencode
yield from _iterencode_list(o, _current_indent_level)
File "C:\Python312\Lib\json\encoder.py", line 326, in _iterencode_list
yield from chunks
File "C:\Python312\Lib\json\encoder.py", line 439, in _iterencode
o = _default(o)
^^^^^^^^^^^
File "C:\Python312\Lib\json\encoder.py", line 180, in default
raise TypeError(f'Object of type {o.class.name} '
TypeError: Object of type Element is not JSON serializable

the JSON library is working so I am not sure what the issue is.

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant