Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

UnicodeDecodeError when docstrings contain ASCII art #99

Closed
rudolfbyker opened this issue Sep 6, 2022 · 2 comments · Fixed by #100
Closed

UnicodeDecodeError when docstrings contain ASCII art #99

rudolfbyker opened this issue Sep 6, 2022 · 2 comments · Fixed by #100

Comments

@rudolfbyker
Copy link
Contributor

rudolfbyker commented Sep 6, 2022

Describe the bug
When some of my docstrings contain ASCII art, I get a UnicodeDecodeError from griffe when trying to run mkdocs serve or mkdocs build.

To Reproduce
See https://github.com/rudolfbyker/repro-griffe-issue-99

Expected behavior
Docs are generated without errors.

Screenshots

PS C:\aa\griffe-bug-repro\docs\dev> mkdocs build
INFO     -  Cleaning site directory
INFO     -  Building documentation to directory: C:\aa\griffe-bug-repro\docs\dev\dist
ERROR    -  Error reading page 'test.md': 'charmap' codec can't decode byte 0x90 in position 105: character maps to <undefined>
Traceback (most recent call last):
  File "C:\Users\rudol\anaconda3\envs\foo\lib\runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\rudol\anaconda3\envs\foo\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\rudol\anaconda3\envs\foo\Scripts\mkdocs.exe\__main__.py", line 7, in <module>
  File "C:\Users\rudol\anaconda3\envs\foo\lib\site-packages\click\core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "C:\Users\rudol\anaconda3\envs\foo\lib\site-packages\click\core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "C:\Users\rudol\anaconda3\envs\foo\lib\site-packages\click\core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "C:\Users\rudol\anaconda3\envs\foo\lib\site-packages\click\core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Users\rudol\anaconda3\envs\foo\lib\site-packages\click\core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "C:\Users\rudol\anaconda3\envs\foo\lib\site-packages\mkdocs\__main__.py", line 192, in build_command
    build.build(config.load_config(**kwargs), dirty=not clean)
  File "C:\Users\rudol\anaconda3\envs\foo\lib\site-packages\mkdocs\commands\build.py", line 292, in build
    _populate_page(file.page, config, files, dirty)
  File "C:\Users\rudol\anaconda3\envs\foo\lib\site-packages\mkdocs\commands\build.py", line 174, in _populate_page
    page.render(config, files)
  File "C:\Users\rudol\anaconda3\envs\foo\lib\site-packages\mkdocs\structure\pages.py", line 175, in render
    self.content = md.convert(self.markdown)
  File "C:\Users\rudol\anaconda3\envs\foo\lib\site-packages\markdown\core.py", line 264, in convert
    root = self.parser.parseDocument(self.lines).getroot()
  File "C:\Users\rudol\anaconda3\envs\foo\lib\site-packages\markdown\blockparser.py", line 90, in parseDocument
    self.parseChunk(self.root, '\n'.join(lines))
  File "C:\Users\rudol\anaconda3\envs\foo\lib\site-packages\markdown\blockparser.py", line 105, in parseChunk
    self.parseBlocks(parent, text.split('\n\n'))
  File "C:\Users\rudol\anaconda3\envs\foo\lib\site-packages\markdown\blockparser.py", line 123, in parseBlocks
    if processor.run(parent, blocks) is not False:
  File "C:\Users\rudol\anaconda3\envs\foo\lib\site-packages\mkdocstrings\extension.py", line 121, in run
    html, handler, data = self._process_block(identifier, block, heading_level)
  File "C:\Users\rudol\anaconda3\envs\foo\lib\site-packages\mkdocstrings\extension.py", line 195, in _process_block
    data: CollectorItem = handler.collect(identifier, options)
  File "C:\Users\rudol\anaconda3\envs\foo\lib\site-packages\mkdocstrings_handlers\python\handler.py", line 191, in collect
    loader.load_module(module_name)
  File "C:\Users\rudol\anaconda3\envs\foo\lib\site-packages\griffe\loader.py", line 137, in load_module
    module_name, package = self.finder.find_spec(module, try_relative_path)
  File "C:\Users\rudol\anaconda3\envs\foo\lib\site-packages\griffe\finder.py", line 117, in find_spec
    return module_name, self.find_package(top_module_name)
  File "C:\Users\rudol\anaconda3\envs\foo\lib\site-packages\griffe\finder.py", line 149, in find_package
    if init_module.exists() and not _is_pkg_style_namespace(init_module):
  File "C:\Users\rudol\anaconda3\envs\foo\lib\site-packages\griffe\finder.py", line 287, in _is_pkg_style_namespace
    code = init_module.read_text()
  File "C:\Users\rudol\anaconda3\envs\foo\lib\pathlib.py", line 1237, in read_text
    return f.read()
  File "C:\Users\rudol\anaconda3\envs\foo\lib\encodings\cp1252.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 105: character maps to <undefined>

System (please complete the following information):

  • griffe version: 0.22.0
  • Python version: 3.8.12
  • OS: Windows 10
@rudolfbyker
Copy link
Contributor Author

I think specifying encoding='utf-8' wherever read_text() is called will solve the problem. The encoding argument is documented here: https://docs.python.org/3/library/functions.html#open

In text mode, if encoding is not specified the encoding used is platform-dependent: locale.getpreferredencoding(False) is called to get the current locale encoding. (For reading and writing raw bytes use binary mode and leave encoding unspecified.)

@rudolfbyker
Copy link
Contributor Author

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant