Skip to content

rustdoc does not expand escape sequences in doc fragments of macros properly #25943

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom
Closed
thelink2012 opened this issue Jun 1, 2015 · 0 comments
Closed

rustdoc does not expand escape sequences in doc fragments of macros properly #25943

thelink2012 opened this issue Jun 1, 2015 · 0 comments
Labels
T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue.

Comments

@thelink2012
Copy link

All of the following I'm going to describe happens only when expanding doc comments in a macro, using meta fragments, in other contexts it works fine, so it indeed is a thing! A sample of doc comments expansion in a macro can be found at #25929 so I'll skip samples and go direct to the point.

If I use a escape sequence of usual strings it seems to expand it right there on rustdoc.

I have found the issue by trying to do something like the following:

 /// `\0`

When I run cargo doc I get this panic:

PS C:\Projects\forks\iup-rust> cargo doc --verbose
       Fresh libc v0.1.6
   Compiling iup v0.0.2 (file:///C:/Projects/forks/iup-rust)
     Running `rustdoc src\lib.rs --crate-name iup -o C:\Projects\forks\iup-rust\target\doc -L dependency=C:\Projects\for
ks\iup-rust\target\debug -L dependency=C:\Projects\forks\iup-rust\target\debug\deps --extern libc=C:\Projects\forks\iup-
rust\target\debug\deps\liblibc-d8d1720964a096ed.rlib --extern libc=C:\Projects\forks\iup-rust\target\debug\deps\liblibc-
d8d1720964a096ed.rlib --extern iup_sys=C:\Projects\forks\iup-rust\target\debug\deps\libiup_sys-16fee22cb642ccd4.rlib --e
xtern iup_sys=C:\Projects\forks\iup-rust\target\debug\deps\libiup_sys-16fee22cb642ccd4.rlib`
       Fresh iup-sys v0.0.3
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: NulError(6, [60, 99, 111, 100, 101, 62, 0,
60, 47, 99, 111, 100, 101, 62])', C:/bot/slave/stable-dist-rustc-win-32/build/src/libcore\result.rs:729
stack backtrace:
   1: 0x63102bdf - rt::unwind::register::ha3686cf8cb60783c3wv
   2: 0x630c56fc - rt::unwind::begin_unwind_inner::h6a42beee3bbc278bduv
   3: 0x630c611f - rt::unwind::begin_unwind_fmt::h2fc1fc2db73eb310Rsv
   4: 0x63102640 - rust_begin_unwind
   5: 0x6311cf28 - panicking::panic_fmt::h52b0d659b0d05042wwy
   6: 0x6c8fcc9f - html::toc::TocBuilder::new::hb7ed7be44c6fc916q6r
   7: 0x6ca514d4 - hoedown_buffer_printf
   8: 0x6ca5185d - hoedown_buffer_printf
   9: 0x76ddb106 - free
thread '<main>' panicked at 'called `Result::unwrap()` on an `Err` value: Any', C:/bot/slave/stable-dist-rustc-win-32/bu
ild/src/libcore\result.rs:729
stack backtrace:
   1: 0x63102bdf - rt::unwind::register::ha3686cf8cb60783c3wv
   2: 0x630c56fc - rt::unwind::begin_unwind_inner::h6a42beee3bbc278bduv
   3: 0x630c611f - rt::unwind::begin_unwind_fmt::h2fc1fc2db73eb310Rsv
   4: 0x63102640 - rust_begin_unwind
   5: 0x6311cf28 - panicking::panic_fmt::h52b0d659b0d05042wwy
   6: 0x6c99b2cc - main::h854d123b237a9daepdu
   7: 0x6313c59c - rust_try
   8:   0x401645 - main
   9:   0x4013de
  10: 0x765b7c04 - BaseThreadInitThunk
  11: 0x76fcad1f - RtlInitializeExceptionChain
Could not document `iup`.

Caused by:
  Process didn't exit successfully: `rustdoc src\lib.rs --crate-name iup -o C:\Projects\forks\iup-rust\target\doc -L dep
endency=C:\Projects\forks\iup-rust\target\debug -L dependency=C:\Projects\forks\iup-rust\target\debug\deps --extern libc
=C:\Projects\forks\iup-rust\target\debug\deps\liblibc-d8d1720964a096ed.rlib --extern libc=C:\Projects\forks\iup-rust\tar
get\debug\deps\liblibc-d8d1720964a096ed.rlib --extern iup_sys=C:\Projects\forks\iup-rust\target\debug\deps\libiup_sys-16
fee22cb642ccd4.rlib --extern iup_sys=C:\Projects\forks\iup-rust\target\debug\deps\libiup_sys-16fee22cb642ccd4.rlib` (exi
t code: 101)

When using the \0 outside a code block it does not happen, but if I try a hexadecimal literal outside the code block instead it still panics.

/// \x0

I get

PS C:\Projects\forks\iup-rust> cargo doc --verbose
   Compiling iup v0.0.2 (file:///C:/Projects/forks/iup-rust)
     Running `rustdoc src\lib.rs --crate-name iup -o C:\Projects\forks\iup-rust\target\doc -L dependency=C:\Projects\for
ks\iup-rust\target\debug -L dependency=C:\Projects\forks\iup-rust\target\debug\deps --extern libc=C:\Projects\forks\iup-
rust\target\debug\deps\liblibc-d8d1720964a096ed.rlib --extern libc=C:\Projects\forks\iup-rust\target\debug\deps\liblibc-
d8d1720964a096ed.rlib --extern iup_sys=C:\Projects\forks\iup-rust\target\debug\deps\libiup_sys-16fee22cb642ccd4.rlib --e
xtern iup_sys=C:\Projects\forks\iup-rust\target\debug\deps\libiup_sys-16fee22cb642ccd4.rlib`
       Fresh libc v0.1.6
       Fresh iup-sys v0.0.3
thread '<unnamed>' panicked at 'lexer should have rejected a bad character escape \x0 Action generated when the text is
edited, but before its value is actually changed.', C:/bot/slave/stable-dist-rustc-win-32/build/src/libcore\option.rs:33
0
stack backtrace:
   1: 0x63102bdf - rt::unwind::register::ha3686cf8cb60783c3wv
   2: 0x630c56fc - rt::unwind::begin_unwind_inner::h6a42beee3bbc278bduv
   3: 0x630c611f - rt::unwind::begin_unwind_fmt::h2fc1fc2db73eb310Rsv
   4: 0x63102640 - rust_begin_unwind
   5: 0x6311cf28 - panicking::panic_fmt::h52b0d659b0d05042wwy
   6: 0x6a1f6a42 - parse::char_lit::h76da3a3875da6b37cXT
   7: 0x6a1f86d2 - parse::str_lit::h85149c8921a41c28h2T
   8: 0x6a1f59b7 - parse::parser::Parser<'a>::lit_from_token::h68fc0ae22c986b30yQG
   9: 0x6a1fa317 - parse::parser::Parser<'a>::parse_lit::h9f70e38391de0fa2yUG
  10: 0x6a2482b4 - parse::attr::Parser<'a>.ParserAttr::parse_meta_item::h1c98783168c92ce14kT
  11: 0x6a423862 - ext::tt::macro_parser::parse_nt::h25dc8b6a526655dfPZf
  12: 0x6a13ad50 - ext::tt::macro_parser::parse::ha0f605ccc4fd9aa7GJf
  13: 0x6a137b8f - ast::TokenTree::parse::h79e7e0468b7d782bdCl
  14: 0x6a42563a - ext::tt::macro_rules::MacroRulesMacroExpander.TTMacroExpander::expand::h96b4da2fbcd98be3Xeg
  15: 0x6a38044b - ext::expand::expand_item_mac::h6351873f0995efe7tib
  16: 0x6a37030d - ext::expand::expand_item::hf68f0012c3b848a1scb
  17: 0x6a36ccd8 - ext::expand::expand_item::hf68f0012c3b848a1scb
  18: 0x6a379e6f - ext::expand::expand_item::hf68f0012c3b848a1scb
  19: 0x6a379ae6 - ext::expand::expand_item::hf68f0012c3b848a1scb
  20: 0x6a374757 - ext::expand::expand_item::hf68f0012c3b848a1scb
  21: 0x6a3cfc65 - ext::expand::PatIdentRenamer<'a>.Folder::fold_mac::h289ea2cd95f4c5b4CMb
  22: 0x6a3cfacf - ext::expand::PatIdentRenamer<'a>.Folder::fold_mac::h289ea2cd95f4c5b4CMb
  23: 0x6a37173b - ext::expand::expand_item::hf68f0012c3b848a1scb
  24: 0x6a36ccd8 - ext::expand::expand_item::hf68f0012c3b848a1scb
  25: 0x6a379e6f - ext::expand::expand_item::hf68f0012c3b848a1scb
  26: 0x6a379ae6 - ext::expand::expand_item::hf68f0012c3b848a1scb
  27: 0x6a374757 - ext::expand::expand_item::hf68f0012c3b848a1scb
  28: 0x6a3cfc65 - ext::expand::PatIdentRenamer<'a>.Folder::fold_mac::h289ea2cd95f4c5b4CMb
  29: 0x6a3cfacf - ext::expand::PatIdentRenamer<'a>.Folder::fold_mac::h289ea2cd95f4c5b4CMb
  30: 0x6a37173b - ext::expand::expand_item::hf68f0012c3b848a1scb
  31: 0x6a36ccd8 - ext::expand::expand_item::hf68f0012c3b848a1scb
  32: 0x6a379e6f - ext::expand::expand_item::hf68f0012c3b848a1scb
  33: 0x6a379ae6 - ext::expand::expand_item::hf68f0012c3b848a1scb
  34: 0x6a374757 - ext::expand::expand_item::hf68f0012c3b848a1scb
  35: 0x6a3cfc65 - ext::expand::PatIdentRenamer<'a>.Folder::fold_mac::h289ea2cd95f4c5b4CMb
  36: 0x6a3cfacf - ext::expand::PatIdentRenamer<'a>.Folder::fold_mac::h289ea2cd95f4c5b4CMb
  37: 0x6a37173b - ext::expand::expand_item::hf68f0012c3b848a1scb
  38: 0x6a36ccd8 - ext::expand::expand_item::hf68f0012c3b848a1scb
  39: 0x6a3d6f01 - ext::expand::expand_crate::ha0bcd83c140e9790Y9b
  40: 0x6cd67820 - driver::collect_crate_metadata::h5feaa081cce22dd0E1a
  41: 0x6cd13930 - driver::phase_2_configure_and_expand::he8982b2f8dc553f7Wsa
  42: 0x6c8cdb2e - core::run_core::h54f67f861d00dcf7wnj
  43: 0x6c9b22c6 - usage::h7829c7327eb005efpgu
  44: 0x6c9b16fb - usage::h7829c7327eb005efpgu
  45: 0x63100e18 - sys::process2::Command::cwd::h292693c1b697cec3Yeu
  46: 0x765b7c04 - BaseThreadInitThunk
  47: 0x76fcad1f - RtlInitializeExceptionChain
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: "rustc failed"', C:/bot/slave/stable-dist-r
ustc-win-32/build/src/libcore\result.rs:729
stack backtrace:
   1: 0x63102bdf - rt::unwind::register::ha3686cf8cb60783c3wv
   2: 0x630c56fc - rt::unwind::begin_unwind_inner::h6a42beee3bbc278bduv
   3: 0x630c611f - rt::unwind::begin_unwind_fmt::h2fc1fc2db73eb310Rsv
   4: 0x63102640 - rust_begin_unwind
   5: 0x6311cf28 - panicking::panic_fmt::h52b0d659b0d05042wwy
   6: 0x6c9a88c8 - usage::h7829c7327eb005efpgu
   7: 0x6c99fef1 - main_args::hacd7685d13170c8aChu
   8: 0x6c99c239 - main::h854d123b237a9daepdu
   9: 0x6c99b6ab - main::h854d123b237a9daepdu
  10: 0x63100e18 - sys::process2::Command::cwd::h292693c1b697cec3Yeu
  11: 0x765b7c04 - BaseThreadInitThunk
  12: 0x76fcad1f - RtlInitializeExceptionChain
thread '<main>' panicked at 'called `Result::unwrap()` on an `Err` value: Any', C:/bot/slave/stable-dist-rustc-win-32/bu
ild/src/libcore\result.rs:729
stack backtrace:
   1: 0x63102bdf - rt::unwind::register::ha3686cf8cb60783c3wv
   2: 0x630c56fc - rt::unwind::begin_unwind_inner::h6a42beee3bbc278bduv
   3: 0x630c611f - rt::unwind::begin_unwind_fmt::h2fc1fc2db73eb310Rsv
   4: 0x63102640 - rust_begin_unwind
   5: 0x6311cf28 - panicking::panic_fmt::h52b0d659b0d05042wwy
   6: 0x6c99b2cc - main::h854d123b237a9daepdu
   7: 0x6313c59c - rust_try
   8:   0x401645 - main
   9:   0x4013de
  10: 0x765b7c04 - BaseThreadInitThunk
  11: 0x76fcad1f - RtlInitializeExceptionChain
Could not document `iup`.

Caused by:
  Process didn't exit successfully: `rustdoc src\lib.rs --crate-name iup -o C:\Projects\forks\iup-rust\target\doc -L dep
endency=C:\Projects\forks\iup-rust\target\debug -L dependency=C:\Projects\forks\iup-rust\target\debug\deps --extern libc
=C:\Projects\forks\iup-rust\target\debug\deps\liblibc-d8d1720964a096ed.rlib --extern libc=C:\Projects\forks\iup-rust\tar
get\debug\deps\liblibc-d8d1720964a096ed.rlib --extern iup_sys=C:\Projects\forks\iup-rust\target\debug\deps\libiup_sys-16
fee22cb642ccd4.rlib --extern iup_sys=C:\Projects\forks\iup-rust\target\debug\deps\libiup_sys-16fee22cb642ccd4.rlib` (exi
t code: 101)

Interesting but let's try something that is not a null character to see what happens:

/// \x61

Ha! I get a a in my html output!

Meta

PS C:\Projects\forks\iup-rust> rustc --version --verbose
rustc 1.0.0 (a59de37e9 2015-05-13) (built 2015-05-14)
binary: rustc
commit-hash: a59de37e99060162a2674e3ff45409ac73595c0e
commit-date: 2015-05-13
build-date: 2015-05-14
host: i686-pc-windows-gnu
release: 1.0.0
PS C:\Projects\forks\iup-rust> cargo --version --verbose
cargo 0.2.0-nightly (efb482d 2015-04-30) (built 2015-04-30)
PS C:\Projects\forks\iup-rust> rustdoc --version --verbose
rustdoc 1.0.0 (a59de37e9 2015-05-13) (built 2015-05-14)
binary: rustdoc
commit-hash: a59de37e99060162a2674e3ff45409ac73595c0e
commit-date: 2015-05-13
build-date: 2015-05-14
host: i686-pc-windows-gnu
release: 1.0.0
@thelink2012 thelink2012 changed the title rustdoc does not expand doc fragments of macros properly rustdoc does not expand escape sequences in doc fragments of macros properly Jun 1, 2015
@steveklabnik steveklabnik added the T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue. label Jun 8, 2015
@barosl barosl mentioned this issue Jul 4, 2015
Merged
barosl added a commit to barosl/rust that referenced this issue Jul 9, 2015
@barosl
Preserve escape sequences in documentation comments on macro expansion
5c60d1d 
Escape sequences in documentation comments must not be parsed as a
normal string when expanding a macro, otherwise some innocent but
invalid-escape-sequence-looking comments will trigger an ICE.

Although this commit replaces normal string literals with raw string
literals in macro expansion, this shouldn't be much a problem
considering documentation comments are converted into attributes before
being passed to a macro anyways.

Fixes rust-lang#25929.
Fixes rust-lang#25943.
Manishearth added a commit to Manishearth/rust that referenced this issue Jul 17, 2015
@Manishearth
Rollup merge of rust-lang#26777 - barosl:macro-doc-escapes, r=pnkfelix
22aa163 
Escape sequences in documentation comments must not be parsed as a normal string when expanding a macro, otherwise some innocent but invalid-escape-sequence-looking comments will trigger an ICE.

Although this commit replaces normal string literals with raw string literals in macro expansion, this shouldn't be much a problem considering documentation comments are converted into attributes before being passed to a macro anyways.

Fixes rust-lang#25929.
Fixes rust-lang#25943.
# for free to join this conversation on GitHub. Already have an account? # to comment
Assignees
No one assigned
Labels
T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@steveklabnik @thelink2012