rustdoc: Markdown headers don't get recognized in certain cases

[panicbit]

#Headers don't seem to get recognized when the header is preceded by some text and a following code block contains hidden lines (///# ...).

Here's a minimal example:

/// Removing this very line makes it render correctly.
///
/// # I will not be rendered correctly
///
/// ```
///# fn foo() {} // A hidden line like this needs to exist
/// ```
pub fn some_item() {}

And this is what it looks like:

I've set up a repository with this example to play around with here: https://github.com/panicbit/rustdoc_bug_33771

cc @Manishearth

[GuillaumeGomez]

There are other similar issues reported about the markdown parser (can't find them however...). Thanks for reporting one more. 😃

[alilleybrinker]

@GuillaumeGomez, it looks like the current Rustdoc Markdown parser is using Hoedown, which hasn't been updated in a while. Is there any interest in switching to a different Markdown parser? It seems to me that writing a good pure-Rust markdown parser that could be used in Rustdoc would also be more broadly useful.

[hanna-kruppe]

Well there's pulldown-cmark, used by mdBook.

[GuillaumeGomez]

@AndrewBrinker: If you find one or write one, then sure.

[steveklabnik]

I would be very interested in moving to pull down-cmark or something else CommonMark compliant.

On Jun 10, 2016, 11:58 +0300, Guillaume Gomeznotifications@github.com, wrote:

@AndrewBrinker(https://github.com/AndrewBrinker): If you find one or write one, then sure.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly,view it on GitHub(#33771 (comment)), ormute the thread(https://github.com/notifications/unsubscribe/AABsikYSELwsHZaAr6ILqNXAceL49MI8ks5qKScegaJpZM4IjwxD).

[Luthaf]

I would be very interested in moving to pull down-cmark or something else CommonMark compliant.

What would be the plan here? I think making a full switch on the first time would not be a good idea, so maybe two backends (hoedown and pulldown-cmark) should work together for a few time. What would be the better way to have both exposed? A rustdoc flag?

[steveklabnik]

I am not 100% sure, but I'm also not 100% sure that it's that big a deal; it's only in edge cases that there are differences. We'd have to experiment and make sure.

[frewsxcv]

Opened an issue for moving to pulldown-cmark: #38400

[BurntSushi]

Within the last couple of weeks, I've seen this bug appear quite frequently. (I have never seen it before.) As in, a significant number of headers in my docs aren't rendering properly as described in the OP.

[GuillaumeGomez]

I'd prefer to wait for the switch to pulldown to be officially complete to check if we still have this issue because it seems to be very specific to hoedown.

[panicbit]

As already suspected, this seems to be fixed with the pulldown parser.

Rustdoc also correctly warns about this change:

WARNING: rendering difference in `Removing this very line makes it render correctly.`
   --> src/lib.rs:8:0
    /html[0]/body[1] Tags differ: expected: `h1`, found: `p`

[GuillaumeGomez]

Then let's close this issue once the switch to pulldown is complete.

[kzys]

#47398 and #48274 have been merged. We probably can close this issue 👍