Skip to content

Derive for items defined inside a function receive incorrect span information #47983

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Closed
sgrif opened this issue Feb 3, 2018 · 2 comments
Closed
Labels
A-attributes Area: Attributes (`#[…]`, `#![…]`) A-decl-macros-1-2 Area: Declarative macros 1.2 A-decl-macros-2-0 Area: Declarative macros 2.0 (#39412) C-bug Category: This is a bug. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Comments

@sgrif
Copy link
Contributor

sgrif commented Feb 3, 2018

Steps to reproduce (rustc 1.25.0-nightly (56733bc9f 2018-02-01))

src/main.rs

#![feature(proc_macro)]

extern crate repro;
use repro::AsChangeset;

fn main() {
    #[derive(AsChangeset)]
    #[table_name = "users"]
    struct UserForm {
        id: i32,
        name: String,
    }
}

src/lib.rs

#![feature(proc_macro)]

extern crate proc_macro;
use proc_macro::TokenStream;

#[proc_macro_derive(AsChangeset, attributes(table_name))]
pub fn derive(input: TokenStream) -> TokenStream {
    for tt in input {
        println!("{:#?}", tt);
    }
    TokenStream::empty()
}

The struct token will have an incorrect span which points at the correct source code, but shows <macro expansion>:1:1 instead of a file/line number when used. Every other token will have a useless span Span { lo: BytePos(0), hi: BytePos(0), ctxt: #0 }

@sgrif sgrif added A-attributes Area: Attributes (`#[…]`, `#![…]`) T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. A-decl-macros-2-0 Area: Declarative macros 2.0 (#39412) C-bug Category: This is a bug. labels Feb 3, 2018
@sgrif
Copy link
Contributor Author

sgrif commented Feb 3, 2018

There also appears to be a bug when one of these dummy spans is received. Span::resolved_at appears to do absolutely nothing.

sgrif added a commit to diesel-rs/diesel that referenced this issue Feb 3, 2018
Since `QueryableByName` is one of the more recently written derives, it
should have been a really straightforward port. Unfortunately, the
tests for this derive hit multiple rustc bugs

- rust-lang/rust#47983
- rust-lang/rust#47311

I love what we were able to do with the error message here. We could
even go so far as to have the `help` lines point at the struct itself
for the `table_name` annotation if we want to.

I also much prefer the workaround for
rust-lang/rust#47311 in this PR to the one I
did in #1529. I'll need to update that PR if this is merged first.
sgrif added a commit to diesel-rs/diesel that referenced this issue Feb 3, 2018
Since `QueryableByName` is one of the more recently written derives, it
should have been a really straightforward port. Unfortunately, the
tests for this derive hit multiple rustc bugs

- rust-lang/rust#47983
- rust-lang/rust#47311

I love what we were able to do with the error message here. We could
even go so far as to have the `help` lines point at the struct itself
for the `table_name` annotation if we want to.

I also much prefer the workaround for
rust-lang/rust#47311 in this PR to the one I
did in #1529. I'll need to update that PR if this is merged first.
sgrif added a commit to diesel-rs/diesel that referenced this issue Feb 3, 2018
Since `QueryableByName` is one of the more recently written derives, it
should have been a really straightforward port. Unfortunately, the
tests for this derive hit multiple rustc bugs

- rust-lang/rust#47983
- rust-lang/rust#47311

I love what we were able to do with the error message here. We could
even go so far as to have the `help` lines point at the struct itself
for the `table_name` annotation if we want to.

I also much prefer the workaround for
rust-lang/rust#47311 in this PR to the one I
did in #1529. I'll need to update that PR if this is merged first.
sgrif added a commit to diesel-rs/diesel that referenced this issue Feb 3, 2018
Since `QueryableByName` is one of the more recently written derives, it
should have been a really straightforward port. Unfortunately, the
tests for this derive hit multiple rustc bugs

- rust-lang/rust#47983
- rust-lang/rust#47311

I love what we were able to do with the error message here. We could
even go so far as to have the `help` lines point at the struct itself
for the `table_name` annotation if we want to.

I also much prefer the workaround for
rust-lang/rust#47311 in this PR to the one I
did in #1529. I'll need to update that PR if this is merged first.
sgrif added a commit to diesel-rs/diesel that referenced this issue Feb 4, 2018
Since `QueryableByName` is one of the more recently written derives, it
should have been a really straightforward port. Unfortunately, the
tests for this derive hit multiple rustc bugs

- rust-lang/rust#47983
- rust-lang/rust#47311

I love what we were able to do with the error message here. We could
even go so far as to have the `help` lines point at the struct itself
for the `table_name` annotation if we want to.

I also much prefer the workaround for
rust-lang/rust#47311 in this PR to the one I
did in #1529. I'll need to update that PR if this is merged first.
@alexcrichton alexcrichton added the A-decl-macros-1-2 Area: Declarative macros 1.2 label May 22, 2018
@alexcrichton
Copy link
Member

This is caused by #43081, but I'm not precisely sure why yet

alexcrichton added a commit to alexcrichton/rust that referenced this issue Jul 22, 2018
Ever plagued by rust-lang#43081 the compiler can return surprising spans in situations
related to procedural macros. This is exhibited by rust-lang#47983 where whenever a
procedural macro is invoked in a nested item context it would fail to have
correct span information.

While rust-lang#43230 provided a "hack" to cache the token stream used for each item in
the compiler it's not a full-blown solution. This commit continues to extend
this "hack" a bit more to work for nested items.

Previously in the parser the `parse_item` method would collect the tokens for an
item into a cache on the item itself. It turned out, however, that nested items
were parsed through the `parse_item_` method, so they didn't receive similar
treatment. To remedy this situation the hook for collecting tokens was moved
into `parse_item_` instead of `parse_item`.

Afterwards the token collection scheme was updated to support nested collection
of tokens. This is implemented by tracking `TokenStream` tokens instead of
`TokenTree` to allow for collecting items into streams at intermediate layers
and having them interleaved in the upper layers.

All in all, this...

Closes rust-lang#47983
bors added a commit that referenced this issue Jul 24, 2018
rustc: Implement tokenization of nested items

Ever plagued by #43081 the compiler can return surprising spans in situations
related to procedural macros. This is exhibited by #47983 where whenever a
procedural macro is invoked in a nested item context it would fail to have
correct span information.

While #43230 provided a "hack" to cache the token stream used for each item in
the compiler it's not a full-blown solution. This commit continues to extend
this "hack" a bit more to work for nested items.

Previously in the parser the `parse_item` method would collect the tokens for an
item into a cache on the item itself. It turned out, however, that nested items
were parsed through the `parse_item_` method, so they didn't receive similar
treatment. To remedy this situation the hook for collecting tokens was moved
into `parse_item_` instead of `parse_item`.

Afterwards the token collection scheme was updated to support nested collection
of tokens. This is implemented by tracking `TokenStream` tokens instead of
`TokenTree` to allow for collecting items into streams at intermediate layers
and having them interleaved in the upper layers.

All in all, this...

Closes #47983
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
A-attributes Area: Attributes (`#[…]`, `#![…]`) A-decl-macros-1-2 Area: Declarative macros 1.2 A-decl-macros-2-0 Area: Declarative macros 2.0 (#39412) C-bug Category: This is a bug. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

No branches or pull requests

2 participants