-
Notifications
You must be signed in to change notification settings - Fork 13.4k
Shrink Span #15594
New issue
Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? # to your account
Comments
Though, I'm not sure if there are few enough ExpnInfos for this to work. Maybe we could have a "phantom span". When the high bit of the base is set, use the rest of it as the index into a table that contains the real span and the expninfo. |
And long term I think we should just have a |
@huonw raises the point of module spans for files over 64K. |
Could extend the idea of phantom span and and use another bit to mean "hugespan", which it then looks up in yet another table. |
But that brings us down from 4G to 1G of max source code size, which isn't very nice. |
Actually I'm not sure that's a problem, given that it can use the rest of the base as an index in the hugespan table. |
I think my original idea is flawed but I believe we can do better in 64 bits than what we do today. |
I wonder if we could do something like struct Span {
start: u32,
is_huge: u1,
len_or_hugelen_index: u7,
expn_info_index: u24
} (stored as i.e. assume that most spans are less than 128 bytes long, and, if it's longer, index into a table using a combination of It might be worth making a compiler with no spans (or making them zerosized), to estimate the absolute largest gain we can get from changing them, since it may turn out that |
That is along the lines of what I was thinking as well. |
@eddyb do you want to do this? |
I wish I had the time to fiddle with it, sorry. |
I'm happy to take a look. |
Ok, I investigated the absolute maximum gain we can get from working on this, by replacing all the spans in the AST with However, I could still compare on the following #![no_std]
#![feature(lang_items, macro_rules)]
// Pull in the system libc library for what crt0.o likely requires
extern crate libc;
pub enum Option<T> { None, Some(T) }
#[inline(never)]
fn f(x: uint) -> uint { x + 1 }
macro_rules! create_exprs {
($e: expr; ) => { ::f(::f($e) + ::f($e)) };
($e: expr; $_x: expr $($xs: expr)*) => {
{
let a = create_exprs!($e; $($xs)*);
let b = create_exprs!($e; $($xs)*);
let c = create_exprs!($e; $($xs)*);
let d = create_exprs!($e; $($xs)*);
let e = create_exprs!($e; $($xs)*);
let f = create_exprs!($e; $($xs)*);
let g = create_exprs!($e; $($xs)*);
let h = create_exprs!($e; $($xs)*);
if a + b == 0 {
c + d
} else if e + f == 0 {
g
} else {
h
}
}
}
}
macro_rules! create {
($i: item; ) => { $i };
($i: item; $_x: expr $($xs: expr)*) => {
mod foo { create!($i; $($xs)*) }
mod bar { create!($i; $($xs)*) }
pub fn baz(x: ::Option<uint>) -> ::Option<uint> {
use None;
let z = match x {
::Some(y) if y > 10 => create_exprs!(y; 2 16 128),
::Some(z) => create_exprs!(z; 2 16 128),
None => create_exprs!(0u; 2 16 128)
};
bar::baz(bar::baz(foo::baz(foo::baz(::Some(z)))))
}
}
}
create!(pub fn baz(x: ::Option<uint>) -> ::Option<uint> {
#![inline(never)]
use None;
match x { ::Some(a) => ::Some(a + 1), None => None }
}; 1 2 4 8 16 32)
// Entry point for this program
#[start]
fn start(_argc: int, _argv: *const *const u8) -> int {
match bar::baz(foo::baz(foo::baz(None))) {
Some(x) => x as int,
None => 0
}
}
// These functions are invoked by the compiler, but not
// for a bare-bones hello world. These are normally
// provided by libstd.
#[lang = "stack_exhausted"] extern fn stack_exhausted() {}
#[lang = "eh_personality"] extern fn eh_personality() {} Anyway, I measured the memory use with @cmr: would it be feasible to run a mem bench of my code on the file above? |
@huonw yes, I can do that later. On Fri, Jul 18, 2014 at 6:48 AM, Huon Wilson notifications@github.com
|
@huonw Can't build the example with stage1 rustc:
|
@cmr hm, it works for me. The bootstrap fails building
|
I've tried to measure span memory consumption by doubling span size instead of setting it to zero (https://github.com/michaelwoerister/rust/tree/span-memory-bench). This way things don't break and one can test against existing codebases. Here are some numbers:
Numbers are max resident set in KB. |
This works out to:
|
Is this still relevant? Is anybody still interested in reducing that 11% of space occupied by spans? Would reducing this number really impact compiletime meaningfully? |
I'll just link this here for documentation: https://internals.rust-lang.org/t/rfc-compiler-refactoring-spans/1357/23 -- One possible strategy for reducing span sizes. |
Compress most of spans to 32 bits As described in https://internals.rust-lang.org/t/rfc-compiler-refactoring-spans/1357/28 Closes #15594 r? @michaelwoerister
Compress most of spans to 32 bits As described in https://internals.rust-lang.org/t/rfc-compiler-refactoring-spans/1357/28 Closes #15594 r? @michaelwoerister
Compress most of spans to 32 bits As described in https://internals.rust-lang.org/t/rfc-compiler-refactoring-spans/1357/28 Closes #15594 r? @michaelwoerister
Compress most of spans to 32 bits As described in https://internals.rust-lang.org/t/rfc-compiler-refactoring-spans/1357/28 Closes #15594 r? @michaelwoerister
…arms, r=Veykril Deunwrap add_missing_match_arms Last subtask of rust-lang#15398
Span is how we know what source a given piece of the AST corresponds to. There are many, many Spans when compiling a crate. It is currently 16 bytes.
Since a span is always contiguous, it can be compressed by storing the base (
u32
) plus a length (u16
). Furthermore,expn_info
is currently occupying a full 8 bytes. It could be replaced with au16
index into someexpn_info
table somewhere.The text was updated successfully, but these errors were encountered: