Tracking issue for Fn traits (unboxed_closures & fn_traits feature)

[aturon]

Tracks stabilization for the Fn* traits.

Random bugs:

• FnOnce doesn't seem to support type-based dispatch (as Add, etc. do) #45510 – type-based dispatch not working
• Cannot call implementations of the Fn* traits on references #42736 – foo() sugar doesn't work where you have &Foo: FnOnce

[nagisa]

Inability to delegate calls to other FnOnce implementors like this:

struct A<T>(T);

impl<T, Args> FnOnce<Args> for A<T>
where T: FnOnce<Args> {
    type Output = <T as FnOnce<Args>>::Output;
    fn call_once(self, args: Args) -> Self::Output { FnOnce::call_once(self.0, args) }
}

is the reason I have a safety issue in libloading.

[petrochenkov]

FnOnce::Output is stabilized in #34365

[nrc]

Could someone summarise what is blocking stabilisation here please?

[nagisa]

@nrc uncertainty about Args being the right thing given the possibility of variadic generics coming along around 2030 is the most major reason this is unstable.

[aturon]

@nrc also possibly some questions around the inheritance relationship, see #19032

[brunoczim]

There should be a way of casting fn(A) -> B to Fn(A) -> B

[SimonSapin]

@brunoczim That coercion already happens implicitly, but Fn() as a type is a trait object so it needs to be behind some kind of pointer:

use std::rc::Rc;
fn main() {
    let _: &Fn() = &main;
    let _: Box<Fn()> = Box::new(main);
    let _: Rc<Fn()> = Rc::new(main);
}

[Michael-F-Bryan]

One issue we've been discussing on TheDan64/inkwell#5 is the ability to mark something as unsafe to use. What about having an UnsafeFn marker trait which could be used to tell the compiler a callable is unsafe to call?

For context, inkwell is a wrapper around LLVM and we're trying to figure out how to return a function pointer to a JIT compiled function, when calling the function is fundamentally unsafe for the same reasons FFI code is unsafe to call.

[CodeSandwich]

There is another reason to add UnsafeFn: currently unsafe fns don't implement Fn. There is no way to pass unsafe fn as a function argument, they are second class citizen: example.

UnsafeFn could be implemented for things implementing Fn, so Fns can be used wherever UnsafeFn is required. There probably also should be UnsafeFnMut and UnsafeFnOnce.

[alexreg]

@Michael-F-Bryan @CodeSandwich This sounds like something for which an RFC would really be appreciated. It probably wouldn't be an overly long or intricate one to write, even. I would support it, for sure, and judging by an issue I saw about this not long ago (a long-standing one), others would too.

[CodeSandwich]

@alexreg Ok, I'll prepare it in spare time. Unfortunately I lack knowledge and experience to actually implement it or even fully understand the idea.

[alexreg]

@CodeSandwich Don't worry, so do I! Maybe get yourself on Rust's Discord (#design channel) and we can discuss it with some people who really know the nitty gritty? You can ping me there, same username.

[RalfJung]

With every unsafe function comes a manually written "contract" saying when that function may or may not be called.

With UnsafeFn, who is setting that contract?

[Michael-F-Bryan]

With UnsafeFn, who is setting that contract?

I'd say this is done on a case-by-case basis. It's really hard to specify the various invariants and assumptions you make in unsafe code in something as rigid as a type system, so anything that accepts an UnsafeFn would probably also need to document its assumptions.

Before writing up a RFC I thought I'd make a post on the internal forum. It'd be nice to hear what opinions other people have on this topic and the different solutions they come up with.

[eira-fransham]

I don't know how one would realistically implement this, but probably the ideal semantics is to have any function that takes an unsafe function as an argument to also be unsafe.

[SimonSapin]

A different interface/API likely requires a separate set of traits. Though multiplying the number of traits doesn’t sound great, especially if we later want to also support const fn closures (and unsafe const fn?).

[audreyality]

Is there a way to implement these as wrapper traits? Unsafe<T> where T : FnOnce, Const<T> where T : FnOnce, Async<T> where T : FnOnce and so on?

[parasyte]

They both happen to point to the heap, but the structs themselves are stack-allocated when used in the argument position like that. Or when assigned to a variable with a let binding, for instance.

[Lokathor]

Anyway, the fact that people can already make up their own trait based operator overloading, and even paper over the arity issue with a do_call! macro or whatever, means that there's not much reason to make the Fn traits themselves specifically and magically reject the possibility of overloading. We're just giving people a hard time.

I found tour first reference link as to why not function overloading most interesting because the second reply is from an actual T-lang member that said:

the desire to not have monomorphization-time errors in generics.

and if overloading is happening strictly through the trait system it should end up preventing the post-monomorph errors.

[nyxtom]

Anyway, the fact that people can already make up their own trait based operator overloading, and even paper over the arity issue with a do_call! macro or whatever, means that there's not much reason to make the Fn traits themselves specifically and magically reject the possibility of overloading. We're just giving people a hard time.

I found tour first reference link as to why not function overloading most interesting because the second reply is from an actual T-lang member that said:

the desire to not have monomorphization-time errors in generics.

and if overloading is happening strictly through the trait system it should end up preventing the post-monomorph errors.

This has been my experience (as I've seen implemented in other places). One macro to implement arity to fix the Fn(A), Fn(A, B)..etc and another to be able to call with do_call!(A, B, C). Having the generics on the actual trait here Fn<Args> makes this less difficult to work with and avoids the extra macro expansion. That being said, it's not strictly required to have an impl Fn<Args> for F since you can do this with the trait based approach but the ability to use the extern "rust-call" hack to turn a struct into a function call is a bit different.

[Maix0]

I agree with the fact you shouldn't be able to have multiple Fn* impl on an item.

This is why I wonder why Args are generic and not a associated type

[rdrpenguin04]

Would it be helpful if I made a PR to move from generic to associated, just to see how it would work?

[Maix0]

Would it be helpful if I made a PR to move from generic to associated, just to see how it would work?

I think it would be helpful, but there would need to be some work inside rustc (don't quote me on that) since Fn* traits are really bypassing the whole "how do we represent arguments" by using a custom syntax.

Tho it would allow you to get the argument out of an Fn* trait type (i don't think it is an issue, and it could be gated behind a perma-unstable flag if we really don't want that to happen).

There is more work to be done, but I believe that we should make this change. Currently it is an non-issue since you can't implement these trait, but with unboxed closure it would allow you to have some kind of function overload because you can implement a generic trait multiple times just with different generics.

[jplatte]

I'm pretty sure that the fact it is generic can already be easily be relied upon, and is relied upon in practice. I can try to cook up am example soon.

edit: No capacity for this, sorry.

[moulins]

Would it be helpful if I made a PR to move from generic to associated, just to see how it would work?

I don't think this can work; if Args were an associated type, function HRTBs wouldn't be possible (e.g. for<'a> Fn(&'a [i32]) -> &'a i32).

[dbsxdbsx]

I just come here for I want some code like this:

  let x = CustomizeStruct;
  let y = x();  // direct call on instance

And seems that it has to do with code like this:

#![feature(unboxed_closures)]
#![feature(fn_traits)]

struct CustomizeStruct;

impl Fn<()> for CustomizeStruct {
    extern "rust-call" fn call(&self, _args: ()) {
        println!("call CustomizeStruct");
    }
}

impl FnMut<()> for CustomizeStruct {
    extern "rust-call" fn call_mut(&mut self, _args: ()) {
        println!("call CustomizeStruct");
    }
}

impl FnOnce<()> for CustomizeStruct {
    type Output = ();

    extern "rust-call" fn call_once(self, _args: ()) {
        println!("call CustomizeStruct");
    }
}

But due to the instability of the 2 features, which led to me here, then I have to worke around by using Deref:

use std::ops::Deref;

struct Tensor {
    value: i32,
    name: String,
}

impl Tensor {
    fn new(value: i32, name: &str) -> Self {
        Tensor {
            value,
            name: name.to_string(),
        }
    }
}

struct CustomizeStruct {
    closure: Box<dyn Fn(&Tensor) -> i32>,
}

impl CustomizeStruct {
    fn new() -> Self {
        CustomizeStruct {
            closure: Box::new(|tensor: &Tensor| {
                println!("call CustomizeStruct");
                println!("Tensor name: {}", tensor.name);
                tensor.value * 2
            }),
        }
    }
}

impl Deref for CustomizeStruct {
    type Target = dyn Fn(&Tensor) -> i32;

    fn deref(&self) -> &Self::Target {
        &*self.closure
    }
}

fn main() {
    let x = CustomizeStruct::new();
    let tensor = Tensor::new(21, "example tensor");
    let y: i32 = x(&tensor);
    println!("y = {}", y);
}

The code above is compilable on v1.72.0.

[ijackson]

I have reread this issue and the following issues seem to have been raised as in some sense blocking:

1. Stabilising extern "rust-call".
2. Open questions about the API of Args, generic variadic despatch, etc.
3. Should there be UnsafeFn*? Answer: no. (not a blocker, therefore)
4. impl FnOnce for &T call syntax Cannot call implementations of the Fn* traits on references #42736 apparently due to lack of autoref?
5. Open questions about the relationship of the blanket impls for these Fn traits (currently there aren't any, but they existed at some point AFAICT).

1, 2 and 5 could be dealt with by desugaring along these lines:

/// plan to stabilise this:

impl FnMut(i32) for F {
    fn call_mut(&mut self, x: i32) { dbg!(x); }
}

// desugars to:

impl FnMut<(i32,)> for F {
    extern "rust-call" fn call_mut(&mut self, (x,): (i32,)) -> Self::Output {
        dbg!(x);
    }
}
impl FnOnce<(i32,)> for F {
    type Output = ();
    extern "rust-call" fn call_once(mut self, args: (i32,)) -> Self::Output {
        FnMut::call_mut(&mut self, args)
    }
}

// Correspondingly:
//  impl Fn(..) => impl Fn<>, impl FnMut<>, impl FnOnce<>
//  impl FnOnce(..) => impl FnOnce<> (only)

This has the following properties:

• Preserves opacity, and expansion/change possibilities, for the Fn traits
• Does not expose "rust-call" or the type of Args
• Makes impl Fn* magic - which is OK because these are already magic
• Prevents a manual implementor providing both (say) FnMut and Fn, but I think preserving ability to make this possible via specialisation later
• Unblocks most of the obvious use cases (including wrappers for functions)

Realistically,it seems to me that there is little else that we could want that impl FnMut to mean in the future.

This disposes of all the blockers except 4, #42736, which is a despatch anti-affordance when you impl FnOnce for &T (or similar). I hope #42736 isn't actually a blocker ?

What am I missing?

[cuviper]

Open questions about the relationship of the blanket impls for these Fn traits (currently there aren't any, but they existed at some point AFAICT).

Isn't that referring to the impls on references and Box? (both #[fundamental])

[ijackson]

Isn't that referring to the impls on references and Box? (both #[fundamental])

I was referring to comments like #29625 (comment) (references #19032). That's about the relationship between Fn, FnMut and FnOnce. AFAICT before #23282 there were some blanket impls.

I don't think there is any issue with references or Box, that applies to the Fn* traits, but only when the trait(s) are manually implemented?

[ijackson]

I had a go at implementing this:

/// plan to stabilise this:

impl FnMut(i32) for F {
    fn call_mut(&mut self, x: i32) { dbg!(x); }
}

// desugars to:

impl FnMut<(i32,)> for F {
    extern "rust-call" fn call_mut(&mut self, (x,): (i32,)) -> Self::Output {
        dbg!(x);
    }
}
impl FnOnce<(i32,)> for F {
    type Output = ();
    extern "rust-call" fn call_once(mut self, args: (i32,)) -> Self::Output {
        FnMut::call_mut(&mut self, args)
    }
}

But I encountered a difficulty. (I'm not very familiar with the compiler innards, so possibly I'm just going about it entirely the wrong way.)

I was proposing this as a desugaring, and so I think probably this wants to be done during AST lowering. I think the right place to do this would be in compiler/rustc_ast_lowering/src/item.rs, lower_item_kind. That's the last place where the information needed to do this transformation all exists together.

But lower_item_kind only gets to produce one output hir::ItemKind and my proposal calls for multiple impls. I wasn't sure how (or whether) to try to give lower_item_kind the ability to produce multiple output items.

Turning one item into many raises a question about what ought to be done about attributes applied to the user-supplied impl block. I think they may need to be copied, since pieces of the input end up in more than one of the outputs, and we might want the user's attributes to affect type Output= as well as fn call_*. Maybe this is a reason this shouldn't be done?

I had a go at inventing a helper trait instead: ie, the lowering would implement not the normal FnMut (say) but helper::FnMut and a blanket impl would provide ops::FnMut. But my blanket impls conflicted with other blanket impls for &impl Fn for example.

Maybe someone else can get this to work or give me some pointers.

[lbfalvy]

I skimmed this thread so I'm sorry if I missed something, but why is there a difference between the expression of Output and Args? I get that Args has to be a parameter to encode generics, but functions can have generic parameters that only appear in their return type. In fact, std::sync::mpsc::channel does exactly that:

fn channel<T>() -> (Sender<T>, Receiver<T>);

This type of generic is fairly common; a trick I use to describe infallible results in a way that is compatible with whatever the user wants to do is to template on the unconstrained error:

fn this_never_fails<E>() -> Result<(), E>

Why isn't Output also a generic parameter?

Edit: it somehow elided me that Output was already standardized. I guess these traits are just not equivalent to function definitions then.

[SimonSapin]

@lbfalvy You can think of fn items as syntactic sugar for of a struct and some impls. For example fn example(arg1: u32, arg2: bool) -> String {…} becomes:

struct example;

impl FnOnce<(u32, bool)> for example {
    type Output = String;

    extern "rust-call" fn call_once(self, args: (u32, bool)) -> String {…}
}

Now if the function itself is generic like fn channel<T>() -> (Sender<T>, Receiver<T>) {…}, the equivalent expansion would have that T be a parameter of the struct. Then the impl can reference it without any problem when defining the associated type:

struct channel<T>(PhantomData<T>);

impl<T> FnOnce<()> for channel<T> {
    type Output = (Sender<T>, Receiver<T>);

    extern "rust-call" fn call_once(self, args: ()) -> (Sender<T>, Receiver<T>) {…}
}

[lbfalvy]

@SimonSapin I see, but why can't the same technique be used to make Args an associated type too?

[NobodyXu]

@SimonSapin I see, but why can't the same technique be used to make Args an associated type too?

I suppose because there's no varadic associated type support?

You would have to use tuple and it'd make things complicated.

[bjorn3]

I believe it is required to handle higher-ranked types. Don't recall exactly why though.

[rdrpenguin04]

I believe this was also waiting on variadic generics, which are waiting on the type system overhaul.