Block syntax for let-binding

[yannham]

Is your feature request related to a problem? Please describe.
Block let-binding are nice to express a series of order-independent bindings. See #218 for more context.

Describe the solution you'd like
As decided during the standardization meeting, have let-binding blocks as in Nix, just replacing semicolon ; by , for consistency with the rest of the Nickel syntax.

Example:

let foo = ...,
    bar = ...,
    blah = ...,
in
...

To be consistent, the inner syntax should be exactly the same as for records, just without the braces. Those let-binding would also be mutually recursive, as records.

Describe alternatives you've considered
See #218 for discussion and alternatives.

[yannham]

As noticed by @litchipi, it may be a bit strange that single let-binding aren't recursive by default, and block let-binding are. That is, turning

let x = x + 1 in
let y = y + 1 in
...

into

let x = x + 1,
    y = y + 1,
in

changes a benign shadowing to infinite recursion. I see three possibilities:

1. Make single let-binding recursive by default.
2. Make block let-binding non recursive by default. Then we could have a rec keyword that would work for both let forms, and turn them into recursive bindings.
3. Do nothing, and have this discrepancy between single lets and let block.

I'm in favor of 2. While I understand that one may want recursive records by default, 1. sounds extreme semantically and totally disallows shadowing altogether. I've never seen a language with such a feature. 3 makes things unexpected, as shown by the example. 2 is consistent, makes recursion explicit for let-binding (which is IMHO sensible, as most of the time you don't write recursive let-binding) while keeping it implicit for records, as we decided to do in #216.

Ping @edolstra @aspiwack

[aspiwack]

I'm partial to option (2) as well.

[mboes]

I'm not a fan of the proposed syntax. The parallel with records is non-obvious. It would be if the braces were there, though they are of course redundant. In records, I can have (for good reason) trailing commas. Here is would look like this:

let x = 2, in x

We could add the braces back, so e.g.

let { x = 1 + y, y = 1 + x, } in x + y

This would have the added benefit of making #494 (comment) a non-issue. But now people will wonder why they can't write

let my_rec = { x = 1 + y, y = 1 + x, } in
let my_rec in x + y

(which incidentally would effectively be the with my_rec; ... of Nix.)

So to rewind, what problem are we trying to solve? The linked standardization meeting notes contains nothing in the way of motivation, and neither does #218, which at this point could probably be closed in favour of this issue. Writing a sequence of let ... in today for multiple bindings works fine, as it has done for OCaml for a very long time. Is mutually recursive bindings the only motivation, or are we looking to kill another bird with the same stone?

[yannham]

I'm not a fan of the proposed syntax. The parallel with records is non-obvious. It would be if the braces were there, though they are of course redundant.

The rationale isn't to be exactly the same as records. We chose to use the Nix syntax, which was otherwise reasonable and in par with other similar features in other languages, but it wouldn't make sense to keep the ; as we don't use it for records or as a separator anywhere in Nickel, as opposed to Nix. The decision is maybe better rephrased as: let's do as in Nix, but use the , as a separator, because we use , as a separator elsewhere (list elements, record fields, switch cases).

In records, I can have (for good reason) trailing commas. Here is would look like this:

let x = 2, in x

For what it worth, I don't find it any worse or uglier than the possibility of writing let r = {foo = 2,} in .... Trailing comma would be optional, and writing let x = 2, in x is slightly bad taste, but it's hard to be bad-taste proof. Concerning the usage of trailing commas, I imagine it would be the same as with records: extensibility by just pasting/deleting lines, without having to care about removing or adding commas.

So to rewind, what problem are we trying to solve? The linked standardization meeting notes contains nothing in the way of motivation, and neither does #218, which at this point could probably be closed in favour of this issue. Writing a sequence of let ... in today for multiple bindings works fine, as it has done for OCaml for a very long time. Is mutually recursive bindings the only motivation, or are we looking to kill another bird with the same stone?

I would say that the main motivation is to write mutually recursive definitions indeed. And it doesn't seem to hurt anyway. It gives aesthetics as a bonus for blocks of definitions, and more generally to convey explicitly that the bindings of the block are independent. As opposed to a sequence of let .. in, a non-recursive let-block is guaranteed to support re-ordering, deletion and insertion of bindings without changing the semantics of the other bindings. My point is, the semantics of a block and a sequence is different (totally irrelevant right now to Nickel but that would also mean that you can evaluate bindings in parallel, I think but I'm not sure that re-ordering changes the semantic hash in Dhall for sequence of let but not for let-blocks, etc.).

A lot of functional languages have it in some form (Nix, Dhall, OCaml, Haskell, Elm, Clojure, and son on). Although for some reason it's not so used in OCaml indeed (maybe because and is one more character than in? But I've always wondered why, as I've often personally found it hard sometimes to extract structure from a sequence of a dozen heading let-in in OCaml functions, mixing independent and dependent ones).

[mboes]

So in short, I think it's premature to be adding sugar like this to the language. I move to close this issue and revisit once the language has stabilized and we have more users and use cases. What do you think?

I call it sugar because we already have recursive let-bindings and the let-bindings are destructuring. Together those features are enough to deal with mutual recursion:

let { x, y } = { x = <expr>, y = <expr> } in <expr>

It's not the most ergonomic thing, but mutually recursive bindings are not that common, and we get to save a little bit of language syntax estate.

Nickel is a configuration language. We don't need perfect ergonomics. We do need a small language that is simple and quick to pick up. Haskell is an example where lots of let-binding styles have proliferated. See #218 for an instance of that.

Regarding Nix, TBH the let is one of my least favourite things about Nix syntax. I always forget to add the ; just before the in, which moreover is redundant with it. We can do better here, but the point still stands that we are adding a number of new ways to write the same thing (even more so if we add rec too), which is as much overhead for new users and decision fatigue for old users.

convey explicitly that the bindings of the block are independent. As opposed to a sequence of let .. in, a non-recursive let-block is guaranteed to support re-ordering, deletion and insertion of bindings without changing the semantics of the other bindings. My point is, the semantics of a block and a sequence is different (totally irrelevant right now to Nickel but that would also mean that you can evaluate bindings in parallel, I think but I'm not sure that re-ordering changes the semantic hash in Dhall for sequence of let but not for let-blocks, etc.).

There is a philosophical fork in the road that I don't think we need to resolve now, but I'm pointing out for the future. Making recursion or lack thereof explicit, and independence between bindings explicit, gives more control to the user but at the cost of a bigger language. We have a choice between, allowing the user to state their intent very precisely, or keeping things simple for the user and trying to recover the same metadata through simple pre-evaluation analyses if/when needed.

[mboes]

It turns out let-bindings are not recursive currently, see #525. This is a change relative to the very early prototypes for Nickel. Nevertheless, even if let-bindings were to remain non-recursive, if we did had let rec, we'd still be able to implement mutual recursion as above.

[yannham]

It turns out let-bindings are not recursive currently, see #525. This is a change relative to the very early prototypes for Nickel.

I wasn't aware of that. When I started, let-binding weren't recursive. Recursive records weren't implemented. When we added them, @eelco made a case for record being recursive by default in in #83 and #216, and so we did, but there was nothing about let-binding.

Back to the original issue, I agree that mutual recursive values are rare enough that it is acceptable to use records + destructuring as long as we have a rec keyword or recursion by default for single lets. It's fine to revisit let-block later I guess. It just comes with the tiny annoyance that if we do end up to add such let-blocks, we'll have a stronger split in code beween before and after that if we had it from the beginning.

[yannham]

Should we apply the same thinking to the addition for OCaml-like function syntax let f x y z = ... ~ let f = fun x y z => ... as per the standardization meeting's decision?
If we do cancel those two, it may make piecewise block signatures also somehow less useful.

[mboes]

@yannham you led me to the conclusion in #525 that actually, recursion is very much an escape hatch (like you said in #83 already). But then mutual recursion should be even rarer. So going back to my earlier question,

Is mutually recursive bindings the only motivation, or are we looking to kill another bird with the same stone?

I would say that the main motivation is to write mutually recursive definitions indeed.

... then this makes the case for let-blocks all the weaker. I'm becoming increasingly convinced that we should punt.

Should we apply the same thinking to the addition for OCaml-like function syntax let f x y z = ... ~ let f = fun x y z => ... as per the standardization meeting's decision?

I think so, yes. I don't think that syntactic sugar improving anyone's QOL in any significant way.

[mboes]

I don't think that syntactic sugar improving anyone's QOL in any significant way.

I take that back. Function definition is so pervasive that it ought to be really short. However, we could achieve the same differently: by removing the fun keyword (mind you, not sure whether that's a good idea).

[yannham]

We discussed that option in the meeting, but I recall attendees didn't like it for a bunch of reasons. IIRC the general feeling was that it's just three characters that make it explicit from the beginning that we are parsing a function definition and not something else, both for the user and the parser. Otherwise, in exp1 ... expn => body, you need to go all the way to the end to decide if what you parsed before is actually valid (until expn it can be both an application and a function definition, but the two allow different things to be there). Related discussion on Ocaml forum.

Nix destructuring looks like it faces the same problem (when you parse {stuff}:, you don't know until the end of stuff that you are defining a function or a record value). Some tests seem to indicate that the syntax for patterns and values are distinct, so it can decides early. In Nickel I don't think we can hope for this, as we have renaming let {foo=foo_} = bar in, ellipsis for values (used for open record contracts), we have inline contract annotations in patterns let {foo | Num} = bar in ..., and so on. So the set of valid expressions for application and function definitions have a non-empty intersection but are not equal.

It is still probably doable by parsing something that looks like an application in a syntax that is a superset of the two, and decide validity once we've reached the end, but it may be painful (without even considering the specific parsing lib we currently use). Some languages choose this path, like ReScript, which dropped the fun keyword. They use parentheses as in (x,y) => x + y and AFAIK they don't support inline destructuring, but I imagine they still have to disambiguate tuples from functions definitions.

We actually discussed doing something like ReScript (we don't have tuple, so it would not be ambiguous) but that looks odd with respect to ML style application. ReScript just did this to be more JavaScript-like, and also changed the function application in consequence. We also discussed using something shorter like Haskell \, but a lambda sounded more idiosyncratic for people working on configuration than a keyword relating to the word "function", and we decided to go with ML-like let-function -definition syntax.

[yannham]

I stand corrected, as ReScript does support destructuring for function arguments.

[mboes]

Do we have a dedicated issue regarding function syntax? Your thoughts above will be more easily found there than in this issue about blocks of let-bindings.

[yannham]

Function syntax was part of #207 originally, but hasn't been updated after the standardization meeting discussions. Let me do that.

[mboes]

FWIW I find issue-specific tickets easier to browse, comment on and interrelate than the original omnibus "dojo" tickets - in case you feel like breaking out some of the discussion into separate tickets. ;)

[litchipi]

My 2 cents on that matter, what I originally understood when taking the feature request was to simply create a syntax sugar at the parser level to make

let a = 3,
    b = 4,
    c = 5,
in

behave exactly the same as

let a = 3 in
let b = 4 in
let c = 5 in

And I think this syntax sugar makes sense.
It doesn't add any complex issue or philosophical thoughts on the direction of the project, and imho makes the langage simpler to read, and use to create easy-to-modify configurations like so:

# Top level user settings
let port = 8080,
   index_page = "/www/index.html",
   security_level = `HIGH,
   debug = true,
in

# Configuration based on top-level user settings
# ...

And I think it would be better to keep this simple syntax for the simplest use case, and complexify the syntax for more complex features (like mutual recursion), that allows simple project configuration to be written in a simple way, and big projects with complex "programming" needs to still have them.

[mboes]

Well, if this is just syntactic sugar for declaration sequences, and not specifically to handle mutual recursion more cleanly like @yannham said above, then I don't think we yet have enough experience to guide us. Syntactic sugar is easy to add later when we do have more Nickel code in the wild. Harder to remove if we realize it isn't paying for itself in the context of a minimal language like Nickel.

[yannham]

Given the conclusion of the previous discussion, let's close this for now.