Typesafe marshal

[mgsloan]

Notes:

• Adds typesafe marshaling as described in Make it conventional to always export JSRef newtype constructors? (Relevant to potential new marshaling API) ghcjs#419
• Adds some typesystem cleverness:
  • Removes the list specific conversion methods, in favor of an auxiliary class
  • Adds a EnsureNotMaybe constraint to make the marshaling of Maybe well behaved
• Renames JSType and JSONType to JSTypeOf and JSONTypeOf, to avoid collision with the new type family.

Things left to do:

• Add instances of the marshaling classes to all of the reference types in ghcjs-base. This means that lots of modules in ghcjs-base would import GHCJS.Marshal. Is that ok?
• Document the behavior of generic deriving marshaling
• Address these FIXMEs: https://github.com/ghcjs/ghcjs-base/pull/34/files#diff-6dc88c79ebb178104238eb0e8385d068R164
• Consider whether all these instances should be in GHCJS.Marshal - it might be better to have the other parts of ghcjs-base import this marshaling stuff. That would allow the stuff here to turn into normal deriving: https://github.com/mgsloan/ghcjs-base/blob/bd055d746b0c38c3d104fca7f89fa137e59d6111/GHCJS/Marshal.hs#L190 .
• Consider whether we should recommend deriving NFData / Data as standard practice for newtype wrappers around references (documented here)

[spl]

What is the reason for having two JSRef conversion approaches, one “pure” and one in IO?

[mgsloan]

Thanks for the suggestions! I've added a commit updating the laws.

One thing that the pure marshaling is used for is this quasiquoter: https://github.com/ghcjs/ghcjs-ffiqq

It makes sense to have pure marshaling for immutable things. So, in javascript this applies to the boolean / number / string types. Pure marshaling doesn't make sense for objects, because they can change.

It does make sense to have pure marshaling like pFromJS :: MyRefType -> MyRefType, though, because it's equivalent to id.

[spl]

It makes sense to have pure marshaling for immutable things. So, in javascript this applies to the boolean / number / string types. Pure marshaling doesn't make sense for objects, because they can change.

Okay, now I understand the differentiation. Thanks. Follow-up question: Is pure marshaling a necessity or an optimization or something else?

[luite]

What is the reason for having two JSRef conversion approaches, one “pure” and one in IO?

With pure conversions you can operate directly on immutable (either guaranteed by the underlying type, like JavaScript string or number, or by the typesystem) data types from pure Haskell code. Normally the impure conversions should be a strict superset of the pure conversions, since you should be able to get an impure one from a pure one by adding return.

I don't really like the fact that it's two typeclasses. Can we do better? Since purity is really a property of an instance, it would be better to indicate for every instance whether the conversion is pure, eliminating the second class.

I played with the following code earlier, but without injectivity of the Purity type family it doesn't work (GHC 8 lets you add an injectivity annotation, but it (correctly) rejects the family as being noninjective). Is there a way to fix this?

{-# LANGUAGE TypeFamilies, DataKinds, OverloadedStrings #-}
module Test where

import           Data.Text (Text)
import qualified Data.Text as T

newtype JSRef = JSRef Int -- dummy

ref0, ref1 :: JSRef
ref0 = JSRef 0
ref1 = JSRef 1

type family Purity (a :: Bool) b where
  Purity True  b = b
  Purity False b = IO b

class FromJSRef a where
  type IsPureFromJSRef a :: Bool
  fromJSRef :: JSRef -> Purity (IsPureFromJSRef a) a

instance FromJSRef Int where
  type IsPureFromJSRef Int = True
  fromJSRef (JSRef x) = x

instance FromJSRef Bool where
  type IsPureFromJSRef Bool = True
  fromJSRef (JSRef x) = x /= 0

instance FromJSRef Text where
  type IsPureFromJSRef Text = False
  fromJSRef (JSRef x) = return (T.replicate x "text")

An alternative route that actually does work:

Using GADTs, and keeping the pFromJSRef and fromJSRef things separate, still eliminating one of the typeclasses (change the Bool kind to a custom Purity kind to get better error messages). Can we simplify this further?

{-# LANGUAGE ScopedTypeVariables, GADTs, DataKinds, KindSignatures, TypeFamilies, ConstraintKinds, OverloadedStrings #-}
module Test where

import           Data.Text (Text)
import qualified Data.Text as T

newtype JSRef = JSRef Int -- dummy

ref0, ref1 :: JSRef
ref0 = JSRef 0
ref1 = JSRef 1

fromJSRef :: forall a. FromJSRef a => JSRef -> IO a
fromJSRef x = let (r :: Result (IsPureFromJSRef a) a) = fromJSRef_ x
              in  case r of Pure   r -> pure r
                            Impure r -> r

pFromJSRef :: forall a. (FromJSRef a, IsPureFromJSRef a ~ True) => JSRef -> a
pFromJSRef x = let (r :: Result True a) = fromJSRef_ x
               in  case r of Pure r -> r

data Result (p :: Bool) a where
  Pure   :: a    -> Result True  a
  Impure :: IO a -> Result False a

class FromJSRef a where
  type IsPureFromJSRef a :: Bool
  fromJSRef_ :: JSRef -> Result (IsPureFromJSRef a) a

instance FromJSRef Int where
  type IsPureFromJSRef Int = True
  fromJSRef_ (JSRef x) = Pure x

instance FromJSRef Bool where
  type IsPureFromJSRef Bool = True
  fromJSRef_ (JSRef x) = Pure $ x /= 0

instance FromJSRef Text where
  type IsPureFromJSRef Text = False
  fromJSRef_ (JSRef x) = Impure $ return (T.replicate x "text")

[luite]

Bikeshedding opportunity: rename JSNumber, JSBool to Number, Boolean respectively?

Looks prettier, but will probably require the use of more qualified imports. I don't really like having half the world prefixed with JS.

[spl]

With pure conversions you can operate directly on immutable (either guaranteed by the underlying type, like JavaScript string or number, or by the typesystem) data types from pure Haskell code. Normally the impure conversions should be a strict superset of the pure conversions, since you should be able to get an impure one from a pure one by adding return.

So this makes me think that the motivation is convenience? If so, is it really worth the effort to differentiate between these types?

[luite]

So this makes me think that the motivation is convenience? If so, is it really worth the effort to differentiate between these types?

The only realistic option would be to drop pure conversions, which would force much more code to be in IO, unless we have a separate way to handle all the pure conversions. The pure conversions were originally introduced for ghcjs-ffiqq, which needed a pure way to convert to/from arbitrary JSRef values.

for example things like this get much less nice if you need IO everywhere, and IO adds considerable overhead, preventing many optimizations:

f :: SomeObject -> Bool -> Int
f o p
  | [js| `o.doSomePureCheck(`p) |] = 1
  | otherwise                      = 2

And I also think it's quite usful to operate on JSRef values in pure code, without converting beforehand.

[spl]

Fair enough. Thanks for the responses.

I hope you don't mind me continuing to flesh out my understanding on this topic here.

What are the scenarios in which we need to convert types? Let me propose the ones that come to mind:

1. Converting any JS type polymorphically
2. Converting a specific JS type monomorphically

Since the most general form of conversion for all types is via IO, I can clearly see the need for that in 1.

We can always convert via IO for 2, but that is, as has been said, inconvenient and inefficient for some types, specifically the primitives. For those types, we want a non-IO conversion.

Now, this prompts the question of whether we need to treat all primitives in the same way. Do we need to convert any primitive JS type polymorphically? (This would be 3 on the list above, if true.)

If the answer is yes, we should have a type class (or some other ad-hoc mechanism) for that. If the answer is no, then I would question the use of an ad-hoc mechanism for something we don't need.

The ghcjs-ffiqq package is an example that uses the primitive JS type polymorphism. Looking at the source and example code, I wonder whether this is a good idea. It appears to me that code using the quasiquoter does not improve type inference and could lead to places requiring type annotations or signatures.

The use of the primitive JS type polymorphism in ghcjs-ffi appears to be a way to reduce the number of symbols one needs to know to use the interface. That is, you can use the overloaded js for Int, Bool, etc. rather than different functions for each (e.g. jsint, jsbool, etc.). Granted, there could be a lot of different functions, but I wonder if there is a better way.

What if, instead of using a type class to select the conversion instance, you used a parameter to the quasiquoter function that specialized the type? You could, for example, use the TH Name as a parameter, allowing you to use name quoting of the type, e.g. ''Int. (Alternatively, you could use an enumeration to specify a closed world of types.) You could end up with the same number of quasiquoter functions you have now, but just give some of them a parameter. (For example, [js'| 'x + 'y |] becomes [js' ''Int| 'x + 'y |].)

This approach removes any concern with type inference and avoids polluting the namespace with type-specific functions. Finally, you wouldn't need to define a class of pure JS type conversions.

Feel free to shoot holes in my understanding or ideas.

[luite]

I do welcome comments. And we should definitely steer towards simplicity unless we have a good reason for more type classes. I'm really short on time though, with preparations for Haskell eXchange taking most of my time at the moment, so I'll come back to this later.

[mgsloan]

I think this is a good way to get rid of the pure marshaling classes:

class PureMarshal a

instance PureMarshal JSRef
instance PureMarshal Text
-- etc

pFromJSRef :: PureMarshal a => JSRef -> a
pFromJSRef = unsafePerformIO . fromJSRefUnchecked
{-# INLINE pFromJSRef #-}

pToJSRef :: PureMarshal a => a -> JSRef
pToJSRef = unsafePerformIO . toJSRef
{-# INLINE pToJSRef #-}

Inlining should make this perform just as well for the common case, right? This leaves out the PureExclusive / PureShared distinction, but I'm not sure what that's useful for.

[mgsloan]

Bikeshedding opportunity: rename JSNumber, JSBool to Number, Boolean respectively?

I initially had it this way, but then backpeddled because it seemed weird to have Bool be a haskell type and Boolean be a javascript ref. I like dropping the JS prefixes, though, so I'll do this.

The ghcjs-ffiqq package is an example that uses the primitive JS type polymorphism. Looking at the source and example code, I wonder whether this is a good idea. It appears to me that code using the quasiquoter does not improve type inference and could lead to places requiring type annotations or signatures.

At some point, I'm hoping to have a typescript FFI quasiquoter - see this https://github.com/mgsloan/ghcjs-typescript#alternative-approach-to-typescript-ffi .

[luite]

Inlining should make this perform just as well for the common case, right?

Nope, at least not with unsafePerformIO

And I really don't like this approach, it's too fragile. You basically insert unsafePerformIO everywhere by creating a empty instances. It would only be acceptable if user's code was never expected to add pure marshalling instances, and we could hide it.

[mgsloan]

Hmm, good point. Darn, I'd like something really simple!

Your GADTs approach is pretty reasonable, but it would obviously be quite a breaking change. I'd like to bundle such a change with the change to using exceptions instead of Maybe. Maybe this is being overly performance sensitive, but would having these result constructors introduce overhead? Since there's only one constructor available for a given type, one would hope it'd be equivalent, if they had strictness annotations. But, as far as I know, GHC doesn't do any analysis like this.

I don't think associated type families should be used, as they don't play well with GND - https://ghc.haskell.org/trac/ghc/ticket/8165 - but things ought to work fine with using unassociated type families for this.

The boilerplate for defining new ref types would get rather big, though:

{-# LANGUAGE DeriveDataTypeable #-}
{-# LANGUAGE GeneralizedNewtypeDeriving #-}
{-# LANGUAGE TypeFamilies #-}

import Data.Typeable (Typeable)
import GHCJS.Marshal
import GHCJS.Marshal.Pure
import GHCJS.Types

newtype Wrapper = Wrapper JSRef
  deriving (Typeable, IsJSRef, ToJSRef, FromJSRef)
type instance JSType Wrapper = Wrapper
type instance IsPureFromJSRef Wrapper = True
type instance IsPureToJSRef Wrapper = True

If the goal is to reduce the redundancy / complexity of having 4 type classes, then this doesn't seem like a very good idea, as it just turns 2 type classes into type families, which need values for every type which supports marshaling. With the current situation, you only need to mention the pure marshaling stuff when you actually have pure marshaling.

So, my conclusion is that there's no good way to avoid the extra type classes for pure conversions.

[spl]

I do welcome comments. And we should definitely steer towards simplicity unless we have a good reason for more type classes. I'm really short on time though, with preparations for Haskell eXchange taking most of my time at the moment, so I'll come back to this later.

Of course! No problem.

[mgsloan]

I've rebased this atop master.

Does this look good? I'm waiting on review of this before addressing the remaining items in the checklist. Next up after this is switching marshaling over to using exceptions instead of Maybe.

[crocket]

It seems you guys want to break and rewrite Marshalling.

I'm going to document FFI and what I know about Marshalling and the relationship between FFI and Marshalling on https://github.com/ghcjs/ghcjs/wiki/GHCJS-User-Guide

I already documented a basic project setup with stack on it.

Since FFI is not documented, it was difficult for me to figure out how it worked. I had to ask people and consult ghcjs-dom.

Coding is not finished until it is documented for users.

I hope you won't break FFI anytime soon because I spent days on poking GHCJS to figure out FFI.