Typebox JSONSchema definition / self-hosting

[ryanswrt]

Apologies if this is more of a suggestion/question than an issue - but doubt it would get answered anywhere else.

I have a data type that would ideally contain a dynamic JSON schema definition - the only approaches I can see how to model this currently would be to either create a Typebox definition for a valid JSON Schema (which is hopefully possible, albeit a bit of work), create a sub-format that can be compiled into a JSON Schema, or just to use the Any type and lose strict typing...

The question I guess is whether JSON Schema would make sense as a base Typebox type to allow for handling of dynamic schemas, and if that is a project worth embarking on - or if the complexity would be too high to be considered feasible.

[sinclairzx81]

Hi, yes, that's fine.

Generally speaking, a 'dynamic schema' is pretty much analogous to a 'dynamic type', so TypeScript isn't going to be able to deal with that particularly well (being a static type system with no runtime), and TypeBox just builds on top of TypeScript's capabilities (as in, it tries to integrate with its static type system for 'known' types only).

The best you could do in situations like this is know "all possible variations of a schema before hand", in that case, you would create a TS union (or Type.Union) that expressed the shape of all the objects you expected. If you don't know the schema before hand, then the best you're going to be able to do is resort to any (or unknown), and that's fine as the schema is dynamic and unknown.

I am not sure what is producing your dynamic schemas, but I am going to assume they are:

1. not completely random
2. generated by some system based on some rules.

I think what you're probably looking for is some kind of code generation tool that reads these schemas and generates the TS types you need. TypeBox isn't really about code generation (unless you wanted to write that functionality yourself), this was briefly discussed in this issue #19 if you're interested. Once you have generated the types (if that's possible), then TypeScript can statically reason about them.

Perhaps this library may be of some interest https://www.npmjs.com/package/json-schema-to-typescript ?

Hope this helps
S

[sinclairzx81]

Um, maybe I read your question wrong, are you asking if its possible to dynamically build TypeBox schemas?

If that's the case, then yeah, I guess its possible to dynamically compose TypeBox types (though i've never tried), but if it were me id probably just generate raw JSONschema instead. I can't really suggest an approach here without knowing more details of what you're trying to achieve.

I'll leave this issue open for a few days if you want to follow up.
Cheers

[ryanswrt]

Thanks for the reply, helpful insight.

Ultimately, I want to use the JSON Schema generated by my Typebox definition to validate data submitted by a user, and some of the data being submitted will either be JSON Schemas in a particular format (your assumptions being correct).

The resulting type of the dynamic schema won't provide safety as you mention; but there is still some benefit in checking whether the submitted data (or generated JSON Schema) conforms to the JSON Schema spec, although there are probably other validators that could be used outside of Typebox.

Initial implementation of my approach:

function makeOrderableLabeledType(def: TProperties) {
  return Type.Object({
    type: Type.Literal("object"),
    properties: Type.Map(Type.Object(def), {maxProperties: 1})
  }, {
    description: "This object should contain one named field of the desired type"
  })
}

const SchemaStringType = makeOrderableLabeledType({type: Type.Literal("string")});
const SchemaNumberType = makeOrderableLabeledType({type: Type.Literal("number")});
// This will need to be expanded for all supported form field types

const SchemaTypes = Type.Union([SchemaStringType, SchemaNumberType])

const SchemaSchema = Type.Object({
  title: Type.Optional(Type.String()),
  $schema: Type.Literal("http://json-schema.org/draft-07/schema#"),
  // We use an array to ensure that ordering remains consistent
  type: Type.Literal("array"),
  items: Type.Array(SchemaTypes)
});

// Usage
const CustomForm = Type.Object({
  formTitle: Type.String(),
  formOwner: Type.String(),
  formSchema: SchemaSchema
})

[sinclairzx81]

Yeah, that sounds like an interesting thing to write. TypeBox will let you express JSON Schema to validate JSON Schema to some degree, but at some point you're going to run up against recursion. In situations like this where recursive schemas are required, the general rule of thumb would be to use a $ref directive to reference some element in some other part of the schema just to keep things flat. However, TypeBox doesn't provide any mechanism to facilitate type resolution for a $ref. (However it may be expressible with custom properties, I don't think there is a way to get the TS type inference working as expected, forcing you down the any route.)

Below is some code on how you might approach building such a schema with TypeBox, and where recursion comes into play (both for the generation of the schema, and for TypeScript that can't infer recursive types in this fashion). Note the Array<T> schema in particular, where the items property could in theory be a Array<Array<Array<Array<Array<Array<Array<Array<...infinity>>>>>>>>

import { Type, Static } from '@sinclair/typebox'

const ObjectSchema  = () => Type.Object({ type: Type.Literal('object'), properties: Type.Map(AnySchema()) }) // explode: recursion
const ArraySchema   = () => Type.Object({ type: Type.Literal('array'), items: AnySchema() })                 // explode: recursion
const StringSchema  = () => Type.Object({ type: Type.Literal('string') })
const NumberSchema  = () => Type.Object({ type: Type.Literal('number') })
const BooleanSchema = () => Type.Object({ type: Type.Literal('boolean') })
const AnySchema = () => Type.Union([
    StringSchema(),
    NumberSchema(),
    BooleanSchema(),
    // ArraySchema(),  // explode: recursion (TS resolves to any)
    // ObjectSchema()  // explode: recursion (TS resolves to any)
])

const JsonSchema = AnySchema()

type Schema = Static<typeof JsonSchema>

function foo(s: Schema) {
    // ...
}

Type Inference through indirection (using $ref) might be an interesting thing to explore just to see if its possible to get it working. But its not supported in TypeBox as of writing. In the interim, my best advise to you would be express your schema with plain JSON Schema, and implement the Schema nodes with vanilla TypeScript interfaces.

Some links that might be helpful

https://json-schema.org/understanding-json-schema/structuring.html#recursion
microsoft/TypeScript#23400
microsoft/TypeScript#14174

Cheers
S

[sinclairzx81]

Might close off this issue as its not tied to anything in particular. However, I may do some looking into providing out of box support for $ref at some point in future (TypeScript permitting)

Best of luck mate, hope you find a good approach that works for you.
Cheers
S