Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Link usage hints #296

Closed
handrews opened this issue Apr 3, 2017 · 16 comments
Closed

Link usage hints #296

handrews opened this issue Apr 3, 2017 · 16 comments
Milestone

Comments

@handrews
Copy link
Contributor

handrews commented Apr 3, 2017

JSON Hyper-Schema should not include keywords that come from specific protocols, and certainly not ones that do not generalize well enough to apply to any protocol.

This would suggest that "allow" as proposed by #73 would not be ideal as it is HTTP-specific, although the concept generalizes well, so it's not an unreasonable proposal either.

However, there are several things such as HTTP's "Allow", which would optimize and document how to use the protocol to interact with a link target, as opposed to the current LDO hints such as "mediaType" which describe the target resource itself, independent of how it is accessed.

A more clear example is the "Accept-Patch" header, which is critical for correctly constructing an HTTP PATCH request (in conjunction with either "targetSchema" or the schema linked in the response to a HEAD or GET to the target). "Accept-Patch" is very specific to HTTP (or, really, an extension to HTTP). No current Hyper-Schema keyword could accommodate it, and adding an "acceptPatch" field to the LDO would be inappropriately specific.


One approach would be a "usageHints" field for hinting at protocol usage meta-data that could otherwise be discovered by interacting with the target resource. This field would not encompass meta-data that describes the target resource independent of protocol.

The value of this keyword would be an object. The exact possible values for the contents of the object are deferred to the protocol and the target resource, but Hyper-Schema would define general rules to provide for interoperability.

Here's a strawman proposal:

  • keys would be the meta-data field names in whatever form is deemed canonical by the relevant standards, e.g. "Accept-Patch" and not a JSON-ized "acceptPatch"
  • values would be as they would exist in the protocol, except that lists and named fields SHOULD be structured using JSON arrays and objects

So if we wanted to hint at the following authentication header (example taken directly from RFC 7235)

WWW-Authenticate: Newauth realm="apps", type=1,
                       title="Login to \\"apps\\"", Basic realm="simple"

along with hinting at available HTTP methods (example taken directly from RFC 7231)

Allow: GET, HEAD, PUT

the result would look like:

becomes:

{
    "rel": "self",
    "href": "/foos/{id}",
    "usageHints": {
        "Allow": ["GET", "HEAD", "PUT"],
        "WWW-Authenticate": {
            "Newauth": {
                "realm": "apps",
                "type": 1,
                "title": "Login to \"apps\""
            },
            "Basic": {
                "realm": "simple"
            }
        }
    }
}
@handrews
Copy link
Contributor Author

handrews commented Apr 3, 2017

Deferring the exact contents of "usageHints" to the protocol + resource also allows resources to define their own meta-data extensions, for instance the "X-*" HTTP headers that are common mechanisms in APIs.

@jdesrosiers
Copy link
Member

This would suggest that "allow" as proposed by #73 would not be ideal as it is HTTP-specific

It occurred to me recently that we could introduce allow in a way that is not HTTP-specific. The href scheme determines the protocol that is used and the protocol determines what values allow can have. Different protocols means different possible values. Therefore, the spec doesn't have to be HTTP specific, it can just defer to the protocol specified in the href.

Same goes for usageHints. You can directly use HTTP headers without the spec necessarily being coupled to HTTP.

@jdesrosiers
Copy link
Member

@handrews This proposal seems like that API description stuff you have been saying doesn't belong in Hyper-Schema. Can you explain why this proposal is not API description?

(I realize this question sounds passive aggressive. Please don't take it as such. I'm still trying to figure out exactly what you mean by API description.)

@handrews
Copy link
Contributor Author

handrews commented Apr 3, 2017

I realize this question sounds passive aggressive. Please don't take it as such.

Given how often I get my tone wrong in these sorts of conversations, I am in no place to throw stones!

And I think it's a fair question anyway, and the answer may even end up being "actually it doesn't belong here", but I'll explain why I went ahead and filed it. When I talk about aspects of API description being undesirable in JSON Hyper-Schema as a media type, I am making the following distinctions:

  • representations & links vs endpoints & operations
  • single representation scope vs API scope

The usage hints work with links rather than operations, meaning that those hints do not need to be changed for different HTTP methods. Yes, some of them ("Accept-Patch" being a particularly obvious one) are only useful with certain operations that can be performed with the link, but we don't need to change them for each operation. They are always the same hints, which may or may not be relevant depending on the operation being performed.

They are also single representation scoped rather than API scoped. With the auth info, this usage hint is saying exactly what the header would say in a HEAD response. It is relevant to the target resource resource as accessed from the context representation. It doesn't say anything one way or the other about auth across an entire API.

I'm not sure I'd want to do auth this way, but it was an interesting example with some complex structure.

Another point is that a mechanism like "usageHints" that defers its exact contents to the protocol will be relatively easy to misuse, but that is better than coupling HTTP and Hyper-Schema. Ultimately, an implementation can completely ignore hints and still be compliant, because they are non-authoritative. It would be a rather lame implementation, but technically correct.

@handrews
Copy link
Contributor Author

handrews commented Apr 3, 2017

The href scheme determines the protocol that is used and the protocol determines what values allow can have. Different protocols means different possible values. Therefore, the spec doesn't have to be HTTP specific, it can just defer to the protocol specified in the href.

Yes, that's also exactly the mechanism I had in mind for anything that would want to determine sensible values for "usageHints". The spec for "usageHints" needn't mention HTTP at all, although as usual some HTTP-based examples are generally helpful.

@dlax
Copy link
Member

dlax commented Apr 17, 2017

I just discovered the Home Documents for HTTP APIs Internet draft which specifies resource hints pretty thoroughly already.
Quoting introduction of Section 5 of the draft:

Resource hints allow clients to find relevant information about
interacting with a resource beforehand, as a means of optimizing
communications, as well as advertising available behaviors (e.g., to
aid in laying out a user interface for consuming the API).
Hints are just that - they are not a "contract", and are to only be
taken as advisory. The runtime behavior of the resource always
overrides hinted information.

Might be a source of inspiration.

@handrews
Copy link
Contributor Author

@dlax see mnot/I-D#211

@timoshisa
Copy link

Having the capability to use both a "hints" LDO keyword in addition to short-forms "allow" and "method" for HTTP-like protocols would be beneficial. For a vast majority of cases JSON Hyper-Schemas will be used for HTTP API description and validation (client and server). With this end in mind it would be good to try to keep things simple for those users, and being able to specify a method directly inside the LDO rather than as part of a "hints" section would be important for both accessibility of the standard and also ease-of-use generally. There are other line-based internet protocols like SIP which share the same semantics. One can also envisage (and I have used in the past) WebSocket based protocols utilising method/path tuples in JSON documents.

Of course, method, allow and hints should be wrapped in a oneOf to prevent confusion over which one should be used.

The previous draft with "method" is in use by several software projects already. Granted this is still a draft spec, but it is functionally a very important feature of the spec that needs to be put back into the draft sooner rather than later. Specifying with "rel" seems to be too vague (please correct me if I am wrong, would be useful to have something more formal in Hyper Schema's docs as a formal mapping for HTTP is not easy to find.) Without the ability to specify HTTP method we are only able to specify the protocol without being able to fully express the address of how to reach the endpoint (in HTTP, href is only one of the fields needed.)

In terms of HTTP-agnosticism, "method" and "allow" don't need to be locked down to defined-at-this-time HTTP methods and can have their interpretation left open to the protocol being targeted. There is little need to validate the keywords in this spec, and validation can be deferred to protocol-specific schemas.

On a side-note: I favour json-home's camel-cased properties, simply for consistency with the JSON Schema specification and current vogue for JSON document design. Certainly it's cosmetic, but the resulting hypermedia documents will look much cleaner and also it brings consistency with json-home. I also prefer either "usage" or "hints" instead of "usageHints", again for cosmetics.

@handrews
Copy link
Contributor Author

@tbehrsin If you haven't already, please look at the Hyper-Schema Draft-06 migration FAQ for how and why "rel" is not too vague in many cases. If you are following HTTP semantics in terms of methods and representations, you do not need to re-specify links for each method. For dealing with non-HTTP-semantics-complying APIs, see #226.

Regarding camelCase, it depends on whether we want to say "the keys are whatever the specific protocol uses", in which case we do not get to determine the formatting, or whether we want to define a mapping from arbitrary protocol syntax into camelCase. I could go either way, but I tend to prefer minimizing parsing / translation so simply saying "it's whatever the protocol expects" is the option that produces the least amount of work for implementations.

@handrews
Copy link
Contributor Author

@tbehrsin as far as shorthand for "allow" or anything else, let's keep that discussion in #73 so this issue stays focused on the generic mechanism. That is a complex topic and one that is not impacted by whether we add specific keywords (if anything, it would be the other way around).

@handrews
Copy link
Contributor Author

OK, I've taken a look at where JSON Home has gone since the last time I checked in (paging @dlax).

  • It establishes a "Resource Hints" registry with an initial set of hints
  • Several are exact HTTP headers (e.g. "allow", "acceptPatch", etc.)
  • Others are extensions of existing ideas ("acceptPost", "acceptPrefer")
  • A few are less obviously tied to existing HTTP concepts ("docs")
  • Each is given its own specification in terms of the JSON value structure

The scope of what is included is:

Generally, [resource hints] ought to be information that would otherwise be discoverable by interacting with the resource.

This seems like a good scope definition for us as well. It stays within the single target resource, rather than assembling multiple resources into a larger structure such as an API. Like JSON Home, we should allow some flexibility to cover notable gaps in existing specifications.

This covers two use cases:

  1. Optimized response headers (or things very much like them)
  2. Advertising a more complex notion acceptable input ("acceptPrefer", "preconditionRequired")

For the first case, we are well aligned. Allow/"allow" should obviously be an array of strings corresponding to methods. Accept-Patch/"acceptPatch" should be an array of strings.

For the second case, JSON Home is more or less faking response headers for features that have no such headers defined. However, we have a much better mechanism for describing acceptable input, which is a validation schema.

At Cloudflare, we actually do this already, although we only make light use of it (for specifying the auth header requirements, mostly). I'm planning to do more with it in the next iteration, though. The obvious benefit of using a schema is that it's consistent with all other data input mechanisms ("hrefSchema", "submissionSchema", "targetSchema" when used to construct a PUT request).

So I think we're close to a proposal

@handrews
Copy link
Contributor Author

I'm uncertain how to name these things. I'm going to call them "targetHints" and "headerSchema" for now, but is "header" even a suitably generic word? Most protocols that I can think of use headers for these purposes. I don't want to use "meta" or "metaData" because it will cause confusion with the meta-schemas. Anyway, try to read for the proposal and not the easily changed name :-)

"targetHints" takes a JSON value representing information about how to interact with the target resource that could otherwise reasonably be discovered by interacting with the resource. For HTTP, that means an object where the keys are the names of headers that would appear in a HEAD and/or OPTIONS response.

"headerSchema" is a schema indicating what information is acceptable for which headers. In HTTP, this is an object schema naming request headers and describing acceptable values for each. It is expected that only headers of particular interest to user-agents will be documented in this way. "headerSchema" SHOULD NOT set "additionalProperties" to false as unknown headers are very likely to appear.

TODO: Use names as is, lowercase, or camelCase? "Accept-Patch", "accept-patch", or "acceptPatch"?

TODO: Define mapping from HTTP header grammar into JSON and back (this perhaps does not need to be in the spec itself? Would it be a mini JSON Schema vocabulary, or would it just be some sort of appendix / web site guide?)

I would also like to have at least one other good example besides HTTP and CoAP. mailto: headers are put in the URI so that's not a good example. Anyone have any ideas?

@handrews
Copy link
Contributor Author

It turns out that there is a draft proposal for JSON serialization of HTTP headers, so with the usual caveats about referencing other drafts, this provides work we can piggy-back off of, as well as validation that we are not the only people thinking of this sort of thing. Even if our use case is different.

Thanks to @dret for pointing this out!

@dret
Copy link
Contributor

dret commented Aug 30, 2017 via email

@handrews
Copy link
Contributor Author

it would be much simpler for most scenarios to have a JSON model for this kind of data.

Definitely. I need to read the draft in detail, but a JSON model would not only make it unambiguous for us to convey header values (for the purposes of hinting) in a JSON environment, but would then make it very natural to write schemas declaring which headers are particularly important, and what value(s) they might take. That would be a lot harder to do if it were not clear how to express certain headers in JSON, much less in schemas.

handrews added a commit to handrews/json-schema-spec that referenced this issue Sep 2, 2017
This addresses the request half of json-schema-org#296, and is a companion
keyword to "targetHints" (json-schema-org#383).

Like "targetHints", this is a rough first pass to get the
discussion of specifics started.  I expect it to be somewhat
underspecified to encourage experimentation.

I intentionally chose an example that is both very complex
and very useful.  It might also be worth putting up a simpler
example.  Although when this appears right after "targetHints",
the simpler header value examples in that section will help
motivate what is going on here.
@handrews handrews added this to the draft-07 (wright-*-02) milestone Sep 3, 2017
handrews added a commit to handrews/json-schema-spec that referenced this issue Sep 13, 2017
This addresses the request half of json-schema-org#296, and is a companion
keyword to "targetHints" (json-schema-org#383).

Like "targetHints", this is a rough first pass to get the
discussion of specifics started.  I expect it to be somewhat
underspecified to encourage experimentation.

I intentionally chose an example that is both very complex
and very useful.  It might also be worth putting up a simpler
example.  Although when this appears right after "targetHints",
the simpler header value examples in that section will help
motivate what is going on here.
handrews added a commit to handrews/json-schema-spec that referenced this issue Sep 14, 2017
This addresses the request half of json-schema-org#296, and is a companion
keyword to "targetHints" (json-schema-org#383).

Like "targetHints", this is a rough first pass to get the
discussion of specifics started.  I expect it to be somewhat
underspecified to encourage experimentation.

I intentionally chose an example that is both very complex
and very useful.  It might also be worth putting up a simpler
example.  Although when this appears right after "targetHints",
the simpler header value examples in that section will help
motivate what is going on here.
handrews added a commit to handrews/json-schema-spec that referenced this issue Sep 14, 2017
This addresses the request half of json-schema-org#296, and is a companion
keyword to "targetHints" (json-schema-org#383).

Like "targetHints", this is a rough first pass to get the
discussion of specifics started.  I expect it to be somewhat
underspecified to encourage experimentation.

I intentionally chose an example that is both very complex
and very useful.  It might also be worth putting up a simpler
example.  Although when this appears right after "targetHints",
the simpler header value examples in that section will help
motivate what is going on here.
handrews added a commit to handrews/json-schema-spec that referenced this issue Sep 14, 2017
This addresses the request half of json-schema-org#296, and is a companion
keyword to "targetHints" (json-schema-org#383).

Like "targetHints", this is a rough first pass to get the
discussion of specifics started.  I expect it to be somewhat
underspecified to encourage experimentation.

I intentionally chose an example that is both very complex
and very useful.  It might also be worth putting up a simpler
example.  Although when this appears right after "targetHints",
the simpler header value examples in that section will help
motivate what is going on here.
@handrews
Copy link
Contributor Author

Merged #383 and #390.

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants