jsonlines.org

[finnp]

Hey,

I just saw this page on JSON Lines, which also is basically ndjson: http://jsonlines.org/
It is more of a good documention on the format than a specification, but I like it.

@wardi I just saw the page. Do you feel like giving input to the spec?

Best,
Finn

[wardi]

Yes. We need an official one of these.

I like the "ndjson" name better than "JSON Lines".
I don't like blank lines (or comment lines as the ndj format has) because it makes referencing objects/lines ambiguous: What does "I want the 3rd record" mean when the 2nd is blank?

[finnp]

Thanks for you input.

I am also not supporting the NDJ use of comments.

Blank lines are not allowed in the format, however we suggest ("SHOULD") that the parser should ignore blank lines (see here). Mostly because empty lines would be an error that could be quite common.

[wardi]

I guess I prefer for things to fail loudly when something isn't right.

This spec suggests that any number of newlines is acceptable between records. I deliberately included language on jsonlines.org about referencing records by index because that's what unix text processing tools do.

How about saying that each newline creates a new record, and that blank records may be treated as null records. This way the application can choose to ignore them, but it's never ambiguous what a record's number is.

[finnp]

Hm, I don't like null records, because "null" is proper JSON. I changed the specification now, to make clear that the parser should fail on errors and only optionally ignore empty lines - and that this behavior should be configurable and clearly documented : #11.

[finnp]

Hi @wardi,

I feel like I closed this issue too quickly last year. The spec actually now reads this:

If the JSON text is not parseable, the parser SHOULD raise an error. The parser MAY silently ignore empty lines, e.g. \n\n. This behavior MUST be documented and SHOULD be configurable by the user of the parser.

This only optionally allows the new line to be ignored. So the default behavior is the one you preferred for JSONLines.

I would love to hear your opinion. Maybe we can merge JSONLines and NDJSON now?

[wardi]

@finnp probably! That would eliminate one of the projects I don't make enough time for.

I was planning to add more examples of use, such as use of jq -c for creating ndjson. would you be willing to incorporate some examples to make your site a little more approachable?

You haven't included any language about record numbers. I feel that it is important to have a common language to reference items within a ndjson stream. Would you consider adding this as a recommendation?

[sandstrom]

@finnp @wardi Please merge them ⛵

It would solve a lot of the confusion with two very similarly named projects. 1 + 1 == 3.

[hugovk]

@finnp What do you think about @wardi's suggestions about adding examples of use (I think this is a great idea, examples always help) and a recommendation about record numbers?

Merging would at least make things simpler for people trying to decide which to reference.

[sandstrom]

friendly ping @finnp @wardi 😄