Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Add an example usage for the inputs builtin #2216

Closed
wants to merge 3 commits into from

Conversation

hexagonrecursion
Copy link

It took me too long to figure out a simple trick: you can use ., inputs to get a stream of all the inputs including the first one. I want to add this to the manual to make this more discoverable.

@coveralls
Copy link

coveralls commented Nov 21, 2020

Coverage Status

Coverage remained the same at 84.134% when pulling 27b0629 on hexagonrecursion:patch-1 into a17dd32 on stedolan:master.

@wtlangford
Copy link
Contributor

This is a great addition- But I think I'd recommend a slightly different example, and possibly show off the use of the -n flag to pass null as the first input and leave the input stream untouched (this is the way I typically use inputs).

$ echo 1 2 3 | jq -n 'reduce inputs as $i (0; . + i)'
6

This shows off the purpose a bit better- it lets you deal with a sequence of JSON values that you don't want to slurp up all at once into a single array. Imagine you were processing structured log output and you had a file with a large number of (potentially also quite large) log entries you wanted to reduce over- you'd want to avoid slurping all of that up into an array for memory reasons.

Regardless, this is definitely a gap in our documentation- If you want to update your PR to use an example like the one I provided, I'd be happy to merge it!

@hexagonrecursion
Copy link
Author

show off the use of the -n flag to pass null as the first input and leave the input stream untouched

You can do that? I did not know. The documentation for the -n option needs an improvement too.

Imagine you were processing structured log output and you had a file with a large number of (potentially also quite large) log entries you wanted to reduce over- you'd want to avoid slurping all of that up into an array for memory reasons.

Something like that. Before I figured out the ., inputs trick to grab all the inputs I used

jq 'select just the info I care about' | jq --slurp 'process it some more e.g. de-duplicate'

when I finally figured out the above trick I switched to

jq '[., inputs | select just the info I care about] | process it some more e.g. de-duplicate'

@hexagonrecursion
Copy link
Author

Thank you @pkoppstein I'm sorry this PR fell through the cracks and I completely forgot about it. I have incorporated your feedback.

@hexagonrecursion
Copy link
Author

CI failed for strange reasons. Close and reopen to try again.

@hexagonrecursion
Copy link
Author

I am 99.8% certain this is an issue on your end (or upstream)

error: mingw32: key "5F944B027F7FE2091985AA2EFA11531AA0AA7F57" is unknown
:: Import PGP key 4096R/5F944B027F7FE2091985AA2EFA11531AA0AA7F57, "Christoph Reiter (MSYS2 development key) <reiter.christoph@gmail.com>", created: 2020-06-22? [Y/n] error: mingw32: signature from "Christoph Reiter (MSYS2 development key) <reiter.christoph@gmail.com>" is unknown trust
error: failed to update mingw32 (invalid or corrupted database (PGP signature))
Error: flex: no bottle available!
Error: bison: no bottle available!
The command "brew install flex bison" failed and exited with 1 during .

@hexagonrecursion
Copy link
Author

@stedolan @wtlangford I have received no feedback on my last patch which I have submitted almost a month ago.

Any of the following would be preferable to complete silence:

  • I an busy right now. I will be able to review your patch by <some time in the reasonably near future>
  • Sorry, but we are not interested in fixing this
  • I like the idea, but your current implementation needs more work. <please specify what needs improvement>

@hexagonrecursion
Copy link
Author

Close and reopen to restart CI

@hexagonrecursion
Copy link
Author

ping?

Copy link
Contributor

@itchyny itchyny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like a nice addition. However, in jq manual you need to use four space indentation instead of a code block for the example.

itchyny added a commit to itchyny/jq that referenced this pull request Jul 22, 2023
- Add error/0 and mentions null input behavior (close jqlang#2231)
- Explain value iterator suffix syntax .foo[] (close jqlang#1047)
- Mention array slicing is also zero-based (close jqlang#2094)
- Add examples of input and inputs filters (close jqlang#2216, close jqlang#2470)
- Improve sort_by about multiple values (close jqlang#2103, close jqlang#2467, close jqlang#2474)
- Improve foreach section and simplify examples (close jqlang#2169, close jqlang#1148)
- Fix recurse/1 document on how it is identical using recurse/2 (close jqlang#2036)
- Add non-string examples of index/1, rindex/1 (close jqlang#1422)
itchyny added a commit to itchyny/jq that referenced this pull request Jul 22, 2023
- Add error/0 and mentions null input behavior (close jqlang#2231)
- Explain value iterator suffix syntax .foo[] (close jqlang#1047)
- Mention array slicing is also zero-based (close jqlang#2094)
- Add examples of input and inputs filters (close jqlang#2216, close jqlang#2470)
- Improve sort_by about multiple values (close jqlang#2103, close jqlang#2467, close jqlang#2474)
- Improve foreach section and simplify examples (close jqlang#2169, close jqlang#1148)
- Fix recurse/1 document on how it is identical using recurse/2 (close jqlang#2036)
- Add non-string examples of index/1, rindex/1 (close jqlang#1422)
itchyny added a commit to itchyny/jq that referenced this pull request Jul 22, 2023
- Add error/0 and mentions null input behavior (close jqlang#2231)
- Explain value iterator suffix syntax .foo[] (close jqlang#1047)
- Mention array slicing is also zero-based (close jqlang#2094)
- Add examples of input and inputs filters (close jqlang#2216, close jqlang#2470)
- Improve sort_by about multiple values (close jqlang#2103, close jqlang#2467, close jqlang#2474)
- Improve foreach section and simplify examples (close jqlang#2169, close jqlang#1148)
- Fix recurse/1 document on how it is identical using recurse/2 (close jqlang#2036)
- Add non-string examples of index/1, rindex/1 (close jqlang#1422)
itchyny added a commit to itchyny/jq that referenced this pull request Jul 22, 2023
- Add error/0 and mentions null input behavior (close jqlang#2231)
- Explain value iterator suffix syntax .foo[] (close jqlang#1047)
- Mention array slicing is also zero-based (close jqlang#2094)
- Add examples of input and inputs filters (close jqlang#2216, close jqlang#2470)
- Improve sort_by about multiple values (close jqlang#2103, close jqlang#2467, close jqlang#2474)
- Improve foreach section and simplify examples (close jqlang#1148, close jqlang#2169)
- Fix recurse/1 document on how it is identical using recurse/2 (close jqlang#2036)
- Add non-string examples of index/1, rindex/1 (close jqlang#1422)
itchyny added a commit to itchyny/jq that referenced this pull request Jul 22, 2023
- Add error/0 and mentions null input behavior (close jqlang#2231)
- Explain value iterator suffix syntax .foo[] (close jqlang#1047)
- Mention array slicing is also zero-based (close jqlang#2094)
- Add examples of input and inputs filters (close jqlang#2216, close jqlang#2470)
- Improve sort_by about multiple values (close jqlang#2103, close jqlang#2467, close jqlang#2474)
- Improve foreach section and simplify examples (close jqlang#1148, close jqlang#2169)
- Fix recurse/1 document on how it is identical using recurse/2 (close jqlang#2036)
- Add non-string examples of index/1, rindex/1 (close jqlang#1422)
- Simplify the example of truncate_stream/1 (close jqlang#1736)
itchyny added a commit to itchyny/jq that referenced this pull request Jul 22, 2023
- Add error/0 and mentions null input behavior (close jqlang#2231)
- Explain value iterator suffix syntax .foo[] (close jqlang#1047)
- Mention array slicing is also zero-based (close jqlang#2094)
- Add examples of input and inputs filters (close jqlang#2216, close jqlang#2470)
- Improve sort_by about multiple values (close jqlang#2103, close jqlang#2467, close jqlang#2474)
- Improve foreach section and simplify examples (close jqlang#1148, close jqlang#2169)
- Fix recurse/1 document on how it is identical using recurse/2 (close jqlang#2036, close jqlang#2412)
- Add non-string examples of index/1, rindex/1 (close jqlang#1422)
- Simplify the example of truncate_stream/1 (close jqlang#1736)
itchyny added a commit to itchyny/jq that referenced this pull request Jul 22, 2023
- Add error/0 and mentions null input behavior (close jqlang#2231)
- Explain value iterator suffix syntax .foo[] (close jqlang#1047)
- Mention array slicing is also zero-based (close jqlang#2094)
- Add examples of input and inputs filters (close jqlang#2216, close jqlang#2470)
- Improve sort_by about multiple values (close jqlang#2103, close jqlang#2467, close jqlang#2474)
- Improve foreach section and simplify examples (close jqlang#1148, close jqlang#2169)
- Fix recurse/1 document on how it is identical using recurse/2 (close jqlang#2036, close jqlang#2412)
- Add non-string examples of index/1, rindex/1 (close jqlang#1422)
- Simplify the example of truncate_stream/1 (close jqlang#1736)
itchyny added a commit to itchyny/jq that referenced this pull request Jul 22, 2023
- Add error/0 and mentions null input behavior (close jqlang#2231)
- Explain value iterator suffix syntax .foo[] (close jqlang#1047)
- Mention array slicing is also zero-based (close jqlang#2094)
- Add examples of input and inputs filters (close jqlang#2216, close jqlang#2470)
- Improve sort_by about multiple values (close jqlang#2103, close jqlang#2467, close jqlang#2474)
- Improve foreach section and simplify examples (close jqlang#1148, close jqlang#2169)
- Fix recurse/1 document on how it is identical using recurse/2 (close jqlang#2036, close jqlang#2412)
- Add non-string examples of index/1, rindex/1 (close jqlang#1422)
- Simplify the example of truncate_stream/1 (close jqlang#1736)
itchyny added a commit to itchyny/jq that referenced this pull request Jul 22, 2023
- Add error/0 and mentions null input behavior (close jqlang#2231)
- Explain value iterator suffix syntax .foo[] (close jqlang#1047)
- Mention array slicing is also zero-based (close jqlang#2094)
- Add examples of input and inputs filters (close jqlang#2216, close jqlang#2470)
- Improve sort_by about multiple values (close jqlang#2103, close jqlang#2467, close jqlang#2474)
- Improve foreach section and simplify examples (close jqlang#1148, close jqlang#2169)
- Fix recurse/1 document on how it is identical using recurse/2 (close jqlang#2036, close jqlang#2412)
- Add non-string examples of index/1, rindex/1 (close jqlang#1422)
- Simplify the example of truncate_stream/1 (close jqlang#1736)
itchyny added a commit to itchyny/jq that referenced this pull request Jul 22, 2023
- Add error/0 and mentions null input behavior (close jqlang#2231)
- Explain value iterator suffix syntax .foo[] (close jqlang#1047)
- Mention array slicing is also zero-based (close jqlang#2094)
- Add examples of input and inputs filters (close jqlang#2216, close jqlang#2470)
- Improve sort_by about multiple values (close jqlang#2103, close jqlang#2467, close jqlang#2474)
- Improve foreach section and simplify examples (close jqlang#1148, close jqlang#2169)
- Fix recurse/1 document on how it is identical using recurse/2 (close jqlang#2036, close jqlang#2412)
- Add non-string examples of index/1, rindex/1 (close jqlang#1422)
- Simplify the example of truncate_stream/1 (close jqlang#1736)
itchyny added a commit to itchyny/jq that referenced this pull request Jul 23, 2023
- Add error/0 and mentions null input behavior (close jqlang#2231)
- Explain value iterator suffix syntax .foo[] (close jqlang#1047)
- Mention array slicing is also zero-based (close jqlang#2094)
- Add examples of input and inputs filters (close jqlang#2216, close jqlang#2470)
- Improve sort_by about multiple values (close jqlang#2103, close jqlang#2467, close jqlang#2474)
- Improve foreach section and simplify examples (close jqlang#1148, close jqlang#2169)
- Fix recurse/1 document on how it is identical using recurse/2 (close jqlang#2036, close jqlang#2412)
- Add non-string examples of index/1, rindex/1 (close jqlang#1422)
- Simplify the example of truncate_stream/1 (close jqlang#1736)
itchyny added a commit to itchyny/jq that referenced this pull request Jul 23, 2023
- Add error/0 and mentions null input behavior (close jqlang#2231)
- Explain value iterator suffix syntax .foo[] (close jqlang#1047)
- Mention array slicing is also zero-based (close jqlang#2094)
- Add examples of input and inputs filters (close jqlang#2216, close jqlang#2470)
- Improve sort_by about multiple values (close jqlang#2103, close jqlang#2467, close jqlang#2474)
- Improve foreach section and simplify examples (close jqlang#1148, close jqlang#2169)
- Fix recurse/1 document on how it is identical using recurse/2 (close jqlang#2036, close jqlang#2412)
- Add non-string examples of index/1, rindex/1 (close jqlang#1422)
- Simplify the example of truncate_stream/1 (close jqlang#1736)
itchyny added a commit to itchyny/jq that referenced this pull request Jul 23, 2023
- Add error/0 and mentions null input behavior (close jqlang#2231)
- Explain value iterator suffix syntax .foo[] (close jqlang#1047)
- Mention array slicing is also zero-based (close jqlang#2094)
- Add examples of input and inputs filters (close jqlang#2216, close jqlang#2470)
- Improve sort_by about multiple values (close jqlang#2103, close jqlang#2467, close jqlang#2474)
- Improve foreach section and simplify examples (close jqlang#1148, close jqlang#2169)
- Fix recurse/1 document on how it is identical using recurse/2 (close jqlang#2036, close jqlang#2412)
- Add non-string examples of index/1, rindex/1 (close jqlang#1422)
- Simplify the example of truncate_stream/1 (close jqlang#1736)
itchyny added a commit to itchyny/jq that referenced this pull request Jul 23, 2023
- Add error/0 and mentions null input behavior (close jqlang#2231)
- Explain value iterator suffix syntax .foo[] (close jqlang#1047)
- Mention array slicing is also zero-based (close jqlang#2094)
- Add examples of input and inputs filters (close jqlang#2216, close jqlang#2470)
- Improve sort_by about multiple values (close jqlang#2103, close jqlang#2467, close jqlang#2474)
- Improve foreach section and simplify examples (close jqlang#1148, close jqlang#2169)
- Fix recurse/1 document on how it is identical using recurse/2 (close jqlang#2036, close jqlang#2412)
- Add non-string examples of index/1, rindex/1 (close jqlang#1422)
- Simplify the example of truncate_stream/1 (close jqlang#1736)
itchyny added a commit to itchyny/jq that referenced this pull request Jul 23, 2023
- Add error/0 and mentions null input behavior (close jqlang#2231)
- Explain value iterator suffix syntax .foo[] (close jqlang#1047)
- Mention array slicing is also zero-based (close jqlang#2094)
- Add examples of input and inputs filters (close jqlang#2216, close jqlang#2470)
- Improve sort_by about multiple values (close jqlang#2103, close jqlang#2467, close jqlang#2474)
- Improve foreach section and simplify examples (close jqlang#1148, close jqlang#2169)
- Fix recurse/1 document on how it is identical using recurse/2 (close jqlang#2036, close jqlang#2412)
- Add non-string examples of index/1, rindex/1 (close jqlang#1422)
- Simplify the example of truncate_stream/1 (close jqlang#1736)
itchyny added a commit to itchyny/jq that referenced this pull request Jul 23, 2023
- Add error/0 and mentions null input behavior (close jqlang#2231)
- Explain value iterator suffix syntax .foo[] (close jqlang#1047)
- Mention array slicing is also zero-based (close jqlang#2094)
- Add examples of input and inputs filters (close jqlang#2216, close jqlang#2470)
- Improve sort_by about multiple values (close jqlang#2103, close jqlang#2467, close jqlang#2474)
- Improve foreach section and simplify examples (close jqlang#1148, close jqlang#2169)
- Fix recurse/1 document on how it is identical using recurse/2 (close jqlang#2036, close jqlang#2412)
- Add non-string examples of index/1, rindex/1 (close jqlang#1422)
- Simplify the example of truncate_stream/1 (close jqlang#1736)
itchyny added a commit to itchyny/jq that referenced this pull request Jul 23, 2023
- Add error/0 and mentions null input behavior (close jqlang#2231)
- Explain value iterator suffix syntax .foo[] (close jqlang#1047)
- Mention array slicing is also zero-based (close jqlang#2094)
- Add examples of input and inputs filters (close jqlang#2216, close jqlang#2470)
- Improve sort_by about multiple values (close jqlang#2103, close jqlang#2467, close jqlang#2474)
- Improve foreach section and simplify examples (close jqlang#1148, close jqlang#2169)
- Fix recurse/1 document on how it is identical using recurse/2 (close jqlang#2036, close jqlang#2412)
- Add non-string examples of index/1, rindex/1 (close jqlang#1422)
- Simplify the example of truncate_stream/1 (close jqlang#1736)
@itchyny itchyny closed this in ed334b5 Jul 24, 2023
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants