Skip to content

xsv 0.13.0 `stats` compared with qsv 1.0.0 `stats`

Joel Natividad edited this page Dec 4, 2024 · 6 revisions

As referenced in Reddit qsv 1.0.0 announcement thread:

% hyperfine 'xsv stats --everything ~/Downloads/NYC_311_SR_2010-2020-sample-1M.csv -o /tmp/xsvstats-everything.csv'
Benchmark 1: xsv stats --everything ~/Downloads/NYC_311_SR_2010-2020-sample-1M.csv -o /tmp/xsvstats-everything.csv
  Time (mean ± σ):      3.246 s ±  0.251 s    [User: 15.675 s, System: 1.192 s]
  Range (min … max):    3.024 s …  3.887 s    10 runs
 
% hyperfine 'qsv stats --everything ~/Downloads/NYC_311_SR_2010-2020-sample-1M.csv -o /tmp/qsvstats-everything.csv'
Benchmark 1: qsv stats --everything ~/Downloads/NYC_311_SR_2010-2020-sample-1M.csv -o /tmp/qsvstats-everything.csv
  Time (mean ± σ):      2.158 s ±  0.181 s    [User: 10.857 s, System: 0.663 s]
  Range (min … max):    1.947 s …  2.469 s    10 runs
 
% hyperfine 'qsv stats --everything --infer-dates ~/Downloads/NYC_311_SR_2010-2020-sample-1M.csv -o /tmp/qsvstats-everything-inferdates.csv'
Benchmark 1: qsv stats --everything --infer-dates ~/Downloads/NYC_311_SR_2010-2020-sample-1M.csv -o /tmp/qsvstats-everything-inferdates.csv
  Time (mean ± σ):      2.801 s ±  0.172 s    [User: 16.112 s, System: 0.753 s]
  Range (min … max):    2.578 s …  3.029 s    10 runs

xsv 0.13.0 stats --everything:

12 columns
3.246 secs
https://github.com/dathere/qsv/blob/master/docs/misc/xsvstats-everything.csv

qsv 1.0.0 stats --everything:

37 columns and four dataset level "stats" (rowcount, colcount, filesize, fingerprint hash)
See https://github.com/dathere/qsv/wiki/Supplemental#stats-command-output-explanation for more info.
2.158 secs https://github.com/dathere/qsv/blob/master/docs/misc/qsvstats-everything.csv

qsv 1.0.0 stats --everything --infer-dates:

same as above, but with date-inferencing
2.801 seconds
https://github.com/dathere/qsv/blob/master/docs/misc/qsvstats-everything-inferdates.csv