-
Notifications
You must be signed in to change notification settings - Fork 104
xsv 0.13.0 `stats` compared with qsv 1.0.0 `stats`
As referenced in Reddit qsv 1.0.0 announcement thread:
% hyperfine 'xsv stats --everything ~/Downloads/NYC_311_SR_2010-2020-sample-1M.csv -o /tmp/xsvstats-everything.csv'
Benchmark 1: xsv stats --everything ~/Downloads/NYC_311_SR_2010-2020-sample-1M.csv -o /tmp/xsvstats-everything.csv
Time (mean ± σ): 3.246 s ± 0.251 s [User: 15.675 s, System: 1.192 s]
Range (min … max): 3.024 s … 3.887 s 10 runs
% hyperfine 'qsv stats --everything ~/Downloads/NYC_311_SR_2010-2020-sample-1M.csv -o /tmp/qsvstats-everything.csv'
Benchmark 1: qsv stats --everything ~/Downloads/NYC_311_SR_2010-2020-sample-1M.csv -o /tmp/qsvstats-everything.csv
Time (mean ± σ): 2.158 s ± 0.181 s [User: 10.857 s, System: 0.663 s]
Range (min … max): 1.947 s … 2.469 s 10 runs
% hyperfine 'qsv stats --everything --infer-dates ~/Downloads/NYC_311_SR_2010-2020-sample-1M.csv -o /tmp/qsvstats-everything-inferdates.csv'
Benchmark 1: qsv stats --everything --infer-dates ~/Downloads/NYC_311_SR_2010-2020-sample-1M.csv -o /tmp/qsvstats-everything-inferdates.csv
Time (mean ± σ): 2.801 s ± 0.172 s [User: 16.112 s, System: 0.753 s]
Range (min … max): 2.578 s … 3.029 s 10 runs- These benchmarks were compiled on an Apple Mac Mini 2023 model with an M2 Pro chip with 12 CPU cores & 32GB of RAM with a 1TB SSD running macOS Sequoia 15.2 Developer Beta, using a million row, 41 column, 520 MB sample of NYC's 311 data.
xsvwas installed by runningcargo install xsv.
12 columns
3.246 secs
https://github.com/dathere/qsv/blob/master/docs/misc/xsvstats-everything.csv
37 columns and four dataset level "stats" (rowcount, colcount, filesize, fingerprint hash)
See https://github.com/dathere/qsv/wiki/Supplemental#stats-command-output-explanation for more info.
2.158 secs
https://github.com/dathere/qsv/blob/master/docs/misc/qsvstats-everything.csv
same as above, but with date-inferencing
2.801 seconds
https://github.com/dathere/qsv/blob/master/docs/misc/qsvstats-everything-inferdates.csv
qsv — GitHub · Releases · Discussions · qsv pro · Try it online · Benchmarks · datHere · DeepWiki · Dual-licensed MIT / Unlicense
Edit this page: Contributing to the Wiki
Home · Why qsv? · Tier legend
- All Commands (index)
- Selection & Inspection
- Transform & Reshape
- Aggregation & Statistics
- Joins & Set Ops
- SQL & Polars
- Validation & Schema
- Metadata Profiling (profile)
- Conversion & I/O
- Geospatial
- HTTP & Web
- Scripting (Luau / Python)
- Indexing, Compression & Diff
- AI & Documentation