Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

Improve PoSt benches #1571

Merged
merged 1 commit into from
Mar 9, 2022
Merged

Improve PoSt benches #1571

merged 1 commit into from
Mar 9, 2022

Conversation

cryptonemo
Copy link
Collaborator

feat: extend winning_post bench to allow fake seals
feat: add a window_post_fake bench for using fake seals
feat: bump default API version in both PoSt benches to current
docs: update documentation on bench usage

@cryptonemo
Copy link
Collaborator Author

WindowPostFake 32GiB:

$ time FIL_PROOFS_VERIFY_PARAMETERS=0 FIL_PROOFS_USE_MULTICORE_SDR=1 FIL_PROOFS_USE_GPU_TREE_BUILDER=0 FIL_PROOFS_USE_GPU_COLUMN_BUILDER=1 RUST_BACKTRACE=full RUST_LOG=info RUSTFLAGS="-C target-cpu=native" cargo run --features cuda --release --bin benchy -- window-post-fake --size 32GiB --fake
... snip ...
{"git":{"hash":"751fb197f72601962918a57658a87c1209d1d47b","date":"2022-03-02T17:12:06Z"},"system":{"system":"Linux","release":"5.4.0-90-generic","version":"#101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021","architecture":"x86_64","processor":"AMD EPYC 7402 24-Core Processor ","processor-base-frequency-hz":0,"processor-max-frequency-hz":0,"processor-features":"FeatureInfo { eax: 8589072, ebx: 1026557952, edx_ecx: SSE3 | PCLMULQDQ | MONITOR | SSSE3 | FMA | CMPXCHG16B | SSE41 | SSE42 | MOVBE | POPCNT | AESNI | XSAVE | OSXSAVE | AVX | F16C | RDRAND | FPU | VME | DE | PSE | TSC | MSR | PAE | MCE | CX8 | APIC | SEP | MTRR | PGE | MCA | CMOV | PAT | PSE36 | CLFSH | MMX | FXSR | SSE | SSE2 | HTT }","processor-cores-logical":96,"processor-cores-physical":10,"memory-total-bytes":1056765012000},"benchmarks":{"inputs":{"sector-size":34359738368,"fake-replica":true},"outputs":{"gen-window-post-cpu-time-ms":1523534,"gen-window-post-wall-time-ms":170963,"verify-window-post-cpu-time-ms":43542,"verify-window-post-wall-time-ms":2140}}}
real 16m17.285s
user 260m40.108s
sys 16m55.279s

WindowPostFake 64GiB:

$ time FIL_PROOFS_VERIFY_PARAMETERS=0 FIL_PROOFS_USE_MULTICORE_SDR=1 FIL_PROOFS_USE_GPU_TREE_BUILDER=0 FIL_PROOFS_USE_GPU_COLUMN_BUILDER=1 RUST_BACKTRACE=full RUST_LOG=info RUSTFLAGS="-C target-cpu=native" cargo run --features cuda --release --bin benchy -- window-post-fake --size 64GiB --fake
... snip ...
{"git":{"hash":"751fb197f72601962918a57658a87c1209d1d47b","date":"2022-03-02T17:12:06Z"},"system":{"system":"Linux","release":"5.4.0-90-generic","version":"#101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021","architecture":"x86_64","processor":"AMD EPYC 7402 24-Core Processor ","processor-base-frequency-hz":0,"processor-max-frequency-hz":0,"processor-features":"FeatureInfo { eax: 8589072, ebx: 3090155520, edx_ecx: SSE3 | PCLMULQDQ | MONITOR | SSSE3 | FMA | CMPXCHG16B | SSE41 | SSE42 | MOVBE | POPCNT | AESNI | XSAVE | OSXSAVE | AVX | F16C | RDRAND | FPU | VME | DE | PSE | TSC | MSR | PAE | MCE | CX8 | APIC | SEP | MTRR | PGE | MCA | CMOV | PAT | PSE36 | CLFSH | MMX | FXSR | SSE | SSE2 | HTT }","processor-cores-logical":96,"processor-cores-physical":10,"memory-total-bytes":1056765012000},"benchmarks":{"inputs":{"sector-size":68719476736,"fake-replica":true},"outputs":{"gen-window-post-cpu-time-ms":1568735,"gen-window-post-wall-time-ms":173015,"verify-window-post-cpu-time-ms":42311,"verify-window-post-wall-time-ms":2102}}}
real 29m35.435s
user 500m47.215s
sys 28m43.033s

@cryptonemo
Copy link
Collaborator Author

WinningPostFake 32GiB:

$ time FIL_PROOFS_VERIFY_PARAMETERS=0 FIL_PROOFS_USE_MULTICORE_SDR=1 FIL_PROOFS_USE_GPU_TREE_BUILDER=0 FIL_PROOFS_USE_GPU_COLUMN_BUILDER=1 RUST_BACKTRACE=full RUST_LOG=info RUSTFLAGS="-C target-cpu=native" cargo run --features cuda --release --bin benchy -- winning-post --size 32GiB --fake
... snip ...
{"git":{"hash":"751fb197f72601962918a57658a87c1209d1d47b","date":"2022-03-02T17:12:06Z"},"system":{"system":"Linux","release":"5.4.0-90-generic","version":"#101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021","architecture":"x86_64","processor":"AMD EPYC 7402 24-Core Processor ","processor-base-frequency-hz":0,"processor-max-frequency-hz":0,"processor-features":"FeatureInfo { eax: 8589072, ebx: 2955937792, edx_ecx: SSE3 | PCLMULQDQ | MONITOR | SSSE3 | FMA | CMPXCHG16B | SSE41 | SSE42 | MOVBE | POPCNT | AESNI | XSAVE | OSXSAVE | AVX | F16C | RDRAND | FPU | VME | DE | PSE | TSC | MSR | PAE | MCE | CX8 | APIC | SEP | MTRR | PGE | MCA | CMOV | PAT | PSE36 | CLFSH | MMX | FXSR | SSE | SSE2 | HTT }","processor-cores-logical":96,"processor-cores-physical":10,"memory-total-bytes":1056765012000},"benchmarks":{"inputs":{"sector-size":34359738368,"fake-replica":true},"outputs":{"gen-winning-post-cpu-time-ms":14645,"gen-winning-post-wall-time-ms":1742,"verify-winning-post-cpu-time-ms":796,"verify-winning-post-wall-time-ms":24,"gen-winning-post-sector-challenge-cpu-time-ms":0,"gen-winning-post-sector-challenge-wall-time-ms":0}}}
real 13m11.989s
user 239m49.681s
sys 11m45.601s

WinningPostFake 64GiB:

s$ time FIL_PROOFS_VERIFY_PARAMETERS=0 FIL_PROOFS_USE_MULTICORE_SDR=1 FIL_PROOFS_USE_GPU_TREE_BUILDER=1 FIL_PROOFS_USE_GPU_COLUMN_BUILDER=1 RUST_BACKTRACE=full RUST_LOG=info RUSTFLAGS="-C target-cpu=native" cargo run --features cuda --release --bin benchy -- winning-post --size 64GiB --fake
... snip ...
{"git":{"hash":"751fb197f72601962918a57658a87c1209d1d47b","date":"2022-03-02T17:12:06Z"},"system":{"system":"Linux","release":"5.4.0-90-generic","version":"#101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021","architecture":"x86_64","processor":"AMD EPYC 7402 24-Core Processor ","processor-base-frequency-hz":0,"processor-max-frequency-hz":0,"processor-features":"FeatureInfo { eax: 8589072, ebx: 2167408640, edx_ecx: SSE3 | PCLMULQDQ | MONITOR | SSSE3 | FMA | CMPXCHG16B | SSE41 | SSE42 | MOVBE | POPCNT | AESNI | XSAVE | OSXSAVE | AVX | F16C | RDRAND | FPU | VME | DE | PSE | TSC | MSR | PAE | MCE | CX8 | APIC | SEP | MTRR | PGE | MCA | CMOV | PAT | PSE36 | CLFSH | MMX | FXSR | SSE | SSE2 | HTT }","processor-cores-logical":96,"processor-cores-physical":10,"memory-total-bytes":1056765012000},"benchmarks":{"inputs":{"sector-size":68719476736,"fake-replica":true},"outputs":{"gen-winning-post-cpu-time-ms":16485,"gen-winning-post-wall-time-ms":1029,"verify-winning-post-cpu-time-ms":801,"verify-winning-post-wall-time-ms":21,"gen-winning-post-sector-challenge-cpu-time-ms":0,"gen-winning-post-sector-challenge-wall-time-ms":0}}}
real 22m26.124s
user 12m25.785s
sys 21m15.350s

@cryptonemo
Copy link
Collaborator Author

Oops, first 3 results had GPU Tree building disabled accidentally ... the final result did not.

vmx
vmx previously approved these changes Mar 3, 2022
View reviewed changes
@cryptonemo
feat: extend winning_post bench to allow fake seals
9cb21ec 
feat: add a window_post_fake bench for using fake seals
feat: bump default API version in both PoSt benches to current
docs: update documentation on bench usage
@cryptonemo cryptonemo force-pushed the improved-post-benches branch from 751fb19 to 9cb21ec Compare March 8, 2022 19:33
@cryptonemo cryptonemo merged commit 2c6ea8b into master Mar 9, 2022
@cryptonemo cryptonemo deleted the improved-post-benches branch March 9, 2022 13:18
vmx pushed a commit that referenced this pull request Jun 28, 2022
@cryptonemo @vmx
feat: extend winning_post bench to allow fake seals (#1571)
b8f0aac 
feat: add a window_post_fake bench for using fake seals
feat: bump default API version in both PoSt benches to current
docs: update documentation on bench usage
vmx pushed a commit that referenced this pull request Jun 28, 2022
@cryptonemo @vmx
feat: extend winning_post bench to allow fake seals (#1571)
49c67a7 
feat: add a window_post_fake bench for using fake seals
feat: bump default API version in both PoSt benches to current
docs: update documentation on bench usage
# for free to join this conversation on GitHub. Already have an account? # to comment
Reviewers

@vmx vmx vmx left review comments

@dignifiedquire dignifiedquire Awaiting requested review from dignifiedquire

@DrPeterVanNostrand DrPeterVanNostrand Awaiting requested review from DrPeterVanNostrand DrPeterVanNostrand is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@cryptonemo @vmx