-
-
Notifications
You must be signed in to change notification settings - Fork 342
New issue
Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? # to your account
test: improve sim tests for some remaining feedback #6654
Conversation
|
Benchmark suite | Current: d1fc151 | Previous: 669239b | Ratio |
---|---|---|---|
phase0 getAttestationDeltas - 250000 worstcase | 19.376 ms/op | 5.7044 ms/op | 3.40 |
Full benchmark results
Benchmark suite | Current: d1fc151 | Previous: 669239b | Ratio |
---|---|---|---|
getPubkeys - index2pubkey - req 1000 vs - 250000 vc | 875.54 us/op | 1.1323 ms/op | 0.77 |
getPubkeys - validatorsArr - req 1000 vs - 250000 vc | 107.61 us/op | 111.90 us/op | 0.96 |
BLS verify - blst-native | 1.3494 ms/op | 1.1196 ms/op | 1.21 |
BLS verifyMultipleSignatures 3 - blst-native | 2.8196 ms/op | 2.3493 ms/op | 1.20 |
BLS verifyMultipleSignatures 8 - blst-native | 6.2012 ms/op | 5.2425 ms/op | 1.18 |
BLS verifyMultipleSignatures 32 - blst-native | 23.484 ms/op | 19.424 ms/op | 1.21 |
BLS verifyMultipleSignatures 64 - blst-native | 46.187 ms/op | 38.217 ms/op | 1.21 |
BLS verifyMultipleSignatures 128 - blst-native | 90.492 ms/op | 75.469 ms/op | 1.20 |
BLS deserializing 10000 signatures | 950.91 ms/op | 823.72 ms/op | 1.15 |
BLS deserializing 100000 signatures | 9.2696 s/op | 8.4494 s/op | 1.10 |
BLS verifyMultipleSignatures - same message - 3 - blst-native | 1.3972 ms/op | 1.1357 ms/op | 1.23 |
BLS verifyMultipleSignatures - same message - 8 - blst-native | 1.6057 ms/op | 1.2905 ms/op | 1.24 |
BLS verifyMultipleSignatures - same message - 32 - blst-native | 2.5016 ms/op | 2.0201 ms/op | 1.24 |
BLS verifyMultipleSignatures - same message - 64 - blst-native | 3.5995 ms/op | 3.8050 ms/op | 0.95 |
BLS verifyMultipleSignatures - same message - 128 - blst-native | 5.6984 ms/op | 4.8925 ms/op | 1.16 |
BLS aggregatePubkeys 32 - blst-native | 28.652 us/op | 23.030 us/op | 1.24 |
BLS aggregatePubkeys 128 - blst-native | 108.98 us/op | 88.903 us/op | 1.23 |
notSeenSlots=1 numMissedVotes=1 numBadVotes=10 | 70.034 ms/op | 48.941 ms/op | 1.43 |
notSeenSlots=1 numMissedVotes=0 numBadVotes=4 | 72.819 ms/op | 47.780 ms/op | 1.52 |
notSeenSlots=2 numMissedVotes=1 numBadVotes=10 | 55.921 ms/op | 30.884 ms/op | 1.81 |
getSlashingsAndExits - default max | 201.98 us/op | 159.62 us/op | 1.27 |
getSlashingsAndExits - 2k | 593.95 us/op | 464.24 us/op | 1.28 |
proposeBlockBody type=full, size=empty | 6.9657 ms/op | 5.2791 ms/op | 1.32 |
isKnown best case - 1 super set check | 569.00 ns/op | 235.00 ns/op | 2.42 |
isKnown normal case - 2 super set checks | 476.00 ns/op | 233.00 ns/op | 2.04 |
isKnown worse case - 16 super set checks | 463.00 ns/op | 222.00 ns/op | 2.09 |
InMemoryCheckpointStateCache - add get delete | 7.7350 us/op | 3.8380 us/op | 2.02 |
validate api signedAggregateAndProof - struct | 2.9380 ms/op | 2.4351 ms/op | 1.21 |
validate gossip signedAggregateAndProof - struct | 2.8784 ms/op | 2.3955 ms/op | 1.20 |
validate gossip attestation - vc 640000 | 1.4002 ms/op | 1.1466 ms/op | 1.22 |
batch validate gossip attestation - vc 640000 - chunk 32 | 169.75 us/op | 141.45 us/op | 1.20 |
batch validate gossip attestation - vc 640000 - chunk 64 | 146.87 us/op | 129.61 us/op | 1.13 |
batch validate gossip attestation - vc 640000 - chunk 128 | 140.92 us/op | 122.57 us/op | 1.15 |
batch validate gossip attestation - vc 640000 - chunk 256 | 135.23 us/op | 114.46 us/op | 1.18 |
pickEth1Vote - no votes | 1.3557 ms/op | 920.86 us/op | 1.47 |
pickEth1Vote - max votes | 8.2638 ms/op | 13.692 ms/op | 0.60 |
pickEth1Vote - Eth1Data hashTreeRoot value x2048 | 16.332 ms/op | 20.103 ms/op | 0.81 |
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 | 25.521 ms/op | 28.073 ms/op | 0.91 |
pickEth1Vote - Eth1Data fastSerialize value x2048 | 809.48 us/op | 521.40 us/op | 1.55 |
pickEth1Vote - Eth1Data fastSerialize tree x2048 | 4.6036 ms/op | 7.8683 ms/op | 0.59 |
bytes32 toHexString | 557.00 ns/op | 382.00 ns/op | 1.46 |
bytes32 Buffer.toString(hex) | 312.00 ns/op | 201.00 ns/op | 1.55 |
bytes32 Buffer.toString(hex) from Uint8Array | 529.00 ns/op | 404.00 ns/op | 1.31 |
bytes32 Buffer.toString(hex) + 0x | 310.00 ns/op | 222.00 ns/op | 1.40 |
Object access 1 prop | 0.17600 ns/op | 0.13000 ns/op | 1.35 |
Map access 1 prop | 0.15600 ns/op | 0.10000 ns/op | 1.56 |
Object get x1000 | 7.5010 ns/op | 4.7520 ns/op | 1.58 |
Map get x1000 | 0.82500 ns/op | 0.71400 ns/op | 1.16 |
Object set x1000 | 56.435 ns/op | 29.365 ns/op | 1.92 |
Map set x1000 | 47.809 ns/op | 22.370 ns/op | 2.14 |
Return object 10000 times | 0.25030 ns/op | 0.21760 ns/op | 1.15 |
Throw Error 10000 times | 4.0204 us/op | 2.7258 us/op | 1.47 |
fastMsgIdFn sha256 / 200 bytes | 3.3960 us/op | 1.9280 us/op | 1.76 |
fastMsgIdFn h32 xxhash / 200 bytes | 352.00 ns/op | 257.00 ns/op | 1.37 |
fastMsgIdFn h64 xxhash / 200 bytes | 397.00 ns/op | 261.00 ns/op | 1.52 |
fastMsgIdFn sha256 / 1000 bytes | 11.691 us/op | 6.0260 us/op | 1.94 |
fastMsgIdFn h32 xxhash / 1000 bytes | 458.00 ns/op | 361.00 ns/op | 1.27 |
fastMsgIdFn h64 xxhash / 1000 bytes | 464.00 ns/op | 343.00 ns/op | 1.35 |
fastMsgIdFn sha256 / 10000 bytes | 104.18 us/op | 51.002 us/op | 2.04 |
fastMsgIdFn h32 xxhash / 10000 bytes | 2.0090 us/op | 1.7560 us/op | 1.14 |
fastMsgIdFn h64 xxhash / 10000 bytes | 1.4120 us/op | 1.1420 us/op | 1.24 |
send data - 1000 256B messages | 20.564 ms/op | 12.427 ms/op | 1.65 |
send data - 1000 512B messages | 28.624 ms/op | 19.675 ms/op | 1.45 |
send data - 1000 1024B messages | 53.731 ms/op | 22.871 ms/op | 2.35 |
send data - 1000 1200B messages | 43.062 ms/op | 30.836 ms/op | 1.40 |
send data - 1000 2048B messages | 52.224 ms/op | 41.642 ms/op | 1.25 |
send data - 1000 4096B messages | 48.384 ms/op | 35.701 ms/op | 1.36 |
send data - 1000 16384B messages | 114.78 ms/op | 103.06 ms/op | 1.11 |
send data - 1000 65536B messages | 486.11 ms/op | 393.54 ms/op | 1.24 |
enrSubnets - fastDeserialize 64 bits | 1.5300 us/op | 870.00 ns/op | 1.76 |
enrSubnets - ssz BitVector 64 bits | 540.00 ns/op | 322.00 ns/op | 1.68 |
enrSubnets - fastDeserialize 4 bits | 258.00 ns/op | 128.00 ns/op | 2.02 |
enrSubnets - ssz BitVector 4 bits | 534.00 ns/op | 345.00 ns/op | 1.55 |
prioritizePeers score -10:0 att 32-0.1 sync 2-0 | 239.59 us/op | 139.42 us/op | 1.72 |
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 | 335.01 us/op | 179.13 us/op | 1.87 |
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 | 416.53 us/op | 287.25 us/op | 1.45 |
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 | 683.27 us/op | 427.23 us/op | 1.60 |
prioritizePeers score 0:0 att 64-1 sync 4-1 | 880.30 us/op | 511.67 us/op | 1.72 |
array of 16000 items push then shift | 1.8187 us/op | 1.5020 us/op | 1.21 |
LinkedList of 16000 items push then shift | 9.8970 ns/op | 6.2450 ns/op | 1.58 |
array of 16000 items push then pop | 110.53 ns/op | 110.52 ns/op | 1.00 |
LinkedList of 16000 items push then pop | 10.177 ns/op | 7.4380 ns/op | 1.37 |
array of 24000 items push then shift | 2.7080 us/op | 1.8335 us/op | 1.48 |
LinkedList of 24000 items push then shift | 9.9800 ns/op | 7.7000 ns/op | 1.30 |
array of 24000 items push then pop | 186.14 ns/op | 144.05 ns/op | 1.29 |
LinkedList of 24000 items push then pop | 9.6460 ns/op | 5.9510 ns/op | 1.62 |
intersect bitArray bitLen 8 | 5.9380 ns/op | 4.8970 ns/op | 1.21 |
intersect array and set length 8 | 80.791 ns/op | 49.848 ns/op | 1.62 |
intersect bitArray bitLen 128 | 35.987 ns/op | 29.251 ns/op | 1.23 |
intersect array and set length 128 | 1.0670 us/op | 713.42 ns/op | 1.50 |
bitArray.getTrueBitIndexes() bitLen 128 | 1.8840 us/op | 1.2970 us/op | 1.45 |
bitArray.getTrueBitIndexes() bitLen 248 | 2.9470 us/op | 2.2290 us/op | 1.32 |
bitArray.getTrueBitIndexes() bitLen 512 | 6.3870 us/op | 3.7470 us/op | 1.70 |
Buffer.concat 32 items | 1.1500 us/op | 919.00 ns/op | 1.25 |
Uint8Array.set 32 items | 2.4800 us/op | 2.4550 us/op | 1.01 |
Set add up to 64 items then delete first | 5.1612 us/op | 1.9929 us/op | 2.59 |
OrderedSet add up to 64 items then delete first | 7.1038 us/op | 2.8534 us/op | 2.49 |
Set add up to 64 items then delete last | 5.4597 us/op | 2.0530 us/op | 2.66 |
OrderedSet add up to 64 items then delete last | 6.2923 us/op | 3.0679 us/op | 2.05 |
Set add up to 64 items then delete middle | 5.2789 us/op | 2.0841 us/op | 2.53 |
OrderedSet add up to 64 items then delete middle | 8.2187 us/op | 5.1237 us/op | 1.60 |
Set add up to 128 items then delete first | 10.367 us/op | 4.7850 us/op | 2.17 |
OrderedSet add up to 128 items then delete first | 13.682 us/op | 7.0097 us/op | 1.95 |
Set add up to 128 items then delete last | 9.4476 us/op | 3.8744 us/op | 2.44 |
OrderedSet add up to 128 items then delete last | 13.486 us/op | 5.8344 us/op | 2.31 |
Set add up to 128 items then delete middle | 9.6241 us/op | 4.0111 us/op | 2.40 |
OrderedSet add up to 128 items then delete middle | 18.676 us/op | 10.897 us/op | 1.71 |
Set add up to 256 items then delete first | 20.343 us/op | 8.0310 us/op | 2.53 |
OrderedSet add up to 256 items then delete first | 30.444 us/op | 14.054 us/op | 2.17 |
Set add up to 256 items then delete last | 21.941 us/op | 9.4283 us/op | 2.33 |
OrderedSet add up to 256 items then delete last | 27.160 us/op | 11.886 us/op | 2.29 |
Set add up to 256 items then delete middle | 21.314 us/op | 9.4149 us/op | 2.26 |
OrderedSet add up to 256 items then delete middle | 52.059 us/op | 35.413 us/op | 1.47 |
transfer serialized Status (84 B) | 2.0000 us/op | 1.4800 us/op | 1.35 |
copy serialized Status (84 B) | 1.5440 us/op | 1.0410 us/op | 1.48 |
transfer serialized SignedVoluntaryExit (112 B) | 2.2780 us/op | 1.5620 us/op | 1.46 |
copy serialized SignedVoluntaryExit (112 B) | 1.6390 us/op | 1.1130 us/op | 1.47 |
transfer serialized ProposerSlashing (416 B) | 3.3890 us/op | 1.9260 us/op | 1.76 |
copy serialized ProposerSlashing (416 B) | 3.2940 us/op | 2.9560 us/op | 1.11 |
transfer serialized Attestation (485 B) | 3.5960 us/op | 2.9380 us/op | 1.22 |
copy serialized Attestation (485 B) | 3.2740 us/op | 2.6380 us/op | 1.24 |
transfer serialized AttesterSlashing (33232 B) | 3.0110 us/op | 2.4970 us/op | 1.21 |
copy serialized AttesterSlashing (33232 B) | 8.1140 us/op | 8.6090 us/op | 0.94 |
transfer serialized Small SignedBeaconBlock (128000 B) | 3.2070 us/op | 2.9360 us/op | 1.09 |
copy serialized Small SignedBeaconBlock (128000 B) | 26.277 us/op | 13.610 us/op | 1.93 |
transfer serialized Avg SignedBeaconBlock (200000 B) | 3.6710 us/op | 2.6510 us/op | 1.38 |
copy serialized Avg SignedBeaconBlock (200000 B) | 40.701 us/op | 15.155 us/op | 2.69 |
transfer serialized BlobsSidecar (524380 B) | 3.9220 us/op | 2.5370 us/op | 1.55 |
copy serialized BlobsSidecar (524380 B) | 105.76 us/op | 138.36 us/op | 0.76 |
transfer serialized Big SignedBeaconBlock (1000000 B) | 4.6360 us/op | 3.1880 us/op | 1.45 |
copy serialized Big SignedBeaconBlock (1000000 B) | 199.65 us/op | 168.36 us/op | 1.19 |
pass gossip attestations to forkchoice per slot | 4.5922 ms/op | 2.7351 ms/op | 1.68 |
forkChoice updateHead vc 100000 bc 64 eq 0 | 704.42 us/op | 489.79 us/op | 1.44 |
forkChoice updateHead vc 600000 bc 64 eq 0 | 5.7267 ms/op | 2.7861 ms/op | 2.06 |
forkChoice updateHead vc 1000000 bc 64 eq 0 | 7.3945 ms/op | 4.7562 ms/op | 1.55 |
forkChoice updateHead vc 600000 bc 320 eq 0 | 4.3306 ms/op | 2.7645 ms/op | 1.57 |
forkChoice updateHead vc 600000 bc 1200 eq 0 | 4.5015 ms/op | 2.9176 ms/op | 1.54 |
forkChoice updateHead vc 600000 bc 7200 eq 0 | 5.9987 ms/op | 4.0037 ms/op | 1.50 |
forkChoice updateHead vc 600000 bc 64 eq 1000 | 11.601 ms/op | 10.121 ms/op | 1.15 |
forkChoice updateHead vc 600000 bc 64 eq 10000 | 12.691 ms/op | 11.143 ms/op | 1.14 |
forkChoice updateHead vc 600000 bc 64 eq 300000 | 17.615 ms/op | 12.090 ms/op | 1.46 |
computeDeltas 500000 validators 300 proto nodes | 7.1578 ms/op | 3.4742 ms/op | 2.06 |
computeDeltas 500000 validators 1200 proto nodes | 7.1279 ms/op | 3.4384 ms/op | 2.07 |
computeDeltas 500000 validators 7200 proto nodes | 6.8696 ms/op | 3.3030 ms/op | 2.08 |
computeDeltas 750000 validators 300 proto nodes | 10.067 ms/op | 5.1525 ms/op | 1.95 |
computeDeltas 750000 validators 1200 proto nodes | 10.357 ms/op | 5.2449 ms/op | 1.97 |
computeDeltas 750000 validators 7200 proto nodes | 10.173 ms/op | 5.0659 ms/op | 2.01 |
computeDeltas 1400000 validators 300 proto nodes | 19.518 ms/op | 10.141 ms/op | 1.92 |
computeDeltas 1400000 validators 1200 proto nodes | 19.669 ms/op | 9.8342 ms/op | 2.00 |
computeDeltas 1400000 validators 7200 proto nodes | 19.362 ms/op | 9.6649 ms/op | 2.00 |
computeDeltas 2100000 validators 300 proto nodes | 27.428 ms/op | 15.607 ms/op | 1.76 |
computeDeltas 2100000 validators 1200 proto nodes | 29.735 ms/op | 15.311 ms/op | 1.94 |
computeDeltas 2100000 validators 7200 proto nodes | 27.840 ms/op | 14.403 ms/op | 1.93 |
altair processAttestation - 250000 vs - 7PWei normalcase | 2.3572 ms/op | 1.6522 ms/op | 1.43 |
altair processAttestation - 250000 vs - 7PWei worstcase | 3.3307 ms/op | 2.9627 ms/op | 1.12 |
altair processAttestation - setStatus - 1/6 committees join | 195.69 us/op | 120.58 us/op | 1.62 |
altair processAttestation - setStatus - 1/3 committees join | 360.13 us/op | 241.47 us/op | 1.49 |
altair processAttestation - setStatus - 1/2 committees join | 487.94 us/op | 288.28 us/op | 1.69 |
altair processAttestation - setStatus - 2/3 committees join | 651.50 us/op | 378.03 us/op | 1.72 |
altair processAttestation - setStatus - 4/5 committees join | 819.19 us/op | 561.54 us/op | 1.46 |
altair processAttestation - setStatus - 100% committees join | 987.22 us/op | 594.66 us/op | 1.66 |
altair processBlock - 250000 vs - 7PWei normalcase | 8.0171 ms/op | 7.6770 ms/op | 1.04 |
altair processBlock - 250000 vs - 7PWei normalcase hashState | 35.951 ms/op | 34.930 ms/op | 1.03 |
altair processBlock - 250000 vs - 7PWei worstcase | 45.264 ms/op | 36.133 ms/op | 1.25 |
altair processBlock - 250000 vs - 7PWei worstcase hashState | 95.812 ms/op | 95.826 ms/op | 1.00 |
phase0 processBlock - 250000 vs - 7PWei normalcase | 2.9811 ms/op | 2.1381 ms/op | 1.39 |
phase0 processBlock - 250000 vs - 7PWei worstcase | 30.600 ms/op | 28.775 ms/op | 1.06 |
altair processEth1Data - 250000 vs - 7PWei normalcase | 576.66 us/op | 280.08 us/op | 2.06 |
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 | 7.7150 us/op | 14.373 us/op | 0.54 |
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 | 68.847 us/op | 32.972 us/op | 2.09 |
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 | 24.318 us/op | 17.727 us/op | 1.37 |
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 | 11.533 us/op | 9.0110 us/op | 1.28 |
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 | 219.85 us/op | 148.76 us/op | 1.48 |
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 | 1.5667 ms/op | 1.0563 ms/op | 1.48 |
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 | 2.5206 ms/op | 1.8290 ms/op | 1.38 |
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 | 2.2495 ms/op | 1.3441 ms/op | 1.67 |
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 | 3.5738 ms/op | 2.6521 ms/op | 1.35 |
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 | 2.9447 ms/op | 1.8722 ms/op | 1.57 |
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 | 5.7831 ms/op | 4.4778 ms/op | 1.29 |
Tree 40 250000 create | 419.00 ms/op | 346.30 ms/op | 1.21 |
Tree 40 250000 get(125000) | 217.82 ns/op | 123.45 ns/op | 1.76 |
Tree 40 250000 set(125000) | 1.2120 us/op | 855.09 ns/op | 1.42 |
Tree 40 250000 toArray() | 22.080 ms/op | 23.259 ms/op | 0.95 |
Tree 40 250000 iterate all - toArray() + loop | 23.737 ms/op | 22.786 ms/op | 1.04 |
Tree 40 250000 iterate all - get(i) | 75.726 ms/op | 50.373 ms/op | 1.50 |
MutableVector 250000 create | 19.335 ms/op | 7.3393 ms/op | 2.63 |
MutableVector 250000 get(125000) | 7.0290 ns/op | 5.6410 ns/op | 1.25 |
MutableVector 250000 set(125000) | 299.41 ns/op | 219.81 ns/op | 1.36 |
MutableVector 250000 toArray() | 3.9607 ms/op | 2.3133 ms/op | 1.71 |
MutableVector 250000 iterate all - toArray() + loop | 3.9429 ms/op | 2.5731 ms/op | 1.53 |
MutableVector 250000 iterate all - get(i) | 1.5877 ms/op | 1.3498 ms/op | 1.18 |
Array 250000 create | 3.5099 ms/op | 2.3406 ms/op | 1.50 |
Array 250000 clone - spread | 1.4063 ms/op | 1.3174 ms/op | 1.07 |
Array 250000 get(125000) | 1.1750 ns/op | 1.0400 ns/op | 1.13 |
Array 250000 set(125000) | 4.4860 ns/op | 1.2480 ns/op | 3.59 |
Array 250000 iterate all - loop | 169.48 us/op | 158.06 us/op | 1.07 |
effectiveBalanceIncrements clone Uint8Array 300000 | 38.545 us/op | 19.311 us/op | 2.00 |
effectiveBalanceIncrements clone MutableVector 300000 | 407.00 ns/op | 397.00 ns/op | 1.03 |
effectiveBalanceIncrements rw all Uint8Array 300000 | 200.95 us/op | 186.25 us/op | 1.08 |
effectiveBalanceIncrements rw all MutableVector 300000 | 96.194 ms/op | 71.802 ms/op | 1.34 |
phase0 afterProcessEpoch - 250000 vs - 7PWei | 123.47 ms/op | 82.578 ms/op | 1.50 |
phase0 beforeProcessEpoch - 250000 vs - 7PWei | 56.127 ms/op | 61.178 ms/op | 0.92 |
altair processEpoch - mainnet_e81889 | 547.40 ms/op | 382.65 ms/op | 1.43 |
mainnet_e81889 - altair beforeProcessEpoch | 87.998 ms/op | 77.872 ms/op | 1.13 |
mainnet_e81889 - altair processJustificationAndFinalization | 17.227 us/op | 19.237 us/op | 0.90 |
mainnet_e81889 - altair processInactivityUpdates | 7.6905 ms/op | 5.1091 ms/op | 1.51 |
mainnet_e81889 - altair processRewardsAndPenalties | 63.875 ms/op | 73.555 ms/op | 0.87 |
mainnet_e81889 - altair processRegistryUpdates | 2.4760 us/op | 2.7590 us/op | 0.90 |
mainnet_e81889 - altair processSlashings | 620.00 ns/op | 692.00 ns/op | 0.90 |
mainnet_e81889 - altair processEth1DataReset | 593.00 ns/op | 670.00 ns/op | 0.89 |
mainnet_e81889 - altair processEffectiveBalanceUpdates | 1.7063 ms/op | 1.5013 ms/op | 1.14 |
mainnet_e81889 - altair processSlashingsReset | 4.5900 us/op | 3.2230 us/op | 1.42 |
mainnet_e81889 - altair processRandaoMixesReset | 9.6090 us/op | 4.0420 us/op | 2.38 |
mainnet_e81889 - altair processHistoricalRootsUpdate | 686.00 ns/op | 606.00 ns/op | 1.13 |
mainnet_e81889 - altair processParticipationFlagUpdates | 2.3830 us/op | 1.2010 us/op | 1.98 |
mainnet_e81889 - altair processSyncCommitteeUpdates | 768.00 ns/op | 588.00 ns/op | 1.31 |
mainnet_e81889 - altair afterProcessEpoch | 128.10 ms/op | 84.536 ms/op | 1.52 |
capella processEpoch - mainnet_e217614 | 2.2150 s/op | 1.8646 s/op | 1.19 |
mainnet_e217614 - capella beforeProcessEpoch | 530.36 ms/op | 408.19 ms/op | 1.30 |
mainnet_e217614 - capella processJustificationAndFinalization | 17.929 us/op | 8.2570 us/op | 2.17 |
mainnet_e217614 - capella processInactivityUpdates | 22.082 ms/op | 18.469 ms/op | 1.20 |
mainnet_e217614 - capella processRewardsAndPenalties | 673.67 ms/op | 419.50 ms/op | 1.61 |
mainnet_e217614 - capella processRegistryUpdates | 38.116 us/op | 23.545 us/op | 1.62 |
mainnet_e217614 - capella processSlashings | 1.1770 us/op | 438.00 ns/op | 2.69 |
mainnet_e217614 - capella processEth1DataReset | 1.1840 us/op | 296.00 ns/op | 4.00 |
mainnet_e217614 - capella processEffectiveBalanceUpdates | 4.7465 ms/op | 3.1207 ms/op | 1.52 |
mainnet_e217614 - capella processSlashingsReset | 7.9920 us/op | 2.0430 us/op | 3.91 |
mainnet_e217614 - capella processRandaoMixesReset | 6.7370 us/op | 3.0940 us/op | 2.18 |
mainnet_e217614 - capella processHistoricalRootsUpdate | 1.5530 us/op | 454.00 ns/op | 3.42 |
mainnet_e217614 - capella processParticipationFlagUpdates | 4.4120 us/op | 1.2140 us/op | 3.63 |
mainnet_e217614 - capella afterProcessEpoch | 398.19 ms/op | 202.73 ms/op | 1.96 |
phase0 processEpoch - mainnet_e58758 | 562.63 ms/op | 405.16 ms/op | 1.39 |
mainnet_e58758 - phase0 beforeProcessEpoch | 181.83 ms/op | 105.55 ms/op | 1.72 |
mainnet_e58758 - phase0 processJustificationAndFinalization | 18.873 us/op | 9.5230 us/op | 1.98 |
mainnet_e58758 - phase0 processRewardsAndPenalties | 84.162 ms/op | 58.536 ms/op | 1.44 |
mainnet_e58758 - phase0 processRegistryUpdates | 13.481 us/op | 6.5720 us/op | 2.05 |
mainnet_e58758 - phase0 processSlashings | 601.00 ns/op | 359.00 ns/op | 1.67 |
mainnet_e58758 - phase0 processEth1DataReset | 715.00 ns/op | 342.00 ns/op | 2.09 |
mainnet_e58758 - phase0 processEffectiveBalanceUpdates | 1.3999 ms/op | 1.5273 ms/op | 0.92 |
mainnet_e58758 - phase0 processSlashingsReset | 4.1710 us/op | 3.1190 us/op | 1.34 |
mainnet_e58758 - phase0 processRandaoMixesReset | 4.9910 us/op | 2.6350 us/op | 1.89 |
mainnet_e58758 - phase0 processHistoricalRootsUpdate | 804.00 ns/op | 493.00 ns/op | 1.63 |
mainnet_e58758 - phase0 processParticipationRecordUpdates | 6.0970 us/op | 3.2140 us/op | 1.90 |
mainnet_e58758 - phase0 afterProcessEpoch | 108.81 ms/op | 64.044 ms/op | 1.70 |
phase0 processEffectiveBalanceUpdates - 250000 normalcase | 1.4499 ms/op | 1.0185 ms/op | 1.42 |
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 | 1.8732 ms/op | 1.0837 ms/op | 1.73 |
altair processInactivityUpdates - 250000 normalcase | 31.433 ms/op | 21.041 ms/op | 1.49 |
altair processInactivityUpdates - 250000 worstcase | 36.239 ms/op | 24.106 ms/op | 1.50 |
phase0 processRegistryUpdates - 250000 normalcase | 15.174 us/op | 6.7000 us/op | 2.26 |
phase0 processRegistryUpdates - 250000 badcase_full_deposits | 492.06 us/op | 482.62 us/op | 1.02 |
phase0 processRegistryUpdates - 250000 worstcase 0.5 | 139.44 ms/op | 132.77 ms/op | 1.05 |
altair processRewardsAndPenalties - 250000 normalcase | 79.941 ms/op | 63.744 ms/op | 1.25 |
altair processRewardsAndPenalties - 250000 worstcase | 74.045 ms/op | 62.377 ms/op | 1.19 |
phase0 getAttestationDeltas - 250000 normalcase | 14.248 ms/op | 5.7884 ms/op | 2.46 |
phase0 getAttestationDeltas - 250000 worstcase | 19.376 ms/op | 5.7044 ms/op | 3.40 |
phase0 processSlashings - 250000 worstcase | 127.67 us/op | 78.709 us/op | 1.62 |
altair processSyncCommitteeUpdates - 250000 | 176.82 ms/op | 110.37 ms/op | 1.60 |
BeaconState.hashTreeRoot - No change | 648.00 ns/op | 267.00 ns/op | 2.43 |
BeaconState.hashTreeRoot - 1 full validator | 147.08 us/op | 136.80 us/op | 1.08 |
BeaconState.hashTreeRoot - 32 full validator | 1.5639 ms/op | 1.5123 ms/op | 1.03 |
BeaconState.hashTreeRoot - 512 full validator | 16.990 ms/op | 16.324 ms/op | 1.04 |
BeaconState.hashTreeRoot - 1 validator.effectiveBalance | 172.95 us/op | 144.95 us/op | 1.19 |
BeaconState.hashTreeRoot - 32 validator.effectiveBalance | 2.3315 ms/op | 2.0147 ms/op | 1.16 |
BeaconState.hashTreeRoot - 512 validator.effectiveBalance | 31.789 ms/op | 32.985 ms/op | 0.96 |
BeaconState.hashTreeRoot - 1 balances | 146.61 us/op | 143.11 us/op | 1.02 |
BeaconState.hashTreeRoot - 32 balances | 1.3115 ms/op | 1.2595 ms/op | 1.04 |
BeaconState.hashTreeRoot - 512 balances | 13.339 ms/op | 11.131 ms/op | 1.20 |
BeaconState.hashTreeRoot - 250000 balances | 236.86 ms/op | 152.33 ms/op | 1.55 |
aggregationBits - 2048 els - zipIndexesInBitList | 41.135 us/op | 19.257 us/op | 2.14 |
byteArrayEquals 32 | 78.449 ns/op | 63.656 ns/op | 1.23 |
Buffer.compare 32 | 59.286 ns/op | 38.507 ns/op | 1.54 |
byteArrayEquals 1024 | 2.1918 us/op | 1.7210 us/op | 1.27 |
Buffer.compare 1024 | 75.553 ns/op | 43.417 ns/op | 1.74 |
byteArrayEquals 16384 | 35.104 us/op | 27.511 us/op | 1.28 |
Buffer.compare 16384 | 283.90 ns/op | 229.25 ns/op | 1.24 |
byteArrayEquals 123687377 | 268.29 ms/op | 189.02 ms/op | 1.42 |
Buffer.compare 123687377 | 9.9790 ms/op | 4.8832 ms/op | 2.04 |
byteArrayEquals 32 - diff last byte | 79.850 ns/op | 58.765 ns/op | 1.36 |
Buffer.compare 32 - diff last byte | 62.062 ns/op | 39.213 ns/op | 1.58 |
byteArrayEquals 1024 - diff last byte | 2.1731 us/op | 1.7524 us/op | 1.24 |
Buffer.compare 1024 - diff last byte | 76.693 ns/op | 43.737 ns/op | 1.75 |
byteArrayEquals 16384 - diff last byte | 34.883 us/op | 27.840 us/op | 1.25 |
Buffer.compare 16384 - diff last byte | 279.42 ns/op | 220.47 ns/op | 1.27 |
byteArrayEquals 123687377 - diff last byte | 261.51 ms/op | 207.19 ms/op | 1.26 |
Buffer.compare 123687377 - diff last byte | 10.223 ms/op | 4.4224 ms/op | 2.31 |
byteArrayEquals 32 - random bytes | 7.1030 ns/op | 4.5760 ns/op | 1.55 |
Buffer.compare 32 - random bytes | 63.331 ns/op | 47.298 ns/op | 1.34 |
byteArrayEquals 1024 - random bytes | 6.2000 ns/op | 4.5920 ns/op | 1.35 |
Buffer.compare 1024 - random bytes | 62.675 ns/op | 47.624 ns/op | 1.32 |
byteArrayEquals 16384 - random bytes | 6.4490 ns/op | 4.7710 ns/op | 1.35 |
Buffer.compare 16384 - random bytes | 63.177 ns/op | 46.098 ns/op | 1.37 |
byteArrayEquals 123687377 - random bytes | 9.2900 ns/op | 7.1700 ns/op | 1.30 |
Buffer.compare 123687377 - random bytes | 74.130 ns/op | 48.150 ns/op | 1.54 |
regular array get 100000 times | 45.978 us/op | 40.895 us/op | 1.12 |
wrappedArray get 100000 times | 47.835 us/op | 41.151 us/op | 1.16 |
arrayWithProxy get 100000 times | 17.640 ms/op | 11.329 ms/op | 1.56 |
ssz.Root.equals | 57.486 ns/op | 55.401 ns/op | 1.04 |
byteArrayEquals | 58.307 ns/op | 54.005 ns/op | 1.08 |
Buffer.compare | 12.744 ns/op | 9.6400 ns/op | 1.32 |
shuffle list - 16384 els | 9.0012 ms/op | 5.6118 ms/op | 1.60 |
shuffle list - 250000 els | 133.81 ms/op | 83.585 ms/op | 1.60 |
processSlot - 1 slots | 19.833 us/op | 11.284 us/op | 1.76 |
processSlot - 32 slots | 4.7799 ms/op | 2.1886 ms/op | 2.18 |
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei | 64.710 ms/op | 47.723 ms/op | 1.36 |
getCommitteeAssignments - req 1 vs - 250000 vc | 2.7845 ms/op | 2.4390 ms/op | 1.14 |
getCommitteeAssignments - req 100 vs - 250000 vc | 4.0478 ms/op | 3.4977 ms/op | 1.16 |
getCommitteeAssignments - req 1000 vs - 250000 vc | 4.4100 ms/op | 3.8150 ms/op | 1.16 |
findModifiedValidators - 10000 modified validators | 417.82 ms/op | 288.80 ms/op | 1.45 |
findModifiedValidators - 1000 modified validators | 240.17 ms/op | 158.39 ms/op | 1.52 |
findModifiedValidators - 100 modified validators | 200.33 ms/op | 159.90 ms/op | 1.25 |
findModifiedValidators - 10 modified validators | 215.86 ms/op | 156.50 ms/op | 1.38 |
findModifiedValidators - 1 modified validators | 180.53 ms/op | 142.32 ms/op | 1.27 |
findModifiedValidators - no difference | 209.72 ms/op | 161.24 ms/op | 1.30 |
compare ViewDUs | 5.0225 s/op | 3.7193 s/op | 1.35 |
compare each validator Uint8Array | 1.2745 s/op | 1.7513 s/op | 0.73 |
compare ViewDU to Uint8Array | 1.3035 s/op | 823.04 ms/op | 1.58 |
migrate state 1000000 validators, 24 modified, 0 new | 835.36 ms/op | 671.93 ms/op | 1.24 |
migrate state 1000000 validators, 1700 modified, 1000 new | 1.2805 s/op | 896.71 ms/op | 1.43 |
migrate state 1000000 validators, 3400 modified, 2000 new | 1.4554 s/op | 1.0667 s/op | 1.36 |
migrate state 1500000 validators, 24 modified, 0 new | 838.77 ms/op | 729.76 ms/op | 1.15 |
migrate state 1500000 validators, 1700 modified, 1000 new | 1.2910 s/op | 1.0584 s/op | 1.22 |
migrate state 1500000 validators, 3400 modified, 2000 new | 1.5086 s/op | 1.2998 s/op | 1.16 |
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei | 5.3000 ns/op | 4.8300 ns/op | 1.10 |
state getBlockRootAtSlot - 250000 vs - 7PWei | 1.1129 us/op | 735.44 ns/op | 1.51 |
computeProposers - vc 250000 | 11.094 ms/op | 8.0411 ms/op | 1.38 |
computeEpochShuffling - vc 250000 | 131.49 ms/op | 88.795 ms/op | 1.48 |
getNextSyncCommittee - vc 250000 | 189.38 ms/op | 147.51 ms/op | 1.28 |
computeSigningRoot for AttestationData | 30.391 us/op | 25.499 us/op | 1.19 |
hash AttestationData serialized data then Buffer.toString(base64) | 2.4286 us/op | 1.4732 us/op | 1.65 |
toHexString serialized data | 1.4059 us/op | 929.93 ns/op | 1.51 |
Buffer.toString(base64) | 255.41 ns/op | 190.56 ns/op | 1.34 |
by benchmarkbot/action
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!! 🎸
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## unstable #6654 +/- ##
============================================
+ Coverage 61.69% 61.81% +0.12%
============================================
Files 556 556
Lines 58820 59050 +230
Branches 1887 1898 +11
============================================
+ Hits 36287 36502 +215
- Misses 22492 22505 +13
- Partials 41 43 +2 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
🎉 This PR is included in v1.18.0 🎉 |
Motivation
Improve the sim tests assertions.
Description
Steps to test or reproduce