Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

test: improve sim tests for some remaining feedback #6654

Merged
merged 5 commits into from
Apr 12, 2024

Conversation

nazarhussain
Copy link
Contributor

Motivation

Improve the sim tests assertions.

Description

  • Split the web3.js plugins structure
  • Verify the blobs as well with the count.

Steps to test or reproduce

  • Run all tests

@nazarhussain nazarhussain requested a review from a team as a code owner April 9, 2024 14:22
@nazarhussain nazarhussain self-assigned this Apr 9, 2024
Copy link
Contributor

github-actions bot commented Apr 9, 2024

⚠️ Performance Alert ⚠️

Possible performance regression was detected for some benchmarks.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold.

Benchmark suite Current: d1fc151 Previous: 669239b Ratio
phase0 getAttestationDeltas - 250000 worstcase 19.376 ms/op 5.7044 ms/op 3.40
Full benchmark results
Benchmark suite Current: d1fc151 Previous: 669239b Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 875.54 us/op 1.1323 ms/op 0.77
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 107.61 us/op 111.90 us/op 0.96
BLS verify - blst-native 1.3494 ms/op 1.1196 ms/op 1.21
BLS verifyMultipleSignatures 3 - blst-native 2.8196 ms/op 2.3493 ms/op 1.20
BLS verifyMultipleSignatures 8 - blst-native 6.2012 ms/op 5.2425 ms/op 1.18
BLS verifyMultipleSignatures 32 - blst-native 23.484 ms/op 19.424 ms/op 1.21
BLS verifyMultipleSignatures 64 - blst-native 46.187 ms/op 38.217 ms/op 1.21
BLS verifyMultipleSignatures 128 - blst-native 90.492 ms/op 75.469 ms/op 1.20
BLS deserializing 10000 signatures 950.91 ms/op 823.72 ms/op 1.15
BLS deserializing 100000 signatures 9.2696 s/op 8.4494 s/op 1.10
BLS verifyMultipleSignatures - same message - 3 - blst-native 1.3972 ms/op 1.1357 ms/op 1.23
BLS verifyMultipleSignatures - same message - 8 - blst-native 1.6057 ms/op 1.2905 ms/op 1.24
BLS verifyMultipleSignatures - same message - 32 - blst-native 2.5016 ms/op 2.0201 ms/op 1.24
BLS verifyMultipleSignatures - same message - 64 - blst-native 3.5995 ms/op 3.8050 ms/op 0.95
BLS verifyMultipleSignatures - same message - 128 - blst-native 5.6984 ms/op 4.8925 ms/op 1.16
BLS aggregatePubkeys 32 - blst-native 28.652 us/op 23.030 us/op 1.24
BLS aggregatePubkeys 128 - blst-native 108.98 us/op 88.903 us/op 1.23
notSeenSlots=1 numMissedVotes=1 numBadVotes=10 70.034 ms/op 48.941 ms/op 1.43
notSeenSlots=1 numMissedVotes=0 numBadVotes=4 72.819 ms/op 47.780 ms/op 1.52
notSeenSlots=2 numMissedVotes=1 numBadVotes=10 55.921 ms/op 30.884 ms/op 1.81
getSlashingsAndExits - default max 201.98 us/op 159.62 us/op 1.27
getSlashingsAndExits - 2k 593.95 us/op 464.24 us/op 1.28
proposeBlockBody type=full, size=empty 6.9657 ms/op 5.2791 ms/op 1.32
isKnown best case - 1 super set check 569.00 ns/op 235.00 ns/op 2.42
isKnown normal case - 2 super set checks 476.00 ns/op 233.00 ns/op 2.04
isKnown worse case - 16 super set checks 463.00 ns/op 222.00 ns/op 2.09
InMemoryCheckpointStateCache - add get delete 7.7350 us/op 3.8380 us/op 2.02
validate api signedAggregateAndProof - struct 2.9380 ms/op 2.4351 ms/op 1.21
validate gossip signedAggregateAndProof - struct 2.8784 ms/op 2.3955 ms/op 1.20
validate gossip attestation - vc 640000 1.4002 ms/op 1.1466 ms/op 1.22
batch validate gossip attestation - vc 640000 - chunk 32 169.75 us/op 141.45 us/op 1.20
batch validate gossip attestation - vc 640000 - chunk 64 146.87 us/op 129.61 us/op 1.13
batch validate gossip attestation - vc 640000 - chunk 128 140.92 us/op 122.57 us/op 1.15
batch validate gossip attestation - vc 640000 - chunk 256 135.23 us/op 114.46 us/op 1.18
pickEth1Vote - no votes 1.3557 ms/op 920.86 us/op 1.47
pickEth1Vote - max votes 8.2638 ms/op 13.692 ms/op 0.60
pickEth1Vote - Eth1Data hashTreeRoot value x2048 16.332 ms/op 20.103 ms/op 0.81
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 25.521 ms/op 28.073 ms/op 0.91
pickEth1Vote - Eth1Data fastSerialize value x2048 809.48 us/op 521.40 us/op 1.55
pickEth1Vote - Eth1Data fastSerialize tree x2048 4.6036 ms/op 7.8683 ms/op 0.59
bytes32 toHexString 557.00 ns/op 382.00 ns/op 1.46
bytes32 Buffer.toString(hex) 312.00 ns/op 201.00 ns/op 1.55
bytes32 Buffer.toString(hex) from Uint8Array 529.00 ns/op 404.00 ns/op 1.31
bytes32 Buffer.toString(hex) + 0x 310.00 ns/op 222.00 ns/op 1.40
Object access 1 prop 0.17600 ns/op 0.13000 ns/op 1.35
Map access 1 prop 0.15600 ns/op 0.10000 ns/op 1.56
Object get x1000 7.5010 ns/op 4.7520 ns/op 1.58
Map get x1000 0.82500 ns/op 0.71400 ns/op 1.16
Object set x1000 56.435 ns/op 29.365 ns/op 1.92
Map set x1000 47.809 ns/op 22.370 ns/op 2.14
Return object 10000 times 0.25030 ns/op 0.21760 ns/op 1.15
Throw Error 10000 times 4.0204 us/op 2.7258 us/op 1.47
fastMsgIdFn sha256 / 200 bytes 3.3960 us/op 1.9280 us/op 1.76
fastMsgIdFn h32 xxhash / 200 bytes 352.00 ns/op 257.00 ns/op 1.37
fastMsgIdFn h64 xxhash / 200 bytes 397.00 ns/op 261.00 ns/op 1.52
fastMsgIdFn sha256 / 1000 bytes 11.691 us/op 6.0260 us/op 1.94
fastMsgIdFn h32 xxhash / 1000 bytes 458.00 ns/op 361.00 ns/op 1.27
fastMsgIdFn h64 xxhash / 1000 bytes 464.00 ns/op 343.00 ns/op 1.35
fastMsgIdFn sha256 / 10000 bytes 104.18 us/op 51.002 us/op 2.04
fastMsgIdFn h32 xxhash / 10000 bytes 2.0090 us/op 1.7560 us/op 1.14
fastMsgIdFn h64 xxhash / 10000 bytes 1.4120 us/op 1.1420 us/op 1.24
send data - 1000 256B messages 20.564 ms/op 12.427 ms/op 1.65
send data - 1000 512B messages 28.624 ms/op 19.675 ms/op 1.45
send data - 1000 1024B messages 53.731 ms/op 22.871 ms/op 2.35
send data - 1000 1200B messages 43.062 ms/op 30.836 ms/op 1.40
send data - 1000 2048B messages 52.224 ms/op 41.642 ms/op 1.25
send data - 1000 4096B messages 48.384 ms/op 35.701 ms/op 1.36
send data - 1000 16384B messages 114.78 ms/op 103.06 ms/op 1.11
send data - 1000 65536B messages 486.11 ms/op 393.54 ms/op 1.24
enrSubnets - fastDeserialize 64 bits 1.5300 us/op 870.00 ns/op 1.76
enrSubnets - ssz BitVector 64 bits 540.00 ns/op 322.00 ns/op 1.68
enrSubnets - fastDeserialize 4 bits 258.00 ns/op 128.00 ns/op 2.02
enrSubnets - ssz BitVector 4 bits 534.00 ns/op 345.00 ns/op 1.55
prioritizePeers score -10:0 att 32-0.1 sync 2-0 239.59 us/op 139.42 us/op 1.72
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 335.01 us/op 179.13 us/op 1.87
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 416.53 us/op 287.25 us/op 1.45
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 683.27 us/op 427.23 us/op 1.60
prioritizePeers score 0:0 att 64-1 sync 4-1 880.30 us/op 511.67 us/op 1.72
array of 16000 items push then shift 1.8187 us/op 1.5020 us/op 1.21
LinkedList of 16000 items push then shift 9.8970 ns/op 6.2450 ns/op 1.58
array of 16000 items push then pop 110.53 ns/op 110.52 ns/op 1.00
LinkedList of 16000 items push then pop 10.177 ns/op 7.4380 ns/op 1.37
array of 24000 items push then shift 2.7080 us/op 1.8335 us/op 1.48
LinkedList of 24000 items push then shift 9.9800 ns/op 7.7000 ns/op 1.30
array of 24000 items push then pop 186.14 ns/op 144.05 ns/op 1.29
LinkedList of 24000 items push then pop 9.6460 ns/op 5.9510 ns/op 1.62
intersect bitArray bitLen 8 5.9380 ns/op 4.8970 ns/op 1.21
intersect array and set length 8 80.791 ns/op 49.848 ns/op 1.62
intersect bitArray bitLen 128 35.987 ns/op 29.251 ns/op 1.23
intersect array and set length 128 1.0670 us/op 713.42 ns/op 1.50
bitArray.getTrueBitIndexes() bitLen 128 1.8840 us/op 1.2970 us/op 1.45
bitArray.getTrueBitIndexes() bitLen 248 2.9470 us/op 2.2290 us/op 1.32
bitArray.getTrueBitIndexes() bitLen 512 6.3870 us/op 3.7470 us/op 1.70
Buffer.concat 32 items 1.1500 us/op 919.00 ns/op 1.25
Uint8Array.set 32 items 2.4800 us/op 2.4550 us/op 1.01
Set add up to 64 items then delete first 5.1612 us/op 1.9929 us/op 2.59
OrderedSet add up to 64 items then delete first 7.1038 us/op 2.8534 us/op 2.49
Set add up to 64 items then delete last 5.4597 us/op 2.0530 us/op 2.66
OrderedSet add up to 64 items then delete last 6.2923 us/op 3.0679 us/op 2.05
Set add up to 64 items then delete middle 5.2789 us/op 2.0841 us/op 2.53
OrderedSet add up to 64 items then delete middle 8.2187 us/op 5.1237 us/op 1.60
Set add up to 128 items then delete first 10.367 us/op 4.7850 us/op 2.17
OrderedSet add up to 128 items then delete first 13.682 us/op 7.0097 us/op 1.95
Set add up to 128 items then delete last 9.4476 us/op 3.8744 us/op 2.44
OrderedSet add up to 128 items then delete last 13.486 us/op 5.8344 us/op 2.31
Set add up to 128 items then delete middle 9.6241 us/op 4.0111 us/op 2.40
OrderedSet add up to 128 items then delete middle 18.676 us/op 10.897 us/op 1.71
Set add up to 256 items then delete first 20.343 us/op 8.0310 us/op 2.53
OrderedSet add up to 256 items then delete first 30.444 us/op 14.054 us/op 2.17
Set add up to 256 items then delete last 21.941 us/op 9.4283 us/op 2.33
OrderedSet add up to 256 items then delete last 27.160 us/op 11.886 us/op 2.29
Set add up to 256 items then delete middle 21.314 us/op 9.4149 us/op 2.26
OrderedSet add up to 256 items then delete middle 52.059 us/op 35.413 us/op 1.47
transfer serialized Status (84 B) 2.0000 us/op 1.4800 us/op 1.35
copy serialized Status (84 B) 1.5440 us/op 1.0410 us/op 1.48
transfer serialized SignedVoluntaryExit (112 B) 2.2780 us/op 1.5620 us/op 1.46
copy serialized SignedVoluntaryExit (112 B) 1.6390 us/op 1.1130 us/op 1.47
transfer serialized ProposerSlashing (416 B) 3.3890 us/op 1.9260 us/op 1.76
copy serialized ProposerSlashing (416 B) 3.2940 us/op 2.9560 us/op 1.11
transfer serialized Attestation (485 B) 3.5960 us/op 2.9380 us/op 1.22
copy serialized Attestation (485 B) 3.2740 us/op 2.6380 us/op 1.24
transfer serialized AttesterSlashing (33232 B) 3.0110 us/op 2.4970 us/op 1.21
copy serialized AttesterSlashing (33232 B) 8.1140 us/op 8.6090 us/op 0.94
transfer serialized Small SignedBeaconBlock (128000 B) 3.2070 us/op 2.9360 us/op 1.09
copy serialized Small SignedBeaconBlock (128000 B) 26.277 us/op 13.610 us/op 1.93
transfer serialized Avg SignedBeaconBlock (200000 B) 3.6710 us/op 2.6510 us/op 1.38
copy serialized Avg SignedBeaconBlock (200000 B) 40.701 us/op 15.155 us/op 2.69
transfer serialized BlobsSidecar (524380 B) 3.9220 us/op 2.5370 us/op 1.55
copy serialized BlobsSidecar (524380 B) 105.76 us/op 138.36 us/op 0.76
transfer serialized Big SignedBeaconBlock (1000000 B) 4.6360 us/op 3.1880 us/op 1.45
copy serialized Big SignedBeaconBlock (1000000 B) 199.65 us/op 168.36 us/op 1.19
pass gossip attestations to forkchoice per slot 4.5922 ms/op 2.7351 ms/op 1.68
forkChoice updateHead vc 100000 bc 64 eq 0 704.42 us/op 489.79 us/op 1.44
forkChoice updateHead vc 600000 bc 64 eq 0 5.7267 ms/op 2.7861 ms/op 2.06
forkChoice updateHead vc 1000000 bc 64 eq 0 7.3945 ms/op 4.7562 ms/op 1.55
forkChoice updateHead vc 600000 bc 320 eq 0 4.3306 ms/op 2.7645 ms/op 1.57
forkChoice updateHead vc 600000 bc 1200 eq 0 4.5015 ms/op 2.9176 ms/op 1.54
forkChoice updateHead vc 600000 bc 7200 eq 0 5.9987 ms/op 4.0037 ms/op 1.50
forkChoice updateHead vc 600000 bc 64 eq 1000 11.601 ms/op 10.121 ms/op 1.15
forkChoice updateHead vc 600000 bc 64 eq 10000 12.691 ms/op 11.143 ms/op 1.14
forkChoice updateHead vc 600000 bc 64 eq 300000 17.615 ms/op 12.090 ms/op 1.46
computeDeltas 500000 validators 300 proto nodes 7.1578 ms/op 3.4742 ms/op 2.06
computeDeltas 500000 validators 1200 proto nodes 7.1279 ms/op 3.4384 ms/op 2.07
computeDeltas 500000 validators 7200 proto nodes 6.8696 ms/op 3.3030 ms/op 2.08
computeDeltas 750000 validators 300 proto nodes 10.067 ms/op 5.1525 ms/op 1.95
computeDeltas 750000 validators 1200 proto nodes 10.357 ms/op 5.2449 ms/op 1.97
computeDeltas 750000 validators 7200 proto nodes 10.173 ms/op 5.0659 ms/op 2.01
computeDeltas 1400000 validators 300 proto nodes 19.518 ms/op 10.141 ms/op 1.92
computeDeltas 1400000 validators 1200 proto nodes 19.669 ms/op 9.8342 ms/op 2.00
computeDeltas 1400000 validators 7200 proto nodes 19.362 ms/op 9.6649 ms/op 2.00
computeDeltas 2100000 validators 300 proto nodes 27.428 ms/op 15.607 ms/op 1.76
computeDeltas 2100000 validators 1200 proto nodes 29.735 ms/op 15.311 ms/op 1.94
computeDeltas 2100000 validators 7200 proto nodes 27.840 ms/op 14.403 ms/op 1.93
altair processAttestation - 250000 vs - 7PWei normalcase 2.3572 ms/op 1.6522 ms/op 1.43
altair processAttestation - 250000 vs - 7PWei worstcase 3.3307 ms/op 2.9627 ms/op 1.12
altair processAttestation - setStatus - 1/6 committees join 195.69 us/op 120.58 us/op 1.62
altair processAttestation - setStatus - 1/3 committees join 360.13 us/op 241.47 us/op 1.49
altair processAttestation - setStatus - 1/2 committees join 487.94 us/op 288.28 us/op 1.69
altair processAttestation - setStatus - 2/3 committees join 651.50 us/op 378.03 us/op 1.72
altair processAttestation - setStatus - 4/5 committees join 819.19 us/op 561.54 us/op 1.46
altair processAttestation - setStatus - 100% committees join 987.22 us/op 594.66 us/op 1.66
altair processBlock - 250000 vs - 7PWei normalcase 8.0171 ms/op 7.6770 ms/op 1.04
altair processBlock - 250000 vs - 7PWei normalcase hashState 35.951 ms/op 34.930 ms/op 1.03
altair processBlock - 250000 vs - 7PWei worstcase 45.264 ms/op 36.133 ms/op 1.25
altair processBlock - 250000 vs - 7PWei worstcase hashState 95.812 ms/op 95.826 ms/op 1.00
phase0 processBlock - 250000 vs - 7PWei normalcase 2.9811 ms/op 2.1381 ms/op 1.39
phase0 processBlock - 250000 vs - 7PWei worstcase 30.600 ms/op 28.775 ms/op 1.06
altair processEth1Data - 250000 vs - 7PWei normalcase 576.66 us/op 280.08 us/op 2.06
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 7.7150 us/op 14.373 us/op 0.54
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 68.847 us/op 32.972 us/op 2.09
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 24.318 us/op 17.727 us/op 1.37
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 11.533 us/op 9.0110 us/op 1.28
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 219.85 us/op 148.76 us/op 1.48
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 1.5667 ms/op 1.0563 ms/op 1.48
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 2.5206 ms/op 1.8290 ms/op 1.38
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 2.2495 ms/op 1.3441 ms/op 1.67
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 3.5738 ms/op 2.6521 ms/op 1.35
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 2.9447 ms/op 1.8722 ms/op 1.57
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 5.7831 ms/op 4.4778 ms/op 1.29
Tree 40 250000 create 419.00 ms/op 346.30 ms/op 1.21
Tree 40 250000 get(125000) 217.82 ns/op 123.45 ns/op 1.76
Tree 40 250000 set(125000) 1.2120 us/op 855.09 ns/op 1.42
Tree 40 250000 toArray() 22.080 ms/op 23.259 ms/op 0.95
Tree 40 250000 iterate all - toArray() + loop 23.737 ms/op 22.786 ms/op 1.04
Tree 40 250000 iterate all - get(i) 75.726 ms/op 50.373 ms/op 1.50
MutableVector 250000 create 19.335 ms/op 7.3393 ms/op 2.63
MutableVector 250000 get(125000) 7.0290 ns/op 5.6410 ns/op 1.25
MutableVector 250000 set(125000) 299.41 ns/op 219.81 ns/op 1.36
MutableVector 250000 toArray() 3.9607 ms/op 2.3133 ms/op 1.71
MutableVector 250000 iterate all - toArray() + loop 3.9429 ms/op 2.5731 ms/op 1.53
MutableVector 250000 iterate all - get(i) 1.5877 ms/op 1.3498 ms/op 1.18
Array 250000 create 3.5099 ms/op 2.3406 ms/op 1.50
Array 250000 clone - spread 1.4063 ms/op 1.3174 ms/op 1.07
Array 250000 get(125000) 1.1750 ns/op 1.0400 ns/op 1.13
Array 250000 set(125000) 4.4860 ns/op 1.2480 ns/op 3.59
Array 250000 iterate all - loop 169.48 us/op 158.06 us/op 1.07
effectiveBalanceIncrements clone Uint8Array 300000 38.545 us/op 19.311 us/op 2.00
effectiveBalanceIncrements clone MutableVector 300000 407.00 ns/op 397.00 ns/op 1.03
effectiveBalanceIncrements rw all Uint8Array 300000 200.95 us/op 186.25 us/op 1.08
effectiveBalanceIncrements rw all MutableVector 300000 96.194 ms/op 71.802 ms/op 1.34
phase0 afterProcessEpoch - 250000 vs - 7PWei 123.47 ms/op 82.578 ms/op 1.50
phase0 beforeProcessEpoch - 250000 vs - 7PWei 56.127 ms/op 61.178 ms/op 0.92
altair processEpoch - mainnet_e81889 547.40 ms/op 382.65 ms/op 1.43
mainnet_e81889 - altair beforeProcessEpoch 87.998 ms/op 77.872 ms/op 1.13
mainnet_e81889 - altair processJustificationAndFinalization 17.227 us/op 19.237 us/op 0.90
mainnet_e81889 - altair processInactivityUpdates 7.6905 ms/op 5.1091 ms/op 1.51
mainnet_e81889 - altair processRewardsAndPenalties 63.875 ms/op 73.555 ms/op 0.87
mainnet_e81889 - altair processRegistryUpdates 2.4760 us/op 2.7590 us/op 0.90
mainnet_e81889 - altair processSlashings 620.00 ns/op 692.00 ns/op 0.90
mainnet_e81889 - altair processEth1DataReset 593.00 ns/op 670.00 ns/op 0.89
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.7063 ms/op 1.5013 ms/op 1.14
mainnet_e81889 - altair processSlashingsReset 4.5900 us/op 3.2230 us/op 1.42
mainnet_e81889 - altair processRandaoMixesReset 9.6090 us/op 4.0420 us/op 2.38
mainnet_e81889 - altair processHistoricalRootsUpdate 686.00 ns/op 606.00 ns/op 1.13
mainnet_e81889 - altair processParticipationFlagUpdates 2.3830 us/op 1.2010 us/op 1.98
mainnet_e81889 - altair processSyncCommitteeUpdates 768.00 ns/op 588.00 ns/op 1.31
mainnet_e81889 - altair afterProcessEpoch 128.10 ms/op 84.536 ms/op 1.52
capella processEpoch - mainnet_e217614 2.2150 s/op 1.8646 s/op 1.19
mainnet_e217614 - capella beforeProcessEpoch 530.36 ms/op 408.19 ms/op 1.30
mainnet_e217614 - capella processJustificationAndFinalization 17.929 us/op 8.2570 us/op 2.17
mainnet_e217614 - capella processInactivityUpdates 22.082 ms/op 18.469 ms/op 1.20
mainnet_e217614 - capella processRewardsAndPenalties 673.67 ms/op 419.50 ms/op 1.61
mainnet_e217614 - capella processRegistryUpdates 38.116 us/op 23.545 us/op 1.62
mainnet_e217614 - capella processSlashings 1.1770 us/op 438.00 ns/op 2.69
mainnet_e217614 - capella processEth1DataReset 1.1840 us/op 296.00 ns/op 4.00
mainnet_e217614 - capella processEffectiveBalanceUpdates 4.7465 ms/op 3.1207 ms/op 1.52
mainnet_e217614 - capella processSlashingsReset 7.9920 us/op 2.0430 us/op 3.91
mainnet_e217614 - capella processRandaoMixesReset 6.7370 us/op 3.0940 us/op 2.18
mainnet_e217614 - capella processHistoricalRootsUpdate 1.5530 us/op 454.00 ns/op 3.42
mainnet_e217614 - capella processParticipationFlagUpdates 4.4120 us/op 1.2140 us/op 3.63
mainnet_e217614 - capella afterProcessEpoch 398.19 ms/op 202.73 ms/op 1.96
phase0 processEpoch - mainnet_e58758 562.63 ms/op 405.16 ms/op 1.39
mainnet_e58758 - phase0 beforeProcessEpoch 181.83 ms/op 105.55 ms/op 1.72
mainnet_e58758 - phase0 processJustificationAndFinalization 18.873 us/op 9.5230 us/op 1.98
mainnet_e58758 - phase0 processRewardsAndPenalties 84.162 ms/op 58.536 ms/op 1.44
mainnet_e58758 - phase0 processRegistryUpdates 13.481 us/op 6.5720 us/op 2.05
mainnet_e58758 - phase0 processSlashings 601.00 ns/op 359.00 ns/op 1.67
mainnet_e58758 - phase0 processEth1DataReset 715.00 ns/op 342.00 ns/op 2.09
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.3999 ms/op 1.5273 ms/op 0.92
mainnet_e58758 - phase0 processSlashingsReset 4.1710 us/op 3.1190 us/op 1.34
mainnet_e58758 - phase0 processRandaoMixesReset 4.9910 us/op 2.6350 us/op 1.89
mainnet_e58758 - phase0 processHistoricalRootsUpdate 804.00 ns/op 493.00 ns/op 1.63
mainnet_e58758 - phase0 processParticipationRecordUpdates 6.0970 us/op 3.2140 us/op 1.90
mainnet_e58758 - phase0 afterProcessEpoch 108.81 ms/op 64.044 ms/op 1.70
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.4499 ms/op 1.0185 ms/op 1.42
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.8732 ms/op 1.0837 ms/op 1.73
altair processInactivityUpdates - 250000 normalcase 31.433 ms/op 21.041 ms/op 1.49
altair processInactivityUpdates - 250000 worstcase 36.239 ms/op 24.106 ms/op 1.50
phase0 processRegistryUpdates - 250000 normalcase 15.174 us/op 6.7000 us/op 2.26
phase0 processRegistryUpdates - 250000 badcase_full_deposits 492.06 us/op 482.62 us/op 1.02
phase0 processRegistryUpdates - 250000 worstcase 0.5 139.44 ms/op 132.77 ms/op 1.05
altair processRewardsAndPenalties - 250000 normalcase 79.941 ms/op 63.744 ms/op 1.25
altair processRewardsAndPenalties - 250000 worstcase 74.045 ms/op 62.377 ms/op 1.19
phase0 getAttestationDeltas - 250000 normalcase 14.248 ms/op 5.7884 ms/op 2.46
phase0 getAttestationDeltas - 250000 worstcase 19.376 ms/op 5.7044 ms/op 3.40
phase0 processSlashings - 250000 worstcase 127.67 us/op 78.709 us/op 1.62
altair processSyncCommitteeUpdates - 250000 176.82 ms/op 110.37 ms/op 1.60
BeaconState.hashTreeRoot - No change 648.00 ns/op 267.00 ns/op 2.43
BeaconState.hashTreeRoot - 1 full validator 147.08 us/op 136.80 us/op 1.08
BeaconState.hashTreeRoot - 32 full validator 1.5639 ms/op 1.5123 ms/op 1.03
BeaconState.hashTreeRoot - 512 full validator 16.990 ms/op 16.324 ms/op 1.04
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 172.95 us/op 144.95 us/op 1.19
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 2.3315 ms/op 2.0147 ms/op 1.16
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 31.789 ms/op 32.985 ms/op 0.96
BeaconState.hashTreeRoot - 1 balances 146.61 us/op 143.11 us/op 1.02
BeaconState.hashTreeRoot - 32 balances 1.3115 ms/op 1.2595 ms/op 1.04
BeaconState.hashTreeRoot - 512 balances 13.339 ms/op 11.131 ms/op 1.20
BeaconState.hashTreeRoot - 250000 balances 236.86 ms/op 152.33 ms/op 1.55
aggregationBits - 2048 els - zipIndexesInBitList 41.135 us/op 19.257 us/op 2.14
byteArrayEquals 32 78.449 ns/op 63.656 ns/op 1.23
Buffer.compare 32 59.286 ns/op 38.507 ns/op 1.54
byteArrayEquals 1024 2.1918 us/op 1.7210 us/op 1.27
Buffer.compare 1024 75.553 ns/op 43.417 ns/op 1.74
byteArrayEquals 16384 35.104 us/op 27.511 us/op 1.28
Buffer.compare 16384 283.90 ns/op 229.25 ns/op 1.24
byteArrayEquals 123687377 268.29 ms/op 189.02 ms/op 1.42
Buffer.compare 123687377 9.9790 ms/op 4.8832 ms/op 2.04
byteArrayEquals 32 - diff last byte 79.850 ns/op 58.765 ns/op 1.36
Buffer.compare 32 - diff last byte 62.062 ns/op 39.213 ns/op 1.58
byteArrayEquals 1024 - diff last byte 2.1731 us/op 1.7524 us/op 1.24
Buffer.compare 1024 - diff last byte 76.693 ns/op 43.737 ns/op 1.75
byteArrayEquals 16384 - diff last byte 34.883 us/op 27.840 us/op 1.25
Buffer.compare 16384 - diff last byte 279.42 ns/op 220.47 ns/op 1.27
byteArrayEquals 123687377 - diff last byte 261.51 ms/op 207.19 ms/op 1.26
Buffer.compare 123687377 - diff last byte 10.223 ms/op 4.4224 ms/op 2.31
byteArrayEquals 32 - random bytes 7.1030 ns/op 4.5760 ns/op 1.55
Buffer.compare 32 - random bytes 63.331 ns/op 47.298 ns/op 1.34
byteArrayEquals 1024 - random bytes 6.2000 ns/op 4.5920 ns/op 1.35
Buffer.compare 1024 - random bytes 62.675 ns/op 47.624 ns/op 1.32
byteArrayEquals 16384 - random bytes 6.4490 ns/op 4.7710 ns/op 1.35
Buffer.compare 16384 - random bytes 63.177 ns/op 46.098 ns/op 1.37
byteArrayEquals 123687377 - random bytes 9.2900 ns/op 7.1700 ns/op 1.30
Buffer.compare 123687377 - random bytes 74.130 ns/op 48.150 ns/op 1.54
regular array get 100000 times 45.978 us/op 40.895 us/op 1.12
wrappedArray get 100000 times 47.835 us/op 41.151 us/op 1.16
arrayWithProxy get 100000 times 17.640 ms/op 11.329 ms/op 1.56
ssz.Root.equals 57.486 ns/op 55.401 ns/op 1.04
byteArrayEquals 58.307 ns/op 54.005 ns/op 1.08
Buffer.compare 12.744 ns/op 9.6400 ns/op 1.32
shuffle list - 16384 els 9.0012 ms/op 5.6118 ms/op 1.60
shuffle list - 250000 els 133.81 ms/op 83.585 ms/op 1.60
processSlot - 1 slots 19.833 us/op 11.284 us/op 1.76
processSlot - 32 slots 4.7799 ms/op 2.1886 ms/op 2.18
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 64.710 ms/op 47.723 ms/op 1.36
getCommitteeAssignments - req 1 vs - 250000 vc 2.7845 ms/op 2.4390 ms/op 1.14
getCommitteeAssignments - req 100 vs - 250000 vc 4.0478 ms/op 3.4977 ms/op 1.16
getCommitteeAssignments - req 1000 vs - 250000 vc 4.4100 ms/op 3.8150 ms/op 1.16
findModifiedValidators - 10000 modified validators 417.82 ms/op 288.80 ms/op 1.45
findModifiedValidators - 1000 modified validators 240.17 ms/op 158.39 ms/op 1.52
findModifiedValidators - 100 modified validators 200.33 ms/op 159.90 ms/op 1.25
findModifiedValidators - 10 modified validators 215.86 ms/op 156.50 ms/op 1.38
findModifiedValidators - 1 modified validators 180.53 ms/op 142.32 ms/op 1.27
findModifiedValidators - no difference 209.72 ms/op 161.24 ms/op 1.30
compare ViewDUs 5.0225 s/op 3.7193 s/op 1.35
compare each validator Uint8Array 1.2745 s/op 1.7513 s/op 0.73
compare ViewDU to Uint8Array 1.3035 s/op 823.04 ms/op 1.58
migrate state 1000000 validators, 24 modified, 0 new 835.36 ms/op 671.93 ms/op 1.24
migrate state 1000000 validators, 1700 modified, 1000 new 1.2805 s/op 896.71 ms/op 1.43
migrate state 1000000 validators, 3400 modified, 2000 new 1.4554 s/op 1.0667 s/op 1.36
migrate state 1500000 validators, 24 modified, 0 new 838.77 ms/op 729.76 ms/op 1.15
migrate state 1500000 validators, 1700 modified, 1000 new 1.2910 s/op 1.0584 s/op 1.22
migrate state 1500000 validators, 3400 modified, 2000 new 1.5086 s/op 1.2998 s/op 1.16
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 5.3000 ns/op 4.8300 ns/op 1.10
state getBlockRootAtSlot - 250000 vs - 7PWei 1.1129 us/op 735.44 ns/op 1.51
computeProposers - vc 250000 11.094 ms/op 8.0411 ms/op 1.38
computeEpochShuffling - vc 250000 131.49 ms/op 88.795 ms/op 1.48
getNextSyncCommittee - vc 250000 189.38 ms/op 147.51 ms/op 1.28
computeSigningRoot for AttestationData 30.391 us/op 25.499 us/op 1.19
hash AttestationData serialized data then Buffer.toString(base64) 2.4286 us/op 1.4732 us/op 1.65
toHexString serialized data 1.4059 us/op 929.93 ns/op 1.51
Buffer.toString(base64) 255.41 ns/op 190.56 ns/op 1.34

by benchmarkbot/action

matthewkeil
matthewkeil previously approved these changes Apr 10, 2024
Copy link
Member

@matthewkeil matthewkeil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!! 🎸

Copy link

codecov bot commented Apr 12, 2024

Codecov Report

Merging #6654 (f75c533) into unstable (6bd9af9) will increase coverage by 0.12%.
Report is 7 commits behind head on unstable.
The diff coverage is n/a.

Additional details and impacted files
@@             Coverage Diff              @@
##           unstable    #6654      +/-   ##
============================================
+ Coverage     61.69%   61.81%   +0.12%     
============================================
  Files           556      556              
  Lines         58820    59050     +230     
  Branches       1887     1898      +11     
============================================
+ Hits          36287    36502     +215     
- Misses        22492    22505      +13     
- Partials         41       43       +2     

Copy link
Member

@nflaig nflaig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@nazarhussain nazarhussain merged commit ae984f0 into unstable Apr 12, 2024
19 of 20 checks passed
@nazarhussain nazarhussain deleted the nh/sim-feedback branch April 12, 2024 13:37
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.18.0 🎉

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants