Skip to content

Delete nonreduced inputs #157

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Merged
merged 4 commits into from
Oct 25, 2023
Merged

Delete nonreduced inputs #157

merged 4 commits into from
Oct 25, 2023

Conversation

maflcko
Copy link
Contributor

@maflcko maflcko commented Oct 23, 2023

As per the usual process to avoid wasted CI resources and timeouts when it runs on large and presumed irrelevant inputs.

Previous: #119

To "reproduce"

Install a fresh VM, as explained in the bash script's doc, and run it:

apt update && apt install curl -y
curl -L -O https://raw.githubusercontent.com/bitcoin-core/bitcoin-maintainer-tools/main/delete_nonreduced_fuzz_inputs.sh
bash delete_nonreduced_fuzz_inputs.sh

To "test"

  • Keep an eye on coverage stats, to ensure it doesn't drop
  • Re-run the script, to ensure it is "reproducible" to some extent
  • Anything else you think is important to test or review

@maflcko
Copy link
Contributor Author

maflcko commented Oct 23, 2023

Did a second run. Some stats:

  • 400k deleted files (this pull):
# git diff --stat origin/main..01a9287310b6c2e6344e28c4ba655d179142f719 | tail -1
 446405 files changed, 5974234 deletions(-)
  • Same on the second run:
# git diff --stat origin/main..HEAD | tail -1
 444894 files changed, 5404772 deletions(-)
  • Cross-diff of 24k files:
# git diff --stat 01a9287310b6c2e6344e28c4ba655d179142f719..HEAD | tail -1
 23461 files changed, 50266 insertions(+), 4085 deletions(-)

@maflcko
Copy link
Contributor Author

maflcko commented Oct 23, 2023

lint fail is expected (obviously)

cc @murchandamus @dergoegge

@maflcko
Copy link
Contributor Author

maflcko commented Oct 24, 2023

Looks like utxo_total_supply is the reason for the 2h timeout. I've removed the large fuzz inputs in this folder for now.

@maflcko
Copy link
Contributor Author

maflcko commented Oct 24, 2023

Looks like the CI passed after 43minutes now: https://cirrus-ci.com/task/4832424811560960?logs=ci#L4022

@dergoegge
Copy link
Member

Looks like utxo_total_supply is the reason for the 2h timeout. I've removed the large fuzz inputs in this folder for now.

We could add options for that target either to oss-fuzz or in the target itself (e.g. if (buffer.size() > N) return;) to avoid large inputs in the future.

@maflcko
Copy link
Contributor Author

maflcko commented Oct 24, 2023

Let's create a separate tracking issue for this target?

@maflcko
Copy link
Contributor Author

maflcko commented Oct 24, 2023

@dergoegge
Copy link
Member

ACK 428a2e7

root@0e503e9c42d6:/qa-assets# git diff --stat origin/main..HEAD | tail -1
 445073 files changed, 5399978 deletions(-)

@fanquake fanquake merged commit 4ae1613 into bitcoin-core:main Oct 25, 2023
@dergoegge dergoegge mentioned this pull request Mar 18, 2024
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants