Skip to content
This repository has been archived by the owner on Jan 22, 2025. It is now read-only.

Improve BPF SDK dependency caching #10434

Merged
merged 1 commit into from
Jun 8, 2020
Merged

Conversation

jackcmay
Copy link
Contributor

@jackcmay jackcmay commented Jun 5, 2020

Problem

BPF dependencies can be expensive to download/untar/clone

Summary of Changes

Improve dependency caching by:

  • Avoid copying by downloading directly to the cache
  • Reduce untar time by untaring into cache
  • Cache cloned repos

Fixes #10369

@jackcmay
Copy link
Contributor Author

jackcmay commented Jun 5, 2020

@mvines You mentioned

cloning https://github.com/solana-labs/rust-bpf-sysroot.git: uncached but also anonymous http

Can you elaborate?

@jackcmay jackcmay requested a review from mvines June 5, 2020 22:38
@mvines
Copy link
Contributor

mvines commented Jun 5, 2020

I github grants a better rate limit if you believe git clone over ssh instead of http, since it can identify you. But if we're caching the clone then this is probably a moot point

@jackcmay jackcmay merged commit 97f9b63 into solana-labs:master Jun 8, 2020
@jackcmay jackcmay deleted the cache branch June 8, 2020 16:20
svenski123 pushed a commit to svenski123/solana that referenced this pull request Jun 8, 2020
@jackcmay jackcmay added the v1.2 label Jun 11, 2020
@jackcmay
Copy link
Contributor Author

@mvines Think we should backport this to 1.1 too?

mergify bot pushed a commit that referenced this pull request Jun 11, 2020
@mvines
Copy link
Contributor

mvines commented Jun 11, 2020

Sure

@jackcmay jackcmay added the v1.1 label Jun 11, 2020
mergify bot pushed a commit that referenced this pull request Jun 11, 2020
solana-grimes pushed a commit that referenced this pull request Jun 11, 2020
jackcmay added a commit that referenced this pull request Jun 12, 2020
(cherry picked from commit 97f9b63)

Co-authored-by: Jack May <jack@solana.com>
# for free to subscribe to this conversation on GitHub. Already have an account? #.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

BPF SDK cloning of dependent repositories may be causing us to be throttled
2 participants