Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

[nv16] flexible bundle loading #8666

Merged
merged 5 commits into from
May 17, 2022
Merged

Conversation

vyzo
Copy link
Contributor

@vyzo vyzo commented May 17, 2022

On top of #8658.

Adds flexibility for bundle fetching, which can use an env var or explicit URL to avoid downloading from github.

cc @travisperson

@vyzo vyzo requested a review from arajasek May 17, 2022 17:41
@vyzo vyzo requested a review from a team as a code owner May 17, 2022 17:41
@vyzo vyzo changed the title flexible bundle loading [nv16] flexible bundle loading May 17, 2022
Copy link
Member

@raulk raulk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having to modify your bundle.toml to specify an envvar which in turn is going to override any other setting seems like a circuitous way of doing things. It also defeats the one of the purposes, which is being able to check out the Lotus repo as-is (or download a pre-built binary) and make it use locally built bundles instead without any changes in sources.

@@ -6,7 +6,7 @@ The bundles are specified in build/bundles.toml using the following syntax:
```toml
[[bundles]]
version = X # actors version
release = tag # release gag
release = tag # release tag
Copy link
Contributor

@arajasek arajasek May 17, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add examples of how to use path / envvar / url here

// this is a local bundle, specified by an env var to load from the filesystem
path := os.Getenv(bd.EnvVar)
if path == "" {
return xerrors.Errorf("bundle envvar is empty: %s", bd.EnvVar)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could just fallthrough in this case, but if the EnvVar field is set, but there's no value provided user is probably just doing something wrong. Up to you.

Copy link
Member

@raulk raulk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion is to remove EnvVar and instead use predefined env var templates:

LOTUS_ACTORS_[ActorVersion]_BUNDLE_PATH

e.g.

LOTUS_ACTORS_V8_BUNDLE_PATH=<path>

@vyzo
Copy link
Contributor Author

vyzo commented May 17, 2022

ok fine.

Copy link
Member

@raulk raulk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having to modify your bundle.toml to specify an envvar which in turn is going to override any other setting seems like a circuitous way of doing things. It also defeats the one of the purposes, which is being able to check out the Lotus repo as-is (or download a pre-built binary) and make it use locally built bundles instead without any changes in sources.

@vyzo vyzo requested a review from raulk May 17, 2022 18:16
Copy link
Member

@raulk raulk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All of this is growing organically at a very rapid rate and I think we're going to have to clean up some of this later, but it's not a big deal because this is all "private" and not exposed over APIs or intended for users to depend on.

build/bundle.go Outdated
Comment on lines 24 to 26
Path map[string]string
// URL is the (optional) bundle URL; takes precdence over github release
URL string
// CHecksum is the bundle sha256 checksume in hex, when specifying a URL.
Checksum string
URL map[string]BundleURL
Copy link
Member

@raulk raulk May 17, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd pluralize these field names.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe, I kind of prefer singular here; but i don't feel strongly either way.

@vyzo
Copy link
Contributor Author

vyzo commented May 17, 2022

Yeah, let's get to its final "works really well" phase, and we can do some cleanup.

@arajasek
Copy link
Contributor

@raulk Strong agree, there is definitely a round of cleanup that needs to happen -- ultimately these bundles are gonna be "never changing" things (except for devs and network upgrades), so something much more static will work.

@vyzo vyzo merged commit 86e1144 into feat/nv16-dev-bundle May 17, 2022
@vyzo vyzo deleted the feat/nv16-flexibundle branch May 17, 2022 19:12
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants