Add instructions to build docker image locally #262
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ds with docker container
chowbao
reviewed
Jul 1, 2024
| @@ -235,7 +235,7 @@ func AddCommonFlags(flags *pflag.FlagSet) { | |||
| flags.Bool("futurenet", false, "If set, will connect to Futurenet instead of Mainnet.") | |||
| flags.StringToStringP("extra-fields", "u", map[string]string{}, "Additional fields to append to output jsons. Used for appending metadata") | |||
| flags.Bool("captive-core", false, "If set, run captive core to retrieve data. Otherwise use TxMeta file datastore.") | |||
| flags.String("datastore-path", "sdf-ledger-close-metas/ledgers", "Datastore bucket path to read txmeta files from.") | |||
chowbao
reviewed
Jul 1, 2024
| @@ -7,7 +7,7 @@ BUILD_DATE := $(shell date -u +%FT%TZ) | |||
| ETLHASH=stellar/stellar-etl:$(shell git rev-parse --short HEAD) | |||
|
|
|||
| docker-build: | |||
| $(SUDO) docker build --pull --no-cache --label org.opencontainers.image.created="$(BUILD_DATE)" \ | |||
| $(SUDO) docker build --platform linux/amd64 --pull --no-cache --label org.opencontainers.image.created="$(BUILD_DATE)" \ | |||
There was a problem hiding this comment.
I recommend not adding --platform to the Makefile as it is used in our https://github.com/stellar/pipelines/blob/master/stellar-etl/Jenkinsfile#L44-L45
Edit: NVM it should be fine to add as it'd be a noop. For more context: we all have macbooks with m* chips so we need to force the x86 (e.g. linux/amd64) build
chowbao
approved these changes
Jul 1, 2024
5 tasks
# for free
to join this conversation on GitHub.
Already have an account?
# to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR Checklist
PR Structure
otherwise).
Thoroughness
Release planning
semver, and I've changed the name of the BRANCH to release/_ , feature/_ or patch/* .
What
This PR adds following changes:
Add instructions in README to build docker image locally and run export commands locally. This is helpful for local testing.
Update docker-build command in Makefile to support Apple M1 chip. See https://stackoverflow.com/questions/66662820/m1-docker-preview-and-keycloak-images-platform-linux-amd64-does-not-match-th
Update datastore path from
sdf-ledger-close-metas/ledgerstosdf-ledger-close-meta/ledgers(Note the removed (s) in metas)Context from slack:
"We recently did a full history backfill for our datalake in GCS in bucket sdf-ledger-close-meta/ledgers
sdf-ledger-close-metas/ledgers is the old bucket. This was used as our frontfilling bucket but should be updated and changed in stellar-etl to the new full history data lake as the default bucket"
Why
Covered in what section
Known limitations