cmd/puppeth: integrate blockscout #18261

rjl493456442 · 2018-12-09T08:16:46Z

This PR integrate an open-source blockchain explorer blockscout into the puppeth.

Now puppeth can provide explorer service for PoA based chain and PoW based chain

5chdn · 2019-02-07T13:40:25Z

That's pretty cool

rocammo · 2019-02-27T13:03:16Z

It works pretty well and it has a very good interface. Good job!

hadv · 2019-02-28T08:41:19Z

@rjl493456442 I want to build my own docker but facing this error. Do you have any idea? Thank you!

* Getting absinthe_phoenix (git@github.com:ayrat555/absinthe_phoenix.git)
fatal: cannot run ssh: No such file or directory
fatal: unable to fork
** (Mix) Command "git --git-dir=.git fetch --force --quiet --progress" failed

rjl493456442 · 2019-02-28T08:43:36Z

@hadv It is an upstream issue. I've already opened an issue here blockscout/blockscout#1478

hadv · 2019-03-01T02:49:19Z

It works pretty well and it has a very good interface. Good job!

yeah, great! Except I cannot run the data node with --ws options. the data node only can sync the data if I disable the web-socket. Otherwise, the data node can only sync some blocks then drop all peers. Any idea, @rjl493456442 ? Thanks!

rocammo · 2019-03-01T07:54:34Z

It works pretty well and it has a very good interface. Good job!

yeah, great! Except I cannot run the data node with --ws options. the data node only can sync the data if I disable the web-socket. Otherwise, the data node can only sync some blocks then drop all peers. Any idea, @rjl493456442 ? Thanks!

I don't know if there is any difference in whether the network is PoA or PoW. In my case, my implementation has been carried out in a network configured as PoW for a demo, and the truth is that I have not had any problems.

hadv · 2019-03-01T08:02:36Z

I don't know if there is any difference in whether the network is PoA or PoW. In my case, my implementation has been carried out in a network configured as PoW for a demo, and the truth is that I have not had any problems.

I'm running a private PoA network with some divert then I need to build my own blockscout docker image and I have to remove compile's --warnings-as-errors flag to build the blockscout project also by the way.

rocammo · 2019-03-01T08:19:35Z

I don't know if there is any difference in whether the network is PoA or PoW. In my case, my implementation has been carried out in a network configured as PoW for a demo, and the truth is that I have not had any problems.

I'm running a private PoA network with some divert then I need to build my own blockscout docker image and I have to remove compile's --warnings-as-errors flag to build the blockscout project also by the way.

I didn't have any compilation problems when I added it just two days ago and I also compiled it from source, so it's strange that you're having it. Could you please provide the explorer container log by docker logs <containerId> or the compilation errors that you're having?

hadv · 2019-03-01T08:49:00Z

I didn't have any compilation problems when I added it just two days ago and I also compiled it from source, so it's strange that you're having it. Could you please provide the explorer container log by docker logs <containerId> or the compilation errors that you're having?

I cannot build the docker image with warning as error option

https://github.com/puppeth/blockscout/blob/9af22a89de29c094acc2349cd5b368a5564fe2ee/apps/block_scout_web/mix.exs#L142

hadv · 2019-03-01T10:26:24Z

@rocammo can you try to run with contract metadata and validator? I enable them but the explorer shutdown out of the blue 😢

if System.get_env("METADATA_CONTRACT") && System.get_env("VALIDATORS_CONTRACT") do
  config :explorer, Explorer.Validator.MetadataRetriever,
    metadata_contract_address: System.get_env("METADATA_CONTRACT"),
    validators_contract_address: System.get_env("VALIDATORS_CONTRACT")

  config :explorer, Explorer.Validator.MetadataProcessor, enabled: true
else
  config :explorer, Explorer.Validator.MetadataProcessor, enabled: false
end

rjl493456442 · 2019-03-01T10:36:33Z

Btw there is a known issue in the current PR found by @mirooy and @Kimjingoo that now the geth node version in blockscout image is different with the wallet image or some other components. So that there could have a consensus error now if you try to deploy a smart contract.

Have to wait the upstream to fix the build issue and then rebuild the puppeth/blockscout.

hadv · 2019-03-02T03:22:11Z

Btw there is a known issue in the current PR found by @mirooy and @Kimjingoo that now the geth node version in blockscout image is different with the wallet image or some other components. So that there could have a consensus error now if you try to deploy a smart contract.

Have to wait the upstream to fix the build issue and then rebuild the puppeth/blockscout.

is this the error?


** (FunctionClauseError) no function clause matching in Logger.level_to_number/1
    (logger) lib/logger.ex:536: Logger.level_to_number(:prod)
    (logger) lib/logger.ex:533: Logger.compare_levels/2
    (logger_file_backend) lib/logger_file_backend.ex:33: LoggerFileBackend.handle_event/2
    (stdlib) gen_event.erl:577: :gen_event.server_update/4
    (stdlib) gen_event.erl:559: :gen_event.server_notify/4
    (stdlib) gen_event.erl:300: :gen_event.handle_msg/6
    (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
Last message: {:debug, #PID<0.499.0>, {Logger, ["QUERY", 32, "OK", "", [32, "db", 61, '1.5', 109, 115], [], [], 10, "commit", 32, "[]"], {{2019, 3, 2}, {3, 11, 47, 758}}, [pid: #PID<0.1086.0>, ansi_color: :magenta, line: 50, function: "log/3", module: Ecto.LogEntry, file: "lib/ecto/log_entry.ex", application: :ecto, transaction_id: 1991, import_id: 1959, block_number: 135, fetcher: :block_realtime]}}
State: %{format: [:time, " ", :metadata, "[", :level, "] ", :message, "\n"], inode: nil, io_device: nil, level: :prod, metadata: [], metadata_filter: [fetcher: :failed_created_addresses], name: :failed_contract_creations, path: "/opt/app/logs/prod/indexer/failed_contract_creations.log", rotate: nil}
2019-03-02T03:11:47.786 application=indexer [info] refetch from consensus was found on block (60). A reorg initiated.
2019-03-02T03:11:47.800 [error] GenServer #PID<0.1186.0> terminating
** (stop) {:EXIT, {:function_clause, [{Logger, :level_to_number, [:prod], [file: 'lib/logger.ex', line: 536]}, {Logger, :compare_levels, 2, [file: 'lib/logger.ex', line: 533]}, {LoggerFileBackend, :handle_event, 2, [file: 'lib/logger_file_backend.ex', line: 33]}, {:gen_event, :server_update, 4, [file: 'gen_event.erl', line: 577]}, {:gen_event, :server_notify, 4, [file: 'gen_event.erl', line: 559]}, {:gen_event, :handle_msg, 6, [file: 'gen_event.erl', line: 300]}, {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 249]}]}}
Last message: {:gen_event_EXIT, {LoggerFileBackend, :failed_contract_creations}, {:EXIT, {:function_clause, [{Logger, :level_to_number, [:prod], [file: 'lib/logger.ex', line: 536]}, {Logger, :compare_levels, 2, [file: 'lib/logger.ex', line: 533]}, {LoggerFileBackend, :handle_event, 2, [file: 'lib/logger_file_backend.ex', line: 33]}, {:gen_event, :server_update, 4, [file: 'gen_event.erl', line: 577]}, {:gen_event, :server_notify, 4, [file: 'gen_event.erl', line: 559]}, {:gen_event, :handle_msg, 6, [file: 'gen_event.erl', line: 300]}, {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 249]}]}}}
State: {Logger, {LoggerFileBackend, :failed_contract_creations}}

hadv · 2019-03-02T03:48:36Z

I don't know if there is any difference in whether the network is PoA or PoW. In my case, my implementation has been carried out in a network configured as PoW for a demo, and the truth is that I have not had any problems.

I'm running a private PoA network with some divert then I need to build my own blockscout docker image and I have to remove compile's --warnings-as-errors flag to build the blockscout project also by the way.

I didn't have any compilation problems when I added it just two days ago and I also compiled it from source, so it's strange that you're having it. Could you please provide the explorer container log by docker logs <containerId> or the compilation errors that you're having?

This is exactly the problem I have and they provide a work around for it

blockscout/blockscout#1391

hadv · 2019-03-02T10:11:59Z

It works pretty well and it has a very good interface. Good job!

yeah, great! Except I cannot run the data node with --ws options. the data node only can sync the data if I disable the web-socket. Otherwise, the data node can only sync some blocks then drop all peers. Any idea, @rjl493456442 ? Thanks!

the docker container shutdown suddenly if connect to local rpc/ws endpoint. If I connect to other endpoint it's okay. Seems problem due to running every things on the same docker container

hadv · 2019-03-03T02:40:23Z

cmd/puppeth/module_explorer.go

-	echo '(cd ../etherchain-light && npm start &)'                      >> explorer.sh && \
-	echo 'exec /parity/parity --chain=/chain.json --port={{.NodePort}} --tracing=on --fat-db=on --pruning=archive' >> explorer.sh
+  echo 'geth --cache 512 init /genesis.json' > explorer.sh && \
+	echo $'exec geth --networkid {{.NetworkID}} --syncmode "full"  --port {{.NodePort}} --bootnodes {{.Bootnodes}} --ethstats \'{{.Ethstats}}\' --cache=512 --rpc --rpccorsdomain "*" --rpcvhosts "*" --ws --wsorigins "*" &' >> explorer.sh && \


we should enable debug rpc api to reading smart contract & internal transaction

Yes, I added it in my local and haven't pushed it. Thanks for catching!

hadv · 2019-03-03T02:41:20Z

cmd/puppeth/module_explorer.go

-	echo '(cd ../etherchain-light && npm start &)'                      >> explorer.sh && \
-	echo 'exec /parity/parity --chain=/chain.json --port={{.NodePort}} --tracing=on --fat-db=on --pruning=archive' >> explorer.sh
+  echo 'geth --cache 512 init /genesis.json' > explorer.sh && \
+	echo $'exec geth --networkid {{.NetworkID}} --syncmode "full"  --port {{.NodePort}} --bootnodes {{.Bootnodes}} --ethstats \'{{.Ethstats}}\' --cache=512 --rpc --rpccorsdomain "*" --rpcvhosts "*" --ws --wsorigins "*" &' >> explorer.sh && \


I prefer to allow user to specify which geth node they want to connect to other than always using localhost

My original idea is: this embedded geth node is only available for blockscout. Probably if a user wants to connect to a geth node, then they can deploy a new node somehow.

Yes, it's a good idea. by the way, when I running embedded geth node the docker container suddenly shutdown. I might be wrong somewhere but still dont know the reason but when connect to outside node, it worked properly without any problem.

hadv · 2019-03-05T07:41:46Z

cmd/puppeth/wizard_explorer.go

+		infos.dbdir = w.readString()
+	} else {
+		fmt.Printf("Where should postgres data be stored on the remote machine? (default = %s)\n", infos.dbdir)
+		infos.dbdir = w.readDefaultString(infos.datadir)


typo seems

Suggested change

infos.dbdir = w.readDefaultString(infos.datadir)

infos.dbdir = w.readDefaultString(infos.dbdir)

rjl493456442 · 2019-03-06T02:54:42Z

Fix the blockscout docker image and now it can work in the clique mode. Yeah

rjl493456442 · 2019-04-12T10:05:04Z

Now the upstream project is much stabler. Appreciate for the support from @ayrat555 and I think this PR is review ready :))

kant777 · 2023-11-11T13:39:48Z

How to run blockscout after this integration?

rjl493456442 force-pushed the integrate-blockscout branch 6 times, most recently from fdd0b10 to adeb320 Compare December 13, 2018 06:36

rjl493456442 changed the title ~~[WIP] cmd/puppeth: integrate blockscout~~ cmd/puppeth: integrate blockscout Dec 13, 2018

rjl493456442 requested a review from karalabe December 13, 2018 06:38

rjl493456442 mentioned this pull request Jan 16, 2019

Hi, I try to run blockscout in docker container, the error log occurs: blockscout/blockscout#1301

Closed

hadv suggested changes Mar 3, 2019

View reviewed changes

hadv reviewed Mar 5, 2019

View reviewed changes

rjl493456442 force-pushed the integrate-blockscout branch from adeb320 to 575bdeb Compare March 5, 2019 07:50

rjl493456442 added this to the 1.9.0 milestone Apr 4, 2019

rjl493456442 force-pushed the integrate-blockscout branch from 575bdeb to cf226af Compare April 11, 2019 03:17

adamschmideg assigned karalabe Apr 11, 2019

rjl493456442 added the pr:review label Apr 12, 2019

rjl493456442 force-pushed the integrate-blockscout branch from cf226af to 34fd1c8 Compare July 8, 2019 15:58

rjl493456442 and others added 6 commits July 8, 2019 19:14

cmd/puppeth: integrate blockscout

88b7dcb

cmd/puppeth: expose debug namespace for blockscout

4fe2574

cmd/puppeth: fix dbdir

fd6c04d

cmd/puppeth: run explorer in archive mode

433104a

cmd/puppeth: ensure node is synced

0298349

cmd/puppeth: fix explorer docker alignment + drop unneeded exec

54e3937

karalabe force-pushed the integrate-blockscout branch from 34fd1c8 to 54e3937 Compare July 8, 2019 16:14

karalabe added 2 commits July 8, 2019 19:59

cmd/puppeth: polish up config saving and reloading

6ccf329

cmd/puppeth: check both web and p2p port for explorer service

34bbfd8

karalabe merged commit 16e3136 into ethereum:master Jul 8, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cmd/puppeth: integrate blockscout #18261

cmd/puppeth: integrate blockscout #18261

rjl493456442 commented Dec 9, 2018 •

edited

Loading

5chdn commented Feb 7, 2019

rocammo commented Feb 27, 2019

hadv commented Feb 28, 2019

rjl493456442 commented Feb 28, 2019

hadv commented Mar 1, 2019 •

edited

Loading

rocammo commented Mar 1, 2019

hadv commented Mar 1, 2019 •

edited

Loading

rocammo commented Mar 1, 2019

hadv commented Mar 1, 2019

hadv commented Mar 1, 2019

rjl493456442 commented Mar 1, 2019 •

edited

Loading

hadv commented Mar 2, 2019

hadv commented Mar 2, 2019

hadv commented Mar 2, 2019

hadv Mar 3, 2019

rjl493456442 Mar 3, 2019

hadv Mar 3, 2019

rjl493456442 Mar 3, 2019

hadv Mar 3, 2019

hadv Mar 5, 2019

rjl493456442 commented Mar 6, 2019

rjl493456442 commented Apr 12, 2019

kant777 commented Nov 11, 2023

	infos.dbdir = w.readDefaultString(infos.datadir)
	infos.dbdir = w.readDefaultString(infos.dbdir)

cmd/puppeth: integrate blockscout #18261

cmd/puppeth: integrate blockscout #18261

Conversation

rjl493456442 commented Dec 9, 2018 • edited Loading

5chdn commented Feb 7, 2019

rocammo commented Feb 27, 2019

hadv commented Feb 28, 2019

rjl493456442 commented Feb 28, 2019

hadv commented Mar 1, 2019 • edited Loading

rocammo commented Mar 1, 2019

hadv commented Mar 1, 2019 • edited Loading

rocammo commented Mar 1, 2019

hadv commented Mar 1, 2019

hadv commented Mar 1, 2019

rjl493456442 commented Mar 1, 2019 • edited Loading

hadv commented Mar 2, 2019

hadv commented Mar 2, 2019

hadv commented Mar 2, 2019

hadv Mar 3, 2019

Choose a reason for hiding this comment

rjl493456442 Mar 3, 2019

Choose a reason for hiding this comment

hadv Mar 3, 2019

Choose a reason for hiding this comment

rjl493456442 Mar 3, 2019

Choose a reason for hiding this comment

hadv Mar 3, 2019

Choose a reason for hiding this comment

hadv Mar 5, 2019

Choose a reason for hiding this comment

rjl493456442 commented Mar 6, 2019

rjl493456442 commented Apr 12, 2019

kant777 commented Nov 11, 2023

rjl493456442 commented Dec 9, 2018 •

edited

Loading

hadv commented Mar 1, 2019 •

edited

Loading

hadv commented Mar 1, 2019 •

edited

Loading

rjl493456442 commented Mar 1, 2019 •

edited

Loading