Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

Gossamer sometimes don't start syncing when starting #3622

Closed
EclesioMeloJunior opened this issue Dec 4, 2023 · 1 comment · Fixed by #3695
Closed

Gossamer sometimes don't start syncing when starting #3622

EclesioMeloJunior opened this issue Dec 4, 2023 · 1 comment · Fixed by #3695
Assignees
@EclesioMeloJunior
Labels
P-critical this must be fixed immediately or contributors or users will be severely impacted. S-sync-westend related to particular network syncing.

Comments

@EclesioMeloJunior
Copy link
Member

Issue summary

Gossamer is failing to connect to bootnodes at startup, facing errors like:

failed to dial: failed to dial 12D3KooWAq2A7UNFS6725XFatD5QW7iYBezTLdAUx1SmRkxN79Ne: no good addresses pkg=network

failed to open connection for peer 12D3KooWByVpK92hMi9CzTjyFg9cPHDU5ariTM3EPMq9vdh5S5Po: failed to dial: context deadline exceeded pkg=network

[/ip4/34.141.212.103/tcp/30334/ws] dial tcp 34.141.212.103:30334: i/o timeout pkg=network

[/ip4/34.141.212.103/tcp/30333] dial tcp4 34.141.212.103:30333: connect: connection refused

2023-12-04T12:27:05Z WARN failed to open connection for peer 12D3KooWGi1tCpKXLMYED9y28QXLnwgD4neYb1Arqq4QpeV1Sv3K: failed to dial: failed to dial 12D3KooWGi1tCpKXLMYED9y28QXLnwgD4neYb1Arqq4QpeV1Sv3K: all dials failed

The following log stream shows the errors in ordered by time

December 04, 2023 at 08:28 (UTC-4:00) | 2023-12-04T12:28:51Z WARN failed to advertise in the DHT: failed to find any peer in table pkg=network | 08f43eb0b5f2465fa89f813d0846918f | gossamer-westend
-- | -- | -- | --
December 04, 2023 at 08:28 (UTC-4:00) | 2023-12-04T12:28:21Z WARN failed to advertise in the DHT: failed to find any peer in table pkg=network | 08f43eb0b5f2465fa89f813d0846918f | gossamer-westend
December 04, 2023 at 08:27 (UTC-4:00) | 2023-12-04T12:27:51Z WARN failed to advertise in the DHT: failed to find any peer in table pkg=network | 08f43eb0b5f2465fa89f813d0846918f | gossamer-westend
December 04, 2023 at 08:27 (UTC-4:00) | 2023-12-04T12:27:21Z WARN failed to advertise in the DHT: failed to find any peer in table pkg=network | 08f43eb0b5f2465fa89f813d0846918f | gossamer-westend
December 04, 2023 at 08:27 (UTC-4:00) | 2023-12-04T12:27:05Z WARN failed to open connection for peer 12D3KooWAq2A7UNFS6725XFatD5QW7iYBezTLdAUx1SmRkxN79Ne: failed to dial: failed to dial 12D3KooWAq2A7UNFS6725XFatD5QW7iYBezTLdAUx1SmRkxN79Ne: no good addresses pkg=network | 08f43eb0b5f2465fa89f813d0846918f | gossamer-westend
December 04, 2023 at 08:27 (UTC-4:00) | 2023-12-04T12:27:05Z WARN failed to open connection for peer 12D3KooWNg8iUqhux7X7voNU9Nty5pzehrFJwkQwg1CJnqN3CTzE: failed to dial: failed to dial 12D3KooWNg8iUqhux7X7voNU9Nty5pzehrFJwkQwg1CJnqN3CTzE: no good addresses pkg=network | 08f43eb0b5f2465fa89f813d0846918f | gossamer-westend
December 04, 2023 at 08:27 (UTC-4:00) | 2023-12-04T12:27:05Z WARN failed to open connection for peer 12D3KooWGi1tCpKXLMYED9y28QXLnwgD4neYb1Arqq4QpeV1Sv3K: failed to dial: failed to dial 12D3KooWGi1tCpKXLMYED9y28QXLnwgD4neYb1Arqq4QpeV1Sv3K: all dials failed | 08f43eb0b5f2465fa89f813d0846918f | gossamer-westend
December 04, 2023 at 08:27 (UTC-4:00) | * [/ip4/34.141.212.103/tcp/30333] dial tcp4 34.141.212.103:30333: connect: connection refused | 08f43eb0b5f2465fa89f813d0846918f | gossamer-westend
December 04, 2023 at 08:27 (UTC-4:00) | * [/ip4/34.141.212.103/tcp/30334/ws] dial tcp 34.141.212.103:30334: i/o timeout pkg=network | 08f43eb0b5f2465fa89f813d0846918f | gossamer-westend
December 04, 2023 at 08:27 (UTC-4:00) | 2023-12-04T12:27:05Z WARN failed to open connection for peer 12D3KooWByVpK92hMi9CzTjyFg9cPHDU5ariTM3EPMq9vdh5S5Po: failed to dial: context deadline exceeded pkg=network | 08f43eb0b5f2465fa89f813d0846918f | gossamer-westend
December 04, 2023 at 08:27 (UTC-4:00) | 2023-12-04T12:27:00Z WARN failed to open connection for peer 12D3KooWPVPzs42GvRBShdUMtFsk4SvnByrSdWqb6aeAAHvLMSLS: failed to dial: context deadline exceeded pkg=network | 08f43eb0b5f2465fa89f813d0846918f | gossamer-westend
December 04, 2023 at 08:26 (UTC-4:00) | 2023-12-04T12:26:55Z WARN failed to open connection for peer 12D3KooWKer94o1REDPtAhjtYR4SdLehnSrN8PEhBnZm5NBoCrMC: failed to dial: context deadline exceeded pkg=network | 08f43eb0b5f2465fa89f813d0846918f | gossamer-westend
December 04, 2023 at 08:26 (UTC-4:00) | 2023-12-04T12:26:51Z WARN failed to advertise in the DHT: failed to find any peer in table pkg=network | 08f43eb0b5f2465fa89f813d0846918f | gossamer-westend
December 04, 2023 at 08:26 (UTC-4:00) | 2023-12-04T12:26:50Z INFO started network service with supported protocols /ipfs/ping/1.0.0, /ipfs/id/1.0.0, /ipfs/id/push/1.0.0, /e143f23803ac50e8f6f8e62695d1ce9e4e1d68aa36c1cd2cfd15340213f3423e/grandpa/1, /wnd2/sync/2, /wnd2/light/2, /wnd2/block-announces/1, /wnd2/transactions/1, /wnd2/kad pkg=network | 08f43eb0b5f2465fa89f813d0846918f | gossamer-westend

December 04, 2023 at 08:28 (UTC-4:00)	2023-12-04T12:28:51Z WARN failed to advertise in the DHT: failed to find any peer in table pkg=network	[08f43eb0b5f2465fa89f813d0846918f](https://us-east-2.console.aws.amazon.com/ecs/v2/clusters/gossamer-stg/services/gossamer-westend-svc/tasks/08f43eb0b5f2465fa89f813d0846918f?region=us-east-2)	gossamer-westend
December 04, 2023 at 08:28 (UTC-4:00)	2023-12-04T12:28:21Z WARN failed to advertise in the DHT: failed to find any peer in table pkg=network	[08f43eb0b5f2465fa89f813d0846918f](https://us-east-2.console.aws.amazon.com/ecs/v2/clusters/gossamer-stg/services/gossamer-westend-svc/tasks/08f43eb0b5f2465fa89f813d0846918f?region=us-east-2)	gossamer-westend
December 04, 2023 at 08:27 (UTC-4:00)	2023-12-04T12:27:51Z WARN failed to advertise in the DHT: failed to find any peer in table pkg=network	[08f43eb0b5f2465fa89f813d0846918f](https://us-east-2.console.aws.amazon.com/ecs/v2/clusters/gossamer-stg/services/gossamer-westend-svc/tasks/08f43eb0b5f2465fa89f813d0846918f?region=us-east-2)	gossamer-westend
December 04, 2023 at 08:27 (UTC-4:00)	2023-12-04T12:27:21Z WARN failed to advertise in the DHT: failed to find any peer in table pkg=network	[08f43eb0b5f2465fa89f813d0846918f](https://us-east-2.console.aws.amazon.com/ecs/v2/clusters/gossamer-stg/services/gossamer-westend-svc/tasks/08f43eb0b5f2465fa89f813d0846918f?region=us-east-2)	gossamer-westend
December 04, 2023 at 08:27 (UTC-4:00)	2023-12-04T12:27:05Z WARN failed to open connection for peer 12D3KooWAq2A7UNFS6725XFatD5QW7iYBezTLdAUx1SmRkxN79Ne: failed to dial: failed to dial 12D3KooWAq2A7UNFS6725XFatD5QW7iYBezTLdAUx1SmRkxN79Ne: no good addresses pkg=network	[08f43eb0b5f2465fa89f813d0846918f](https://us-east-2.console.aws.amazon.com/ecs/v2/clusters/gossamer-stg/services/gossamer-westend-svc/tasks/08f43eb0b5f2465fa89f813d0846918f?region=us-east-2)	gossamer-westend
December 04, 2023 at 08:27 (UTC-4:00)	2023-12-04T12:27:05Z WARN failed to open connection for peer 12D3KooWNg8iUqhux7X7voNU9Nty5pzehrFJwkQwg1CJnqN3CTzE: failed to dial: failed to dial 12D3KooWNg8iUqhux7X7voNU9Nty5pzehrFJwkQwg1CJnqN3CTzE: no good addresses pkg=network	[08f43eb0b5f2465fa89f813d0846918f](https://us-east-2.console.aws.amazon.com/ecs/v2/clusters/gossamer-stg/services/gossamer-westend-svc/tasks/08f43eb0b5f2465fa89f813d0846918f?region=us-east-2)	gossamer-westend
December 04, 2023 at 08:27 (UTC-4:00)	2023-12-04T12:27:05Z WARN failed to open connection for peer 12D3KooWGi1tCpKXLMYED9y28QXLnwgD4neYb1Arqq4QpeV1Sv3K: failed to dial: failed to dial 12D3KooWGi1tCpKXLMYED9y28QXLnwgD4neYb1Arqq4QpeV1Sv3K: all dials failed	[08f43eb0b5f2465fa89f813d0846918f](https://us-east-2.console.aws.amazon.com/ecs/v2/clusters/gossamer-stg/services/gossamer-westend-svc/tasks/08f43eb0b5f2465fa89f813d0846918f?region=us-east-2)	gossamer-westend
December 04, 2023 at 08:27 (UTC-4:00)	* [/ip4/34.141.212.103/tcp/30333] dial tcp4 34.141.212.103:30333: connect: connection refused	[08f43eb0b5f2465fa89f813d0846918f](https://us-east-2.console.aws.amazon.com/ecs/v2/clusters/gossamer-stg/services/gossamer-westend-svc/tasks/08f43eb0b5f2465fa89f813d0846918f?region=us-east-2)	gossamer-westend
December 04, 2023 at 08:27 (UTC-4:00)	* [/ip4/34.141.212.103/tcp/30334/ws] dial tcp 34.141.212.103:30334: i/o timeout pkg=network	[08f43eb0b5f2465fa89f813d0846918f](https://us-east-2.console.aws.amazon.com/ecs/v2/clusters/gossamer-stg/services/gossamer-westend-svc/tasks/08f43eb0b5f2465fa89f813d0846918f?region=us-east-2)	gossamer-westend
December 04, 2023 at 08:27 (UTC-4:00)	2023-12-04T12:27:05Z WARN failed to open connection for peer 12D3KooWByVpK92hMi9CzTjyFg9cPHDU5ariTM3EPMq9vdh5S5Po: failed to dial: context deadline exceeded pkg=network	[08f43eb0b5f2465fa89f813d0846918f](https://us-east-2.console.aws.amazon.com/ecs/v2/clusters/gossamer-stg/services/gossamer-westend-svc/tasks/08f43eb0b5f2465fa89f813d0846918f?region=us-east-2)	gossamer-westend
December 04, 2023 at 08:27 (UTC-4:00)	2023-12-04T12:27:00Z WARN failed to open connection for peer 12D3KooWPVPzs42GvRBShdUMtFsk4SvnByrSdWqb6aeAAHvLMSLS: failed to dial: context deadline exceeded pkg=network	[08f43eb0b5f2465fa89f813d0846918f](https://us-east-2.console.aws.amazon.com/ecs/v2/clusters/gossamer-stg/services/gossamer-westend-svc/tasks/08f43eb0b5f2465fa89f813d0846918f?region=us-east-2)	gossamer-westend
December 04, 2023 at 08:26 (UTC-4:00)	2023-12-04T12:26:55Z WARN failed to open connection for peer 12D3KooWKer94o1REDPtAhjtYR4SdLehnSrN8PEhBnZm5NBoCrMC: failed to dial: context deadline exceeded pkg=network	[08f43eb0b5f2465fa89f813d0846918f](https://us-east-2.console.aws.amazon.com/ecs/v2/clusters/gossamer-stg/services/gossamer-westend-svc/tasks/08f43eb0b5f2465fa89f813d0846918f?region=us-east-2)	gossamer-westend
December 04, 2023 at 08:26 (UTC-4:00)	2023-12-04T12:26:51Z WARN failed to advertise in the DHT: failed to find any peer in table pkg=network	[08f43eb0b5f2465fa89f813d0846918f](https://us-east-2.console.aws.amazon.com/ecs/v2/clusters/gossamer-stg/services/gossamer-westend-svc/tasks/08f43eb0b5f2465fa89f813d0846918f?region=us-east-2)	gossamer-westend
December 04, 2023 at 08:26 (UTC-4:00)	2023-12-04T12:26:50Z INFO started network service with supported protocols /ipfs/ping/1.0.0, /ipfs/id/1.0.0, /ipfs/id/push/1.0.0, /e143f23803ac50e8f6f8e62695d1ce9e4e1d68aa36c1cd2cfd15340213f3423e/grandpa/1, /wnd2/sync/2, /wnd2/light/2, /wnd2/block-announces/1, /wnd2/transactions/1, /wnd2/kad pkg=network	[08f43eb0b5f2465fa89f813d0846918f](https://us-east-2.console.aws.amazon.com/ecs/v2/clusters/gossamer-stg/services/gossamer-westend-svc/tasks/08f43eb0b5f2465fa89f813d0846918f?region=us-east-2)	gossamer-westend

Other information and links
@EclesioMeloJunior EclesioMeloJunior self-assigned this Jan 12, 2024
@P1sar P1sar removed the Sync label Jan 15, 2024
@EclesioMeloJunior EclesioMeloJunior changed the title Gossamer node cannot connect to bootnodes Gossamer sometimes don't start syncing when starting Jan 15, 2024
@EclesioMeloJunior EclesioMeloJunior added P-critical this must be fixed immediately or contributors or users will be severely impacted. S-sync-westend related to particular network syncing. labels Jan 15, 2024
@EclesioMeloJunior EclesioMeloJunior mentioned this issue Jan 15, 2024
Merged
@q9f
Copy link
Contributor

q9f commented Jan 16, 2024

@P1sar we need a networking or p2p label

# for free to join this conversation on GitHub. Already have an account? # to comment
Assignees

@EclesioMeloJunior EclesioMeloJunior

Labels
P-critical this must be fixed immediately or contributors or users will be severely impacted. S-sync-westend related to particular network syncing.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix(dot/sync): execute p2p handshake when there is no target ChainSafe/gossamer
3 participants
@P1sar @EclesioMeloJunior @q9f