Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Prevent myhoard starting more than one backup stream #155

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

nicois
Copy link
Contributor

@nicois nicois commented Apr 4, 2023

Multiple backup streams add complexity and instability, increasing load on the server for negligible benefit. It is better to complete the current backup before devoting resources to beginning a second stream.

This also has the side-effect of better handling situations where one stream appears to fail, then recovers, and interferes with a new backup stream: until the first one is finished, a second one will not start, after which the first one is gone.

A "terminated" backup stream (ie: one which is completed, closed or broken) does not count towards this limit; only one which is not yet in one of these terminal states is.

@nicois nicois force-pushed the nicois/disable-multiple-backup-streams branch 2 times, most recently from 464ef49 to b99efd5 Compare April 4, 2023 04:25
Multiple backup streams add complexity and instability,
increasing load on the server for negligible benefit. It is
better to complete the current backup before devoting resources
to beginning a second stream.

This also has the side-effect of better handling situations where
one stream appears to fail, then recovers, and interferes with a
new backup stream: until the first one is finished, a second one
will not start, after which the first one is gone.
@nicois nicois force-pushed the nicois/disable-multiple-backup-streams branch from b99efd5 to 0c30e54 Compare April 4, 2023 05:38
@nicois nicois marked this pull request as ready for review April 4, 2023 07:51
@nicois nicois requested a review from a team as a code owner April 4, 2023 07:51
Copy link
Contributor

@rikonen rikonen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what the intended behavior with the streams with this change is exactly as the streams are marked as completed / closed by the controller when we have multiple concurrent streams and certain conditions are met and it looks to me like this would just not start multiple streams at all, i.e. would not create new backups. But also worth keeping in mind that in order for point-in-time recovery to work we must have the previous stream in active state and uploading binlogs until the new one is fully complete and we must not break PITR so I'm not convinced we can make any changes related to this.

@nicois
Copy link
Contributor Author

nicois commented May 4, 2023

@rikonen , when I create a PITR fork of a service in a test environment, I do not see a second stream being created on the primary service. The only activity appears to be a forced binlog flush.
Are there specific situations where a second backup stream is required, or could it be that these days a single stream is sufficient?

May 04 00:30:33 nick-farrell-test-n1017 myhoard[3169]: 2023-05-04 00:30:33,358        MainThread        Controller        INFO        Rotating binlog due to external request
May 04 00:30:33 nick-farrell-test-n1017 myhoard[3169]: 2023-05-04 00:30:33,365        MainThread        WebServer        INFO        Waiting up to 3.0 seconds for upload of 5 to complete
May 04 00:30:33 nick-farrell-test-n1017 myhoard[3169]: 2023-05-04 00:30:33,366        Thread-1        BinlogScanner        INFO        New binlog '/srv/aiven-persistent/1017-service/mysqlhome/8/binlogs/binlog.000005' (241 bytes) in 0.00 seconds, found 0 GTID ranges: []
May 04 00:30:33 nick-farrell-test-n1017 myhoard[3169]: 2023-05-04 00:30:33,824        Thread-3        BackupStream/2023-05-04T00:20:30.782913+00:00Z_bf5060b2-94bc-4693-b5b9-22e2efad652f        INFO        Successfully uploaded local binlog index 5 as remote index 4 in 0.5 seconds
May 04 00:30:33 nick-farrell-test-n1017 myhoard[3169]: 2023-05-04 00:30:33,867        MainThread        WebServer        INFO        Log 5 was backed up in 0.5 seconds

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants