Skip to content
This repository has been archived by the owner on Mar 2, 2022. It is now read-only.

Reduce Restic Progress FPS #82

Merged
merged 1 commit into from
Mar 30, 2021
Merged

Reduce Restic Progress FPS #82

merged 1 commit into from
Mar 30, 2021

Conversation

cimnine
Copy link
Contributor

@cimnine cimnine commented Mar 29, 2021

By defining the environment variable RESTIC_PROGRESS_FPS it is possible to reduce restic's output.

The reason to introduce this change is that we suspect that the buffer of the scanner is eventually filling up. This PR does not fix that permanently, but it reduces the output of restic dramatically, which should at least should delay the occurence of such errors until a more permament is found.
Also, wrestic does not need the progress to be reported every 1/60th of a second.

The behavior now – as implemented – is that if RESTIC_PROGRESS_FPS is defined in the environment variables that are passed to wrestic, the given value will be used. Otherwise, wrestic will set the value to about one progress report line per minute (i.e. 1.0f/60.0f per Second).

Originally discussed in #79 (comment).

@cimnine cimnine added the enhancement New feature or request label Mar 29, 2021
@cimnine cimnine requested a review from Kidswiss March 29, 2021 12:57
@cimnine cimnine self-assigned this Mar 29, 2021
Copy link
Contributor

@Kidswiss Kidswiss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@cimnine cimnine force-pushed the ReduceResticOutputFrequency branch from e0513e1 to 13128a4 Compare March 30, 2021 07:19
@cimnine
Copy link
Contributor Author

cimnine commented Mar 30, 2021

@Kidswiss can you have another look.

I've changed this function from:

func (c *Command) setResticProgressFPSIfNotDefined(givenEnv []string) []string {
	newEnv := make([]string, len(givenEnv))
	copy(newEnv, givenEnv)

	for i := range newEnv {
		if strings.HasPrefix("RESTIC_PROGRESS_FPS=", newEnv[i]) {
			return newEnv
		}
	}

	const frequency = 1.0 / 60.0
	c.cmdLogger.Info("Defining RESTIC_PROGRESS_FPS", "frequency", frequency)
	return append(newEnv, fmt.Sprintf("RESTIC_PROGRESS_FPS=%f", frequency))
}

to

func (c *Command) setResticProgressFPSIfNotDefined(givenEnv []string) []string {
	for _, envVar := range givenEnv {
		if strings.HasPrefix("RESTIC_PROGRESS_FPS=", envVar) {
			return givenEnv
		}
	}

	const frequency = 1.0 / 60.0
	c.cmdLogger.Info("Defining RESTIC_PROGRESS_FPS", "frequency", frequency)
	return append(givenEnv, fmt.Sprintf("RESTIC_PROGRESS_FPS=%f", frequency))
}

I.e. I removed the copy because I don't think it is needed: When we don't change the array, it's definitely not needed. And when we append, we'll get a new reference anyway.

@cimnine cimnine requested a review from Kidswiss March 30, 2021 07:22
Copy link
Contributor

@Kidswiss Kidswiss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, it's not needed to create a copy. LGTM

@cimnine cimnine merged commit 7cbd01a into master Mar 30, 2021
@cimnine cimnine deleted the ReduceResticOutputFrequency branch March 30, 2021 07:48
# for free to subscribe to this conversation on GitHub. Already have an account? #.
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants