Progress bar for restore

[vshalts]

Backup have nice progress bar, but restore is not.
In real life I expect to do restore not that often (now I play with restic and with restore a bit too often).
But it will be useful to have progress for restore too (for big repo I would like to see some estimation about how much time it will take).

[fd0]

Thanks for raising this issue. It's not a high priority right now, because (as you already said) restoring is important, but not done so often.

[fd0]

Still relevant, will be addressed by #2003

[fortinmike]

@fd0 Has development of this feature stalled? Seems like #2003 will never see the light of day, at least in its current form.

IMHO having restore progress (especially for big restores) is important to convey the fact that restore is working (did not freeze) and to provide at least a basic idea of how much work remains to be done to restore something. At a minimum, it would be nice to display a "spinner" (indeterminate progress) indicator to at least indicate that the restore is being performed and is not stalled.

When performing time-critical restores (e.g. a last-resort restore to bring a service back online as quickly as possible), not being able to give a time estimate to whoever you report to could be quite problematic.

[peixotorms]

+1 restoring multiple GB of data without any sort of feedback is not very friendly.

[rawtaz]

For anyone looking for this; While you can't see how much is still left to be done during restore, you can definitely see whether or not restic is working on the restore by inspecting the CPU usage and foremost the network traffic usage that it's causing.

[peixotorms]

I am using watch -n 10 du -sh to roughly check the restore size, but a progress bar (or a way to pipe the data transfer through pv) would be most welcome. Thanks

[patrickuhlmann]

1. you run the restic restore operation in one terminal/session
2. you open another termin/session and run the command watch -n 10 du -sh /your/restore/folder
   In the second terminal you will see the size of the resored folder and it will refresh every 10 seconds. That way you can see how much data was already restored and it should climb over the time.

[fermulator]

@fd0 , is there any particular "requirements" or maintainer "suggestions for implementation"? -- if someone from the community would like to pick it up I mean, it would be good to have a list of requirements and implementation preferences maybe?

[MichaelEischer]

As suggested in #2438, the --verbose flag could be used to select how much information should be printed, just like with the backup command.

[darth-veitcher]

Is there still no progress reporting availability on restoring from a backup? Feels like a fairly critical requirement.

[rawtaz]

@darth-veitcher Correct, there currently is not. Please refer to simply checking how much data was restored in your filesystem.

[darth-veitcher]

How can we tell the total that is going to be restored to calculate % and progress? Is there a way to do this based on the repository itself. So manually query repo for total size and then query how much transferred to the local filesystem?

Also what's the level of effort required to achieve this inside the tool? Really feels like a big missing feature to still have around after this time.

[rawtaz]

How can we tell the total that is going to be restored to calculate % and progress?

Probably using the stats command on the snapshot you intend to restore, which can tell you how much data there is in the given snapshot.

Also what's the level of effort required to achieve this inside the tool?

Looking at #2003 which was start but then closed it seems that it isn't very simple or straight forward to just add this.

Really feels like a big missing feature to still have around after this time.

Sure, it would be nice to have a fancy %. But if you consider how many other things there are to attend, this is low priority. Pretty much every issue/PR/problem/etc is by those who it affects considered a priority, while if you look at the bigger picture it's not that simple to just implement it.

[patrickuhlmann]

I agree that it is inconvenient and I did have hanging restore processes (problems with OneDrive). I also understand that it is hard to do estimations of remaining time. On the other hand it is one area where restic is lagging behind other solutions like borg and it is an important part of the user experience. Also I think restore is done relatively often because every serious individual/organisation will verify that restore still works regularly.

Would it be a possibility to show the % of blocks restored vs. the total number of blocks to be read? This seems like a simple solution based on information that most likely is already available and gives a rough idea of the current state.

[darth-veitcher]

Out of interest (and without looking at the code too deeply) isn't it a case of hooking into this for each loop of nodes and then printing to stdout?

internal/restorer/restorer.go

Line 60 seems to build a total number of nodes and then we iterate through, write to the target and verify the resulting output.

This section builds a list of nodes to restore and defines their source and target.

restic/internal/restorer/restorer.go

Lines 60 to 75 in bf9c877

	for _, node := range tree.Nodes {
	// ensure that the node name does not contain anything that refers to a
	// top-level directory.
	nodeName := filepath.Base(filepath.Join(string(filepath.Separator), node.Name))
	if nodeName != node.Name {
	debug.Log("node %q has invalid name %q", node.Name, nodeName)
	err := res.Error(location, errors.Errorf("invalid child node name %s", node.Name))
	if err != nil {
	return hasRestored, err
	}
	continue
	}
	nodeTarget := filepath.Join(target, nodeName)
	nodeLocation := filepath.Join(location, nodeName)

And then we return hasRestored at the end.

restic/internal/restorer/restorer.go

Line 154 in bf9c877

return hasRestored, nil

tree.Nodes is a slice of Nodes

restic/internal/restic/node.go

Line 57 in bf9c877

type Nodes []*Node

Maybe I'm missing something though. Happy to be told otherwise 🤷

[pabs3]

There have already been two pull requests for this feature (#671 #2003) but they never got merged. Probably the way to go is read through the discussions on them, then read through the patches, then rebase them on git master, then address the problems discussed with them.

[asm0dey]

I'm restoring a 200 GB backup right now, and even du is slow on it after 116 of them were restored. Please add something at least fundamental

[MichaelEischer]

Implemented in #3991