Restore¶
$ backy2 restore --help
usage: backy2 restore [-h] [-s] [-f] [-c CONTINUE_FROM] version_uid target
positional arguments:
version_uid
target Source (url-like, e.g. file:///dev/sda or
rbd://pool/imagename)
optional arguments:
-h, --help show this help message and exit
-s, --sparse Faster. Restore only existing blocks (works only with
file- and rbd-restore, not with lvm)
-f, --force Force overwrite of existing files/devices/images
-c CONTINUE_FROM, --continue-from CONTINUE_FROM
Continue from this block (only use this for partially
failed restores!)
Full restore¶
A full restore happens either into a file (i.e. an image file), to a device (e.g. /dev/hda) or to a ceph/rbd-volume.
The target is specified by the URI schema. Examples:
$ backy2 restore <version_uid> file:///var/lib/vms/myvm.img
$ backy2 restore -f <version_uid> file:///dev/hdm
$ backy2 restore <version_uid> rbd://pool/myvm_restore
If the target already exists as
it is a device file
the rbd volume exists
the image file exists
you need to --force the restore. Without --force backy2 will break with
an error and give you a hint:
backy2 restore 90fbbeb6-1fbe-11e7-9f25-a44e314f9270 file://tmp/T
INFO: $ /home/dk/develop/backy2/env/bin/backy2 restore 90fbbeb6-1fbe-11e7-9f25-a44e314f9270 file://tmp/T
ERROR: Target already exists: file://tmp/T
Error opening restore target. You must force the restore.
Note
When restoring to a ceph/rbd volume, backy2 will create this rbd volume for you if it does not exist.
If the restore-target is full of 0x00 bytes, you can use the -s or --sparse
option for faster restores. With -s backy2 will not write (i.e. skip) empty
blocks or blocks that contain only 0x00 bytes.
Usually you can use -s if your restore target is
a new/non-existing ceph/rbd volume
a new/non-existing image file (backy2 will then create a sparse file)
Caution
If you use -s on existing images, devices or files, restore-blocks which
do not exist or contain only 0x00 bytes will not be written, so whatever
random data was in there before the restore will remain.
Live-mount with FUSE¶
FUSE is a filesystem in userspace. backy2 can create a mountpoint for its
backups. Example:
$ backy2 fuse /mnt
All existing backups are then available in /mnt:
$ tree /mnt
mnt
├── by_name
│ └── sometest1
│ ├── 0c44841a-8d47-11ea-8b2d-3dc6919c2aca -> ../../by_version_uid/0c44841a-8d47-11ea-8b2d-3dc6919c2aca
│ ├── 60ae794e-8d46-11ea-8b2d-3dc6919c2aca -> ../../by_version_uid/60ae794e-8d46-11ea-8b2d-3dc6919c2aca
│ └── 9d8cfe80-8d46-11ea-8b2d-3dc6919c2aca -> ../../by_version_uid/9d8cfe80-8d46-11ea-8b2d-3dc6919c2aca
└── by_version_uid
├── 0c44841a-8d47-11ea-8b2d-3dc6919c2aca
│ ├── data
│ ├── expire
│ ├── name
│ ├── snapshot_name
│ ├── tags
│ └── valid
├── 60ae794e-8d46-11ea-8b2d-3dc6919c2aca
│ ├── data
│ ├── expire
│ ├── name
│ ├── snapshot_name
│ ├── tags
│ └── valid
└── 9d8cfe80-8d46-11ea-8b2d-3dc6919c2aca
├── data
├── expire
├── name
├── snapshot_name
├── tags
└── valid
$ ls -l /mnt/by_version_uid/0c44841a-8d47-11ea-8b2d-3dc6919c2aca/data
-rw------- 1 root root 293529518 Mai 3 14:04 /mnt/by_version_uid/0c44841a-8d47-11ea-8b2d-3dc6919c2aca/data
Here’s what the directories/files contain:
by_name: Unique names for the backups by_name/<name>: Symlinks to the version directories by_version_uid: All existing version_uids. The direcories’ date is the version’s date by_version_uid/<uid>/data: The binary data by_version_uid/<uid>/expire: The expiration date (example: 2020-04-20T01:02:03) by_version_uid/<uid>/name: The backup’s name by_version_uid/<uid>/snapshot_name: The snapshot name by_version_uid/<uid>/tags: Comma separated list of given tags for the version by_version_uid/<uid>/valid: Only exists (and contains ‘valid’) when the version is valid by_version_uid/<uid>/invalid: Only exists (and contains ‘invalid’) when the version is invalid by_version_uid/<uid>/protected: Only exists (and contains ‘protected’) when the version is protected
If the data contains partitions, you may make them accessible by creating a loop device and
then partprobe’ing the partitions:
# losetup /dev/loop100 /mnt/by_version_uid/0c44841a-8d47-11ea-8b2d-3dc6919c2aca/data
# partprobe /dev/loop100
# mount /dev/loop100p1 /mnt2
Note
All data files are made writeable by backy2. Writes will be buffered
into a temporary file (the file will be created in the configured cachedir
in backy2.cfg). Please make sure the file has enough place to grow
for your changes.
No writes will ever change the version. Writes only exist while backy2 fuse
is running.
If the data contains a filesystem, you may directly mount it:
# mount /mnt/by_version_uid/0c44841a-8d47-11ea-8b2d-3dc6919c2aca/data /mnt2
To finish the fuse mount, press ctrl+c on the backy2 fuse command.
backy2 will then cleanup the temporary file and the mountpoint will be
removed.
Tip
If you want to fusemount as user, you must enable user_allow_other in
/etc/fuse.conf.
Restore continuation¶
If your backup target (=restore source) or your restore target are unstable, it can happen that you’re unable to restore a full backup due to service interruptions on these storages.
In that case you can resume a stopped restore process, even if backy2 stops working in strange ways (actually you may also just kill the backy2 process by pressing ctrl+c or killing the process).
For hinting from where to start writing to the restore-target again, backy2
outputs Last ID as the hint which was the last block until which all
blocks have been written successfully to the restore-target.
Note
This especially works even with asynchronuous writes as we have them when using the simultanuous_writes config variable.
You can then pass the last block-ID that you have seen during the restore
process to backy2 restore. It’s important however that you use the exactly
same parameters as before when restoring:
$ backy2 restore 30d53cea-7ff8-11ea-9466-8931a4889813 file://test.img
INFO: $ backy2 restore 30d53cea-7ff8-11ea-9466-8931a4889813 file://test.img
INFO: Restore phase 1/2 (sparse) to null:// Read Queue [ ] Write Queue [ ] (0.4% 0.0MB/sØ ETA 1s)
INFO: Restore phase 2/2 (data) to null:// Read Queue [= ] Write Queue [ ] (0.4% 0.0MB/sØ ETA 13s) Last ID: None
INFO: Restore phase 2/2 (data) to null:// Read Queue [==========] Write Queue [ ] (20.7% 197.3MB/sØ ETA 4s) Last ID: 42
^CTraceback (most recent call last):
File "/home/dk/develop/backy2/env/bin/backy2", line 11, in <module>
load_entry_point('backy2', 'console_scripts', 'backy2')()
File "/home/dk/develop/backy2/src/backy2/scripts/backy.py", line 749, in main
func(**func_args)
File "/home/dk/develop/backy2/src/backy2/scripts/backy.py", line 103, in restore
backy.restore(version_uid, target, sparse, force, int(continue_from))
File "/home/dk/develop/backy2/src/backy2/backy.py", line 380, in restore
data_checksum = self.hash_function(data).hexdigest()
KeyboardInterrupt
$ backy2 restore 30d53cea-7ff8-11ea-9466-8931a4889813 file://test.img -c 42
INFO: $ backy2 restore 30d53cea-7ff8-11ea-9466-8931a4889813 file://test.img -c 42
INFO: Restore phase 1/2 (sparse) to null:// Read Queue [ ] Write Queue [ ] (16.8% 0.0MB/sØ ETA 0s)
INFO: Restore phase 2/2 (data) to null:// Read Queue [==== ] Write Queue [ ] (0.5% 0.0MB/sØ ETA 23s) Last ID: None
INFO: Restore phase 2/2 (data) to null:// Read Queue [==========] Write Queue [ ] (38.3% 292.1MB/sØ ETA 2s) Last ID: 122
INFO: Backy complete.
Hint
If you don’t see why -c 42 has been used in the above command,
you will need to scroll to the right as output is cut in this documentation.
Then you’ll see Last ID: 42.
Edge-Cases¶
Invalid blocks / versions¶
During full restore, backy2 will compare each restored block’s checksum to the stored checksum from the metadata store no matter if the block has been marked invalid previously. If it encounters a difference, the block and all versions containing this block will be marked invalid and a warning will be given:
ERROR: Checksum mismatch during restore for block 2558 (is: a134381a9760b5569cdaa2b1496483a5cef0428b2de351593599aaac4a5525e8dc97610a3cf515bce685d1e2a1e76c054c3438382588669a909e2073dac1a25c should-be: asd, block-valid: 0). Block restored is invalid. Continuing.
INFO: Marked block invalid (UID 73923236eb5ttp7DJ6BdynMfwMnZamTF, Checksum asd. Affected versions: 90fbbeb6-1fbe-11e7-9f25-a44e314f9270, bc1af5e0-2f1c-11e7-b961-a44e314f9270, 0ffaf1ba-2f1d-11e7-b961-a44e314f9270, 2563c0a4-2f1d-11e7-b961-a44e314f9270, 430d4f4e-2f1d-11e7-b961-a44e314f9270
Even when encountering such an error, backy2 will continue to restore.
Note
The philosophy behind this is that restores must allways succeed, even if there are errors in the data. Most times invalid data is on irrelevant places or can be fixed later. It’s always worse to crash/break the restore process when an error occurs.
The null:// target¶
As there’s the null://<size> source, there’s also a null:// target. You may use this for performance-testing or bottleneck-tests, but of course all “written” data will just be lost. Example:
$ backy2 restore 30d53cea-7ff8-11ea-9466-8931a4889813 null://