In order to get rid of old backup versions, backy2 uses a garbage collection algorithm which is a 2-step process in backy2.
Remove old versions¶
In order to remove an old version, use
$ backy2 rm --help usage: backy2 rm [-h] [-f] version_uid positional arguments: version_uid optional arguments: -h, --help show this help message and exit -f, --force Force removal of version, even if it's younger than the configured disallow_rm_when_younger_than_days.
$ backy2 rm 2453fe90-2f22-11e7-b961-a44e314f9270 INFO: $ /home/dk/develop/backy2/env/bin/backy2 rm 2453fe90-2f22-11e7-b961-a44e314f9270 INFO: Removed backup version 2453fe90-2f22-11e7-b961-a44e314f9270 with 25600 blocks. INFO: Backy complete.
There is a config option called
defaults to 6. If the to-be-removed backup version is younger than this value,
backy2 throws an error and exits with code 100:
backy2 rm 2453fe90-2f22-11e7-b961-a44e314f9270 INFO: $ /home/dk/develop/backy2/env/bin/backy2 rm 2453fe90-2f22-11e7-b961-a44e314f9270 ERROR: Unexpected exception ERROR: 'Version 2453fe90-2f22-11e7-b961-a44e314f9270 is too young. Will not delete.' … backy2.backy.LockError: 'Version 2453fe90-2f22-11e7-b961-a44e314f9270 is too young. Will not delete.' INFO: Backy failed. $ echo $? 100
Of course you can
--force backy2 to delete this version:
$ backy2 rm 2453fe90-2f22-11e7-b961-a44e314f9270 --force INFO: $ /home/dk/develop/backy2/env/bin/backy2 rm 2453fe90-2f22-11e7-b961-a44e314f9270 --force INFO: Removed backup version 2453fe90-2f22-11e7-b961-a44e314f9270 with 25600 blocks. INFO: Backy complete.
backy2 rm removes version and corrosponding blocks from the metadata store.
It also adds the removed metadata entries into a delete-candidates - List.
In order to really delete blocks from the backup target, you’ll need
Using the expire feature¶
If you have added expiration to your versions as shown in the backup section, you can get the expired versions (based on you server time) via:
$ backy2 ls -e
If you want to remove versions automatically by scripting, you might use this:
$ for version in `backy2 -ms ls -e -f uid`; do backy2 rm $version; done $ backy2 cleanup
To free up space on the backup target, you need to cleanup. There are two different cleanup methods, but you’ll usually only need the so-called fast-cleanup.
$ backy2 cleanup --help usage: backy2 cleanup [-h] [-f] [-p PREFIX] [--dangerous-force] optional arguments: -h, --help show this help message and exit -f, --full Do a full cleanup. This will read the full metadata from the data backend (i.e. backup storage) and compare it to the metadata in the meta backend. Unused data will then be deleted. This is a slow, but complete process. A full cleanup must not be run parallel to ANY other backy jobs. -p PREFIX, --prefix PREFIX If you perform a full cleanup, you may add --prefix to only cleanup block uids starting with this prefix. This is for iterative cleanups. Example: cleanup --full --prefix=a --dangerous-force Seriously, do not use this outside of testing and development.
backy2 cleanup will go through the list of delete-candidates and check if
there are blocks which are not referenced from any other version anymore.
These blocks are then deleted from the backup-target. The still-in-use blocks are removed from the list of delete-candidates.
In order to provide parallelism (i.e. multiple backy2 processes at the same time), backy2 needs to prevent race-conditions between adding a delete-candidate to the list and actually removing its data. That’s why a cleanup will only remove data blocks once they’re on the list of delete-candidates for more than 1 hour.
There are times (e.g. when your database is corrupted or when you create a new database based on export/import) when backy2 does not know if the blocks in the backend storage are all known by the metadata store.
-f option of cleanup comes into play. With this option, backy2 will
read all block UIDs from the backend storage (which can take very long) and
compare it to the list of known block UIDs in the metadata store.
Blocks unknown to the metadata store are then deleted from the backend storage.
Because this can be a very slow process, you can use the
-p option to
provide a prefix. The first 10 characters of block UIDs are hexadecimal numbers
and small letters (0-9, a-f).
So a typical cleanup process after a desaster looks like this:
$ backy2 cleanup -f -p 00 $ backy2 cleanup -f -p 01 … $ backy2 cleanup -f -p fe $ backy2 cleanup -f -p ff
Or even something like
$ for p in $(printf %02x'\n' `seq -f %1.f 0 255`); do backy2 cleanup -f -p $p; done