BorgBackup: Difference between revisions

From miki
Jump to navigation Jump to search
 
(8 intermediate revisions by the same user not shown)
Line 1: Line 1:
== References ==
== References ==
* [https://github.com/borgbackup/borg Borg on GitHub]
* [https://github.com/borgbackup/borg Borg on GitHub]
* {{red|New}} [https://github.com/borgmatic-collective/borgmatic Borgmatic] A tool to deploy BorgBackup easily
* [https://news.ycombinator.com/item?id=21642364 A discussion thread on Hacker News about BorgBackup]
: Mentioning bup and restic as possible alternatives, or Bacula for extreme reliability at very high backups size.
* {{red|New}} [https://news.ycombinator.com/item?id=41702315 More discussion on Borg 2.0 supporting rclone, for cloud storage]
: Mentioning restic, kopia, rustic as possible alternative. Note that Borg 2.X is still beta.
: Also mentioning Hetzner storage box, 4€ / month for 1TB data.

== To Do ==
Benchmarks:
* DONE - Client/server zavcvl0005, storage on lacie-cloudbox via CIFS.
* DONE - Client zavcxl0005, server griffin, storage on lacie-cloudbox via CIFS.
* Client zavcxl0005, server lacie-cloudbox, local storage.
* Client zavcxl0005, server raspberry-pi, storage on lacie-cloudbox via CIFS.

Config:
* How best configure Borg for multiple clients backing up to same repo (see ticket [https://github.com/borgbackup/borg/issues/2313 #2313]).
* How sync. backup time on multiple client? Triggered by server? or triggered by client? How prevent concurrent session?
* Collect stats on client and server. Report these stats via email or such.

To Do:
* Install borg on lacie-cloudbox, collect bench and stats.


== Install on Ubuntu 16.04 ==
== Install on Ubuntu 16.04 ==
Line 10: Line 31:
sudo apt install libfuse-dev fuse pkg-config # optional, for FUSE support
sudo apt install libfuse-dev fuse pkg-config # optional, for FUSE support
sudo pip3 install "borgbackup==1.1.0b5"
sudo pip3 install "borgbackup==1.1.0b5"
</source>

Since '''Ubuntu 18.04 Bionic''':
<source lang=bash>
sudo apt install borgbackup # v1.1.5-1 on Bionic
</source>
</source>


Line 16: Line 42:
* [https://github.com/borgbackup/borg/issues/2313 borg create without syncing cache]
* [https://github.com/borgbackup/borg/issues/2313 borg create without syncing cache]
: Implements [https://github.com/borgbackup/borg/issues/916 #916], and [https://github.com/borgbackup/borg/issues/474 Borgception].
: Implements [https://github.com/borgbackup/borg/issues/916 #916], and [https://github.com/borgbackup/borg/issues/474 Borgception].

=== Foresee disk full ===
Reference:
* [https://github.com/borgbackup/borg/issues/2756 Issue#2756borg delete requires free disk space]
* https://borgbackup.readthedocs.io/en/stable/quickstart.html#important-note-about-free-space

BorgBackup doesn't always handle disk full scenarios. To overcome this:
* Create a dummy file on the repo volume that can be deleted when necessary (to run <code>purge</code> commands, etc).
<source lang="bash">
dd if=/dev/zero of=DELETE_ME_IF_DISK_FULL bs=1M count=512
</source>
* Tell borg to leave some free space
<source lang="bash">
borg config REPO additional_free_space 2G
</source>

=== See snapshot disk usage ===
* Use this python tool [https://github.com/KrystianD/borg-ncdu-analyzer borg-ncdu-analyzer]
<source lang=bash>
export BORG_REPO=...
python3 borg_ncdu_analyzer.py ::some_snapshot --full-path
</source>

== Troubleshoot ==
=== Fix Corrupted segment reference count ===
We have this error:
AssertionError: Corrupted segment reference count - corrupted index or hints

This may be due to an old version bug [https://borgbackup.readthedocs.io/en/stable/changes.html]. To fix:

<source lang="bash">
# On all clients and all repos, delete the cache
borg delete --cache-only REPO

# For all repos:
borg check --repair REPO
</source>

Latest revision as of 10:28, 1 October 2024

References

Mentioning bup and restic as possible alternatives, or Bacula for extreme reliability at very high backups size.
Mentioning restic, kopia, rustic as possible alternative. Note that Borg 2.X is still beta.
Also mentioning Hetzner storage box, 4€ / month for 1TB data.

To Do

Benchmarks:

  • DONE - Client/server zavcvl0005, storage on lacie-cloudbox via CIFS.
  • DONE - Client zavcxl0005, server griffin, storage on lacie-cloudbox via CIFS.
  • Client zavcxl0005, server lacie-cloudbox, local storage.
  • Client zavcxl0005, server raspberry-pi, storage on lacie-cloudbox via CIFS.

Config:

  • How best configure Borg for multiple clients backing up to same repo (see ticket #2313).
  • How sync. backup time on multiple client? Triggered by server? or triggered by client? How prevent concurrent session?
  • Collect stats on client and server. Report these stats via email or such.

To Do:

  • Install borg on lacie-cloudbox, collect bench and stats.

Install on Ubuntu 16.04

sudo apt install python3 python3-dev python3-pip python-virtualenv libssl-dev openssl libacl1-dev libacl1 liblz4-dev liblz4-1 build-essential
sudo apt install libfuse-dev fuse pkg-config    # optional, for FUSE support
sudo pip3 install "borgbackup==1.1.0b5"

Since Ubuntu 18.04 Bionic: 

sudo apt install borgbackup        # v1.1.5-1 on Bionic

Tips

Multiple client

Implements #916, and Borgception.

Foresee disk full

Reference:

BorgBackup doesn't always handle disk full scenarios. To overcome this:

  • Create a dummy file on the repo volume that can be deleted when necessary (to run purge commands, etc).
dd if=/dev/zero of=DELETE_ME_IF_DISK_FULL bs=1M count=512
  • Tell borg to leave some free space
borg config REPO additional_free_space 2G

See snapshot disk usage

export BORG_REPO=...
python3 borg_ncdu_analyzer.py ::some_snapshot --full-path

Troubleshoot

Fix Corrupted segment reference count

We have this error: 

AssertionError: Corrupted segment reference count - corrupted index or hints

This may be due to an old version bug [1]. To fix: 

# On all clients and all repos, delete the cache
borg delete --cache-only REPO

# For all repos:
borg check --repair REPO
Retrieved from "https://miki.immie.org/mediawiki/index.php?title=BorgBackup&oldid=17758"