UrBackup

References

• https://www.urbackup.org/, the website.
• UrBackup administration manual.

Tuning

• Performance considerations for larger UrBackup server instances
• Backup server for the raspberry pi
• Internet Mode

Install on Ubuntu

• Install instructions.

Server

sudo add-apt-repository ppa:uroni/urbackup
sudo apt update
sudo apt install urbackup-server

Client

TF=`mktemp` && wget "https://hndl.urbackup.org/Client/2.1.15/UrBackup%20Client%20Linux%202.1.15.sh" -O $TF && sh $TF; rm $TF
# Verifying archive integrity... All good.
# Uncompressing UrBackup Client Installer for Linux  100%  
# Installation of UrBackup Client 2.1.15 to /usr/local ... Proceed ? [Y/n]
# y
# Uncompressing install data...
# Detected Debian (derivative) system
# Detected systemd
# Detected architecture x86_64-linux-glibc
# Installed daemon configuration at /etc/default/urbackupclient...
# Info: Restoring from web interface is disabled per default. Enable by modifying /etc/default/urbackupclient.
# Installing systemd unit...
# Created symlink from /etc/systemd/system/multi-user.target.wants/urbackupclientbackend.service to /lib/systemd/system/urbackupclientbackend.service.
# Starting UrBackup Client service...
# Successfully started client service. Installation complete.
# +Detected Ubuntu LTS. Dattobd supported
# 
# -Detected no btrfs filesystem
# +Detected LVM volumes
# Please select the snapshot mechanism to be used for backups:
# 1) dattobd volume snapshot kernel module from https://github.com/datto/dattobd
# 2) LVM - Logical Volume Manager snapshots
# 4) Use no snapshot mechanism
# 1
# Configured dattobd. Please install dattobd following the instructions at https://github.com/datto/dattobd
# Configured snapshot mechanism via /usr/local/etc/urbackup/snapshot.cfg

# Install dattobd
sudo apt-key adv --recv-keys --keyserver keyserver.ubuntu.com 29FF164C
echo "deb https://cpkg.datto.com/repositories $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/datto-linux-agent.list
sudo apt-get update
sudo apt-get install dattobd-dkms dattobd-utils

By default, server connects to client using ports 35621, 35622, 35623. For this to work, we must add the following ufw rules:

--- a/ufw/user.rules
+++ b/ufw/user.rules
@@ -17,6 +17,12 @@
 :ufw-user-limit-accept - [0:0]
 ### RULES ###
 
+### tuple ### allow tcp 35621,35623 0.0.0.0/0 any 0.0.0.0/0 in
+-A ufw-user-input -p tcp -m multiport --dports 35621,35623 -j ACCEPT
+
+### tuple ### allow udp 35622 0.0.0.0/0 any 0.0.0.0/0 in
+-A ufw-user-input -p udp --dport 35622 -j ACCEPT
+
 ### tuple ### allow tcp 135,139,445 0.0.0.0/0 any 0.0.0.0/0 in
 -A ufw-user-input -p tcp -m multiport --dports 135,139,445 -j ACCEPT
 
--- a/ufw/user6.rules
+++ b/ufw/user6.rules
@@ -17,6 +17,12 @@
 :ufw6-user-limit-accept - [0:0]
 ### RULES ###
 
+### tuple ### allow tcp 35621,35623 ::/0 any ::/0 in
+-A ufw6-user-input -p tcp -m multiport --dports 35621,35623 -j ACCEPT
+
+### tuple ### allow udp 35622 ::/0 any ::/0 in
+-A ufw6-user-input -p udp --dport 35622 -j ACCEPT
+
 ### tuple ### allow tcp 135,139,445 ::/0 any ::/0 in
 -A ufw6-user-input -p tcp -m multiport --dports 135,139,445 -j ACCEPT

However, this means that any device can connect and read the files on the client! So, on untrusted network, it is recommended to enable Internet only mode [1]):

echo "internet_server=example.com
internet_server_port=55415
internet_authkey=foobar
internet_mode_enabled=true" > /usr/local/var/urbackup/data/settings.cfg

UrBackup client is not installed as a package. Files are found here:

/etc/default/urbackupclient
/etc/systemd/system/multi-user.target.wants/urbackupclientbackend.service
/lib/systemd/system/urbackupclientbackend.service
/usr/local/bin/urbackupclientctl
/usr/local/etc/urbackup
/usr/local/sbin/uninstall_urbackupclient
/usr/local/sbin/urbackupclientbackend
/usr/local/share/urbackup
/usr/local/var/urbackup
/var/log/urbackupclient.log

Dattobd on client

To enable dattobd snapshot, edit the file /usr/local/etc/urbackup/snapshot.cfg:

#This is a key=value config file for determining the scripts/programs to create snapshots

create_filesystem_snapshot=/usr/local/share/urbackup/dattobd_create_filesystem_snapshot
remove_filesystem_snapshot=/usr/local/share/urbackup/dattobd_remove_filesystem_snapshot

To disable snapshot, remove the file or set the keys to empty value:

#This is a key=value config file for determining the scripts/programs to create snapshots

#create_filesystem_snapshot=/usr/local/share/urbackup/dattobd_create_filesystem_snapshot
#remove_filesystem_snapshot=/usr/local/share/urbackup/dattobd_remove_filesystem_snapshot
create_filesystem_snapshot=
remove_filesystem_snapshot=

To remove snapshot / overlay files left by dattobd /usr/local/share/urbackup/dattobd_remove_filesystem_snapshot:

cat /proc/datto-info
# # cat /proc/datto-info 
# {
# 	"version": "0.9.16",
# 	"devices": [
# 		{
# 			"minor": 0,
# 			"cow_file": "/.datto_3d41c58e-6724-4d47-8981-11c766a08a24_63a1cea102479ac8f40726939d656624778f1a2f7fb6714f",
# 			"block_device": "/dev/dm-1",
# 			"max_cache": 314572800,
# 			"fallocate": 2147483648,
# 			"seq_id": 1,
# 			"uuid": "e8f7f4d4b11549458da6c8cfdad6f93f",
# 			"state": 3
# 		}
# 	]
# }
l /mnt/urbackup_snaps/
# total 4.0K
# drwxr-xr-x 2 root root 48 May  8 05:29 /mnt/urbackup_snaps/63a1cea102479ac8f40726939d656624778f1a2f7fb6714f/
# -rw-r--r-- 1 root root  2 May  8 05:29 /mnt/urbackup_snaps/63a1cea102479ac8f40726939d656624778f1a2f7fb6714f-num
sudo umount /mnt/urbackup_snaps/63a1cea102479ac8f40726939d656624778f1a2f7fb6714f
sudo rmdir /mnt/urbackup_snaps/63a1cea102479ac8f40726939d656624778f1a2f7fb6714f
sudo rm /mnt/urbackup_snaps/63a1cea102479ac8f40726939d656624778f1a2f7fb6714f-num
losetup
# NAME       SIZELIMIT OFFSET AUTOCLEAR RO BACK-FILE
# /dev/loop0         0      0         1  0 /.overlay_2fefd007-3e48-4162-b2c6-45ccdda22f37_63a1cea102479ac8f40726939d656624778f1a2f7fb6714f
losetup -D
rm /.overlay_2fefd007-3e48-4162-b2c6-45ccdda22f37_63a1cea102479ac8f40726939d656624778f1a2f7fb6714f 
dmsetup ls
# sda5_crypt	(252:0)
# ubuntu--vg-home	(252:3)
# ubuntu--vg-swap_1	(252:2)
# wsnap-63a1cea102479ac8f40726939d656624778f1a2f7fb6714f	(252:4)
# ubuntu--vg-root	(252:1)
dmsetup remove wsnap-63a1cea102479ac8f40726939d656624778f1a2f7fb6714f
dbdctl destroy 0

Benchmarks

First backup

• Server: Griffin, backup to NFS /smb/lacie-cloudbox/beq06659/urbackup.
• Client: zavcxl0005, backup of /home/data /home/peetersm

• 129247M (135525883904), 80260 directories, 697949 files.
• Using dattobd.

• Default urbackup configuration.
• Log:

• Activity: Required time 185 min, Used storage 133.56 GB.
• Log: 142.388 GB, average speed: 110.523MBit/s
• Log: Time taken for backing up client zavcxl0005: 5h 27m 11s
• BACKUP INTERRUPTED because backup storage disk FULL!

Note:

• After reboot of Griffin, UrBackup took ~10min to rebuild index and delete the last backup! However the files are still there. Hopefully next backup would be faster...
• Stats on lacie-cloudbox:

• Collecting these stats is very slow.
• There are 3.5x more files in the repository.
• Storage is not compressed

du -hs /shares/beq06659/urbackup
# 144G    .                     # Took >10min to get
find -type f|wc -l
# 2426985                       # Took >10min to get! This is 3.5x the number of files to backup!!!

Todo:

• Disable following symlinks.
• Backup with and without dattobd.