Burp

From miki
Revision as of 21:32, 26 April 2017 by Mip (talk | contribs) (→‎Troubleshoot)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

Burp is a network backup and restore program. It attempts to reduce network traffic and the amount of space that is used by each backup.

References

Build on Lacie-CloudBox

Build on Optware

Links:

Example of install log on QNAP

Install:

  • Clone git
# Clone git repository
git clone https://github.com/grke/burp.git
cd burp
git checkout 2.1.6

# Generate the autoconf files
autoreconf -vif
# ...
# Compilation failed in require at /opt/share/automake-1.12/Automake/ChannelDefs.pm line 23.
# ...
# autoreconf: aclocal failed with exit status: 255
  • We run autoreconf on ubuntu, and import files back on lacie:
autoreconf -vif
find -name .gitignore|xargs rm
git add -A
git checkout HEAD -- .gitignore m4/.gitignore src/win32/.gitignore test/.gitignore
git commit -m "add results of autoreconf -vif"
git branch -f master
  • Install dependencies, from log linked above. Note can use ipkg list to find package.
ipkg install openssl-dev librsync zlib
ipkg install gcc make automake autoconf libtool binutils autoconf
ipkg install optware-devel
  • Install uthash headers in /opt/include:
git clone https://github.com/troydhanson/uthash.git
cp uthash/src/ut* /opt/include/
  • OpenSSL error: unable to find OpenSLL library
./configure --prefix=/opt --sysconfdir=/opt/etc/burp --localstatedir=/opt/var
# checking For OpenSSL... no
# configure: error: Unable to find OpenSSL library
This is due to undefined reference:
grep -iC 10 openssl config.log|less
# /opt/arm-none-linux-gnueabi/lib/libdl.so.2: undefined reference to `_dl_tls_get_addr_soft@GLIBC_PRIVATE'
We relink some libraries as suggested here. Let's relink them all:
cd /opt/arm-none-linux-gnueabi/
mv lib lib.original
ln -sf /lib lib
cd -
  • Configure again, now fine:
./configure --prefix=/opt --sysconfdir=/opt/etc/burp --localstatedir=/opt/var
  • Make
make
# ... error about missing aclocal...

Digging further, we see that aclocal does not work because it requires Perl built with threads enabled. For this we need Perl 5.22 from Optware-ng

Build on Optware-ng

To install 

# Install dependencies
ipkg install openssl-dev librsync zlib
ipkg install gcc
ipkg install make automake autoconf libtool binutils autoconf
ipkg install optware-devel

# Install uthash headers in /opt/include:
git clone https://github.com/troydhanson/uthash.git
cp uthash/src/ut* /opt/include/

# Clone git repository
git clone https://github.com/grke/burp.git
cd burp
git checkout 2.1.6

# Generate the autoconf files
autoreconf -vif

# Configure
./configure --prefix=/opt --sysconfdir=/opt/etc/burp --localstatedir=/opt/var

# Make
make

# Install
make install
# Copied/created the following files/directories:
#   /opt/bin/vss_strip
#   /opt/sbin/burp
#   /opt/sbin/burp_ca
#   /opt/share/burp
#   /opt/share/doc/burp
#   /opt/share/man/man8/{bedup.8,bsigs.8,bsparse.8,burp.8,burp_ca.8,vss_strip.8}

# Install config
make install-configs
# Copied/created the following files/directories:
#     /opt/etc/burp

# If we want to create a .deb package:
# (Note: if fail, run make install install-configs w/o checkinstall first)
sudo checkinstall make install install-configs

Edit the following files:

  • /etc/burp/burp-server.conf (for backup server)
  • /etc/burp/burp.conf (for backup client)

For each client cname on the server (see cname in burp.conf), edit the file:

  • /etc/burp/clientconfdir/cname.

Start the server with: 

sudo burp -c /etc/burp/burp-server.conf

On the client, start a backup with: 

sudo burp -a b

Monitor ongoing backups with: 

sudo burp -s

Benchmarks

### On the server
egrep "(protocol|ratelimit|comp|ssl_ciphers).*=" /opt/etc/burp/burp-server.conf
pkill burp && rm -rf /opt/var/spool/burp/* && burp -c /opt/etc/burp/burp-server.conf

### On the client
egrep "(protocol|ratelimit|comp|ssl_ciphers).*=" /etc/burp/burp.conf
sudo burp -a b
/var/www/datasheet on zavcxl0005 -- no compression, protocol 2
du --apparent-size -BM -s /var/www/datasheet/
# 92M	/var/www/datasheet/
du -BM -s /var/www/datasheet/
# 92M	/var/www/datasheet/

### On the server
# protocol = 2
# compression = zlib0
# ssl_compression = zlib0
### On the client
# protocol = 0
# ssl_compression = zlib0
sudo burp -a b
# --------------------------------------------------------------------------------
# Start time: 2017-04-26 22:35:07
#   End time: 2017-04-26 22:36:24
# Time taken: 01:17
#                          New   Changed Duplicate   Deleted     Total |  Scanned
#                    ------------------------------------------------------------
#              Files:        0         0         0         0         0 |       11
#        Directories:        0         0         0         0         0 |        6
#             Blocks:    14005         0     28811         0     42816 |        0
#        Grand total:    14005         0     28811         0     42816 |       17
#                    ------------------------------------------------------------
# 
#              Messages:             0
#              Warnings:             0
# 
#       Bytes estimated:      95598630 (91.17 MB)
#       Bytes in backup:             0
#        Bytes received:        109011 (106.46 KB)
#            Bytes sent:      96078512 (91.63 MB)
# --------------------------------------------------------------------------------
# 2017-04-26 22:36:24: burp[18005] End backup
# 2017-04-26 22:36:24: burp[18005] backup finished ok
  • CPU usage on server: 95% average.
  • transfer time: 77s.
  • transfer speed: 1.2MB/s.
/var/www/datasheet on zavcxl0005 - no compression, no encryption, protocol 2
### On the server
# protocol = 2
# compression = zlib0
# compression = zlib0
# ssl_ciphers = eNULL
# ssl_compression = zlib0
### On the client
# protocol = 0
# ssl_ciphers = eNULL
# ssl_compression = zlib0
sudo burp -a b
# --------------------------------------------------------------------------------
# Start time: 2017-04-26 22:52:53
#   End time: 2017-04-26 22:53:58
# Time taken: 01:05
#                          New   Changed Duplicate   Deleted     Total |  Scanned
#                    ------------------------------------------------------------
#              Files:        0         0         0         0         0 |       11
#        Directories:        0         0         0         0         0 |        6
#             Blocks:    14005         0     28023         0     42028 |        0
#        Grand total:    14005         0     28023         0     42028 |       17
#                    ------------------------------------------------------------
# 
#              Messages:             0
#              Warnings:             0
# 
#       Bytes estimated:      95598630 (91.17 MB)
#       Bytes in backup:             0
#        Bytes received:        109011 (106.46 KB)
#            Bytes sent:      96078512 (91.63 MB)
# --------------------------------------------------------------------------------
# 2017-04-26 22:53:58: burp[22408] End backup
# 2017-04-26 22:53:58: burp[22408] backup finished ok
  • CPU usage on server: 95% average.
  • transfer time: 65s.
  • transfer speed: 1.4MB/s.
### On the server
# protocol = 1
# compression = zlib0
# ssl_ciphers = eNULL
# ssl_compression = zlib0
### On the client
# protocol = 0
# ssl_ciphers = eNULL
# ssl_compression = zlib0
burp -a b
# --------------------------------------------------------------------------------
# Start time: 2017-04-26 23:12:07
#   End time: 2017-04-26 23:12:23
# Time taken: 00:16
#                          New   Changed Duplicate   Deleted     Total |  Scanned
#                    ------------------------------------------------------------
#              Files:       11         0         0         0        11 |       11
#        Directories:        0         0         0         0         0 |        6
#        Grand total:       11         0         0         0        11 |       17
#                    ------------------------------------------------------------
# 
#              Messages:             0
#              Warnings:             0
# 
#       Bytes estimated:      95598630 (91.17 MB)
#       Bytes in backup:      95598630 (91.17 MB)
#        Bytes received:          1844 (1.80 KB)
#            Bytes sent:      95720309 (91.29 MB)
# --------------------------------------------------------------------------------
# 2017-04-26 23:12:23: burp[23561] Phase 2 end (send file data)
# 2017-04-26 23:12:23: burp[23561] backup finished ok
  • CPU usage on server: 95% average.
  • transfer time: 16s.
  • transfer speed: 5.7MB/s.
Increasing ratelimit
  • No effect on any protocol.
Protocol 2 - Backup again after duplicating all files
### On the server
# protocol = 2
# compression = zlib0
# ssl_ciphers = eNULL
# ssl_compression = zlib0
### On the client
# protocol = 0
# ssl_ciphers = eNULL
# ssl_compression = zlib0
sudo cp -r /var/www/datasheet/arm /var/www/datasheet/arm2    # We make a redundant copy inside
sudo burp -a b
# --------------------------------------------------------------------------------
# Start time: 2017-04-26 23:22:22
#   End time: 2017-04-26 23:22:35
# Time taken: 00:13
#                          New   Changed Duplicate   Deleted     Total |  Scanned
#                    ------------------------------------------------------------
#              Files:        0         0         0         0         0 |       22
#        Directories:        0         0         0         0         0 |       11
#             Blocks:        0         0     18024         0     18024 |        0
#        Grand total:        0         0     18024         0     18024 |       33
#                    ------------------------------------------------------------
# 
#              Messages:             0
#              Warnings:             0
# 
#       Bytes estimated:     191197260 (182.34 MB)
#       Bytes in backup:             0
#        Bytes received:          1156 (1.13 KB)
#            Bytes sent:        412007 (402.35 KB)
# --------------------------------------------------------------------------------
# 2017-04-26 23:22:35: burp[24000] End backup
# 2017-04-26 23:22:35: burp[24000] backup finished ok
  • CPU usage on server: 95% average.
  • transfer time: 13s.
  • transfer speed: 14.0MB/s.
Protocol 1 - Backup again after duplicating all files
### On the server
# protocol = 1
# compression = zlib0
# ssl_ciphers = eNULL
# ssl_compression = zlib0
### On the client
# protocol = 0
# ssl_ciphers = eNULL
# ssl_compression = zlib0
sudo cp -r /var/www/datasheet/arm /var/www/datasheet/arm2    # We make a redundant copy inside
sudo burp -a b
# --------------------------------------------------------------------------------
# Start time: 2017-04-26 23:28:13
#   End time: 2017-04-26 23:28:30
# Time taken: 00:17
#                          New   Changed Duplicate   Deleted     Total |  Scanned
#                    ------------------------------------------------------------
#              Files:       11         0         0         0        11 |       22
#        Directories:        0         0         0         0         0 |       11
#        Grand total:       11         0         0         0        11 |       33
#                    ------------------------------------------------------------
# 
#              Messages:             0
#              Warnings:             0
# 
#       Bytes estimated:     191197260 (182.34 MB)
#       Bytes in backup:      95598630 (91.17 MB)
#        Bytes received:          1844 (1.80 KB)
#            Bytes sent:      95722406 (91.29 MB)
# --------------------------------------------------------------------------------
# 2017-04-26 23:28:30: burp[24200] Phase 2 end (send file data)
# 2017-04-26 23:28:30: burp[24200] backup finished ok
#
  • CPU usage on server: 95% average.
  • transfer time: 17s.
  • transfer speed: 10.7MB/s.

Troubleshoot

SSL connect error

sudo burp -a b
# 2017-04-25 23:09:47: burp[30776] SSL connect error
# 140585554515608:error:1409441B:SSL routines:ssl3_read_bytes:tlsv1 alert decrypt error:s3_pkt.c:1487:SSL alert number 51

This happens when using old certificates on backup client, typically after changing the configuration to point to another backup server [1]. 

cd /etc/burp
rm ssl_cert_ca.pem ssl_cert-client.key ssl_cert-client.pem
Retrieved from "https://miki.immie.org/mediawiki/index.php?title=Burp&oldid=9309"