BackupPC: Difference between revisions

From miki
Jump to navigation Jump to search
m (Mip moved page Backuppc to BackupPC: Use the correct case)
 
(28 intermediate revisions by the same user not shown)
Line 5: Line 5:
* [http://serverfault.com/questions/48368/rsync-take-too-long-to-run A patch for rsync about computing and caching checksums]
* [http://serverfault.com/questions/48368/rsync-take-too-long-to-run A patch for rsync about computing and caching checksums]


BackupPC links:
== Issues ==
* Doc:
:http://backuppc.sourceforge.net/faq/BackupPC.html
:http://backuppc.sourceforge.net/faq/
* https://wiki.archlinux.org/index.php/BackupPC#The_webserver_user_and_the_suid_problem
* http://redmine.lighttpd.net/projects/lighttpd/wiki/Docs_ModCGI
: (for config cgi in lighttpd, including alias)
* http://redmine.lighttpd.net/projects/lighttpd/wiki/TutorialConfiguration
* http://redmine.lighttpd.net/projects/1/wiki/Docs_Configuration
: (basic config)
* http://www.cyberciti.biz/tips/configure-lighttpd-alias-mod_alias.html
* http://www.cyberciti.biz/tips/lighttpd-setup-a-password-protected-directory-directories.html
: To enable user authentication
* http://redmine.lighttpd.net/projects/1/wiki/HowToRedirectHttpToHttps
: To force https
* http://redmine.lighttpd.net/projects/1/wiki/Docs_ModAuth
: For setting up htpasswd
: See also http://www.cyberciti.biz/tips/lighttpd-setup-a-password-protected-directory-directories.html
* http://backuppc.sourceforge.net/faq/BackupPC.html
* Info on rsync:
: https://help.ubuntu.com/community/rsync#Rsync_Daemon
: man rsyncd.conf
* How to setup ssh+rsync securely:
: http://backuppc.sourceforge.net/faq/ssh.html#how_can_client_access_as_root_be_avoided
: http://www.systemajik.com/blog/setting-up-backuppc-on-ubuntu/
: http://adsm.org/PRIVOXY-FORCE/lists/html/BackupPC-users/2012-02/msg00087.html
* A look at rsync performance
: http://lwn.net/Articles/400489/


== Install and configuration ==
* Backing up a ntfs-3g mount with <code>rsync_bpc</code> keeps transfering all files. See below for details.

: Solution: Use <code>tar</code> method instead?
=== ntfs-3g backups ===
When backing up ntfs-3g shares, '''the option <code>--links</code> and <code>--hard-links</code> must be removed from <code>RsyncArgs</code> and <code>RsyncRestoreArgs</code>''' (see troubleshooting section below)

Since these options are common to a same client, one must create a separate backuppc client if a given physical host contains both ntfs-3g and non-ntfs-3g partitions.
In that case, one client will match the name of the host, where others will have different names. For these clients, set the variable <code>ClientNameAlias</code> to the name of the physical host in the client configuration.

== Install on Windows ==
<source lang=text>
Reference: http://www.systemajik.com/blog/setting-up-backuppc-on-windows/

Download and install cygwin-rsync from http://sourceforge.net/projects/backuppc/files/cygwin-rsyncd/3.0.9.0/
Edit c:\rsyncd\rsyncd.conf
Edit c:\rsyncd\rsyncd.secrets

Update the firewall rules
Test connection with telnet from backuppc server

Create exclusion list
- Make sure we exclude locked file and junction
- To get list of junctions:
dir /aL /s > JunctionPoints.txt
Check backuppc charset setting


TODO:
Test changing (for all win pc, incl. pc-112-597, mandala, griffin, ...)

'/Windows',

Into

'+ /Windows/System32',
'+ /Windows/System32/config',
'+ /Windows/System32/config/**',
'/Windows/**',
</source>

== Install on Synology NAS ==
<source lang=text>
Mnemosyne> mkdir .ssh
Mnemosyne> cd .ssh
Mnemosyne> cat >authorized_keys
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCrpFEALD473OqeplM+qyBx+46KMquWK9utwNmvIy3iBatE4S2oFrHVjeQjSNNDK9qaVh2cHPNzzB0UkV9y174ZLn9qeatU032ieKgFfdTdgQCe88BM8eSiVxTkVWe/bawQwn0qqghPtQl7v6/Bof9H9pAGeWLPTfUFj/+CQTf2vevBZF7iuh/RhaokJ75EY29E2lMlAgfIsLv8OVt71LBJNzvXaZIvQgFn03et44UZuZpoqYlZZgAuVgJbHG88QRxcvciBl3H7adkphEkoGUn05JpG2G01Yw2orxpKuMAkCs6t+Z7LjzQhDe+vmKYC4ZUNr5APDwSUxovI3HWJ10Xv mip@home
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzEuVrk/Od38pnN4x0mM5DwdqnBlQlDZucJGnmjdgUrAnpNXaCuI+dMZSJ/q4m+2yH2icc69jOKNyvHC0DAce9vStyHS61itzhIN2fETB6EoF11+4VzyOBPR2n/drfgBFo9rX7h8FO5em2KrfgysRsJpNW63HqmsMZWhNHjfAwvfpauks6TJVtUctnHu8n8zV5o6Z6CPHq7b+MuLfWa+RMQyE+dUpk9JM3CgrBdumEYMgNWzmTLrMd6VdzHocU3i/6XHWfo+ZzbaN3aKVx/dKiEZQJUTavyXTiVrzWQBE/g+MmMsyb+6MdKarR5Hpct1BLUhIOv7C1QZna1xIjuICr mip@work
Mnemosyne> mount
/dev/root on / type ext3 (rw,relatime,errors=continue,data=ordered)
/tmp on /tmp type tmpfs (0)
none on /dev/pts type devpts (gid=4,mode=620)
/sys on /sys type sysfs (0)
/proc/bus/usb on /proc/bus/usb type usbfs (0)
/dev/md2 on /volume1 type ext3 (usrquota,grpquota)
/volume1/@optware on /opt type bind (bind)
none on /proc/fs/nfsd type nfsd (0)
Mnemosyne> cd /opt
Mnemosyne> ipkg update
Downloading http://ipkg.nslu2-linux.org/feeds/optware/syno-e500/cross/unstable/Packages.gz
Inflating http://ipkg.nslu2-linux.org/feeds/optware/syno-e500/cross/unstable/Packages.gz
Updated list of available packages in /opt/lib/ipkg/lists/cross
Successfully terminated.
Mnemosyne> df -h
Filesystem Size Used Available Use% Mounted on
/dev/md0 2.3G 449.7M 1.9G 19% /
/tmp 251.7M 372.0K 251.3M 0% /tmp
/dev/md2 5.4T 4.7T 643.6G 88% /volume1
/volume1/@optware 5.4T 4.7T 643.6G 88% /opt
# rsync already 3.0.9
Mnemosyne> which rsync
/usr/syno/bin/rsync
Mnemosyne> ipkg install perl par2cmdline tar
Mnemosyne> ipkg install perl-compress-zlib perl-archive-zip
Mnemosyne> which perl
/opt/bin/perl
Mnemosyne> ipkg install optware-devel
Installing optware-devel (6.8-10) to root...
...
Configuring tcl
An error ocurred, return value: 1.
Collected errors:
ERROR: The following packages conflict with wget-ssl:
wget
Mnemosyne> ipkg remove wget
Removing package wget from root...
Successfully terminated.
Mnemosyne> ipkg install optware-devel
Installing optware-devel (6.8-10) to root...
Downloading http://ipkg.nslu2-linux.org/feeds/optware/syno-e500/cross/unstable/optware-devel_6.8-10_powerpc.ipk
file_move: ERROR: failed to rename /opt/ipkg-eY7pMk/optware-devel_6.8-10_powerpc.ipk to /opt/ipkg-eY7pMk/optware-devel_6.8-10_powerpc.ipk: No such file or directory
Nothing to be done
An error ocurred, return value: -1.
Collected errors:
Failed to download optware-devel. Perhaps you need to run 'ipkg update'?
Mnemosyne> ipkg update
Downloading http://ipkg.nslu2-linux.org/feeds/optware/syno-e500/cross/unstable/Packages.gz
ipkg: /opt/ipkg-z1g4Tt/Packages.gz: No such file or directory
file_copy: ERROR: failed to copy /opt/ipkg-z1g4Tt/Packages.gz to /tmp/ipkg.WbgbbX/cross.gz
An error ocurred, return value: 1.
Mnemosyne> ipkg install wget-ssl
Installing wget-ssl (1.12-2) to root...
Downloading http://ipkg.nslu2-linux.org/feeds/optware/syno-e500/cross/unstable/wget-ssl_1.12-2_powerpc.ipk
file_move: ERROR: failed to rename /opt/ipkg-5qgjat/wget-ssl_1.12-2_powerpc.ipk to /opt/ipkg-5qgjat/wget-ssl_1.12-2_powerpc.ipk: No such file or directory
Nothing to be done
An error ocurred, return value: -1.
Collected errors:
Failed to download wget-ssl. Perhaps you need to run 'ipkg update'?
Mnemosyne> ipkg install wget-ssl_1.12-2_powerpc.ipk
Installing wget-ssl (1.12-2) to root...
Installing libidn (1.25-1) to root...
Downloading http://ipkg.nslu2-linux.org/feeds/optware/syno-e500/cross/unstable/libidn_1.25-1_powerpc.ipk
file_move: ERROR: failed to rename /opt/ipkg-9pn57X/libidn_1.25-1_powerpc.ipk to /opt/ipkg-9pn57X/libidn_1.25-1_powerpc.ipk: No such file or directory
Nothing to be done
An error ocurred, return value: -1.
Collected errors:
Failed to download libidn. Perhaps you need to run 'ipkg update'?
Mnemosyne> ipkg install libidn_1.25-1_powerpc.ipk
Installing libidn (1.25-1) to root...
Configuring libidn
Successfully terminated.
Mnemosyne> Connection to mnemosyne closed.

beq06659@nxl67170ux /d/nxp/outlook
$ ssh root@mnemosyne
X11 forwarding request failed on channel 0


BusyBox v1.16.1 (2014-09-04 13:59:18 CST) built-in shell (ash)
Enter 'help' for a list of built-in commands.

Mnemosyne> cd /opt/bin
Mnemosyne> rm wget
Mnemosyne> cd
Mnemosyne> ipkg remove wget
No packages removed.
Nothing to be done
Successfully terminated.
Mnemosyne> ipkg remove wget-ssl
No packages removed.
Nothing to be done
Successfully terminated.
Mnemosyne> ipkg install wget-ssl_1.12-2_powerpc.ipk
Installing wget-ssl (1.12-2) to root...
Configuring wget-ssl
Successfully terminated.
Mnemosyne> ipkg install libidn_1.25-1_powerpc.ipk
Installing libidn (1.25-1) to root...
Configuring libidn
Successfully terminated.
Mnemosyne> Connection to mnemosyne closed.

beq06659@nxl67170ux /d/nxp/outlook
$ ssh root@mnemosyne
X11 forwarding request failed on channel 0


BusyBox v1.16.1 (2014-09-04 13:59:18 CST) built-in shell (ash)
Enter 'help' for a list of built-in commands.

Mnemosyne> which wget
/opt/bin/wget
Mnemosyne> ipkg update
Downloading http://ipkg.nslu2-linux.org/feeds/optware/syno-e500/cross/unstable/Packages.gz
Inflating http://ipkg.nslu2-linux.org/feeds/optware/syno-e500/cross/unstable/Packages.gz
Updated list of available packages in /opt/lib/ipkg/lists/cross
Successfully terminated.
Mnemosyne> ipkg upgrade
Nothing to be done
Successfully terminated.
Mnemosyne> ipkg install optware-devel
...
Configuring optware-devel
Successfully terminated.
Mnemosyne> rm libidn_1.25-1_powerpc.ipk
Mnemosyne> rm wget-ssl_1.12-2_powerpc.ipk
Mnemosyne> rm Packages.gz

# To bypass build error:
powerpc-linux-gnuspe-ar cr libzlib.a adler32.o compress.o crc32.o deflate.o inffast.o inflate.o inftrees.o trees.o zutil.o
make[1]: powerpc-linux-gnuspe-ar: Command not found
make[1]: *** [libzlib.a] Error 127
make[1]: Leaving directory `/root/BackupPC-XS-0.30/zlib'
make: *** [zlib/libzlib.a] Error 2
Mnemosyne> cd /opt/bin
Mnemosyne> for f in addr2line ar as c++ c++filt cpp g++ gcc gccbug gcov ld nm objcopy objdump ranlib readelf size strings strip; do if [ -e /opt/bin/$f ]; then F=powerpc-linux-gnuspe-$f; if [ -e /opt/bin/$F ]; then echo $F exists; else echo Creating symlink $F to $f; ln -sf $f /opt/bin/$F; fi ; else echo $f does not exists; fi; done
Creating symlink powerpc-linux-gnuspe-addr2line to addr2line
powerpc-linux-gnuspe-c++ exists
Creating symlink powerpc-linux-gnuspe-c++filt to c++filt
Creating symlink powerpc-linux-gnuspe-cpp to cpp
powerpc-linux-gnuspe-g++ exists
powerpc-linux-gnuspe-gcc exists
Creating symlink powerpc-linux-gnuspe-gcov to gcov
Creating symlink powerpc-linux-gnuspe-ld to ld
Creating symlink powerpc-linux-gnuspe-nm to nm
Creating symlink powerpc-linux-gnuspe-objcopy to objcopy
Creating symlink powerpc-linux-gnuspe-objdump to objdump
Creating symlink powerpc-linux-gnuspe-ranlib to ranlib
Creating symlink powerpc-linux-gnuspe-readelf to readelf
Creating symlink powerpc-linux-gnuspe-size to size
Creating symlink powerpc-linux-gnuspe-strings to strings
Creating symlink powerpc-linux-gnuspe-strip to strip



Mnemosyne> cd
Mnemosyne> wget http://sourceforge.net/projects/backuppc/files/backuppc-beta/4.0.0alpha3/BackupPC-XS-0.30.tar.gz/download
Mnemosyne> wget http://sourceforge.net/projects/backuppc/files/backuppc-beta/4.0.0alpha3/rsync-bpc-3.0.9.3.tar.gz/download
Mnemosyne> wget http://sourceforge.net/projects/backuppc/files/backuppc-beta/4.0.0alpha3/BackupPC-4.0.0alpha3.tar.gz/download
Mnemosyne> tar xvzf BackupPC-XS-0.30.tar.gz
Mnemosyne> cd BackupPC-XS-0.30/
Mnemosyne> perl Makefile.PL
Mnemosyne> make
Mnemosyne> make test
Mnemosyne> make install

Mnemosyne> cd
Mnemosyne> tar -xvzf rsync-bpc-3.0.9.3.tar.gz
Mnemosyne> cd rsync-bpc-3.0.9.3/
# Should have done Mnemosyne> ./configure.sh --prefix=/opt
Mnemosyne> ./configure.sh
Mnemosyne> make
Mnemosyne> make install

Mnemosyne> ipkg install rrdtool
Mnemosyne> cat /etc/passwd
...
backuppc:x:1033:100:User for backuppc:/var/services/homes/backuppc:/sbin/nologin
Mnemosyne> mkdir /opt/var/lib/backuppc
Mnemosyne> chown backuppc /opt/var/lib/backuppc
Mnemosyne> mkdir /usr/syno/synoman/backuppc

Mnemosyne> cd
Mnemosyne> tar -xvzf BackupPC-4.0.0alpha3.tar.gz
Mnemosyne> cd BackupPC-4.0.0alpha3/
Mnemosyne> ./configure.pl

Is this a new installation or upgrade for BackupPC? If this is
an upgrade please tell me the full path of the existing BackupPC
configuration file (eg: /etc/BackupPC/config.pl). Otherwise, just
hit return.

--> Full path to existing main config.pl []?

I found the following locations for these programs:

bzip2 => /opt/bin/bzip2
cat => /opt/bin/cat
df => /opt/bin/df
gtar/tar => /opt/bin/tar
gzip => /opt/bin/gzip
hostname => /bin/hostname
nmblookup => /usr/syno/bin/nmblookup
par2 => /opt/bin/par2
perl => /opt/bin/perl
ping => /bin/ping
rrdtool => /opt/bin/rrdtool
rsync => /opt/bin/rsync
rsync_bpc => /usr/local/bin/rsync_bpc
sendmail =>
smbclient => /usr/syno/bin/smbclient
split => /opt/bin/split
ssh/ssh2 => /usr/syno/bin/ssh

--> Are these paths correct? [y]?

Please tell me the hostname of the machine that BackupPC will run on.

--> BackupPC will run on host [Mnemosyne]?

BackupPC should run as a dedicated user with limited privileges. You
need to create a user. This user will need read/write permission on
the main data directory and read/execute permission on the install
directory (these directories will be setup shortly).

The primary group for this user should also be chosen carefully.
The data directories and files will have group read permission,
so group members can access backup files.

--> BackupPC should run as user [backuppc]?

Please specify an install directory for BackupPC. This is where the
BackupPC scripts, library and documentation will be installed.

--> Install directory (full path) [/usr/local/BackupPC]? /opt/local/backuppc

Please specify a data directory for BackupPC. This is where all the
PC backup data is stored. This file system needs to be big enough to
accommodate all the PCs you expect to backup (eg: at least several GB
per machine).

--> Data directory (full path) [/data/BackupPC]? /opt/var/lib/backuppc

BackupPC has SCGI and CGI perl interfaces that run under Apache. You need
to pick which one to run.

For SCGI, Apache uses the scgi_mod module to communicate with BackupPC_Admin_SCGI,
which handles the requests. This allows Apache to run as a different user as
backuppc. To use SCGI you need to set SCGIServerPort to any spare
non-privileged TCP port number. A negative value disables SCGI.

Important security warning!! The SCGIServerPort must not be accessible by
anyone untrusted. That means you can't allow untrusted users access to the
BackupPC server, and you should block the SCGIServerPort TCP port from
network access.

The traditional alternative is to use CGI. In this case, an executable needs
to be installed Apache's cgi-bin directory. This executable needs to run as
set-uid backuppc, or it can be run under mod_perl with Apache
running as user backuppc.

--> SCGI port (-1 to disable) [-1]?
--> CGI bin directory (full path, or empty for no CGI) []? /usr/syno/synoman/backuppc

BackupPC's CGI and SCGI script need to display various PNG/GIF
images that should be stored where Apache can serve them. They
should be placed somewhere under Apache's DocumentRoot. BackupPC
also needs to know the URL to access these images. Example:

Apache image directory: /var/www/htdocs/BackupPC
URL for image directory: /BackupPC

The URL for the image directory should start with a slash.

--> Apache image directory (full path, or empty for no S/CGI) []? /usr/syno/synoman/backuppc
--> URL for image directory (omit http://host; starts with '/', or empty for no S/CGI) []? /backuppc

Ok, we're about to:

- install the binaries, lib and docs in /opt/local/backuppc,
- create the data directory /opt/var/lib/backuppc,
- create/update the config.pl file /etc/BackupPC/config.pl,
- optionally install the cgi-bin interface.

--> Do you want to continue? [y]?
Created /opt/local/backuppc/bin
Created /opt/local/backuppc/share/doc/BackupPC
Created /opt/local/backuppc/lib/BackupPC/CGI
Created /opt/local/backuppc/lib/BackupPC/Config
Created /opt/local/backuppc/lib/BackupPC/Lang
Created /opt/local/backuppc/lib/BackupPC/Storage
Created /opt/local/backuppc/lib/BackupPC/Xfer
Created /opt/local/backuppc/lib/BackupPC/Zip
Created /opt/local/backuppc/lib/Net/FTP
Created /opt/var/lib/backuppc
Created /opt/var/lib/backuppc/pool
Created /opt/var/lib/backuppc/cpool
Created /opt/var/lib/backuppc/pc
Created /etc/BackupPC
Created /var/log/BackupPC
Created /var/run/BackupPC
Installing binaries in /opt/local/backuppc/bin
Installing library in /opt/local/backuppc/lib
Installing images in /usr/syno/synoman/backuppc
Making init.d scripts
Making Apache configuration file for suid-perl
Installing docs in /opt/local/backuppc/share/doc/BackupPC
Installing config.pl and hosts in /etc/BackupPC
PING localhost (127.0.0.1): 56 data bytes
64 bytes from 127.0.0.1: seq=0 ttl=64 time=0.087 ms

--- localhost ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.087/0.087/0.087 ms
Installing cgi script BackupPC_Admin in /usr/syno/synoman/backuppc

Ok, it looks like we are finished. There are several more things you
will need to do:

- Browse through the config file, /etc/BackupPC/config.pl,
and make sure all the settings are correct. In particular,
you will need to set $Conf{CgiAdminUsers} so you have
administration privileges in the CGI interface.

- Edit the list of hosts to backup in /etc/BackupPC/hosts.

- Read the documentation in /opt/local/backuppc/doc/BackupPC.html.
Please pay special attention to the security section.

- Verify that the CGI script BackupPC_Admin runs correctly. You might
need to change the permissions or group ownership of BackupPC_Admin.
If this is an upgrade and you are using mod_perl, you will need
to restart Apache. Otherwise it will have stale code.

- BackupPC should be ready to start. Don't forget to run it
as user backuppc! The installation also contains an
init.d/backuppc script that can be copied to /etc/init.d
so that BackupPC can auto-start on boot. This will also enable
administrative users to start the server from the CGI interface.
See init.d/README.

Enjoy!
Mnemosyne> cd /usr/syno/synoman/backuppc
Mnemosyne> mv BackupPC_Admin real-BackupPC_Admin.pl
Mnemosyne> chmod 550 real-BackupPC_Admin.pl
Mnemosyne> cat wrapper.c
#include <unistd.h>
#define REAL_PATH "/usr/syno/synoman/backuppc/real-BackupPC_Admin.pl"
int main(ac, av)
char **av;
{
execv(REAL_PATH, av);
return 0;
}
Mnemosyne> gcc -o BackupPC_Admin wrapper.c
Mnemosyne> chown backuppc BackupPC_Admin
Mnemosyne> chmod 4550 BackupPC_Admin
Mnemosyne> chown backuppc:users BackupPC_Admin
Mnemosyne> mv BackupPC_Admin BackupPC_Admin.cgi
Mnemosyne> chmod 4550 BackupPC_Admin.cgi
-r-sr-x--- 1 backuppc users 8.7K Nov 10 16:20 BackupPC_Admin.cgi
-r-xr-x--- 1 backuppc users 4.0K Nov 10 16:16 real-BackupPC_Admin.pl
-rwxr-xr-x 1 root root 163 Nov 10 16:19 wrapper.c

# import backuppc start up script to /opt/sbin/backuppc
# backuppc requires package procps

Mnemosyne> backuppc status
Mnemosyne> ipkg install procps
Mnemosyne> vi config.pl
Mnemosyne> rmdir pc
Mnemosyne> ln -sf . pc
Mnemosyne> backuppc start
Mnemosyne> cd /etc
Mnemosyne> vi hosts
Mnemosyne> cat hosts
127.0.0.1 localhost
172.19.100.99 Mnemosyne
172.19.100.18 xbmcbuntu
Mnemosyne> curl -k -L http://cpanmin.us | perl - App::cpanminus
Mnemosyne> cpanm File::Listing
Mnemosyne> ipkg install sudo
Mnemosyne> cd /etc/BackupPC/
Mnemosyne> l
total 228K
-rw-r----- 1 backuppc root 0 Nov 10 18:23 LOCK
-rw-r----- 1 backuppc users 82K Nov 10 17:35 config.pl
-rw-r----- 1 root root 82K Nov 10 17:32 config.pl.orig
-rwxr-xr-x 1 root root 838 Nov 10 17:39 griffin.pl
-rw-r--r-- 1 backuppc users 2.3K Nov 10 17:54 hosts
-rwxr-xr-x 1 root root 3.2K Nov 10 17:39 mandala.pl
-rwxr-xr-x 1 root root 821 Nov 10 17:39 nxl67170ux.pl
lrwxrwxrwx 1 root root 1 Nov 10 17:50 pc -> .
-rwxr-xr-x 1 root root 811 Nov 10 17:39 xbmcbuntu.pl
Mnemosyne> vi config.pl
# Edit PingCmd line to use 'sudo'
Mnemosyne> cd /etc/sudoers.d
Mnemosyne> vi backuppc
Mnemosyne> chmod 440 backuppc
Mnemosyne> backuppc start
backuppc started
Mnemosyne>



################################################################""
TODO:
- Delete backuppc db on mnemosyne
</source>

== Install on Lacie-Cloudbox ==
* See http://wiki.qnap.com/wiki/How_to_install_the_BackupPC_application

* Must create a user 'backuppc', and install in the share {{file|/shares/backuppc}}. This include the directory BackupPC and www!
: If we don't, everything is deleted at next boot.

This is for BackupPC 4.0 Alpha 3:
<source lang=text>
#################################
Backuppc 4.0

# Install some modules
# ipkg install samba2 tar rsync # because already available
# Not installing openssh client - let's see the one we have
ipkg install perl par2cmdline
ipkg install perl-compress-zlib perl-archive-zip
ipkg install optware-devel

cd /opt/bin
for f in addr2line ar as c++ c++filt cpp g++ gcc gccbug gcov ld nm objcopy objdump ranlib readelf size strings strip; do if [ -e /opt/bin/$f ]; then F=arm-none-linux-gnueabi-$f; if [ -e /opt/bin/$F ]; then echo $F exists; else echo Creating symlink $F to $f; ln -sf $f /opt/bin/$F; fi ; else echo $f does not exists; fi; done


## >>>>>>>>>>>> ADDED FOR BACKUPPC
cgi.assign += ( "/opt/local/backuppc/cgi-bin/index.cgi" => "" )
#alias.url += ("/backuppc/index.cgi" => "/opt/local/backuppc/cgi-bin/index.cgi")
alias.url += ("/backuppc" => "/opt/local/backuppc/cgi-bin" )
#alias.url += ("/BackupPC_Admin" => "/shares/backuppc/www/cgi-bin/BackupPC_Admin")
auth.debug = 2
auth.backend = "htpasswd"
auth.backend.htpasswd.userfile = "/etc/lighttpd/lighttpd-htpasswd.user"
auth.require = ( "/backuppc" =>
( "method" => "basic", "realm" => "Password protected area", "require" => "user=backuppc" )
)

index-file.names = (
"index.html", "index.php", "index.cgi"
)

# Force HTTPS
$HTTP["scheme"] == "http" {
# capture vhost name with regex conditiona -> %0 in redirect pattern
# must be the most inner block to the redirect rule
$HTTP["host"] =~ ".*" {
url.redirect = (".*" => "https://%0$0")
}
}



########################################################################################################
########################################################################################################
########################################################################################################
########################################################################################################
########################################################################################################
########################################################################################################
########################################################################################################
########################################################################################################
########################################################################################################
########################################################################################################
########################################################################################################

beq06659@nxl67170ux /data/home/beq06659/.ssh (master)
$ ssh lacie
[root@LaCie-CloudBox ~]# cd /opt
[root@LaCie-CloudBox opt]# cd etc
[root@LaCie-CloudBox etc]# l
[root@LaCie-CloudBox etc]# cd backuppc/
[root@LaCie-CloudBox backuppc]# l
[root@LaCie-CloudBox backuppc]# cd ..
[root@LaCie-CloudBox etc]# l
[root@LaCie-CloudBox etc]# mv backuppc backuppc3
[root@LaCie-CloudBox etc]# cd ..
[root@LaCie-CloudBox opt]# l
[root@LaCie-CloudBox opt]# cd local
[root@LaCie-CloudBox local]# l
[root@LaCie-CloudBox local]# rm -rf backuppc
[root@LaCie-CloudBox local]# cd /opt
[root@LaCie-CloudBox opt]# cd var
[root@LaCie-CloudBox var]# l
[root@LaCie-CloudBox var]# cd log
[root@LaCie-CloudBox log]# l
[root@LaCie-CloudBox log]# cd backuppc/
[root@LaCie-CloudBox backuppc]# l
[root@LaCie-CloudBox backuppc]# cd ..
[root@LaCie-CloudBox log]# l
[root@LaCie-CloudBox log]# mv backuppc backuppc3
[root@LaCie-CloudBox log]# l
[root@LaCie-CloudBox log]# cd ..
[root@LaCie-CloudBox var]# cd
[root@LaCie-CloudBox ~]# l
[root@LaCie-CloudBox ~]# mkdir build
[root@LaCie-CloudBox ~]# cd build
[root@LaCie-CloudBox build]# l
[root@LaCie-CloudBox build]# tar
[root@LaCie-CloudBox build]# which tar
[root@LaCie-CloudBox build]# l
[root@LaCie-CloudBox build]# l
[root@LaCie-CloudBox build]# l
total 1.6M
-rw-r--r-- 1 root root 571K Dec 2 2013 BackupPC-4.0.0alpha3.tar.gz
-rw-r--r-- 1 root root 275K Dec 2 2013 BackupPC-XS-0.30.tar.gz
-rw-r--r-- 1 root root 757K Dec 2 2013 rsync-bpc-3.0.9.3.tar.gz
[root@LaCie-CloudBox build]# tar -xvzf BackupPC-4.0.0alpha3.tar.gz
[root@LaCie-CloudBox build]# ls
[root@LaCie-CloudBox build]# tar -xvzf BackupPC-XS-0.30.tar.gz
[root@LaCie-CloudBox build]# tar -xvzf rsync-bpc-3.0.9.3.tar.gz
[root@LaCie-CloudBox build]# ls
[root@LaCie-CloudBox build]# cd BackupPC-XS-0.30
[root@LaCie-CloudBox BackupPC-XS-0.30]# perl Makefile.PL
[root@LaCie-CloudBox BackupPC-XS-0.30]# make
[1]: arm-none-linux-gnueabi-ar: Command not found
make[1]: *** [libzlib.a] Error 127
make[1]: Leaving directory `/root/build/BackupPC-XS-0.30/zlib'
make: *** [zlib/libzlib.a] Error 2
[root@LaCie-CloudBox BackupPC-XS-0.30]# cd /opt/bin
[root@LaCie-CloudBox bin]# l
[root@LaCie-CloudBox bin]# ls ar
ar
[root@LaCie-CloudBox bin]# ipkg install rddtool
Nothing to be done
An error ocurred, return value: 4.
Collected errors:
Cannot find package rddtool.
Check the spelling or perhaps run 'ipkg update'
[root@LaCie-CloudBox bin]# cd /opt
[root@LaCie-CloudBox opt]# cd bin
[root@LaCie-CloudBox bin]# ls
[root@LaCie-CloudBox bin]# l gcc*
[root@LaCie-CloudBox bin]# cd
[root@LaCie-CloudBox ~]# l
[root@LaCie-CloudBox ~]# cd build
[root@LaCie-CloudBox build]# l
[root@LaCie-CloudBox build]# cd BackupPC-XS-0.30
[root@LaCie-CloudBox BackupPC-XS-0.30]# l
[root@LaCie-CloudBox BackupPC-XS-0.30]# make
[root@LaCie-CloudBox bin]# l
[root@LaCie-CloudBox bin]# ls
[root@LaCie-CloudBox bin]# ls powerpc*
[root@LaCie-CloudBox bin]# rm powerpc*
[root@LaCie-CloudBox bin]# ls arm*
[root@LaCie-CloudBox bin]# cd
[root@LaCie-CloudBox ~]# l
[root@LaCie-CloudBox ~]# cd buil
[root@LaCie-CloudBox ~]# cd build
[root@LaCie-CloudBox build]# l
[root@LaCie-CloudBox build]# cd BackupPC-XS-0.30
[root@LaCie-CloudBox BackupPC-XS-0.30]# l
[root@LaCie-CloudBox BackupPC-XS-0.30]# make
[root@LaCie-CloudBox BackupPC-XS-0.30]# make test
[root@LaCie-CloudBox BackupPC-XS-0.30]# make install
[root@LaCie-CloudBox BackupPC-XS-0.30]# cd ..
[root@LaCie-CloudBox build]# l
[root@LaCie-CloudBox build]# cd rsync-bpc-3.0.9.3
[root@LaCie-CloudBox rsync-bpc-3.0.9.3]# l
[root@LaCie-CloudBox rsync-bpc-3.0.9.3]# ./configure
[root@LaCie-CloudBox rsync-bpc-3.0.9.3]# ./configure.sh --prefix=/opt
[root@LaCie-CloudBox rsync-bpc-3.0.9.3]# make
[root@LaCie-CloudBox rsync-bpc-3.0.9.3]# make install
[root@LaCie-CloudBox rsync-bpc-3.0.9.3]# which rsync_bpc
[root@LaCie-CloudBox rsync-bpc-3.0.9.3]# cd ..
[root@LaCie-CloudBox build]# history
973 wget http://sourceforge.net/projects/backuppc/files/backuppc-beta/4.0.0alpha3/BackupPC-XS-0.30.tar.gz/d
974 wget http://sourceforge.net/projects/backuppc/files/backuppc-beta/4.0.0alpha3/BackupPC-XS-0.30.tar.gz/download
975 wget http://sourceforge.net/projects/backuppc/files/backuppc-beta/4.0.0alpha3/rsync-bpc-3.0.9.3.tar.gz/download
976 Mnemosyne> wget http://sourceforge.net/projects/backuppc/files/backuppc-beta/4.0.0alpha3/BackupPC-4.0.0alpha3.tar.gz/download
977 wget http://sourceforge.net/projects/backuppc/files/backuppc-beta/4.0.0alpha3/BackupPC-4.0.0alpha3.tar.gz/download
978 rm rsync-bpc-3.0.9.3.tar.gz.1
979 rm wget
980 tar -xvzf BackupPC-4.0.0alpha3.tar.gz
981 tar -xvzf BackupPC-XS-0.30.tar.gz
982 tar -xvzf rsync-bpc-3.0.9.3.tar.gz
983 perl Makefile.PL
984 ls ar
985 ipkg install rddtool
986 cd /opt
987 cd bin
988 l gcc*
989 for f in addr2line ar as c++ c++filt cpp g++ gcc gccbug gcov ld nm objcopy objdump ranlib readelf size strings strip; do if [ -e /opt/bin/$f ]; then F=powerpc-linux-gnuspe-$f; if [ -e /opt/bin/$F ]; then echo $F exists; else echo Creating symlink $F to $f; ln -sf $f /opt/bin/$F; fi ; else echo $f does not exists; fi; done
990 cd /opt/bin
991 ls
992 ls powerpc*
993 rm powerpc*
994 ls arm*
995 for f in addr2line ar as c++ c++filt cpp g++ gcc gccbug gcov ld nm objcopy objdump ranlib readelf size strings strip; do if [ -e /opt/bin/$f ]; then F=arm-none-linux-gnueabi-$f; if [ -e /opt/bin/$F ]; then echo $F exists; else echo Creating symlink $F to $f; ln -sf $f /opt/bin/$F; fi ; else echo $f does not exists; fi; done
996 cd
997 cd buil
998 cd build
999 cd BackupPC-XS-0.30
1000 make test
1001 cd rsync-bpc-3.0.9.3
1002 l
1003 ./configure.sh --prefix=/opt
1004 make
1005 make install
1006 which rsync_bpc
1007 cd ..
1008 history
[root@LaCie-CloudBox build]# cd /opt
[root@LaCie-CloudBox opt]# cd var
[root@LaCie-CloudBox var]# cd lib
[root@LaCie-CloudBox var]# l
[root@LaCie-CloudBox var]# cd run
[root@LaCie-CloudBox run]# l
[root@LaCie-CloudBox run]# cd ..
[root@LaCie-CloudBox var]# l
[root@LaCie-CloudBox var]# cd
[root@LaCie-CloudBox ~]# cd build
[root@LaCie-CloudBox build]# l
[root@LaCie-CloudBox build]# cd BackupPC-4.0.0alpha3
[root@LaCie-CloudBox BackupPC-4.0.0alpha3]# mkdir -p /opt/var/lib/backuppc
[root@LaCie-CloudBox BackupPC-4.0.0alpha3]# l /opt/var/lib/backuppc
[root@LaCie-CloudBox BackupPC-4.0.0alpha3]# chown backuppc /opt/var/lib/backuppc
[root@LaCie-CloudBox BackupPC-4.0.0alpha3]# l /opt/var/lib
[root@LaCie-CloudBox BackupPC-4.0.0alpha3]# l
[root@LaCie-CloudBox BackupPC-4.0.0alpha3]# ./
[root@LaCie-CloudBox BackupPC-4.0.0alpha3]# ./conf
[root@LaCie-CloudBox BackupPC-4.0.0alpha3]# ./conf
[root@LaCie-CloudBox BackupPC-4.0.0alpha3]# ./conf
[root@LaCie-CloudBox BackupPC-4.0.0alpha3]# ./configure.pl
[root@LaCie-CloudBox BackupPC-4.0.0alpha3]# ipkg install rrdtool
[root@LaCie-CloudBox BackupPC-4.0.0alpha3]# ./configure.pl
[root@LaCie-CloudBox BackupPC-4.0.0alpha3]# ./configure.pl

Is this a new installation or upgrade for BackupPC? If this is
an upgrade please tell me the full path of the existing BackupPC
configuration file (eg: /etc/BackupPC/config.pl). Otherwise, just
hit return.

--> Full path to existing main config.pl []?

I found the following locations for these programs:

bzip2 => /opt/bin/bzip2
cat => /opt/bin/cat
df => /opt/bin/df
gtar/tar => /opt/bin/tar
gzip => /opt/bin/gzip
hostname => /bin/hostname
nmblookup => /usr/bin/nmblookup
par2 => /opt/bin/par2
perl => /opt/bin/perl
ping => /bin/ping
rrdtool => /opt/bin/rrdtool
rsync => /opt/bin/rsync
rsync_bpc => /opt/bin/rsync_bpc
sendmail =>
smbclient =>
split => /opt/bin/split
ssh/ssh2 => /usr/bin/ssh

--> Are these paths correct? [y]?

Please tell me the hostname of the machine that BackupPC will run on.

--> BackupPC will run on host [LaCie-CloudBox]?

BackupPC should run as a dedicated user with limited privileges. You
need to create a user. This user will need read/write permission on
the main data directory and read/execute permission on the install
directory (these directories will be setup shortly).

The primary group for this user should also be chosen carefully.
The data directories and files will have group read permission,
so group members can access backup files.

--> BackupPC should run as user [backuppc]?

Please specify an install directory for BackupPC. This is where the
BackupPC scripts, library and documentation will be installed.

--> Install directory (full path) [/usr/local/BackupPC]? /opt/local/backuppc

Please specify a data directory for BackupPC. This is where all the
PC backup data is stored. This file system needs to be big enough to
accommodate all the PCs you expect to backup (eg: at least several GB
per machine).

--> Data directory (full path) [/data/BackupPC]? /opt/var/lib/backuppc

BackupPC has SCGI and CGI perl interfaces that run under Apache. You need
to pick which one to run.

For SCGI, Apache uses the scgi_mod module to communicate with BackupPC_Admin_SCGI,
which handles the requests. This allows Apache to run as a different user as
backuppc. To use SCGI you need to set SCGIServerPort to any spare
non-privileged TCP port number. A negative value disables SCGI.

Important security warning!! The SCGIServerPort must not be accessible by
anyone untrusted. That means you can't allow untrusted users access to the
BackupPC server, and you should block the SCGIServerPort TCP port from
network access.

The traditional alternative is to use CGI. In this case, an executable needs
to be installed Apache's cgi-bin directory. This executable needs to run as
set-uid backuppc, or it can be run under mod_perl with Apache
running as user backuppc.

--> SCGI port (-1 to disable) [-1]?
--> CGI bin directory (full path, or empty for no CGI) []? /opt/lib/backuppc

BackupPC's CGI and SCGI script need to display various PNG/GIF
images that should be stored where Apache can serve them. They
should be placed somewhere under Apache's DocumentRoot. BackupPC
also needs to know the URL to access these images. Example:

Apache image directory: /var/www/htdocs/BackupPC
URL for image directory: /BackupPC

The URL for the image directory should start with a slash.

--> Apache image directory (full path, or empty for no S/CGI) []? /opt/local/backuppc/cgi-bin
--> URL for image directory (omit http://host; starts with '/', or empty for no S/CGI) []? /backuppc

Ok, we're about to:

- install the binaries, lib and docs in /opt/local/backuppc,
- create the data directory /opt/var/lib/backuppc,
- create/update the config.pl file /etc/BackupPC/config.pl,
- optionally install the cgi-bin interface.

--> Do you want to continue? [y]?
Created /opt/local/backuppc/bin
Created /opt/local/backuppc/share/doc/BackupPC
Created /opt/local/backuppc/lib/BackupPC/CGI
Created /opt/local/backuppc/lib/BackupPC/Config
Created /opt/local/backuppc/lib/BackupPC/Lang
Created /opt/local/backuppc/lib/BackupPC/Storage
Created /opt/local/backuppc/lib/BackupPC/Xfer
Created /opt/local/backuppc/lib/BackupPC/Zip
Created /opt/local/backuppc/lib/Net/FTP
Created /opt/local/backuppc/cgi-bin
Created /opt/var/lib/backuppc
Created /opt/var/lib/backuppc/pool
Created /opt/var/lib/backuppc/cpool
Created /opt/var/lib/backuppc/pc
Created /etc/BackupPC
Created /var/log/BackupPC
Created /var/run/BackupPC
Installing binaries in /opt/local/backuppc/bin
Installing library in /opt/local/backuppc/lib
Installing images in /opt/local/backuppc/cgi-bin
Making init.d scripts
Making Apache configuration file for suid-perl
Installing docs in /opt/local/backuppc/share/doc/BackupPC
Installing config.pl and hosts in /etc/BackupPC
PING localhost (127.0.0.1): 56 data bytes
64 bytes from 127.0.0.1: seq=0 ttl=64 time=0.325 ms

--- localhost ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.325/0.325/0.325 ms
Installing cgi script BackupPC_Admin in /opt/lib/backuppc

Ok, it looks like we are finished. There are several more things you
will need to do:

- Browse through the config file, /etc/BackupPC/config.pl,
and make sure all the settings are correct. In particular,
you will need to set $Conf{CgiAdminUsers} so you have
administration privileges in the CGI interface.

- Edit the list of hosts to backup in /etc/BackupPC/hosts.

- Read the documentation in /opt/local/backuppc/doc/BackupPC.html.
Please pay special attention to the security section.

- Verify that the CGI script BackupPC_Admin runs correctly. You might
need to change the permissions or group ownership of BackupPC_Admin.
If this is an upgrade and you are using mod_perl, you will need
to restart Apache. Otherwise it will have stale code.

- BackupPC should be ready to start. Don't forget to run it
as user backuppc! The installation also contains an
init.d/backuppc script that can be copied to /etc/init.d
so that BackupPC can auto-start on boot. This will also enable
administrative users to start the server from the CGI interface.
See init.d/README.

Enjoy!
[root@LaCie-CloudBox BackupPC-4.0.0alpha3]# cd /etc
[root@LaCie-CloudBox etc]# l
[root@LaCie-CloudBox etc]# ls
[root@LaCie-CloudBox etc]# cd BackupPC/
[root@LaCie-CloudBox BackupPC]# l
[root@LaCie-CloudBox BackupPC]# cd /opt/local
[root@LaCie-CloudBox local]# l
[root@LaCie-CloudBox local]# cd backuppc/
[root@LaCie-CloudBox backuppc]# l
[root@LaCie-CloudBox backuppc]# cd bin
[root@LaCie-CloudBox bin]# ls
[root@LaCie-CloudBox bin]# cd ../cgi-bin/
[root@LaCie-CloudBox cgi-bin]# l
[root@LaCie-CloudBox cgi-bin]# ls
[root@LaCie-CloudBox cgi-bin]# cd /var
[root@LaCie-CloudBox var]# cd log
[root@LaCie-CloudBox log]# l
[root@LaCie-CloudBox log]# cd
[root@LaCie-CloudBox ~]# cd /etc
[root@LaCie-CloudBox etc]# rm -rf BackupPC/
[root@LaCie-CloudBox etc]# cd /var/log
[root@LaCie-CloudBox log]# l
[root@LaCie-CloudBox log]# rm -rf BackupPC/
[root@LaCie-CloudBox log]# ls
[root@LaCie-CloudBox log]# cd /var/run
[root@LaCie-CloudBox run]# l
[root@LaCie-CloudBox run]# ls
[root@LaCie-CloudBox run]# rm -rf BackupPC/
[root@LaCie-CloudBox run]# cd /opt
[root@LaCie-CloudBox opt]# cd local
[root@LaCie-CloudBox local]# l
[root@LaCie-CloudBox local]# rm -rf backuppc/
[root@LaCie-CloudBox local]# cd ../lib
[root@LaCie-CloudBox lib]# l
[root@LaCie-CloudBox lib]# ls
[root@LaCie-CloudBox lib]# cd backuppc/
[root@LaCie-CloudBox backuppc]# m
[root@LaCie-CloudBox backuppc]# l
[root@LaCie-CloudBox backuppc]# cd ..
[root@LaCie-CloudBox lib]# rm -rf backuppc
[root@LaCie-CloudBox lib]# cd
[root@LaCie-CloudBox ~]# cd biuld
[root@LaCie-CloudBox ~]# cd build
[root@LaCie-CloudBox build]# cd BackupPC-4.0.0alpha3
[root@LaCie-CloudBox BackupPC-4.0.0alpha3]# ./conf
[root@LaCie-CloudBox BackupPC-4.0.0alpha3]# cd /opt
[root@LaCie-CloudBox opt]# cd etc
[root@LaCie-CloudBox etc]# cd backuppc
[root@LaCie-CloudBox backuppc]# l
[root@LaCie-CloudBox backuppc]# ln -sf . pc
[root@LaCie-CloudBox backuppc]# cd /opt/var/log/
[root@LaCie-CloudBox log]# l
[root@LaCie-CloudBox log]# cd ..
[root@LaCie-CloudBox var]# ls
[root@LaCie-CloudBox var]# cd backuppc/
[root@LaCie-CloudBox backuppc]# l
[root@LaCie-CloudBox backuppc]# cd ..
[root@LaCie-CloudBox var]# history
978 for f in addr2line ar as c++ c++filt cpp g++ gcc gccbug gcov ld nm objcopy objdump ranlib readelf size strings strip; do if [ -e /opt/bin/$f ]; then F=arm-none-linux-gnueabi-$f; if [ -e /opt/bin/$F ]; then echo $F exists; else echo Creating symlink $F to $f; ln -sf $f /opt/bin/$F; fi ; else echo $f does not exists; fi; done
979 cd buil
980 cd BackupPC-XS-0.30
981 make test
982 cd rsync-bpc-3.0.9.3
983 ./configure.sh --prefix=/opt
984 make
985 make install
986 which rsync_bpc
987 cd var
988 cd lib
989 cd run
990 mkdir -p /opt/var/lib/backuppc
991 l /opt/var/lib/backuppc
992 chown backuppc /opt/var/lib/backuppc
993 l /opt/var/lib
994 ipkg install rrdtool
995 ./configure.pl
996 cd BackupPC/
997 cd /opt/local
998 cd bin
999 cd ../cgi-bin/
1000 cd /var
1001 cd log
1002 cd /etc
1003 cd /var/log
1004 cd /var/run
1005 rm -rf BackupPC/
1006 cd local
1007 rm -rf backuppc/
1008 cd ../lib
1009 m
1010 rm -rf backuppc
1011 cd
1012 cd biuld
1013 cd build
1014 cd BackupPC-4.0.0alpha3
1015 ./configure.pl --config-dir=/opt/etc/backuppc --log-dir=/opt/var/backuppc --run-dir=/opt/var/run/backuppc
1016 cd /opt
1017 cd etc
1018 cd backuppc
1019 ln -sf . pc
1020 cd /opt/var/log/
1021 ls
1022 cd backuppc/
1023 l
1024 cd ..
1025 history
[root@LaCie-CloudBox var]# cd /opt
[root@LaCie-CloudBox opt]# cd etc
[root@LaCie-CloudBox etc]# rm -rf backuppc
[root@LaCie-CloudBox etc]# cd /opt
[root@LaCie-CloudBox opt]# cd var
[root@LaCie-CloudBox var]# l
[root@LaCie-CloudBox var]# rm -rf backuppc/
[root@LaCie-CloudBox var]# cd run
[root@LaCie-CloudBox run]# l
[root@LaCie-CloudBox run]# rm -rf backuppc/
[root@LaCie-CloudBox run]# l
[root@LaCie-CloudBox run]# cd /opt/local/
[root@LaCie-CloudBox local]# l
[root@LaCie-CloudBox local]# rm -rf backuppc/
[root@LaCie-CloudBox local]# cd ../lib
[root@LaCie-CloudBox lib]# l
[root@LaCie-CloudBox lib]# ls
[root@LaCie-CloudBox lib]# rm -rf backuppc/
[root@LaCie-CloudBox lib]# cd /shares/
[root@LaCie-CloudBox shares]# cd backuppc/
[root@LaCie-CloudBox backuppc]# rm -rf *
[root@LaCie-CloudBox backuppc]# cd
[root@LaCie-CloudBox ~]# cd build
[root@LaCie-CloudBox build]# cd BackupPC-4.0.0alpha3
[root@LaCie-CloudBox BackupPC-4.0.0alpha3]# ./configure.pl --config-dir=/opt/etc/backuppc --log-dir=/opt/var/log/backuppc --run-dir=/opt/var/run/bac>

Is this a new installation or upgrade for BackupPC? If this is
an upgrade please tell me the full path of the existing BackupPC
configuration file (eg: /etc/BackupPC/config.pl). Otherwise, just
hit return.

--> Full path to existing main config.pl []?

I found the following locations for these programs:

bzip2 => /opt/bin/bzip2
cat => /opt/bin/cat
df => /opt/bin/df
gtar/tar => /opt/bin/tar
gzip => /opt/bin/gzip
hostname => /bin/hostname
nmblookup => /usr/bin/nmblookup
par2 => /opt/bin/par2
perl => /opt/bin/perl
ping => /bin/ping
rrdtool => /opt/bin/rrdtool
rsync => /opt/bin/rsync
rsync_bpc => /opt/bin/rsync_bpc
sendmail =>
smbclient =>
split => /opt/bin/split
ssh/ssh2 => /usr/bin/ssh

--> Are these paths correct? [y]?

Please tell me the hostname of the machine that BackupPC will run on.

--> BackupPC will run on host [LaCie-CloudBox]?

BackupPC should run as a dedicated user with limited privileges. You
need to create a user. This user will need read/write permission on
the main data directory and read/execute permission on the install
directory (these directories will be setup shortly).

The primary group for this user should also be chosen carefully.
The data directories and files will have group read permission,
so group members can access backup files.

--> BackupPC should run as user [backuppc]?

Please specify an install directory for BackupPC. This is where the
BackupPC scripts, library and documentation will be installed.

--> Install directory (full path) [/usr/local/BackupPC]? /opt/local/backuppc

Please specify a data directory for BackupPC. This is where all the
PC backup data is stored. This file system needs to be big enough to
accommodate all the PCs you expect to backup (eg: at least several GB
per machine).

--> Data directory (full path) [/data/BackupPC]? /shares/backuppc

BackupPC has SCGI and CGI perl interfaces that run under Apache. You need
to pick which one to run.

For SCGI, Apache uses the scgi_mod module to communicate with BackupPC_Admin_SCGI,
which handles the requests. This allows Apache to run as a different user as
backuppc. To use SCGI you need to set SCGIServerPort to any spare
non-privileged TCP port number. A negative value disables SCGI.

Important security warning!! The SCGIServerPort must not be accessible by
anyone untrusted. That means you can't allow untrusted users access to the
BackupPC server, and you should block the SCGIServerPort TCP port from
network access.

The traditional alternative is to use CGI. In this case, an executable needs
to be installed Apache's cgi-bin directory. This executable needs to run as
set-uid backuppc, or it can be run under mod_perl with Apache
running as user backuppc.

--> SCGI port (-1 to disable) [-1]?
--> CGI bin directory (full path, or empty for no CGI) []? /opt/lib/backuppc

BackupPC's CGI and SCGI script need to display various PNG/GIF
images that should be stored where Apache can serve them. They
should be placed somewhere under Apache's DocumentRoot. BackupPC
also needs to know the URL to access these images. Example:

Apache image directory: /var/www/htdocs/BackupPC
URL for image directory: /BackupPC

The URL for the image directory should start with a slash.

--> Apache image directory (full path, or empty for no S/CGI) []? /opt/local/backuppc/cgi-bin
--> URL for image directory (omit http://host; starts with '/', or empty for no S/CGI) []? /backuppc

Ok, we're about to:

- install the binaries, lib and docs in /opt/local/backuppc,
- create the data directory /shares/backuppc,
- create/update the config.pl file /opt/etc/backuppc/config.pl,
- optionally install the cgi-bin interface.

--> Do you want to continue? [y]?
Created /opt/local/backuppc/bin
Created /opt/local/backuppc/share/doc/BackupPC
Created /opt/local/backuppc/lib/BackupPC/CGI
Created /opt/local/backuppc/lib/BackupPC/Config
Created /opt/local/backuppc/lib/BackupPC/Lang
Created /opt/local/backuppc/lib/BackupPC/Storage
Created /opt/local/backuppc/lib/BackupPC/Xfer
Created /opt/local/backuppc/lib/BackupPC/Zip
Created /opt/local/backuppc/lib/Net/FTP
Created /opt/local/backuppc/cgi-bin
Created /shares/backuppc
Created /shares/backuppc/pool
Created /shares/backuppc/cpool
Created /shares/backuppc/pc
Created /opt/etc/backuppc
Created /opt/var/log/backuppc
Created /opt/var/run/backuppc
Installing binaries in /opt/local/backuppc/bin
Installing library in /opt/local/backuppc/lib
Installing images in /opt/local/backuppc/cgi-bin
Making init.d scripts
Making Apache configuration file for suid-perl
Installing docs in /opt/local/backuppc/share/doc/BackupPC
Installing config.pl and hosts in /opt/etc/backuppc
PING localhost (127.0.0.1): 56 data bytes
64 bytes from 127.0.0.1: seq=0 ttl=64 time=0.336 ms

--- localhost ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.336/0.336/0.336 ms
Installing cgi script BackupPC_Admin in /opt/lib/backuppc

Ok, it looks like we are finished. There are several more things you
will need to do:

- Browse through the config file, /opt/etc/backuppc/config.pl,
and make sure all the settings are correct. In particular,
you will need to set $Conf{CgiAdminUsers} so you have
administration privileges in the CGI interface.

- Edit the list of hosts to backup in /opt/etc/backuppc/hosts.

- Read the documentation in /opt/local/backuppc/doc/BackupPC.html.
Please pay special attention to the security section.

- Verify that the CGI script BackupPC_Admin runs correctly. You might
need to change the permissions or group ownership of BackupPC_Admin.
If this is an upgrade and you are using mod_perl, you will need
to restart Apache. Otherwise it will have stale code.

- BackupPC should be ready to start. Don't forget to run it
as user backuppc! The installation also contains an
init.d/backuppc script that can be copied to /etc/init.d
so that BackupPC can auto-start on boot. This will also enable
administrative users to start the server from the CGI interface.
See init.d/README.

Enjoy!
[root@LaCie-CloudBox BackupPC-4.0.0alpha3]# cd /opt/etc/backuppc
[root@LaCie-CloudBox backuppc]# ln -sf . pc
[root@LaCie-CloudBox backuppc]# cd /opt/sbin
[root@LaCie-CloudBox sbin]# vi backuppc
[root@LaCie-CloudBox backuppc]# cd /etc
[root@LaCie-CloudBox etc]# cd lighttpd/
[root@LaCie-CloudBox lighttpd]# cp lighttpd.conf lighttpd.conf-bpc3
[root@LaCie-CloudBox lighttpd]# vi lighttpd.conf
[root@LaCie-CloudBox lighttpd]# cd /opt/local/backuppc/cgi-bin/
[root@LaCie-CloudBox cgi-bin]# vi wrapper.c
[root@LaCie-CloudBox cgi-bin]# cat wrapper.c
#include <unistd.h>
#define REAL_PATH "/opt/lib/backuppc/BackupPC_Admin"
int main(ac, av)
char **av;
{
execv(REAL_PATH, av);
return 0;
}
[root@LaCie-CloudBox cgi-bin]# chmod 550 /opt/lib/backuppc/BackupPC_Admin
[root@LaCie-CloudBox cgi-bin]# gcc -o index.cgi wrapper.c
/opt/lib/gcc/arm-none-linux-gnueabi/4.2.3/../../../../arm-none-linux-gnueabi/bin/ld: Warning: /lib/libc.so.6: Unknown EABI object attribute 44
[root@LaCie-CloudBox cgi-bin]# l
[root@LaCie-CloudBox cgi-bin]# chown backuppc:users index.cgi
[root@LaCie-CloudBox cgi-bin]# chmod 4550 index.cgi
[root@LaCie-CloudBox cgi-bin]# vi /etc/lighttpd/lighttpd.conf
[root@LaCie-CloudBox cgi-bin]# ngc --stop http
[root@LaCie-CloudBox cgi-bin]# ngc --start http
[root@LaCie-CloudBox cgi-bin]# vi /etc/lighttpd/lighttpd.conf
[root@LaCie-CloudBox cgi-bin]# ngc --restart http
[root@LaCie-CloudBox cgi-bin]# ngc --stop http
[root@LaCie-CloudBox cgi-bin]# ngc --start http
[root@LaCie-CloudBox cgi-bin]# vi /etc/lighttpd/lighttpd.conf
[root@LaCie-CloudBox cgi-bin]# cat /etc/lighttpd/lighttpd.conf
server.modules = (
"mod_redirect",
"mod_cgi",
"mod_expire",
"mod_compress",
"mod_proxy",
"mod_fastcgi",
"mod_rewrite",
"mod_setenv",
"mod_secdownload",
"mod_access",
"mod_alias",
"mod_auth",
)

include "secure_password.inc"

server.document-root = "/www"
server.errorlog = "/var/log/lighttpd-error.log"
server.pid-file = "/var/run/lighttpd.pid"
server.upload-dirs = ( "/lacie/tmp" )
server.errorfile-prefix = "/www/errors/status-"

compress.allowed-encodings = ("gzip", "deflate")
compress.cache-dir = "/var/tmp/lighttpd"
compress.filetype = ("text/plain", "text/html", "text/javascript", "text/css", "text/xml")

# Provide a "static" url to retrieve information usefull for production tool for instance.
url.rewrite-once = ("^/info$" => "/api/external/1.0/system_info","^/diagnostics$" => "/api/external/1.0/diagnostics")

index-file.names = (
"index.html", "index.php", "index.cgi"
)

# Force HTTPS
$HTTP["scheme"] == "http" {
# capture vhost name with regex conditiona -> %0 in redirect pattern
# must be the most inner block to the redirect rule
$HTTP["host"] =~ ".*" {
url.redirect = (".*" => "https://%0$0")
}
}

$HTTP["url"] =~ "index"{
setenv.add-response-header = ( "Cache-Control" => "no-cache, no-store" )
}

# For GWT
$HTTP["url"] =~ "\.nocache\."{
setenv.add-response-header = ( "Cache-Control" => "no-cache, no-store" )
}

# For GWT
$HTTP["url"] =~ "\.cache\."{
expire.url = ( "" => "access 1 months" )
setenv.add-response-header = ( "Cache-Control" => "public" )
}


$HTTP["url"] =~ "(gif|png|jpg|css)$"{
expire.url = ( "" => "access 1 months" )
setenv.add-response-header = ( "Cache-Control" => "public" )
}

$HTTP["url"] =~ "/javascripts/"{
expire.url = ( "" => "access 1 months" )
setenv.add-response-header = ( "Cache-Control" => "public" )
}

$HTTP["url"] =~ "/api/local/"{
url.access-deny = ("")
}

## >>>>>>>>>>>> ADDED FOR BACKUPPC
cgi.assign += ( "/opt/local/backuppc/cgi-bin/index.cgi" => "" )
#alias.url += ("/backuppc/index.cgi" => "/opt/local/backuppc/cgi-bin/index.cgi")
alias.url += ("/backuppc" => "/opt/local/backuppc/cgi-bin" )
#alias.url += ("/BackupPC_Admin" => "/shares/backuppc/www/cgi-bin/BackupPC_Admin")
auth.debug = 2
auth.backend = "htpasswd"
auth.backend.htpasswd.userfile = "/etc/lighttpd/lighttpd-htpasswd.user"
auth.require = ( "/backuppc" =>
( "method" => "basic", "realm" => "Password protected area", "require" => "user=backuppc" )
)

proxy.server = (
"/api" => (
"tornado" => (
"host" => "127.0.0.1",
"port" => 8888
))
)

fastcgi.server = (
".php"=>
((
"bin-path" => "/usr/bin/php-cgi",
"socket"=> "/tmp/php.socket",
"max-procs" => 1,
"bin-environment" => (
"PHP_FCGI_CHILDREN" => "2",
"PHP_FCGI_MAX_REQUESTS" => "1500"
),
"allow-x-send-file" => "enable",
)),
)

mimetype.assign = (
".pdf" => "application/pdf",
".sig" => "application/pgp-signature",
".spl" => "application/futuresplash",
".class" => "application/octet-stream",
".ps" => "application/postscript",
".torrent" => "application/x-bittorrent",
".dvi" => "application/x-dvi",
".gz" => "application/x-gzip",
".pac" => "application/x-ns-proxy-autoconfig",
".swf" => "application/x-shockwave-flash",
".tar.gz" => "application/x-tgz",
".tgz" => "application/x-tgz",
".tar" => "application/x-tar",
".zip" => "application/zip",
".mp3" => "audio/mpeg",
".m3u" => "audio/x-mpegurl",
".wma" => "audio/x-ms-wma",
".wax" => "audio/x-ms-wax",
".ogg" => "application/ogg",
".wav" => "audio/x-wav",
".gif" => "image/gif",
".jpg" => "image/jpeg",
".jpeg" => "image/jpeg",
".png" => "image/png",
".xbm" => "image/x-xbitmap",
".xpm" => "image/x-xpixmap",
".xwd" => "image/x-xwindowdump",
".css" => "text/css",
".html" => "text/html",
".htm" => "text/html",
".js" => "text/javascript",
".asc" => "text/plain",
".c" => "text/plain",
".cpp" => "text/plain",
".log" => "text/plain",
".conf" => "text/plain",
".text" => "text/plain",
".txt" => "text/plain",
".dtd" => "text/xml",
".xml" => "text/xml",
".mpeg" => "video/mpeg",
".mpg" => "video/mpeg",
".mov" => "video/quicktime",
".qt" => "video/quicktime",
".avi" => "video/x-msvideo",
".asf" => "video/x-ms-asf",
".asx" => "video/x-ms-asf",
".wmv" => "video/x-ms-wmv",
".bz2" => "application/x-bzip",
".tbz" => "application/x-bzip-compressed-tar",
".tar.bz2" => "application/x-bzip-compressed-tar",
# default mime type
"" => "application/octet-stream",
)

# Redirect all requests to HTTPS, except cgi calls to stay compatible with LEA
#$SERVER["socket"] == ":80" {
# $HTTP["host"] =~ "(.*)" {
# url.redirect = ( "^/(?!cgi)(.*)" => "https://%1/$1" )
# }
#}

# Activate HTTPS
$SERVER["socket"] == ":443" {
ssl.engine = "enable"
ssl.pemfile = "/etc/ssl/private/lighttpd.pem"
}
[root@LaCie-CloudBox cgi-bin]# cd /opt
[root@LaCie-CloudBox opt]# cd etc
[root@LaCie-CloudBox etc]# cd backuppc
[root@LaCie-CloudBox backuppc]# vi hosts
[root@LaCie-CloudBox etc]# cd backuppc3
[root@LaCie-CloudBox backuppc3]# cp griffin.pl mandala.pl nxl67170ux.pl pc-112-597.pl ../backuppc/
[root@LaCie-CloudBox backuppc3]# cp xbmcbuntu.pl ../backuppc
[root@LaCie-CloudBox backuppc3]# cd ../backuppc
[root@LaCie-CloudBox backuppc]# cp config.pl config.pl.orig
[root@LaCie-CloudBox backuppc]# chown backuppc:users *
[root@LaCie-CloudBox backuppc]# chown backuppc:root *
[root@LaCie-CloudBox backuppc]# chmod 640 *
[root@LaCie-CloudBox backuppc]# backuppc start
[root@LaCie-CloudBox backuppc]# cd /etc
[root@LaCie-CloudBox etc]# cd /opt/etc/backuppc
[root@LaCie-CloudBox backuppc]# vi config.pl
[root@LaCie-CloudBox backuppc]# cd /opt/local/backuppc/lib
[root@LaCie-CloudBox lib]# cd BackupPC/
[root@LaCie-CloudBox BackupPC]# vi Lib.pm
[root@LaCie-CloudBox BackupPC-4.0.0alpha3]# echo ############### THIS IS IT
[root@LaCie-CloudBox cgi-bin]# ./index
YOU HAVEN'T DISABLED SET-ID SCRIPTS IN THE KERNEL YET!
FIX YOUR KERNEL, PUT A C WRAPPER AROUND THIS SCRIPT, OR USE -u AND UNDUMP!
[root@LaCie-CloudBox cgi-bin]# cat wrapper.c
#include <unistd.h>
#define REAL_PATH "/opt/lib/backuppc/BackupPC_Admin"
int main(ac, av)
char **av;
{
execv(REAL_PATH, av);
return 0;
}

[root@LaCie-CloudBox cgi-bin]# l /opt/lib/backuppc/BackupPC_Admin
-r-sr-xr-- 1 backuppc users 4.0K Nov 11 00:28 /opt/lib/backuppc/BackupPC_Admin
[root@LaCie-CloudBox cgi-bin]# mv index index.cgi
[root@LaCie-CloudBox cgi-bin]# chmod 0550 /opt/lib/backuppc/BackupPC_Admin
[root@LaCie-CloudBox cgi-bin]# ./index.cgi
[root@LaCie-CloudBox cgi-bin]# backuppc start
[root@LaCie-CloudBox backuppc]# sudo chown backuppc:root *
[root@LaCie-CloudBox backuppc]# sudo chmod 640 *
[root@LaCie-CloudBox backuppc]# su backuppc -s /bin/bash
[root@LaCie-CloudBox etc]#
</source>

== Install BackupPC clients ==
Choices:
; rsyncd
: Can do a full backup w/o giving access to root. access can be restricted to read-only so that backup server cannot do damage to client if compromised. Requires setting up a server and install xinetd. Authentication is weak though, but not much a problem (files are already available on the backup server anyway). Quite simple to configure. I don't like xinetd/inetd though.
; ssh+rsync
: Login on client as user 'backuppc'. SSH private key on backuppc is password-less, but similar to storing a secret password in some file on the backup server. Root access on the client given via sudo, and restricted to run "rsync --server --sender *", such that to enforce read-only access. All traffic goes via ssh, and hence is encrypted. Strong authentication. A bit more complex to configure, but no xinetd, and no daemon.
: See http://www.systemajik.com/blog/setting-up-backuppc-on-ubuntu/ for more information.

ssh+rsync is more difficult to setup; also on the NAS, the backuppc does not have dedicated home directory. So we can't easily store .ssh/id_rsa.pub and .ssh/known_hosts file. Also, on Windows client, it seems the default solution is to use rsyncd. Also, I don't really need link encryption.

We choose '''rsyncd''':
* See https://help.ubuntu.com/community/rsync#Rsync_Daemon .
* http://www.systemajik.com/blog/setting-up-backuppc-on-ubuntu/ (for exclusion of some paths).

Install log:
<source lang="bash">
vi /etc/default/rsync
sudo apt-get install xinetd
vi /etc/xinetd.d/rsync
# service rsync
# {
# disable = no
# socket_type = stream
# wait = no
# user = root
# server = /usr/bin/rsync
# server_args = --daemon
# log_on_failure += USERID
# flags = IPv6
# }
sudo vi /etc/rsyncd.conf
# max connections = 2
# log file = /var/log/rsync.log
# timeout = 6000
#
# [share]
# comment = Public Share
# path = /home/share
# read only = yes
# list = yes
# uid = nobody
# gid = nogroup
# auth users = backuppc
# secrets file = /etc/rsyncd.secrets

# (Note: default timeout 300 is not enough for our NAS - we get child exited prematurely errors)

sudo vi /etc/rsyncd.secrets
# user:password
sudo chmod 600 /etc/rsyncd.secrets
sudo /etc/init.d/xinetd restart

# Testing
sudo rsync backuppc@hostname::share
</source>

== Benchmarks ==

Some statistics:

<source lang="text">
0 full yes 0 11/5 20:19 1.8 0.0 /shares/backuppc/data/pc/nxl67170ux/0
1 incr no 1 11/5 20:58 2.7 0.0 /shares/backuppc/data/pc/nxl67170ux/1

0 full 3565 73.2 0.68 884 4.5 3425 68.8
1 incr 270 201.8 1.25 29 0.0 480 201.8

0 full 3 4.5 2.9 35.4% 68.8 56.8 17.5%
1 incr 3 0.0 0.0 -6.4% 201.8 199.1 1.3%
</source>

Performance not very good. Let's disable compression and do again. I get 1.25MB/s for first full backup.
When pinging nxl67170ux, I get my wlan0 address!
Let's disable WIFI and start again...


== Troubleshooting ==
== Troubleshooting ==
Line 69: Line 1,548:
;Notes
;Notes
* Still use the old parameter <code>--log-format</code>, which has been deprecated and renamed to <code>--out-format</code> in newer rsync version.
* Still use the old parameter <code>--log-format</code>, which has been deprecated and renamed to <code>--out-format</code> in newer rsync version.

;Out-of-memory
* Look at <code>dmesg</code> output, and see if there was an Out of memory error:
<source lang=bash>
dmesg|grep -iC 3 "out of memory"
grep -iC 3 "out of memory" /var/log/messages # To get real timestamps
</source>
<pre>
[3421299.359987] 3205 slab pages
[3421299.359993] 7010 pages shared
[3421299.360000] 8723 pages swap cached
[3421299.360010] Out of memory: kill process 15163 (BackupPC_dump) score 277779 or a child
[3421299.360049] Killed process 15168 (rsync_bpc)
[3421299.734728] rsync_bpc: page allocation failure. order:0, mode:0x20058
[3421299.734750] [<c0031b2c>] (unwind_backtrace+0x0/0xd0) from [<c0080eec>] (__alloc_pages_nodemask+0x49c/0x500)
</pre>
:If so, free some memory. Possibly remove hard-link detection which consumes memory [https://rsync.samba.org/FAQ.html].


==== Zombie rsync_bpc processes ====
==== Zombie rsync_bpc processes ====
Line 125: Line 1,621:
</source>
</source>


=== rsync_bpc keep transfering same file on ntfs-3g mount ===
=== FIXED - rsync_bpc keep transfering same file on ntfs-3g mount ===
{| class=wikitable
To test this, add to {{file|/etc/rsyncd.conf}}:
|-
|'''{{green|1=SOLUTION: Remove options <code>--links</code> and <code>--hard-links</code> when backing up ntfs-3g partitions !!!}}'''
|}


;Setup
* {{file|/c}} is a ntfs-3g mount. Content of {{file|/etc/fstab}}:
UUID=445E1B975E1B813A /c ntfs ro,defaults,nls=utf8,umask=007,gid=46 0 0
* To test this, add to {{file|/etc/rsyncd.conf}}:
<source lang=diff>
<source lang=diff>
+[test1]
+[test1]
Line 132: Line 1,637:
+path = /c/Boot
+path = /c/Boot
</source>
</source>
* In a separate window, monitor the rsyncd log file:
tail -f /var/log/rsync.log
* Create rsync password file:
sudo -v; sudo sed -r 's/^.*://' /etc/rsyncd.secrets | sudo tee /tmp/rsyncd.secrets > /dev/null; sudo chmod 600 /tmp/rsyncd.secrets


;Test case one - PASS - rsync + ntfs-3g
{{file|/c}} is a ntfs-3g mount:
* Initialize:
UUID=445E1B975E1B813A /c ntfs defaults,nls=utf8,umask=007,gid=46 0 0
sudo rm -rf /tmp/rsynctest; sudo mkdir /tmp/rsynctest

* Repeat the following step, and observe that files are only txfed the first time:
In a separate window, monitor the rsyncd log file:
tail -f /var/log/rsyncd.log

With <code>rsync</code>, the sync is done correctly. Files are no longer transfered on second run. To verify, repeat the command below twice:
mkdir /tmp/rsynctest
sudo rsync --super --recursive --protect-args --numeric-ids --perms --owner --group -D --times \
sudo rsync --super --recursive --protect-args --numeric-ids --perms --owner --group -D --times \
--links --hard-links --delete --partial --log-format='log: %o %i %B %8U,%8G %9l %f%L' --stats \
--links --hard-links --delete --partial --log-format='log: %o %i %B %8U,%8G %9l %f%L' --stats \
--modify-window=5 --one-file-system --password-file=/tmp/rsyncd.secrets \
--modify-window=5 --one-file-system --password-file=/tmp/rsyncd.secrets \
--checksum backuppc@griffin::test1 /tmp/rsynctest
--checksum backuppc@griffin::test1 /tmp/rsynctest


;Test case two - FAIL - rsync_bpc + ntfs-3g
But with <code>rsync_bpc</code>, files are always transfered, and reported as identical after transfer.
* Initialize:
i=0
cd /tmp/rsynctest
sudo rm -rf /tmp/rsynctest; sudo mkdir -p /tmp/rsynctest/cpool; i=0
* Repeat the following step, and observe that files are always txfed:
mkdir cpool
cd /tmp/rsynctest; i=$((i+1)); echo backup $i; sudo cp -ar pc/griffin/$((i-1)) pc/griffin/$i; \

sudo rsync_bpc --bpc-top-dir /tmp/rsynctest --bpc-host-name griffin --bpc-share-name test1 --bpc-bkup-num $i \
Repeat the command below:
--bpc-bkup-comp 3 --bpc-bkup-prevnum $((i-1)) --bpc-bkup-prevcomp 3 --bpc-log-level 3 --super --recursive \

cd /tmp/rsynctest; i=$((i+1)); echo backup $i; sudo cp -ar pc/griffin/$((i-1)) pc/griffin/$i;
rsync_bpc --bpc-top-dir /tmp/rsynctest --bpc-host-name griffin --bpc-share-name test1 --bpc-bkup-num $i \
--bpc-bkup-comp 3 --bpc-bkup-prevnum $((i-1)) --bpc-bkup-prevcomp 3 --bpc-log-level 9 --super --recursive \
--protect-args --numeric-ids --perms --owner --group -D --times --links --hard-links --delete --partial \
--protect-args --numeric-ids --perms --owner --group -D --times --links --hard-links --delete --partial \
--log-format='log: %o %i %B %8U,%8G %9l %f%L' --stats --modify-window=5 --one-file-system \
--log-format='log: %o %i %B %8U,%8G %9l %f%L' --stats --modify-window=5 --one-file-system \
--password-file=/tmp/rsyncd.secrets backuppc@griffin::test1 /
--password-file=/tmp/rsyncd.secrets backuppc@griffin::test1 /


Increase <code>--bpc-log-level</code> (say set it to ''9'') to get more debug output.
This only happens when share ''test1'' is on an ntfs-3g mount point. We can verify this does not happen on other file system by editing {{file|/etc/rsyncd.conf}}. For instance:

;Test case three - PASS - rsync_bpc + ntfs_3g fixed
* Edit {{file|/etc/rsyncd.conf}} such that share ''test1'' no longer points to an ntfs-3g mount point. For instance:
<source lang=diff>
<source lang=diff>
[test1]
[test1]
Line 167: Line 1,672:
+path = /boot
+path = /boot
</source>
</source>
* Run step above, and observe that files are only txfed once.
And run the <code>rsync_bpc</code> command above twice.
* If running other cases, reset share to ntfs-3g:
<source lang=diff>
[test1]
comment = Test share 1
-path = /boot
+path = /c/Boot
</source>

;Test case fourth - rsync_bpc + ntfs-3g that works
* Initialize:
sudo rm -rf /tmp/rsynctest; sudo mkdir -p /tmp/rsynctest/cpool; i=0
* Repeat the following step, and observe that files are only txfed the first time when removing parameters <code>--links --hard-links</code>:
cd /tmp/rsynctest; i=$((i+1)); echo backup $i; sudo cp -ar pc/griffin/$((i-1)) pc/griffin/$i; \
sudo rsync_bpc --bpc-top-dir /tmp/rsynctest --bpc-host-name griffin --bpc-share-name test1 --bpc-bkup-num $i \
--bpc-bkup-comp 3 --bpc-bkup-prevnum $((i-1)) --bpc-bkup-prevcomp 3 --bpc-log-level 3 --super --recursive \
--protect-args --numeric-ids --perms --owner --group -D --times --delete --partial \
--log-format='log: %o %i %B %8U,%8G %9l %f%L' --stats --modify-window=5 --one-file-system \
--password-file=/tmp/rsyncd.secrets backuppc@griffin::test1 /

Latest revision as of 05:42, 27 April 2017

References

rsync

BackupPC links:

  • Doc:
http://backuppc.sourceforge.net/faq/BackupPC.html
http://backuppc.sourceforge.net/faq/
(for config cgi in lighttpd, including alias)
(basic config)
To enable user authentication
To force https
For setting up htpasswd
See also http://www.cyberciti.biz/tips/lighttpd-setup-a-password-protected-directory-directories.html
https://help.ubuntu.com/community/rsync#Rsync_Daemon
man rsyncd.conf
  • How to setup ssh+rsync securely:
http://backuppc.sourceforge.net/faq/ssh.html#how_can_client_access_as_root_be_avoided
http://www.systemajik.com/blog/setting-up-backuppc-on-ubuntu/
http://adsm.org/PRIVOXY-FORCE/lists/html/BackupPC-users/2012-02/msg00087.html
  • A look at rsync performance
http://lwn.net/Articles/400489/

Install and configuration

ntfs-3g backups

When backing up ntfs-3g shares, the option --links and --hard-links must be removed from RsyncArgs and RsyncRestoreArgs (see troubleshooting section below)

Since these options are common to a same client, one must create a separate backuppc client if a given physical host contains both ntfs-3g and non-ntfs-3g partitions. In that case, one client will match the name of the host, where others will have different names. For these clients, set the variable ClientNameAlias to the name of the physical host in the client configuration.

Install on Windows

Reference: http://www.systemajik.com/blog/setting-up-backuppc-on-windows/

Download and install cygwin-rsync from http://sourceforge.net/projects/backuppc/files/cygwin-rsyncd/3.0.9.0/
Edit c:\rsyncd\rsyncd.conf
Edit c:\rsyncd\rsyncd.secrets

Update the firewall rules
Test connection with telnet from backuppc server

Create exclusion list
 - Make sure we exclude locked file and junction
 - To get list of junctions:
   dir /aL /s > JunctionPoints.txt
Check backuppc charset setting


TODO:
Test changing (for all win pc, incl. pc-112-597, mandala, griffin, ...)

    '/Windows',

Into

    '+ /Windows/System32',
    '+ /Windows/System32/config',
    '+ /Windows/System32/config/**',
    '/Windows/**',

Install on Synology NAS

Mnemosyne> mkdir .ssh
Mnemosyne> cd .ssh
Mnemosyne> cat >authorized_keys
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCrpFEALD473OqeplM+qyBx+46KMquWK9utwNmvIy3iBatE4S2oFrHVjeQjSNNDK9qaVh2cHPNzzB0UkV9y174ZLn9qeatU032ieKgFfdTdgQCe88BM8eSiVxTkVWe/bawQwn0qqghPtQl7v6/Bof9H9pAGeWLPTfUFj/+CQTf2vevBZF7iuh/RhaokJ75EY29E2lMlAgfIsLv8OVt71LBJNzvXaZIvQgFn03et44UZuZpoqYlZZgAuVgJbHG88QRxcvciBl3H7adkphEkoGUn05JpG2G01Yw2orxpKuMAkCs6t+Z7LjzQhDe+vmKYC4ZUNr5APDwSUxovI3HWJ10Xv mip@home
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzEuVrk/Od38pnN4x0mM5DwdqnBlQlDZucJGnmjdgUrAnpNXaCuI+dMZSJ/q4m+2yH2icc69jOKNyvHC0DAce9vStyHS61itzhIN2fETB6EoF11+4VzyOBPR2n/drfgBFo9rX7h8FO5em2KrfgysRsJpNW63HqmsMZWhNHjfAwvfpauks6TJVtUctnHu8n8zV5o6Z6CPHq7b+MuLfWa+RMQyE+dUpk9JM3CgrBdumEYMgNWzmTLrMd6VdzHocU3i/6XHWfo+ZzbaN3aKVx/dKiEZQJUTavyXTiVrzWQBE/g+MmMsyb+6MdKarR5Hpct1BLUhIOv7C1QZna1xIjuICr mip@work
Mnemosyne> mount
/dev/root on / type ext3 (rw,relatime,errors=continue,data=ordered)
/tmp on /tmp type tmpfs (0)
none on /dev/pts type devpts (gid=4,mode=620)
/sys on /sys type sysfs (0)
/proc/bus/usb on /proc/bus/usb type usbfs (0)
/dev/md2 on /volume1 type ext3 (usrquota,grpquota)
/volume1/@optware on /opt type bind (bind)
none on /proc/fs/nfsd type nfsd (0)
Mnemosyne> cd /opt
Mnemosyne> ipkg update
Downloading http://ipkg.nslu2-linux.org/feeds/optware/syno-e500/cross/unstable/Packages.gz
Inflating http://ipkg.nslu2-linux.org/feeds/optware/syno-e500/cross/unstable/Packages.gz
Updated list of available packages in /opt/lib/ipkg/lists/cross
Successfully terminated.
Mnemosyne> df -h
Filesystem                Size      Used Available Use% Mounted on
/dev/md0                  2.3G    449.7M      1.9G  19% /
/tmp                    251.7M    372.0K    251.3M   0% /tmp
/dev/md2                  5.4T      4.7T    643.6G  88% /volume1
/volume1/@optware         5.4T      4.7T    643.6G  88% /opt
# rsync already 3.0.9
Mnemosyne> which rsync
/usr/syno/bin/rsync
Mnemosyne> ipkg install perl par2cmdline tar
Mnemosyne> ipkg install perl-compress-zlib perl-archive-zip
Mnemosyne> which perl
/opt/bin/perl
Mnemosyne> ipkg install optware-devel
Installing optware-devel (6.8-10) to root...
...
Configuring tcl
An error ocurred, return value: 1.
Collected errors:
ERROR: The following packages conflict with wget-ssl:
	 wget
Mnemosyne> ipkg remove wget
Removing package wget from root...
Successfully terminated.
Mnemosyne> ipkg install optware-devel
Installing optware-devel (6.8-10) to root...
Downloading http://ipkg.nslu2-linux.org/feeds/optware/syno-e500/cross/unstable/optware-devel_6.8-10_powerpc.ipk
file_move: ERROR: failed to rename /opt/ipkg-eY7pMk/optware-devel_6.8-10_powerpc.ipk to /opt/ipkg-eY7pMk/optware-devel_6.8-10_powerpc.ipk: No such file or directory
Nothing to be done
An error ocurred, return value: -1.
Collected errors:
Failed to download optware-devel. Perhaps you need to run 'ipkg update'?
Mnemosyne> ipkg update
Downloading http://ipkg.nslu2-linux.org/feeds/optware/syno-e500/cross/unstable/Packages.gz
ipkg: /opt/ipkg-z1g4Tt/Packages.gz: No such file or directory
file_copy: ERROR: failed to copy /opt/ipkg-z1g4Tt/Packages.gz to /tmp/ipkg.WbgbbX/cross.gz
An error ocurred, return value: 1.
Mnemosyne> ipkg install wget-ssl
Installing wget-ssl (1.12-2) to root...
Downloading http://ipkg.nslu2-linux.org/feeds/optware/syno-e500/cross/unstable/wget-ssl_1.12-2_powerpc.ipk
file_move: ERROR: failed to rename /opt/ipkg-5qgjat/wget-ssl_1.12-2_powerpc.ipk to /opt/ipkg-5qgjat/wget-ssl_1.12-2_powerpc.ipk: No such file or directory
Nothing to be done
An error ocurred, return value: -1.
Collected errors:
Failed to download wget-ssl. Perhaps you need to run 'ipkg update'?
Mnemosyne> ipkg install wget-ssl_1.12-2_powerpc.ipk 
Installing wget-ssl (1.12-2) to root...
Installing libidn (1.25-1) to root...
Downloading http://ipkg.nslu2-linux.org/feeds/optware/syno-e500/cross/unstable/libidn_1.25-1_powerpc.ipk
file_move: ERROR: failed to rename /opt/ipkg-9pn57X/libidn_1.25-1_powerpc.ipk to /opt/ipkg-9pn57X/libidn_1.25-1_powerpc.ipk: No such file or directory
Nothing to be done
An error ocurred, return value: -1.
Collected errors:
Failed to download libidn. Perhaps you need to run 'ipkg update'?
Mnemosyne> ipkg install libidn_1.25-1_powerpc.ipk 
Installing libidn (1.25-1) to root...
Configuring libidn
Successfully terminated.
Mnemosyne> Connection to mnemosyne closed.

beq06659@nxl67170ux /d/nxp/outlook
$ ssh root@mnemosyne
X11 forwarding request failed on channel 0


BusyBox v1.16.1 (2014-09-04 13:59:18 CST) built-in shell (ash)
Enter 'help' for a list of built-in commands.

Mnemosyne> cd /opt/bin
Mnemosyne> rm wget
Mnemosyne> cd
Mnemosyne> ipkg remove wget
No packages removed.
Nothing to be done
Successfully terminated.
Mnemosyne> ipkg remove wget-ssl
No packages removed.
Nothing to be done
Successfully terminated.
Mnemosyne> ipkg install wget-ssl_1.12-2_powerpc.ipk 
Installing wget-ssl (1.12-2) to root...
Configuring wget-ssl
Successfully terminated.
Mnemosyne> ipkg install libidn_1.25-1_powerpc.ipk 
Installing libidn (1.25-1) to root...
Configuring libidn
Successfully terminated.
Mnemosyne> Connection to mnemosyne closed.

beq06659@nxl67170ux /d/nxp/outlook
$ ssh root@mnemosyne
X11 forwarding request failed on channel 0


BusyBox v1.16.1 (2014-09-04 13:59:18 CST) built-in shell (ash)
Enter 'help' for a list of built-in commands.

Mnemosyne> which wget
/opt/bin/wget
Mnemosyne> ipkg update
Downloading http://ipkg.nslu2-linux.org/feeds/optware/syno-e500/cross/unstable/Packages.gz
Inflating http://ipkg.nslu2-linux.org/feeds/optware/syno-e500/cross/unstable/Packages.gz
Updated list of available packages in /opt/lib/ipkg/lists/cross
Successfully terminated.
Mnemosyne> ipkg upgrade
Nothing to be done
Successfully terminated.
Mnemosyne> ipkg install optware-devel
...
Configuring optware-devel
Successfully terminated.
Mnemosyne> rm libidn_1.25-1_powerpc.ipk 
Mnemosyne> rm wget-ssl_1.12-2_powerpc.ipk 
Mnemosyne> rm Packages.gz 

# To bypass build error:
powerpc-linux-gnuspe-ar cr libzlib.a adler32.o compress.o crc32.o deflate.o inffast.o inflate.o inftrees.o trees.o zutil.o
make[1]: powerpc-linux-gnuspe-ar: Command not found
make[1]: *** [libzlib.a] Error 127
make[1]: Leaving directory `/root/BackupPC-XS-0.30/zlib'
make: *** [zlib/libzlib.a] Error 2
Mnemosyne> cd /opt/bin
Mnemosyne> for f in addr2line ar as c++ c++filt cpp g++ gcc gccbug gcov ld nm objcopy objdump ranlib readelf size strings strip; do if [ -e /opt/bin/$f ]; then F=powerpc-linux-gnuspe-$f; if [ -e /opt/bin/$F ]; then echo $F exists; else echo Creating symlink $F to $f; ln -sf $f /opt/bin/$F; fi ; else echo $f does not exists; fi; done
Creating symlink powerpc-linux-gnuspe-addr2line to addr2line
powerpc-linux-gnuspe-c++ exists
Creating symlink powerpc-linux-gnuspe-c++filt to c++filt
Creating symlink powerpc-linux-gnuspe-cpp to cpp
powerpc-linux-gnuspe-g++ exists
powerpc-linux-gnuspe-gcc exists
Creating symlink powerpc-linux-gnuspe-gcov to gcov
Creating symlink powerpc-linux-gnuspe-ld to ld
Creating symlink powerpc-linux-gnuspe-nm to nm
Creating symlink powerpc-linux-gnuspe-objcopy to objcopy
Creating symlink powerpc-linux-gnuspe-objdump to objdump
Creating symlink powerpc-linux-gnuspe-ranlib to ranlib
Creating symlink powerpc-linux-gnuspe-readelf to readelf
Creating symlink powerpc-linux-gnuspe-size to size
Creating symlink powerpc-linux-gnuspe-strings to strings
Creating symlink powerpc-linux-gnuspe-strip to strip



Mnemosyne> cd
Mnemosyne> wget http://sourceforge.net/projects/backuppc/files/backuppc-beta/4.0.0alpha3/BackupPC-XS-0.30.tar.gz/download
Mnemosyne> wget http://sourceforge.net/projects/backuppc/files/backuppc-beta/4.0.0alpha3/rsync-bpc-3.0.9.3.tar.gz/download
Mnemosyne> wget http://sourceforge.net/projects/backuppc/files/backuppc-beta/4.0.0alpha3/BackupPC-4.0.0alpha3.tar.gz/download
Mnemosyne> tar xvzf BackupPC-XS-0.30.tar.gz 
Mnemosyne> cd BackupPC-XS-0.30/
Mnemosyne> perl Makefile.PL 
Mnemosyne> make 
Mnemosyne> make test
Mnemosyne> make install

Mnemosyne> cd
Mnemosyne> tar -xvzf rsync-bpc-3.0.9.3.tar.gz 
Mnemosyne> cd rsync-bpc-3.0.9.3/
# Should have done Mnemosyne> ./configure.sh --prefix=/opt
Mnemosyne> ./configure.sh
Mnemosyne> make
Mnemosyne> make install

Mnemosyne> ipkg install rrdtool
Mnemosyne> cat /etc/passwd 
...
backuppc:x:1033:100:User for backuppc:/var/services/homes/backuppc:/sbin/nologin
Mnemosyne> mkdir /opt/var/lib/backuppc
Mnemosyne> chown backuppc /opt/var/lib/backuppc
Mnemosyne> mkdir /usr/syno/synoman/backuppc

Mnemosyne> cd
Mnemosyne> tar -xvzf BackupPC-4.0.0alpha3.tar.gz 
Mnemosyne> cd BackupPC-4.0.0alpha3/
Mnemosyne> ./configure.pl 

Is this a new installation or upgrade for BackupPC?  If this is
an upgrade please tell me the full path of the existing BackupPC
configuration file (eg: /etc/BackupPC/config.pl).  Otherwise, just
hit return.

--> Full path to existing main config.pl []? 

I found the following locations for these programs:

    bzip2        => /opt/bin/bzip2
    cat          => /opt/bin/cat
    df           => /opt/bin/df
    gtar/tar     => /opt/bin/tar
    gzip         => /opt/bin/gzip
    hostname     => /bin/hostname
    nmblookup    => /usr/syno/bin/nmblookup
    par2         => /opt/bin/par2
    perl         => /opt/bin/perl
    ping         => /bin/ping
    rrdtool      => /opt/bin/rrdtool
    rsync        => /opt/bin/rsync
    rsync_bpc    => /usr/local/bin/rsync_bpc
    sendmail     => 
    smbclient    => /usr/syno/bin/smbclient
    split        => /opt/bin/split
    ssh/ssh2     => /usr/syno/bin/ssh

--> Are these paths correct? [y]? 

Please tell me the hostname of the machine that BackupPC will run on.

--> BackupPC will run on host [Mnemosyne]? 

BackupPC should run as a dedicated user with limited privileges.  You
need to create a user.  This user will need read/write permission on
the main data directory and read/execute permission on the install
directory (these directories will be setup shortly).

The primary group for this user should also be chosen carefully.
The data directories and files will have group read permission,
so group members can access backup files.

--> BackupPC should run as user [backuppc]? 

Please specify an install directory for BackupPC.  This is where the
BackupPC scripts, library and documentation will be installed.

--> Install directory (full path) [/usr/local/BackupPC]? /opt/local/backuppc

Please specify a data directory for BackupPC.  This is where all the
PC backup data is stored.  This file system needs to be big enough to
accommodate all the PCs you expect to backup (eg: at least several GB
per machine).

--> Data directory (full path) [/data/BackupPC]? /opt/var/lib/backuppc

BackupPC has SCGI and CGI perl interfaces that run under Apache.  You need
to pick which one to run.

For SCGI, Apache uses the scgi_mod module to communicate with BackupPC_Admin_SCGI,
which handles the requests.  This allows Apache to run as a different user as
backuppc.  To use SCGI you need to set SCGIServerPort to any spare
non-privileged TCP port number.  A negative value disables SCGI.

Important security warning!!  The SCGIServerPort must not be accessible by
anyone untrusted.  That means you can't allow untrusted users access to the
BackupPC server, and you should block the SCGIServerPort TCP port from
network access.

The traditional alternative is to use CGI.  In this case, an executable needs
to be installed Apache's cgi-bin directory.  This executable needs to run as
set-uid backuppc, or it can be run under mod_perl with Apache
running as user backuppc.

--> SCGI port (-1 to disable) [-1]? 
--> CGI bin directory (full path, or empty for no CGI) []? /usr/syno/synoman/backuppc

BackupPC's CGI and SCGI script need to display various PNG/GIF
images that should be stored where Apache can serve them.  They
should be placed somewhere under Apache's DocumentRoot.  BackupPC
also needs to know the URL to access these images.  Example:

    Apache image directory:  /var/www/htdocs/BackupPC
    URL for image directory: /BackupPC

The URL for the image directory should start with a slash.

--> Apache image directory (full path, or empty for no S/CGI) []? /usr/syno/synoman/backuppc
--> URL for image directory (omit http://host; starts with '/', or empty for no S/CGI) []? /backuppc

Ok, we're about to:

  - install the binaries, lib and docs in /opt/local/backuppc,
  - create the data directory /opt/var/lib/backuppc,
  - create/update the config.pl file /etc/BackupPC/config.pl,
  - optionally install the cgi-bin interface.

--> Do you want to continue? [y]? 
Created /opt/local/backuppc/bin
Created /opt/local/backuppc/share/doc/BackupPC
Created /opt/local/backuppc/lib/BackupPC/CGI
Created /opt/local/backuppc/lib/BackupPC/Config
Created /opt/local/backuppc/lib/BackupPC/Lang
Created /opt/local/backuppc/lib/BackupPC/Storage
Created /opt/local/backuppc/lib/BackupPC/Xfer
Created /opt/local/backuppc/lib/BackupPC/Zip
Created /opt/local/backuppc/lib/Net/FTP
Created /opt/var/lib/backuppc
Created /opt/var/lib/backuppc/pool
Created /opt/var/lib/backuppc/cpool
Created /opt/var/lib/backuppc/pc
Created /etc/BackupPC
Created /var/log/BackupPC
Created /var/run/BackupPC
Installing binaries in /opt/local/backuppc/bin
Installing library in /opt/local/backuppc/lib
Installing images in /usr/syno/synoman/backuppc
Making init.d scripts
Making Apache configuration file for suid-perl
Installing docs in /opt/local/backuppc/share/doc/BackupPC
Installing config.pl and hosts in /etc/BackupPC
PING localhost (127.0.0.1): 56 data bytes
64 bytes from 127.0.0.1: seq=0 ttl=64 time=0.087 ms

--- localhost ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.087/0.087/0.087 ms
Installing cgi script BackupPC_Admin in /usr/syno/synoman/backuppc

Ok, it looks like we are finished.  There are several more things you
will need to do:

  - Browse through the config file, /etc/BackupPC/config.pl,
    and make sure all the settings are correct.  In particular,
    you will need to set $Conf{CgiAdminUsers} so you have
    administration privileges in the CGI interface.

  - Edit the list of hosts to backup in /etc/BackupPC/hosts.

  - Read the documentation in /opt/local/backuppc/doc/BackupPC.html.
    Please pay special attention to the security section.

  - Verify that the CGI script BackupPC_Admin runs correctly.  You might
    need to change the permissions or group ownership of BackupPC_Admin.
    If this is an upgrade and you are using mod_perl, you will need
    to restart Apache.  Otherwise it will have stale code.

  - BackupPC should be ready to start.  Don't forget to run it
    as user backuppc!  The installation also contains an
    init.d/backuppc script that can be copied to /etc/init.d
    so that BackupPC can auto-start on boot.  This will also enable
    administrative users to start the server from the CGI interface.
    See init.d/README.

Enjoy!
Mnemosyne> cd /usr/syno/synoman/backuppc
Mnemosyne> mv BackupPC_Admin real-BackupPC_Admin.pl
Mnemosyne> chmod 550 real-BackupPC_Admin.pl 
Mnemosyne> cat wrapper.c 
#include <unistd.h>
#define REAL_PATH "/usr/syno/synoman/backuppc/real-BackupPC_Admin.pl"
int main(ac, av)
char **av;
{
	execv(REAL_PATH, av);
	return 0;
}
      
Mnemosyne> gcc -o BackupPC_Admin wrapper.c
Mnemosyne> chown backuppc BackupPC_Admin 
Mnemosyne> chmod 4550 BackupPC_Admin 
Mnemosyne> chown backuppc:users BackupPC_Admin 
Mnemosyne> mv BackupPC_Admin BackupPC_Admin.cgi
Mnemosyne> chmod 4550 BackupPC_Admin.cgi 
-r-sr-x--- 1 backuppc users 8.7K Nov 10 16:20 BackupPC_Admin.cgi
-r-xr-x--- 1 backuppc users 4.0K Nov 10 16:16 real-BackupPC_Admin.pl
-rwxr-xr-x 1 root     root   163 Nov 10 16:19 wrapper.c

# import backuppc start up script to /opt/sbin/backuppc
# backuppc requires package procps

Mnemosyne> backuppc status
Mnemosyne> ipkg install procps 
Mnemosyne> vi config.pl
Mnemosyne> rmdir pc
Mnemosyne> ln -sf . pc
Mnemosyne> backuppc start
Mnemosyne> cd /etc
Mnemosyne> vi hosts
Mnemosyne> cat hosts
127.0.0.1	localhost
172.19.100.99	Mnemosyne
172.19.100.18	xbmcbuntu
Mnemosyne> curl -k -L http://cpanmin.us | perl -  App::cpanminus
Mnemosyne> cpanm File::Listing
Mnemosyne> ipkg install sudo
Mnemosyne> cd /etc/BackupPC/
Mnemosyne> l
total 228K
-rw-r----- 1 backuppc root     0 Nov 10 18:23 LOCK
-rw-r----- 1 backuppc users  82K Nov 10 17:35 config.pl
-rw-r----- 1 root     root   82K Nov 10 17:32 config.pl.orig
-rwxr-xr-x 1 root     root   838 Nov 10 17:39 griffin.pl
-rw-r--r-- 1 backuppc users 2.3K Nov 10 17:54 hosts
-rwxr-xr-x 1 root     root  3.2K Nov 10 17:39 mandala.pl
-rwxr-xr-x 1 root     root   821 Nov 10 17:39 nxl67170ux.pl
lrwxrwxrwx 1 root     root     1 Nov 10 17:50 pc -> .
-rwxr-xr-x 1 root     root   811 Nov 10 17:39 xbmcbuntu.pl
Mnemosyne> vi config.pl
# Edit PingCmd line to use 'sudo'
Mnemosyne> cd /etc/sudoers.d
Mnemosyne> vi backuppc
Mnemosyne> chmod 440 backuppc 
Mnemosyne> backuppc start
backuppc started
Mnemosyne> 



################################################################""
TODO:
- Delete backuppc db on mnemosyne

Install on Lacie-Cloudbox

  • Must create a user 'backuppc', and install in the share /shares/backuppc. This include the directory BackupPC and www!
If we don't, everything is deleted at next boot.

This is for BackupPC 4.0 Alpha 3: 

#################################
Backuppc 4.0

# Install some modules
# ipkg install samba2 tar rsync # because already available
# Not installing openssh client - let's see the one we have
ipkg install perl par2cmdline 
ipkg install perl-compress-zlib perl-archive-zip
ipkg install optware-devel

cd /opt/bin
for f in addr2line ar as c++ c++filt cpp g++ gcc gccbug gcov ld nm objcopy objdump ranlib readelf size strings strip; do if [ -e /opt/bin/$f ]; then F=arm-none-linux-gnueabi-$f; if [ -e /opt/bin/$F ]; then echo $F exists; else echo Creating symlink $F to $f; ln -sf $f /opt/bin/$F; fi ; else echo $f does not exists; fi; done


    ## >>>>>>>>>>>> ADDED FOR BACKUPPC
    cgi.assign += ( "/opt/local/backuppc/cgi-bin/index.cgi" => "" )
    #alias.url += ("/backuppc/index.cgi" => "/opt/local/backuppc/cgi-bin/index.cgi")
    alias.url += ("/backuppc" => "/opt/local/backuppc/cgi-bin" )
    #alias.url += ("/BackupPC_Admin" => "/shares/backuppc/www/cgi-bin/BackupPC_Admin")
    auth.debug = 2
    auth.backend = "htpasswd"
    auth.backend.htpasswd.userfile = "/etc/lighttpd/lighttpd-htpasswd.user"
    auth.require = ( "/backuppc" =>
	    ( "method" => "basic", "realm" => "Password protected area", "require" => "user=backuppc" )
    )

    index-file.names            = (
      "index.html", "index.php", "index.cgi"
    )

    # Force HTTPS
    $HTTP["scheme"] == "http" {
	    # capture vhost name with regex conditiona -> %0 in redirect pattern
	    # must be the most inner block to the redirect rule
	    $HTTP["host"] =~ ".*" {
		    url.redirect = (".*" => "https://%0$0")
	    }
    }



########################################################################################################
########################################################################################################
########################################################################################################
########################################################################################################
########################################################################################################
########################################################################################################
########################################################################################################
########################################################################################################
########################################################################################################
########################################################################################################
########################################################################################################

beq06659@nxl67170ux /data/home/beq06659/.ssh (master)
$ ssh lacie
[root@LaCie-CloudBox ~]# cd /opt
[root@LaCie-CloudBox opt]# cd etc
[root@LaCie-CloudBox etc]# l
[root@LaCie-CloudBox etc]# cd backuppc/
[root@LaCie-CloudBox backuppc]# l
[root@LaCie-CloudBox backuppc]# cd ..
[root@LaCie-CloudBox etc]# l
[root@LaCie-CloudBox etc]# mv backuppc backuppc3
[root@LaCie-CloudBox etc]# cd ..
[root@LaCie-CloudBox opt]# l
[root@LaCie-CloudBox opt]# cd local
[root@LaCie-CloudBox local]# l
[root@LaCie-CloudBox local]# rm -rf backuppc 
[root@LaCie-CloudBox local]# cd /opt
[root@LaCie-CloudBox opt]# cd var
[root@LaCie-CloudBox var]# l
[root@LaCie-CloudBox var]# cd log
[root@LaCie-CloudBox log]# l
[root@LaCie-CloudBox log]# cd backuppc/
[root@LaCie-CloudBox backuppc]# l
[root@LaCie-CloudBox backuppc]# cd ..
[root@LaCie-CloudBox log]# l
[root@LaCie-CloudBox log]# mv backuppc backuppc3
[root@LaCie-CloudBox log]# l
[root@LaCie-CloudBox log]# cd ..
[root@LaCie-CloudBox var]# cd
[root@LaCie-CloudBox ~]# l
[root@LaCie-CloudBox ~]# mkdir build
[root@LaCie-CloudBox ~]# cd build
[root@LaCie-CloudBox build]# l
[root@LaCie-CloudBox build]# tar
[root@LaCie-CloudBox build]# which tar
[root@LaCie-CloudBox build]# l
[root@LaCie-CloudBox build]# l
[root@LaCie-CloudBox build]# l
total 1.6M
-rw-r--r-- 1 root root 571K Dec  2  2013 BackupPC-4.0.0alpha3.tar.gz
-rw-r--r-- 1 root root 275K Dec  2  2013 BackupPC-XS-0.30.tar.gz
-rw-r--r-- 1 root root 757K Dec  2  2013 rsync-bpc-3.0.9.3.tar.gz
[root@LaCie-CloudBox build]# tar -xvzf BackupPC-4.0.0alpha3.tar.gz 
[root@LaCie-CloudBox build]# ls
[root@LaCie-CloudBox build]# tar -xvzf BackupPC-XS-0.30.tar.gz 
[root@LaCie-CloudBox build]# tar -xvzf rsync-bpc-3.0.9.3.tar.gz 
[root@LaCie-CloudBox build]# ls
[root@LaCie-CloudBox build]# cd BackupPC-XS-0.30
[root@LaCie-CloudBox BackupPC-XS-0.30]# perl Makefile.PL 
[root@LaCie-CloudBox BackupPC-XS-0.30]# make
[1]: arm-none-linux-gnueabi-ar: Command not found
make[1]: *** [libzlib.a] Error 127
make[1]: Leaving directory `/root/build/BackupPC-XS-0.30/zlib'
make: *** [zlib/libzlib.a] Error 2
[root@LaCie-CloudBox BackupPC-XS-0.30]# cd /opt/bin
[root@LaCie-CloudBox bin]# l
[root@LaCie-CloudBox bin]# ls ar
ar
[root@LaCie-CloudBox bin]# ipkg install rddtool
Nothing to be done
An error ocurred, return value: 4.
Collected errors:
Cannot find package rddtool.
Check the spelling or perhaps run 'ipkg update'
[root@LaCie-CloudBox bin]# cd /opt
[root@LaCie-CloudBox opt]# cd bin
[root@LaCie-CloudBox bin]# ls
[root@LaCie-CloudBox bin]# l gcc*
[root@LaCie-CloudBox bin]# cd
[root@LaCie-CloudBox ~]# l
[root@LaCie-CloudBox ~]# cd build
[root@LaCie-CloudBox build]# l
[root@LaCie-CloudBox build]# cd BackupPC-XS-0.30
[root@LaCie-CloudBox BackupPC-XS-0.30]# l
[root@LaCie-CloudBox BackupPC-XS-0.30]# make                              
[root@LaCie-CloudBox bin]# l
[root@LaCie-CloudBox bin]# ls
[root@LaCie-CloudBox bin]# ls powerpc*
[root@LaCie-CloudBox bin]# rm powerpc*
[root@LaCie-CloudBox bin]# ls arm*       
[root@LaCie-CloudBox bin]# cd
[root@LaCie-CloudBox ~]# l     
[root@LaCie-CloudBox ~]# cd buil
[root@LaCie-CloudBox ~]# cd build
[root@LaCie-CloudBox build]# l
[root@LaCie-CloudBox build]# cd BackupPC-XS-0.30
[root@LaCie-CloudBox BackupPC-XS-0.30]# l
[root@LaCie-CloudBox BackupPC-XS-0.30]# make
[root@LaCie-CloudBox BackupPC-XS-0.30]# make test
[root@LaCie-CloudBox BackupPC-XS-0.30]# make install
[root@LaCie-CloudBox BackupPC-XS-0.30]# cd ..
[root@LaCie-CloudBox build]# l
[root@LaCie-CloudBox build]# cd rsync-bpc-3.0.9.3
[root@LaCie-CloudBox rsync-bpc-3.0.9.3]# l
[root@LaCie-CloudBox rsync-bpc-3.0.9.3]# ./configure
[root@LaCie-CloudBox rsync-bpc-3.0.9.3]# ./configure.sh --prefix=/opt
[root@LaCie-CloudBox rsync-bpc-3.0.9.3]# make
[root@LaCie-CloudBox rsync-bpc-3.0.9.3]# make install
[root@LaCie-CloudBox rsync-bpc-3.0.9.3]# which rsync_bpc
[root@LaCie-CloudBox rsync-bpc-3.0.9.3]# cd ..
[root@LaCie-CloudBox build]# history
  973  wget http://sourceforge.net/projects/backuppc/files/backuppc-beta/4.0.0alpha3/BackupPC-XS-0.30.tar.gz/d
  974  wget http://sourceforge.net/projects/backuppc/files/backuppc-beta/4.0.0alpha3/BackupPC-XS-0.30.tar.gz/download
  975  wget http://sourceforge.net/projects/backuppc/files/backuppc-beta/4.0.0alpha3/rsync-bpc-3.0.9.3.tar.gz/download
  976  Mnemosyne> wget http://sourceforge.net/projects/backuppc/files/backuppc-beta/4.0.0alpha3/BackupPC-4.0.0alpha3.tar.gz/download
  977  wget http://sourceforge.net/projects/backuppc/files/backuppc-beta/4.0.0alpha3/BackupPC-4.0.0alpha3.tar.gz/download
  978  rm rsync-bpc-3.0.9.3.tar.gz.1 
  979  rm wget
  980  tar -xvzf BackupPC-4.0.0alpha3.tar.gz 
  981  tar -xvzf BackupPC-XS-0.30.tar.gz 
  982  tar -xvzf rsync-bpc-3.0.9.3.tar.gz 
  983  perl Makefile.PL 
  984  ls ar
  985  ipkg install rddtool
  986  cd /opt
  987  cd bin
  988  l gcc*
  989  for f in addr2line ar as c++ c++filt cpp g++ gcc gccbug gcov ld nm objcopy objdump ranlib readelf size strings strip; do if [ -e /opt/bin/$f ]; then F=powerpc-linux-gnuspe-$f; if [ -e /opt/bin/$F ]; then echo $F exists; else echo Creating symlink $F to $f; ln -sf $f /opt/bin/$F; fi ; else echo $f does not exists; fi; done
  990  cd /opt/bin
  991  ls
  992  ls powerpc*
  993  rm powerpc*
  994  ls arm*
  995  for f in addr2line ar as c++ c++filt cpp g++ gcc gccbug gcov ld nm objcopy objdump ranlib readelf size strings strip; do if [ -e /opt/bin/$f ]; then F=arm-none-linux-gnueabi-$f; if [ -e /opt/bin/$F ]; then echo $F exists; else echo Creating symlink $F to $f; ln -sf $f /opt/bin/$F; fi ; else echo $f does not exists; fi; done
  996  cd
  997  cd buil
  998  cd build
  999  cd BackupPC-XS-0.30
 1000  make test
 1001  cd rsync-bpc-3.0.9.3
 1002  l
 1003  ./configure.sh --prefix=/opt
 1004  make
 1005  make install
 1006  which rsync_bpc
 1007  cd ..
 1008  history
[root@LaCie-CloudBox build]# cd /opt
[root@LaCie-CloudBox opt]# cd var
[root@LaCie-CloudBox var]# cd lib
[root@LaCie-CloudBox var]# l
[root@LaCie-CloudBox var]# cd run
[root@LaCie-CloudBox run]# l
[root@LaCie-CloudBox run]# cd ..
[root@LaCie-CloudBox var]# l
[root@LaCie-CloudBox var]# cd
[root@LaCie-CloudBox ~]# cd build
[root@LaCie-CloudBox build]# l
[root@LaCie-CloudBox build]# cd BackupPC-4.0.0alpha3
[root@LaCie-CloudBox BackupPC-4.0.0alpha3]# mkdir -p /opt/var/lib/backuppc
[root@LaCie-CloudBox BackupPC-4.0.0alpha3]# l /opt/var/lib/backuppc
[root@LaCie-CloudBox BackupPC-4.0.0alpha3]# chown backuppc /opt/var/lib/backuppc
[root@LaCie-CloudBox BackupPC-4.0.0alpha3]# l /opt/var/lib         
[root@LaCie-CloudBox BackupPC-4.0.0alpha3]# l
[root@LaCie-CloudBox BackupPC-4.0.0alpha3]# ./
[root@LaCie-CloudBox BackupPC-4.0.0alpha3]# ./conf
[root@LaCie-CloudBox BackupPC-4.0.0alpha3]# ./conf
[root@LaCie-CloudBox BackupPC-4.0.0alpha3]# ./conf
[root@LaCie-CloudBox BackupPC-4.0.0alpha3]# ./configure.pl 
[root@LaCie-CloudBox BackupPC-4.0.0alpha3]# ipkg install rrdtool
[root@LaCie-CloudBox BackupPC-4.0.0alpha3]# ./configure.pl 
[root@LaCie-CloudBox BackupPC-4.0.0alpha3]# ./configure.pl 

Is this a new installation or upgrade for BackupPC?  If this is
an upgrade please tell me the full path of the existing BackupPC
configuration file (eg: /etc/BackupPC/config.pl).  Otherwise, just
hit return.

--> Full path to existing main config.pl []? 

I found the following locations for these programs:

    bzip2        => /opt/bin/bzip2
    cat          => /opt/bin/cat
    df           => /opt/bin/df
    gtar/tar     => /opt/bin/tar
    gzip         => /opt/bin/gzip
    hostname     => /bin/hostname
    nmblookup    => /usr/bin/nmblookup
    par2         => /opt/bin/par2
    perl         => /opt/bin/perl
    ping         => /bin/ping
    rrdtool      => /opt/bin/rrdtool
    rsync        => /opt/bin/rsync
    rsync_bpc    => /opt/bin/rsync_bpc
    sendmail     => 
    smbclient    => 
    split        => /opt/bin/split
    ssh/ssh2     => /usr/bin/ssh

--> Are these paths correct? [y]? 

Please tell me the hostname of the machine that BackupPC will run on.

--> BackupPC will run on host [LaCie-CloudBox]? 

BackupPC should run as a dedicated user with limited privileges.  You
need to create a user.  This user will need read/write permission on
the main data directory and read/execute permission on the install
directory (these directories will be setup shortly).

The primary group for this user should also be chosen carefully.
The data directories and files will have group read permission,
so group members can access backup files.

--> BackupPC should run as user [backuppc]? 

Please specify an install directory for BackupPC.  This is where the
BackupPC scripts, library and documentation will be installed.

--> Install directory (full path) [/usr/local/BackupPC]? /opt/local/backuppc

Please specify a data directory for BackupPC.  This is where all the
PC backup data is stored.  This file system needs to be big enough to
accommodate all the PCs you expect to backup (eg: at least several GB
per machine).

--> Data directory (full path) [/data/BackupPC]? /opt/var/lib/backuppc

BackupPC has SCGI and CGI perl interfaces that run under Apache.  You need
to pick which one to run.

For SCGI, Apache uses the scgi_mod module to communicate with BackupPC_Admin_SCGI,
which handles the requests.  This allows Apache to run as a different user as
backuppc.  To use SCGI you need to set SCGIServerPort to any spare
non-privileged TCP port number.  A negative value disables SCGI.

Important security warning!!  The SCGIServerPort must not be accessible by
anyone untrusted.  That means you can't allow untrusted users access to the
BackupPC server, and you should block the SCGIServerPort TCP port from
network access.

The traditional alternative is to use CGI.  In this case, an executable needs
to be installed Apache's cgi-bin directory.  This executable needs to run as
set-uid backuppc, or it can be run under mod_perl with Apache
running as user backuppc.

--> SCGI port (-1 to disable) [-1]? 
--> CGI bin directory (full path, or empty for no CGI) []? /opt/lib/backuppc

BackupPC's CGI and SCGI script need to display various PNG/GIF
images that should be stored where Apache can serve them.  They
should be placed somewhere under Apache's DocumentRoot.  BackupPC
also needs to know the URL to access these images.  Example:

    Apache image directory:  /var/www/htdocs/BackupPC
    URL for image directory: /BackupPC

The URL for the image directory should start with a slash.

--> Apache image directory (full path, or empty for no S/CGI) []? /opt/local/backuppc/cgi-bin
--> URL for image directory (omit http://host; starts with '/', or empty for no S/CGI) []? /backuppc

Ok, we're about to:

  - install the binaries, lib and docs in /opt/local/backuppc,
  - create the data directory /opt/var/lib/backuppc,
  - create/update the config.pl file /etc/BackupPC/config.pl,
  - optionally install the cgi-bin interface.

--> Do you want to continue? [y]? 
Created /opt/local/backuppc/bin
Created /opt/local/backuppc/share/doc/BackupPC
Created /opt/local/backuppc/lib/BackupPC/CGI
Created /opt/local/backuppc/lib/BackupPC/Config
Created /opt/local/backuppc/lib/BackupPC/Lang
Created /opt/local/backuppc/lib/BackupPC/Storage
Created /opt/local/backuppc/lib/BackupPC/Xfer
Created /opt/local/backuppc/lib/BackupPC/Zip
Created /opt/local/backuppc/lib/Net/FTP
Created /opt/local/backuppc/cgi-bin
Created /opt/var/lib/backuppc
Created /opt/var/lib/backuppc/pool
Created /opt/var/lib/backuppc/cpool
Created /opt/var/lib/backuppc/pc
Created /etc/BackupPC
Created /var/log/BackupPC
Created /var/run/BackupPC
Installing binaries in /opt/local/backuppc/bin
Installing library in /opt/local/backuppc/lib
Installing images in /opt/local/backuppc/cgi-bin
Making init.d scripts
Making Apache configuration file for suid-perl
Installing docs in /opt/local/backuppc/share/doc/BackupPC
Installing config.pl and hosts in /etc/BackupPC
PING localhost (127.0.0.1): 56 data bytes
64 bytes from 127.0.0.1: seq=0 ttl=64 time=0.325 ms

--- localhost ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.325/0.325/0.325 ms
Installing cgi script BackupPC_Admin in /opt/lib/backuppc

Ok, it looks like we are finished.  There are several more things you
will need to do:

  - Browse through the config file, /etc/BackupPC/config.pl,
    and make sure all the settings are correct.  In particular,
    you will need to set $Conf{CgiAdminUsers} so you have
    administration privileges in the CGI interface.

  - Edit the list of hosts to backup in /etc/BackupPC/hosts.

  - Read the documentation in /opt/local/backuppc/doc/BackupPC.html.
    Please pay special attention to the security section.

  - Verify that the CGI script BackupPC_Admin runs correctly.  You might
    need to change the permissions or group ownership of BackupPC_Admin.
    If this is an upgrade and you are using mod_perl, you will need
    to restart Apache.  Otherwise it will have stale code.

  - BackupPC should be ready to start.  Don't forget to run it
    as user backuppc!  The installation also contains an
    init.d/backuppc script that can be copied to /etc/init.d
    so that BackupPC can auto-start on boot.  This will also enable
    administrative users to start the server from the CGI interface.
    See init.d/README.

Enjoy!
[root@LaCie-CloudBox BackupPC-4.0.0alpha3]# cd /etc
[root@LaCie-CloudBox etc]# l
[root@LaCie-CloudBox etc]# ls
[root@LaCie-CloudBox etc]# cd BackupPC/
[root@LaCie-CloudBox BackupPC]# l
[root@LaCie-CloudBox BackupPC]# cd /opt/local
[root@LaCie-CloudBox local]# l
[root@LaCie-CloudBox local]# cd backuppc/
[root@LaCie-CloudBox backuppc]# l
[root@LaCie-CloudBox backuppc]# cd bin
[root@LaCie-CloudBox bin]# ls
[root@LaCie-CloudBox bin]# cd ../cgi-bin/
[root@LaCie-CloudBox cgi-bin]# l
[root@LaCie-CloudBox cgi-bin]# ls
[root@LaCie-CloudBox cgi-bin]# cd /var
[root@LaCie-CloudBox var]# cd log
[root@LaCie-CloudBox log]# l
[root@LaCie-CloudBox log]# cd
[root@LaCie-CloudBox ~]# cd /etc
[root@LaCie-CloudBox etc]# rm -rf BackupPC/
[root@LaCie-CloudBox etc]# cd /var/log 
[root@LaCie-CloudBox log]# l
[root@LaCie-CloudBox log]# rm -rf BackupPC/
[root@LaCie-CloudBox log]# ls
[root@LaCie-CloudBox log]# cd /var/run
[root@LaCie-CloudBox run]# l
[root@LaCie-CloudBox run]# ls
[root@LaCie-CloudBox run]# rm -rf BackupPC/
[root@LaCie-CloudBox run]# cd /opt
[root@LaCie-CloudBox opt]# cd local
[root@LaCie-CloudBox local]# l
[root@LaCie-CloudBox local]# rm -rf backuppc/
[root@LaCie-CloudBox local]# cd ../lib
[root@LaCie-CloudBox lib]# l
[root@LaCie-CloudBox lib]# ls
[root@LaCie-CloudBox lib]# cd backuppc/
[root@LaCie-CloudBox backuppc]# m
[root@LaCie-CloudBox backuppc]# l
[root@LaCie-CloudBox backuppc]# cd ..
[root@LaCie-CloudBox lib]# rm -rf backuppc 
[root@LaCie-CloudBox lib]# cd
[root@LaCie-CloudBox ~]# cd biuld
[root@LaCie-CloudBox ~]# cd build
[root@LaCie-CloudBox build]# cd BackupPC-4.0.0alpha3
[root@LaCie-CloudBox BackupPC-4.0.0alpha3]# ./conf
[root@LaCie-CloudBox BackupPC-4.0.0alpha3]# cd /opt
[root@LaCie-CloudBox opt]# cd etc
[root@LaCie-CloudBox etc]# cd backuppc
[root@LaCie-CloudBox backuppc]# l
[root@LaCie-CloudBox backuppc]# ln -sf . pc
[root@LaCie-CloudBox backuppc]# cd /opt/var/log/          
[root@LaCie-CloudBox log]# l
[root@LaCie-CloudBox log]# cd ..
[root@LaCie-CloudBox var]# ls
[root@LaCie-CloudBox var]# cd backuppc/
[root@LaCie-CloudBox backuppc]# l
[root@LaCie-CloudBox backuppc]# cd ..
[root@LaCie-CloudBox var]# history
  978  for f in addr2line ar as c++ c++filt cpp g++ gcc gccbug gcov ld nm objcopy objdump ranlib readelf size strings strip; do if [ -e /opt/bin/$f ]; then F=arm-none-linux-gnueabi-$f; if [ -e /opt/bin/$F ]; then echo $F exists; else echo Creating symlink $F to $f; ln -sf $f /opt/bin/$F; fi ; else echo $f does not exists; fi; done
  979  cd buil
  980  cd BackupPC-XS-0.30
  981  make test
  982  cd rsync-bpc-3.0.9.3
  983  ./configure.sh --prefix=/opt
  984  make
  985  make install
  986  which rsync_bpc
  987  cd var
  988  cd lib
  989  cd run
  990  mkdir -p /opt/var/lib/backuppc
  991  l /opt/var/lib/backuppc
  992  chown backuppc /opt/var/lib/backuppc
  993  l /opt/var/lib
  994  ipkg install rrdtool
  995  ./configure.pl 
  996  cd BackupPC/
  997  cd /opt/local
  998  cd bin
  999  cd ../cgi-bin/
 1000  cd /var
 1001  cd log
 1002  cd /etc
 1003  cd /var/log
 1004  cd /var/run
 1005  rm -rf BackupPC/
 1006  cd local
 1007  rm -rf backuppc/
 1008  cd ../lib
 1009  m
 1010  rm -rf backuppc
 1011  cd
 1012  cd biuld
 1013  cd build
 1014  cd BackupPC-4.0.0alpha3
 1015  ./configure.pl --config-dir=/opt/etc/backuppc --log-dir=/opt/var/backuppc --run-dir=/opt/var/run/backuppc
 1016  cd /opt
 1017  cd etc
 1018  cd backuppc
 1019  ln -sf . pc
 1020  cd /opt/var/log/
 1021  ls
 1022  cd backuppc/
 1023  l
 1024  cd ..
 1025  history
[root@LaCie-CloudBox var]# cd /opt
[root@LaCie-CloudBox opt]# cd etc
[root@LaCie-CloudBox etc]# rm -rf backuppc
[root@LaCie-CloudBox etc]# cd /opt
[root@LaCie-CloudBox opt]# cd var
[root@LaCie-CloudBox var]# l
[root@LaCie-CloudBox var]# rm -rf backuppc/
[root@LaCie-CloudBox var]# cd run 
[root@LaCie-CloudBox run]# l
[root@LaCie-CloudBox run]# rm -rf backuppc/
[root@LaCie-CloudBox run]# l
[root@LaCie-CloudBox run]# cd /opt/local/
[root@LaCie-CloudBox local]# l
[root@LaCie-CloudBox local]# rm -rf backuppc/
[root@LaCie-CloudBox local]# cd ../lib
[root@LaCie-CloudBox lib]# l
[root@LaCie-CloudBox lib]# ls
[root@LaCie-CloudBox lib]# rm -rf backuppc/
[root@LaCie-CloudBox lib]# cd /shares/
[root@LaCie-CloudBox shares]# cd backuppc/
[root@LaCie-CloudBox backuppc]# rm -rf *
[root@LaCie-CloudBox backuppc]# cd
[root@LaCie-CloudBox ~]# cd build
[root@LaCie-CloudBox build]# cd BackupPC-4.0.0alpha3
[root@LaCie-CloudBox BackupPC-4.0.0alpha3]#  ./configure.pl --config-dir=/opt/etc/backuppc --log-dir=/opt/var/log/backuppc --run-dir=/opt/var/run/bac>

Is this a new installation or upgrade for BackupPC?  If this is
an upgrade please tell me the full path of the existing BackupPC
configuration file (eg: /etc/BackupPC/config.pl).  Otherwise, just
hit return.

--> Full path to existing main config.pl []? 

I found the following locations for these programs:

    bzip2        => /opt/bin/bzip2
    cat          => /opt/bin/cat
    df           => /opt/bin/df
    gtar/tar     => /opt/bin/tar
    gzip         => /opt/bin/gzip
    hostname     => /bin/hostname
    nmblookup    => /usr/bin/nmblookup
    par2         => /opt/bin/par2
    perl         => /opt/bin/perl
    ping         => /bin/ping
    rrdtool      => /opt/bin/rrdtool
    rsync        => /opt/bin/rsync
    rsync_bpc    => /opt/bin/rsync_bpc
    sendmail     => 
    smbclient    => 
    split        => /opt/bin/split
    ssh/ssh2     => /usr/bin/ssh

--> Are these paths correct? [y]? 

Please tell me the hostname of the machine that BackupPC will run on.

--> BackupPC will run on host [LaCie-CloudBox]? 

BackupPC should run as a dedicated user with limited privileges.  You
need to create a user.  This user will need read/write permission on
the main data directory and read/execute permission on the install
directory (these directories will be setup shortly).

The primary group for this user should also be chosen carefully.
The data directories and files will have group read permission,
so group members can access backup files.

--> BackupPC should run as user [backuppc]? 

Please specify an install directory for BackupPC.  This is where the
BackupPC scripts, library and documentation will be installed.

--> Install directory (full path) [/usr/local/BackupPC]? /opt/local/backuppc

Please specify a data directory for BackupPC.  This is where all the
PC backup data is stored.  This file system needs to be big enough to
accommodate all the PCs you expect to backup (eg: at least several GB
per machine).

--> Data directory (full path) [/data/BackupPC]? /shares/backuppc

BackupPC has SCGI and CGI perl interfaces that run under Apache.  You need
to pick which one to run.

For SCGI, Apache uses the scgi_mod module to communicate with BackupPC_Admin_SCGI,
which handles the requests.  This allows Apache to run as a different user as
backuppc.  To use SCGI you need to set SCGIServerPort to any spare
non-privileged TCP port number.  A negative value disables SCGI.

Important security warning!!  The SCGIServerPort must not be accessible by
anyone untrusted.  That means you can't allow untrusted users access to the
BackupPC server, and you should block the SCGIServerPort TCP port from
network access.

The traditional alternative is to use CGI.  In this case, an executable needs
to be installed Apache's cgi-bin directory.  This executable needs to run as
set-uid backuppc, or it can be run under mod_perl with Apache
running as user backuppc.

--> SCGI port (-1 to disable) [-1]? 
--> CGI bin directory (full path, or empty for no CGI) []? /opt/lib/backuppc

BackupPC's CGI and SCGI script need to display various PNG/GIF
images that should be stored where Apache can serve them.  They
should be placed somewhere under Apache's DocumentRoot.  BackupPC
also needs to know the URL to access these images.  Example:

    Apache image directory:  /var/www/htdocs/BackupPC
    URL for image directory: /BackupPC

The URL for the image directory should start with a slash.

--> Apache image directory (full path, or empty for no S/CGI) []? /opt/local/backuppc/cgi-bin 
--> URL for image directory (omit http://host; starts with '/', or empty for no S/CGI) []? /backuppc

Ok, we're about to:

  - install the binaries, lib and docs in /opt/local/backuppc,
  - create the data directory /shares/backuppc,
  - create/update the config.pl file /opt/etc/backuppc/config.pl,
  - optionally install the cgi-bin interface.

--> Do you want to continue? [y]? 
Created /opt/local/backuppc/bin
Created /opt/local/backuppc/share/doc/BackupPC
Created /opt/local/backuppc/lib/BackupPC/CGI
Created /opt/local/backuppc/lib/BackupPC/Config
Created /opt/local/backuppc/lib/BackupPC/Lang
Created /opt/local/backuppc/lib/BackupPC/Storage
Created /opt/local/backuppc/lib/BackupPC/Xfer
Created /opt/local/backuppc/lib/BackupPC/Zip
Created /opt/local/backuppc/lib/Net/FTP
Created /opt/local/backuppc/cgi-bin
Created /shares/backuppc
Created /shares/backuppc/pool
Created /shares/backuppc/cpool
Created /shares/backuppc/pc
Created /opt/etc/backuppc
Created /opt/var/log/backuppc
Created /opt/var/run/backuppc
Installing binaries in /opt/local/backuppc/bin
Installing library in /opt/local/backuppc/lib
Installing images in /opt/local/backuppc/cgi-bin
Making init.d scripts
Making Apache configuration file for suid-perl
Installing docs in /opt/local/backuppc/share/doc/BackupPC
Installing config.pl and hosts in /opt/etc/backuppc
PING localhost (127.0.0.1): 56 data bytes
64 bytes from 127.0.0.1: seq=0 ttl=64 time=0.336 ms

--- localhost ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.336/0.336/0.336 ms
Installing cgi script BackupPC_Admin in /opt/lib/backuppc

Ok, it looks like we are finished.  There are several more things you
will need to do:

  - Browse through the config file, /opt/etc/backuppc/config.pl,
    and make sure all the settings are correct.  In particular,
    you will need to set $Conf{CgiAdminUsers} so you have
    administration privileges in the CGI interface.

  - Edit the list of hosts to backup in /opt/etc/backuppc/hosts.

  - Read the documentation in /opt/local/backuppc/doc/BackupPC.html.
    Please pay special attention to the security section.

  - Verify that the CGI script BackupPC_Admin runs correctly.  You might
    need to change the permissions or group ownership of BackupPC_Admin.
    If this is an upgrade and you are using mod_perl, you will need
    to restart Apache.  Otherwise it will have stale code.

  - BackupPC should be ready to start.  Don't forget to run it
    as user backuppc!  The installation also contains an
    init.d/backuppc script that can be copied to /etc/init.d
    so that BackupPC can auto-start on boot.  This will also enable
    administrative users to start the server from the CGI interface.
    See init.d/README.

Enjoy!
[root@LaCie-CloudBox BackupPC-4.0.0alpha3]# cd /opt/etc/backuppc
[root@LaCie-CloudBox backuppc]# ln -sf . pc
[root@LaCie-CloudBox backuppc]# cd /opt/sbin
[root@LaCie-CloudBox sbin]# vi backuppc 
[root@LaCie-CloudBox backuppc]# cd /etc
[root@LaCie-CloudBox etc]# cd lighttpd/
[root@LaCie-CloudBox lighttpd]# cp lighttpd.conf lighttpd.conf-bpc3
[root@LaCie-CloudBox lighttpd]# vi lighttpd.conf
[root@LaCie-CloudBox lighttpd]# cd /opt/local/backuppc/cgi-bin/
[root@LaCie-CloudBox cgi-bin]# vi wrapper.c 
[root@LaCie-CloudBox cgi-bin]# cat wrapper.c 
#include <unistd.h>
#define REAL_PATH "/opt/lib/backuppc/BackupPC_Admin"
int main(ac, av)
char **av;
{
	execv(REAL_PATH, av);
	return 0;
}
[root@LaCie-CloudBox cgi-bin]# chmod 550 /opt/lib/backuppc/BackupPC_Admin 
[root@LaCie-CloudBox cgi-bin]# gcc -o index.cgi wrapper.c                 
/opt/lib/gcc/arm-none-linux-gnueabi/4.2.3/../../../../arm-none-linux-gnueabi/bin/ld: Warning: /lib/libc.so.6: Unknown EABI object attribute 44
[root@LaCie-CloudBox cgi-bin]# l 
[root@LaCie-CloudBox cgi-bin]# chown backuppc:users index.cgi 
[root@LaCie-CloudBox cgi-bin]# chmod 4550 index.cgi 
[root@LaCie-CloudBox cgi-bin]# vi /etc/lighttpd/lighttpd.conf
[root@LaCie-CloudBox cgi-bin]# ngc --stop http
[root@LaCie-CloudBox cgi-bin]# ngc --start http
[root@LaCie-CloudBox cgi-bin]# vi /etc/lighttpd/lighttpd.conf
[root@LaCie-CloudBox cgi-bin]# ngc --restart http            
[root@LaCie-CloudBox cgi-bin]# ngc --stop http   
[root@LaCie-CloudBox cgi-bin]# ngc --start http
[root@LaCie-CloudBox cgi-bin]# vi /etc/lighttpd/lighttpd.conf
[root@LaCie-CloudBox cgi-bin]# cat /etc/lighttpd/lighttpd.conf
server.modules              = (
  "mod_redirect",
  "mod_cgi",
  "mod_expire",
  "mod_compress",
  "mod_proxy",
  "mod_fastcgi",
  "mod_rewrite",
  "mod_setenv",
  "mod_secdownload",
  "mod_access",
  "mod_alias",
  "mod_auth",
)

include "secure_password.inc"

server.document-root        = "/www"
server.errorlog             = "/var/log/lighttpd-error.log"
server.pid-file             = "/var/run/lighttpd.pid"
server.upload-dirs          =  ( "/lacie/tmp" )
server.errorfile-prefix     = "/www/errors/status-"

compress.allowed-encodings  = ("gzip", "deflate")
compress.cache-dir          = "/var/tmp/lighttpd"
compress.filetype           = ("text/plain", "text/html", "text/javascript", "text/css", "text/xml")

# Provide a "static" url to retrieve information usefull for production tool for instance.
url.rewrite-once = ("^/info$" => "/api/external/1.0/system_info","^/diagnostics$" => "/api/external/1.0/diagnostics")

index-file.names            = (
  "index.html", "index.php", "index.cgi"
)

# Force HTTPS
$HTTP["scheme"] == "http" {
	# capture vhost name with regex conditiona -> %0 in redirect pattern
	# must be the most inner block to the redirect rule
	$HTTP["host"] =~ ".*" {
		url.redirect = (".*" => "https://%0$0")
	}
}

$HTTP["url"] =~ "index"{
      setenv.add-response-header = ( "Cache-Control" => "no-cache, no-store" )
}

# For GWT
$HTTP["url"] =~ "\.nocache\."{
      setenv.add-response-header = ( "Cache-Control" => "no-cache, no-store" )
}

# For GWT
$HTTP["url"] =~ "\.cache\."{
      expire.url = ( "" => "access 1 months" )
      setenv.add-response-header = ( "Cache-Control" => "public" )
}


$HTTP["url"] =~ "(gif|png|jpg|css)$"{
      expire.url = ( "" => "access 1 months" )
      setenv.add-response-header = ( "Cache-Control" => "public" )
}

$HTTP["url"] =~ "/javascripts/"{
      expire.url = ( "" => "access 1 months" )
      setenv.add-response-header = ( "Cache-Control" => "public" )
}

$HTTP["url"] =~ "/api/local/"{                                    
      url.access-deny = ("")                                      
}

## >>>>>>>>>>>> ADDED FOR BACKUPPC
cgi.assign += ( "/opt/local/backuppc/cgi-bin/index.cgi" => "" )
#alias.url += ("/backuppc/index.cgi" => "/opt/local/backuppc/cgi-bin/index.cgi")
alias.url += ("/backuppc" => "/opt/local/backuppc/cgi-bin" )
#alias.url += ("/BackupPC_Admin" => "/shares/backuppc/www/cgi-bin/BackupPC_Admin")
auth.debug = 2
auth.backend = "htpasswd"
auth.backend.htpasswd.userfile = "/etc/lighttpd/lighttpd-htpasswd.user"
auth.require = ( "/backuppc" =>
	( "method" => "basic", "realm" => "Password protected area", "require" => "user=backuppc" )
)

proxy.server = (
"/api" => (
  "tornado" => (
    "host" => "127.0.0.1",
    "port" => 8888
  ))
)

fastcgi.server = (
".php"=>
  ((
    "bin-path" => "/usr/bin/php-cgi",
    "socket"=> "/tmp/php.socket",
    "max-procs" => 1,
    "bin-environment" => (
        "PHP_FCGI_CHILDREN" => "2",
        "PHP_FCGI_MAX_REQUESTS" => "1500"
    ),
    "allow-x-send-file" => "enable",
   )),
)

mimetype.assign             = (
  ".pdf"          =>      "application/pdf",
  ".sig"          =>      "application/pgp-signature",
  ".spl"          =>      "application/futuresplash",
  ".class"        =>      "application/octet-stream",
  ".ps"           =>      "application/postscript",
  ".torrent"      =>      "application/x-bittorrent",
  ".dvi"          =>      "application/x-dvi",
  ".gz"           =>      "application/x-gzip",
  ".pac"          =>      "application/x-ns-proxy-autoconfig",
  ".swf"          =>      "application/x-shockwave-flash",
  ".tar.gz"       =>      "application/x-tgz",
  ".tgz"          =>      "application/x-tgz",
  ".tar"          =>      "application/x-tar",
  ".zip"          =>      "application/zip",
  ".mp3"          =>      "audio/mpeg",
  ".m3u"          =>      "audio/x-mpegurl",
  ".wma"          =>      "audio/x-ms-wma",
  ".wax"          =>      "audio/x-ms-wax",
  ".ogg"          =>      "application/ogg",
  ".wav"          =>      "audio/x-wav",
  ".gif"          =>      "image/gif",
  ".jpg"          =>      "image/jpeg",
  ".jpeg"         =>      "image/jpeg",
  ".png"          =>      "image/png",
  ".xbm"          =>      "image/x-xbitmap",
  ".xpm"          =>      "image/x-xpixmap",
  ".xwd"          =>      "image/x-xwindowdump",
  ".css"          =>      "text/css",
  ".html"         =>      "text/html",
  ".htm"          =>      "text/html",
  ".js"           =>      "text/javascript",
  ".asc"          =>      "text/plain",
  ".c"            =>      "text/plain",
  ".cpp"          =>      "text/plain",
  ".log"          =>      "text/plain",
  ".conf"         =>      "text/plain",
  ".text"         =>      "text/plain",
  ".txt"          =>      "text/plain",
  ".dtd"          =>      "text/xml",
  ".xml"          =>      "text/xml",
  ".mpeg"         =>      "video/mpeg",
  ".mpg"          =>      "video/mpeg",
  ".mov"          =>      "video/quicktime",
  ".qt"           =>      "video/quicktime",
  ".avi"          =>      "video/x-msvideo",
  ".asf"          =>      "video/x-ms-asf",
  ".asx"          =>      "video/x-ms-asf",
  ".wmv"          =>      "video/x-ms-wmv",
  ".bz2"          =>      "application/x-bzip",
  ".tbz"          =>      "application/x-bzip-compressed-tar",
  ".tar.bz2"      =>      "application/x-bzip-compressed-tar",
  # default mime type
  ""              =>      "application/octet-stream",
 )

# Redirect all requests to HTTPS, except cgi calls to stay compatible with LEA
#$SERVER["socket"] == ":80" {
#  $HTTP["host"] =~ "(.*)" {
#    url.redirect = ( "^/(?!cgi)(.*)" => "https://%1/$1" )
#  }
#}

# Activate HTTPS
$SERVER["socket"] == ":443" {
  ssl.engine = "enable"
  ssl.pemfile = "/etc/ssl/private/lighttpd.pem"
}
[root@LaCie-CloudBox cgi-bin]# cd /opt
[root@LaCie-CloudBox opt]# cd etc
[root@LaCie-CloudBox etc]# cd backuppc
[root@LaCie-CloudBox backuppc]# vi hosts
[root@LaCie-CloudBox etc]# cd backuppc3
[root@LaCie-CloudBox backuppc3]# cp griffin.pl mandala.pl nxl67170ux.pl pc-112-597.pl ../backuppc/
[root@LaCie-CloudBox backuppc3]# cp xbmcbuntu.pl ../backuppc
[root@LaCie-CloudBox backuppc3]# cd ../backuppc
[root@LaCie-CloudBox backuppc]# cp config.pl config.pl.orig
[root@LaCie-CloudBox backuppc]# chown backuppc:users *
[root@LaCie-CloudBox backuppc]# chown backuppc:root * 
[root@LaCie-CloudBox backuppc]# chmod 640 *
[root@LaCie-CloudBox backuppc]# backuppc start
[root@LaCie-CloudBox backuppc]# cd /etc
[root@LaCie-CloudBox etc]# cd /opt/etc/backuppc
[root@LaCie-CloudBox backuppc]# vi config.pl 
[root@LaCie-CloudBox backuppc]# cd /opt/local/backuppc/lib
[root@LaCie-CloudBox lib]# cd BackupPC/
[root@LaCie-CloudBox BackupPC]# vi Lib.pm 
[root@LaCie-CloudBox BackupPC-4.0.0alpha3]# echo ############### THIS IS IT
[root@LaCie-CloudBox cgi-bin]# ./index 
YOU HAVEN'T DISABLED SET-ID SCRIPTS IN THE KERNEL YET!
FIX YOUR KERNEL, PUT A C WRAPPER AROUND THIS SCRIPT, OR USE -u AND UNDUMP!
[root@LaCie-CloudBox cgi-bin]# cat wrapper.c 
#include <unistd.h>
#define REAL_PATH "/opt/lib/backuppc/BackupPC_Admin"
int main(ac, av)
char **av;
{
	execv(REAL_PATH, av);
	return 0;
}

[root@LaCie-CloudBox cgi-bin]# l /opt/lib/backuppc/BackupPC_Admin 
-r-sr-xr-- 1 backuppc users 4.0K Nov 11 00:28 /opt/lib/backuppc/BackupPC_Admin
[root@LaCie-CloudBox cgi-bin]# mv index index.cgi
[root@LaCie-CloudBox cgi-bin]# chmod 0550 /opt/lib/backuppc/BackupPC_Admin 
[root@LaCie-CloudBox cgi-bin]# ./index.cgi 
[root@LaCie-CloudBox cgi-bin]# backuppc start
[root@LaCie-CloudBox backuppc]# sudo chown backuppc:root *
[root@LaCie-CloudBox backuppc]# sudo chmod 640 *
[root@LaCie-CloudBox backuppc]# su backuppc -s /bin/bash
[root@LaCie-CloudBox etc]#

Install BackupPC clients

Choices:

rsyncd
Can do a full backup w/o giving access to root. access can be restricted to read-only so that backup server cannot do damage to client if compromised. Requires setting up a server and install xinetd. Authentication is weak though, but not much a problem (files are already available on the backup server anyway). Quite simple to configure. I don't like xinetd/inetd though.
ssh+rsync
Login on client as user 'backuppc'. SSH private key on backuppc is password-less, but similar to storing a secret password in some file on the backup server. Root access on the client given via sudo, and restricted to run "rsync --server --sender *", such that to enforce read-only access. All traffic goes via ssh, and hence is encrypted. Strong authentication. A bit more complex to configure, but no xinetd, and no daemon.
See http://www.systemajik.com/blog/setting-up-backuppc-on-ubuntu/ for more information.

ssh+rsync is more difficult to setup; also on the NAS, the backuppc does not have dedicated home directory. So we can't easily store .ssh/id_rsa.pub and .ssh/known_hosts file. Also, on Windows client, it seems the default solution is to use rsyncd. Also, I don't really need link encryption.

We choose rsyncd:

Install log: 

vi /etc/default/rsync
sudo apt-get install xinetd
vi /etc/xinetd.d/rsync
#    service rsync
#    {
#        disable = no
#        socket_type = stream
#        wait = no
#        user = root
#        server = /usr/bin/rsync
#        server_args = --daemon
#        log_on_failure += USERID
#        flags = IPv6
#    }
sudo vi /etc/rsyncd.conf
#    max connections = 2
#    log file = /var/log/rsync.log
#    timeout = 6000
#
#    [share]
#    comment = Public Share
#    path = /home/share
#    read only = yes
#    list = yes
#    uid = nobody
#    gid = nogroup
#    auth users = backuppc
#    secrets file = /etc/rsyncd.secrets

# (Note: default timeout 300 is not enough for our NAS - we get child exited prematurely errors)

sudo vi /etc/rsyncd.secrets
# user:password
sudo chmod 600 /etc/rsyncd.secrets
sudo /etc/init.d/xinetd restart

# Testing
sudo rsync backuppc@hostname::share

Benchmarks

Some statistics: 

0 	 full 	 yes 	 0 	 11/5 20:19 	 1.8 	 0.0 	/shares/backuppc/data/pc/nxl67170ux/0 
1 	 incr 	 no 	 1 	 11/5 20:58 	 2.7 	 0.0 	/shares/backuppc/data/pc/nxl67170ux/1

0 	 full 	 3565 	 73.2 	 0.68 	 884 	 4.5 	 3425 	 68.8 
1 	 incr 	 270 	 201.8 	 1.25 	 29 	 0.0 	 480 	 201.8

0 	 full 	 3 	 4.5 	 2.9 	 35.4% 	 68.8 	 56.8 	 17.5% 
1 	 incr 	 3 	 0.0 	 0.0 	 -6.4% 	 201.8 	 199.1 	 1.3%

Performance not very good. Let's disable compression and do again. I get 1.25MB/s for first full backup. When pinging nxl67170ux, I get my wlan0 address! Let's disable WIFI and start again...

Troubleshooting

Broken pipe

Typical errors: 

rsync_bpc: writefd_unbuffered failed to write 426 bytes to message fd [receiver]: Broken pipe (32)
Possible causes and solutions
  • Happens more frequently with big files, but some small files fail too. Not always the same file
  • Timeout issue
  • Firewall
  • Lots of packet loss / reject. The following iptable route might help [1]:
iptables -I {line} -p tcp --tcp-flags SYN,RST,ACK,FIN ACK -j ACCEPT


Links
  • Interesting discussion on samba.org, with troubleshooting ideas.
  strace -tt -f -o /tmp/rsync-$$.out rsync \
        --verbose \
        --exclude "stor/hosts" \
        --exclude "var/run" \
        --exclude "var/spool" \
        --rsh="ssh -p $backoff_port" \
        --rsync-path="sudo /usr/local/bin/rsync-debug" \
        --recursive \
        --links \
        --perms \
        --times \
        --owner \
        --group \
        --numeric-ids \
        --xattrs \
        --delete \
        --compress \
        --human-readable \
        --stats \
        --exclude='udat/backup*/*' \
        --exclude='udat/migrate/*' \
        /mnt/stor \
        $userid@$backoff_url:$destdir 2>&1

Issues on Lacie-Cloud

Broken pipe, etc

I get failed backup on the lacie-cloud server; I log my findings / tests here.

The current rsync command line is: 

/opt/bin/rsync_bpc --bpc-top-dir /shares/backuppc --bpc-host-name zavcxl0005 --bpc-share-name home \
--bpc-bkup-num 1 --bpc-bkup-comp 3 --bpc-bkup-prevnum -1 --bpc-bkup-prevcomp -1 --bpc-bkup-inode0 567 \
--bpc-log-level 3 --super --recursive --protect-args --numeric-ids --perms --owner --group -D --times \
--links --hard-links --delete --partial --log-format=log: %o %i %B %8U,%8G %9l %f%L --stats \
--modify-window=5 --one-file-system --password-file=/shares/backuppc/pc/zavcxl0005/.rsyncdpw6801 \
--exclude=.cache --exclude=.recoll --exclude=.gvfs --exclude=.thumbnails --exclude=.Trash* \
--exclude=*.nobak --exclude=.*.nobak --exclude=.opera/cache --exclude=Snapshots --exclude=Cache \
--exclude=Dropbox --exclude=$RECYCLE.BIN --exclude=$Recycle.Bin --exclude=System Volume Information \
--exclude=lost+found backuppc@zavcxl0005::home /
Notes
  • Still use the old parameter --log-format, which has been deprecated and renamed to --out-format in newer rsync version.
Out-of-memory
  • Look at dmesg output, and see if there was an Out of memory error:
dmesg|grep -iC 3 "out of memory"
grep -iC 3 "out of memory" /var/log/messages     # To get real timestamps

[3421299.359987] 3205 slab pages
[3421299.359993] 7010 pages shared
[3421299.360000] 8723 pages swap cached
[3421299.360010] Out of memory: kill process 15163 (BackupPC_dump) score 277779 or a child
[3421299.360049] Killed process 15168 (rsync_bpc)
[3421299.734728] rsync_bpc: page allocation failure. order:0, mode:0x20058
[3421299.734750] [<c0031b2c>] (unwind_backtrace+0x0/0xd0) from [<c0080eec>] (__alloc_pages_nodemask+0x49c/0x500)
If so, free some memory. Possibly remove hard-link detection which consumes memory [2].

Zombie rsync_bpc processes

I have two processes still running on lacie-cloudbox: 

[root@LaCie-CloudBox ~]# /opt/bin/ps faux | grep rsync
Unknown HZ value! (73) Assume 100.
root      2249  0.0  0.0   4748     4 ?        Ss   Jun14   0:00 /usr/bin/rsync --daemon --config=/etc/rsyncd.conf --no-detach
root     26783  0.0  0.3   3608   916 pts/0    S+   20:46   0:00          \_ grep rsync
backuppc  5746  1.5  0.0  85764    56 ?        S    10:24   9:41      \_ /opt/bin/rsync_bpc --bpc-top-dir /shares/backuppc --bpc-host-name graphicdaemon --bpc-share-name d --bpc-bkup-num 213 --bpc-bkup-comp 3 --bpc-bkup-prevnum 212 --bpc-bkup-prevcomp 3 --bpc-bkup-inode0 9109 --bpc-log-level 3 --super --recursive --protect-args --numeric-ids --perms --owner --group -D --times --links --hard-links --delete --partial --log-format=log: %o %i %B %8U,%8G %9l %f%L --stats --modify-window=5 --checksum --one-file-system --password-file=/shares/backuppc/pc/graphicdaemon/.rsyncdpw3449 --exclude=.cache --exclude=.recoll --exclude=.gvfs --exclude=.thumbnails --exclude=.Trash* --exclude=*.nobak --exclude=.*.nobak --exclude=.opera/cache --exclude=Cache --exclude=Dropbox --exclude=Snapshots --exclude=$RECYCLE.BIN --exclude=$Recycle.Bin --exclude=System Volume Information --exclude=lost+found backuppc@graphicdaemon::d /
backuppc  5779 12.7  0.0  77872    56 ?        S    10:25  78:57          \_ /opt/bin/rsync_bpc --bpc-top-dir /shares/backuppc --bpc-host-name graphicdaemon --bpc-share-name d --bpc-bkup-num 213 --bpc-bkup-comp 3 --bpc-bkup-prevnum 212 --bpc-bkup-prevcomp 3 --bpc-bkup-inode0 9109 --bpc-log-level 3 --super --recursive --protect-args --numeric-ids --perms --owner --group -D --times --links --hard-links --delete --partial --log-format=log: %o %i %B %8U,%8G %9l %f%L --stats --modify-window=5 --checksum --one-file-system --password-file=/shares/backuppc/pc/graphicdaemon/.rsyncdpw3449 --exclude=.cache --exclude=.recoll --exclude=.gvfs --exclude=.thumbnails --exclude=.Trash* --exclude=*.nobak --exclude=.*.nobak --exclude=.opera/cache --exclude=Cache --exclude=Dropbox --exclude=Snapshots --exclude=$RECYCLE.BIN --exclude=$Recycle.Bin --exclude=System Volume Information --exclude=lost+found backuppc@graphicdaemon::d /

[root@LaCie-CloudBox ~]# netstat -ane | grep 100.117        
netstat: /proc/net/tcp6: No such file or directory
tcp        0      0 172.19.100.98:49947     172.19.100.117:873      ESTABLISHED 
netstat: /proc/net/udp6: No such file or directory
netstat: /proc/net/raw6: No such file or directory

But the rsyncd daemon on the server failed since 10:46 am: 

$ sudo netstat -pna|grep 100.98

$ tail /var/log/rsync.log 
2015/06/19 10:04:31 [2917] send lacie-cloudbox [172.19.100.98] d (backuppc) Games/World_of_Tanks/res/packages/hangar_premium_v2.pkg 28914772
2015/06/19 10:04:31 [2917] send lacie-cloudbox [172.19.100.98] d (backuppc) Games/World_of_Tanks/res/packages/hangar_v2.pkg 27720752
2015/06/19 10:07:43 [2917] send lacie-cloudbox [172.19.100.98] d (backuppc) Games/World_of_Tanks/res/packages/misc.pkg 752868269
2015/06/19 10:12:46 [2917] send lacie-cloudbox [172.19.100.98] d (backuppc) Games/World_of_Tanks/res/packages/shaders.pkg 4879357
2015/06/19 10:17:52 [2917] send lacie-cloudbox [172.19.100.98] d (backuppc) Games/World_of_Tanks/res/packages/shared_content.pkg 950719654
2015/06/19 10:17:52 [2917] send lacie-cloudbox [172.19.100.98] d (backuppc) Games/World_of_Tanks/res/packages/shared_content.prefetch 394119
2015/06/19 10:24:52 [2917] send lacie-cloudbox [172.19.100.98] d (backuppc) Games/World_of_Tanks/res/packages/vehicles_level_01.pkg 36603536
2015/06/19 10:27:20 [2917] send lacie-cloudbox [172.19.100.98] d (backuppc) Games/World_of_Tanks/res/packages/vehicles_level_02.pkg 276573347
2015/06/19 10:46:28 [2917] rsync: [sender] write error: Invalid argument (22)
2015/06/19 10:46:28 [2917] rsync error: error in socket IO (code 10) at io.c(837) [sender=3.1.0]

$ ps faux | grep rsync
isma      8126  0.0  0.0  14776   944 pts/0    S+   20:51   0:00              \_ grep rsync

Checking the client log on lacie-cloudbox (using BackupPC_zcat to decompress), we actually observe that the log is truncated much before the rsyncd log. So it seems the client broke but did not exit. So adding param --timeout to rsync client might not help here.

bpc_fileOpen: can't open pool file

I get for each backup the same error message: 

     same   recv >f......... rwxrwx---        0,      46       640 Windows/winsxs/FileMaps/$$_ime_fr-fr_0d349902e45b4cea.cdf-ms
 G bpc_fileOpen: can't open pool file /shares/backuppc/cpool/f8/02/f803002e8050e1931403001000000000 (from Windows/winsxs/Manifests/amd64_prnin004.inf-languagepack_31bf3856ad364e35_6.1.7600.16385_en-us_426884f5a0a5f5a4.manifest, 3, 16)
 rsync_bpc: failed to open "/Windows/winsxs/Manifests/amd64_prnin004.inf-languagepack_31bf3856ad364e35_6.1.7600.16385_en-us_426884f5a0a5f5a4.manifest", continuing: No such file or directory (2)
     same   recv >f......... rwxrwx---        0,      46      2136 Windows/winsxs/FileMaps/$$_ime_imejp10_dicts_281006c600450618.cdf-ms

The problem is not solved at next backup. A similar problem is reported by another user on backuppc-users mailing list [3].

Stale transfer

Some troubleshooting commands [4]: 

strace -p $PID
lsof -f | grep rsync                          # and the following to confirm)

FIXED - rsync_bpc keep transfering same file on ntfs-3g mount

SOLUTION: Remove options --links and --hard-links when backing up ntfs-3g partitions !!!


Setup
  • /c is a ntfs-3g mount. Content of /etc/fstab:
UUID=445E1B975E1B813A 				/c              ntfs    ro,defaults,nls=utf8,umask=007,gid=46 0       0
  • To test this, add to /etc/rsyncd.conf:
+[test1]
+comment = Test share 1
+path = /c/Boot
  • In a separate window, monitor the rsyncd log file:
tail -f /var/log/rsync.log
  • Create rsync password file:
sudo -v; sudo sed -r 's/^.*://' /etc/rsyncd.secrets | sudo tee /tmp/rsyncd.secrets > /dev/null; sudo chmod 600 /tmp/rsyncd.secrets
Test case one - PASS - rsync + ntfs-3g
  • Initialize:
sudo rm -rf /tmp/rsynctest; sudo mkdir /tmp/rsynctest
  • Repeat the following step, and observe that files are only txfed the first time:
sudo rsync --super --recursive --protect-args --numeric-ids --perms --owner --group -D --times \
--links --hard-links --delete --partial --log-format='log: %o %i %B %8U,%8G %9l %f%L' --stats \
--modify-window=5 --one-file-system --password-file=/tmp/rsyncd.secrets \
--checksum backuppc@griffin::test1 /tmp/rsynctest
Test case two - FAIL - rsync_bpc + ntfs-3g
  • Initialize:
sudo rm -rf /tmp/rsynctest; sudo mkdir -p /tmp/rsynctest/cpool; i=0
  • Repeat the following step, and observe that files are always txfed:
cd /tmp/rsynctest; i=$((i+1)); echo backup $i; sudo cp -ar pc/griffin/$((i-1)) pc/griffin/$i; \
sudo rsync_bpc --bpc-top-dir /tmp/rsynctest --bpc-host-name griffin --bpc-share-name test1 --bpc-bkup-num $i \
--bpc-bkup-comp 3 --bpc-bkup-prevnum $((i-1)) --bpc-bkup-prevcomp 3 --bpc-log-level 3  --super --recursive \
--protect-args --numeric-ids --perms --owner --group -D --times --links --hard-links --delete --partial \
--log-format='log: %o %i %B %8U,%8G %9l %f%L' --stats --modify-window=5 --one-file-system \
--password-file=/tmp/rsyncd.secrets backuppc@griffin::test1 /

Increase --bpc-log-level (say set it to 9) to get more debug output.

Test case three - PASS - rsync_bpc + ntfs_3g fixed
  • Edit /etc/rsyncd.conf such that share test1 no longer points to an ntfs-3g mount point. For instance:
 [test1]
 comment = Test share 1
-path = /c/Boot
+path = /boot
  • Run step above, and observe that files are only txfed once.
  • If running other cases, reset share to ntfs-3g:
 [test1]
 comment = Test share 1
-path = /boot
+path = /c/Boot
Test case fourth - rsync_bpc + ntfs-3g that works
  • Initialize:
sudo rm -rf /tmp/rsynctest; sudo mkdir -p /tmp/rsynctest/cpool; i=0
  • Repeat the following step, and observe that files are only txfed the first time when removing parameters --links --hard-links:
cd /tmp/rsynctest; i=$((i+1)); echo backup $i; sudo cp -ar pc/griffin/$((i-1)) pc/griffin/$i; \
sudo rsync_bpc --bpc-top-dir /tmp/rsynctest --bpc-host-name griffin --bpc-share-name test1 --bpc-bkup-num $i \
--bpc-bkup-comp 3 --bpc-bkup-prevnum $((i-1)) --bpc-bkup-prevcomp 3 --bpc-log-level 3  --super --recursive \
--protect-args --numeric-ids --perms --owner --group -D --times --delete --partial \
--log-format='log: %o %i %B %8U,%8G %9l %f%L' --stats --modify-window=5 --one-file-system \
--password-file=/tmp/rsyncd.secrets backuppc@griffin::test1 /
Retrieved from "https://miki.immie.org/mediawiki/index.php?title=BackupPC&oldid=9313"