Configuration LaCie-CloudBox

Reference information

• Device: Lacie-CloudBox 3TB
• Name: lacie-cloudbox
• Lacie-CloudBox dashboard: http://lacie-cloudbox/

• uname

uname -a
# Linux LaCie-CloudBox 2.6.31.14-svn7493 #1 Fri Oct 17 14:00:51 UTC 2014 armv5tel GNU/Linux

Mounting the drive in another pc, we see the following structure:

sudo parted -l

Model: ATA ST3000DM001-1CH1 (scsi)
Disk /dev/sdb: 3001GB
Sector size (logical/physical): 512B/4096B
Partition Table: gpt

Number  Start   End     Size    File system     Name         Flags
 1      1049kB  2097kB  1049kB                  grub_core    bios_grub
 2      2097kB  203MB   201MB   ext2            boot_rescue
 3      203MB   204MB   1049kB  ext2            nv_data
 4      204MB   1815MB  1611MB  ext2            root_1       raid
 5      1815MB  3426MB  1611MB  ext2            root_2       raid
 6      3426MB  4499MB  1074MB  ext3            var          raid
 7      4499MB  4768MB  268MB   linux-swap(v1)  swap         raid
 8      4768MB  3001GB  2996GB                  user_data    raid


Model: Linux Software RAID Array (md)
Disk /dev/md4: 1611MB
Sector size (logical/physical): 512B/4096B
Partition Table: loop

Number  Start  End     Size    File system  Flags
 1      0.00B  1611MB  1611MB  ext2


Model: Linux Software RAID Array (md)
Disk /dev/md5: 1611MB
Sector size (logical/physical): 512B/4096B
Partition Table: loop

Number  Start  End     Size    File system  Flags
 1      0.00B  1611MB  1611MB  ext2


Model: Linux Software RAID Array (md)
Disk /dev/md6: 1074MB
Sector size (logical/physical): 512B/4096B
Partition Table: loop

Number  Start  End     Size    File system  Flags
 1      0.00B  1074MB  1074MB  ext3


Model: Linux Software RAID Array (md)
Disk /dev/md7: 268MB
Sector size (logical/physical): 512B/4096B
Partition Table: loop

Number  Start  End    Size   File system     Flags
 1      0.00B  268MB  268MB  linux-swap(v1)

The disk is organized as follows :

• /dev/sdb4 and /dev/sdb5 are raid volumes that contains two copies of the same root partition. The other copy is used when doing a firmware update (ping-pong approach, see [1]). We mount them with:

mdadm --assemble --run /dev/md4 /dev/sdb4       # Done automatically on Ubuntu
mdadm --assemble --run /dev/md5 /dev/sdb5       # Done automatically on Ubuntu
mount /dev/md4 /mount/md4
mount /dev/md5 /mount/md5

• /dev/sdb6 contains another raid partition, mounted as an additional layer over root partition above. It contains:

/dev/md6
0/
    etc/
    root/
    var/
1/
    etc/
    root/
    var/
lost+found/
tmp/

Configuration

First install

• Network share available at smb://lacie-cloudbox/.
• Cloudbox dashboard available at http://lacie-cloudbox.local.
• setup smtp access (smtp.scarlet.be, port 25)
• IP Configuration — fixed IP address 172.19.100.98
• Enable the netbackup-share (port 873, encrypted port 22)
• Create user beq06659, with private share.

MAC	00:d0:4b:96:34:32
IP	172.19.100.98
hostname	lacie-cloudbox

Rooting

• Guide: http://lacie.nas-central.org/wiki/Category:CloudBox#Enabling_Secure_Shell
• Log: See Lacie NAS rooting for install log first rooting.

After firmware upgrade, we need to root it again.

• Create telnetd.sh in a share root

cat > /smb/lacie-cloudbox/family/telnetd.sh <<- __END__
#!/bin/sh
/usr/sbin/telnetd -l /bin/sh
__END__

• Get and build clunc:

# CLUNC Home: http://lacie-nas.org/doku.php?id=clunc
git clone http://git.lacie-nas.org/clunc.git
cd clunc
make

• Run clunc, and reboot lacie:

./clunc -i 192.168.1.98

... Doesn't work anymore.

• We mount the drive in another pc. There are 8 partitions, partitions /dev/sdb4 and above are raid.

sudo mdadm --assemble --run /dev/md0 /dev/sdb4
#mdadm: /dev/md0 has been started with 1 drive
sudo mount /dev/md0 /mnt/any

• Enable SSH:

• Edit /etc/initng/runlevel/default.runlevel.
• Edit /etc/unicorn/unicorn_conf/unicorn.sharing.ssh.conf.
• Edit /etc/ssh/ssd_config/.

• Afterwards we unmount:

sudo umount /mnt/any
sudo mdadm --stop /dev/md0

• ... we reboot. And it doesn't work! :-(.

We changed the following file:

• Edit file /etc/initng/runlevel/default.runlevel:

-#sshd
+sshd

• Edit file /etc/unicorn/unicorn_conf/unicorn.sharing.ssh.conf

-enabled: false
+enabled: true

• Edit file /etc/ssh/sshd_config

-PermitRootLogin no
-AllowUsers netbackup
+PermitRootLogin yes
+AllowUsers root netbackup git
+Match User root
+Match User git

• Add to file /etc/environment:

PATH=/opt/bin:/opt/sbin

• Edit file /etc/hosts
• Edit file /etc/lighttpd/lighttpd.conf
• Edit file /etc/passwd (for user git)

Install Git

To enable git, we must:

• Enable git logging in /etc/passwd.
• Enable git ssh logging in /etc/ssh/sshd_config

Git needs a lot of memory when compressing its database (occurs at each fetch!). To limit this, run as root:

git config --system pack.windowmemory 128m
git config --system pack.packsizelimit 1g
git config --system pack.threads 1
git config --system core.bigFileThreshold 16m

Then as user git, in all relevant projects (typically those with big files)

cd ~/git/minecraft_nosaves.git
git config --local core.bigFileThreshold 10m

Install rsync

We do via user git, so:

• Set /bin/bash as shell for user git in /etc/passwd
• Set PermitUserEnvironment yes in /etc/ssh/sshd_config
• Create file /shares/git/.ssh/environment to add rsync into path:

PATH=/opt/bin:/opt/sbin:/bin:/usr/bin:/sbin:/usr/sbin

Startup script

Create the file /opt/etc/init.d/S80config (chmod 755):

#! /bin/sh

# Enable /bin/bash as shell for user git to also allow rsync + git in ssh login
sed -ri '/^git/{s_/home_/shares/git_; s_/bin/false_/bin/bash_}' /etc/passwd
# sed -ri '/^git/{s_/home_/shares/git_; s_/bin/false_/opt/bin/git-shell_}' /etc/passwd

# Change permission for /shares/git or sshd will complain (or add 'StrictModes off to /etc/ssh/ssh_config)
chmod go-w /shares/git

# Configure sshd
if ! grep -q git /etc/ssh/sshd_config; then 
    # Enable ssh login with user git
    sed -ri '/^AllowUsers/s/$/ git/;$aMatch User git' /etc/ssh/sshd_config
    # Enable user environment for user git (in order to set PATH)
    sed -ri $'/^PermitRootLogin/{a PermitUserEnvironment yes\n}' /etc/ssh/sshd_config
    # Restart sshd
    ngc --restart sshd
fi

# Add git to path
if ! grep -q /opt/bin /etc/environment; then echo "PATH=/opt/bin:/opt/sbin" >> /etc/environment; fi

# Add extra addresses to /etc/hosts
if ! grep -q "###hosts.inc###" /etc/hosts; then cat /opt/etc/hosts.inc >> /etc/hosts; fi

Create the hosts include file /opt/etc/hosts.inc:

# ###hosts.inc### additional hosts for lacie-cloudbox
172.19.3.1      local.gateway            # Static
172.19.3.2      manticore                # Static?
172.19.100.1	gryphon
172.19.100.2	minimoy                  # DHCP Static        00:24:54:02:98:8d
172.19.100.3	unicorn                  # obsolete
172.19.100.4	beqleunxp1nb103          # obsolete
172.19.100.5	nxl67002ux               # obsolete
172.19.100.6	griffin                  # DHCP Static        68:b5:99:f2:4d:99
172.19.100.7	lambada                  # obsolete
172.19.100.8	reddragon
172.19.100.9    leviathan                # obsolete
172.19.100.10   nxl67063                 # obsolete
172.19.100.11   mandala                  # DHCP Static        00:1b:24:fa:a2:9b
172.19.100.12   nxl67002                 # obsolete
172.19.100.13   nxl67063ux               # obsolete
172.19.100.14   nxl67170ux               # DHCP Static        b8:ca:3a:cc:07:bf
172.19.100.15   pc-112-597               # DHCP Static        b8:ca:3a:93:91:27
172.19.100.16   zyratastic
172.19.100.17   graphicdaemon
172.19.100.18   xbmcbuntu                # DHCP Static        00:01:2e:4e:46:05 00:01:2e:4e:46:06
172.19.100.19   zavcxl0005               # DHCP Static        48:0f:cf:27:ad:ad
# 172.19.100.98	lacie-cloudbox           # Static
172.19.100.99	mnemosyne                # Static
172.19.100.102	minimoy-wlan             # DHCP Static        00:26:5e:37:83:ea
172.19.100.105	nxl67002ux-wlan          # DHCP Static
172.19.100.106	griffin-wlan             # DHCP Static        00:24:d7:7d:4e:94
172.19.100.107	apple-pi                 # Static
172.19.100.111  mandala-wlan             # DHCP Static        00:1d:92:13:d2:7d
172.19.100.114  nxl67170ux-wlan          # DHCP Static        84:3a:4b:22:0d:0c
172.19.100.116  zyratastic-wlan
172.19.100.118  xbmcbuntu-wlan           # DHCP Static        0c:d2:92:54:c1:53
172.19.100.119  zavcxl0005-wlan          # DHCP Static        ac:fd:ce:40:c5:31
172.19.100.120	galaxy

Install Unison

• ipkg provides unison 2.27.57. It can be installed via ipkg install unison.

ipkg list | grep unison
# unison - 2.27.57-2 - A cross-platform file-synchronization tool.

• To get a more recent version, see Unison to compile from source. v2.40.102 is the same as on Ubuntu Trusty 14.04.

Netconsole server

On lacie-cloudbox:

• First install netcat:

ipkg install netcat

• Then start the server:

mkdir -p /shares/beq06659/netconsole
nc -l -p 6666 -u >/shares/beq06659/netconsole/zavcxl0005-netconsole&

On the client, at reboot/wake-up:

NETCON_TGTHOST=lacie-cloudbox
NETCON_PORT=6666
echo "[------------] Redocking on $(date +"%Y-%m-%d %H:%M:%S")..." | nc -w 1 -u -p 6665 $NETCON_TGTHOST $NETCON_PORT
if [ -n "$NETCON_TGTHOST" ]; then 
    NETCON_TGTIP=$(getent hosts $NETCON_TGTHOST|awk '{print $1}')
    ping -c 1 $NETCON_TGTIP >/dev/null
    NETCON_TGTMAC=$(arp -n $NETCON_TGTIP|awk '/ether/{print $3}')
    echo "Setting up netconsole for location '$LOCATION' ($NETCON_TGTHOST,$NETCON_TGTIP/$NETCON_TGTMAC)"
    sudo rmmod netconsole 2> /dev/null
    sudo modprobe netconsole netconsole=@/,$NETCON_PORT@$NETCON_TGTIP/$NETCON_TGTMAC
else 
    echo "netconsole - Unknown location $LOCATION"
fi

Shares

family

• Mount with

sudo mount //lacie-cloudbox/beq06659 /mnt/any -o noperm,iocharset=utf8,sec=ntlm,username=public,password=

beq06659

• Mount with

sudo mount //lacie-cloudbox/beq06659 /mnt/any -o noperm,iocharset=utf8,sec=ntlm,username=beq06659

Backup

Backup Mnemosyne

We use unison.

/root/.unison/default.prf (<file name="default.prf" tag="source">download</file>)

# Unison default preferences file

# Fastcheck - by default false on windows machine, and true on Unix machine.
# When true, uses modification date + file size as inone number. Otherwise do a full scan
fastcheck = true

# If any new preferences are added by Unison (e.g. 'ignore'
# preferences added via the graphical UI), then store them in the
# default profile file
addprefsto = default.prf

# Ignore file permissions (necessary on samba/CIFS fs)
# perms = 0

# Make sure that both roots are mounted
mountpoint = .

# Common Ignore specification
ignore = Name System Volume Information

/root/.unison/musics.prf (<file name="musics.prf" tag="source">download</file>)

# Common include
include default.prf

root = /shares/Family/musics
root = ssh://root@mnemosyne//volume1/musics

# Where to find unison on remote server
servercmd = /opt/bin/unison

# don't ask for non-conflicting changes
auto = true

# We *MUST* sync permissions because unison defaults would prevent access to files/folders
# via samba shares (since unison runs as root)
# perms = 0

/root/.unison/archives.prf (<file name="archives.prf" tag="source">download</file>)

# Common include
include default.prf

root = /shares/Family/archives
root = ssh://root@mnemosyne//volume1/archives

# Where to find unison on remote server
servercmd = /opt/bin/unison

# don't ask for non-conflicting changes
auto = true

# We *MUST* sync permissions because unison defaults would prevent access to files/folders
# via samba shares (since unison runs as root)
# perms = 0

BackupPC

• See also Backuppc

Fix broken pipe errors

Backuppc requires lot of memory (rsync). To free as much as possible, I kill all python stuff (except ublock-daemon) and all media stuff:

pkill -SIGSTOP -f unicorn            # I stop it first so that it can't react
pkill -SIGKILL -f unicorn            # then kill it
sleep 1 && pkill -SIGSTOP -f unicorn # then stop it again because it respawns
pkill -SIGSTOP -f mt_daapd
pkill -SIGSTOP twonky
pkill -SIGSTOP php-cgi
pkill -SIGKILL -f mt-daapd           # media server stuff
pkill -SIGKILL twonky                # media server stuff
pkill -SIGKILL php-cgi               # php-cgi server launcher by lighttpd

Duplicity (Obsolete)

This section is OBSOLETE — I no longer use duplicity

• Backup done with deja-dup, which use duplicity as back-end
• Duplicity can't access samba share. Instead mount samba share locally, and use file:///smb/lacie-cloudbox/sharename instead. Note the triple slash.
• View available backup:

duplicity collection-status file:///smb/lacie-cloudbox/beq06659

• List file in current backup

duplicity list-current-files file:///smb/lacie-cloudbox/beq06659

Metadata information is stored in ~/.cache/duplicity. Attention, it seems this can grow quite large.

DHCP Server

• install

# Install the service
ipkg install dhcp
mkdir /opt/var/run
# Edit the configuration
vi /opt/etc/dhcpd.conf
# Restart the service
/opt/etc/init.d/S56dhcpd

• DHCP daemon configuration: /opt/etc/dhcpd.conf
• Current leases are at /opt/etc/dhcpd.leases

• Now I disabled the dhcpd, and will try again the one on vDSL+ router:

cp /opt/etc/dhcpd.conf /opt/etc/dhcpd.conf.bak     # Backup config if we want to reinstall
ipkg remove dhcp

• DHCP server on the VDSL+ router is a huge pile of sh*te, so I enable it again:

ipkg install dhcp
cp /opt/etc/dhcpd.conf.bak /opt/etc/dhcpd.conf
/opt/etc/init.d/S56dhcpd

To do

• Current backup only includes files that are readable by user beq06659. It should include all files.
• Exclude some unnecessary folders. Organize these folders to ease future backup.
• Backup / filesystem (at least /etc, /usr/local)

Backup solutions

• deja-dup

• Need one setup for each pc
• No centralized management. Backup job could happen all at the same time. Same files between pc are stored multiple times.
• Easy to setup
• How to backup root filesystem?

• duplicity
• backuppc

• How to install?
• Will the NAS have enough memory?

• bacula

• Package readily available in OptWare

• rsnapshot

See question on bacula/backuppc/rsnapshot at http://serverfault.com/questions/282228/bacula-vs-backuppc

Rooting NAS and install new app

• TBC

Troubleshoot

sshd

• See log at /var/log/messages
• Check sshd config /etc/ssh/sshd_config:

• Make sure user is authorized:

 AllowUsers root netbackup git

• If user home is on /shares mount, then disable StrictModes

StrictModes no

• Enable user environment

PermitUserEnvironment yes

then

env > ~/.ssh/environment
vi ~/.ssh/environment                # Remove useless variables

• Edit /etc/passwd:

git:x:503:100:None:/shares/git:/opt/bin/git-shell

• Restart sshd:

ngc --restart sshd

git

• Check that environment is ok [2]

ssh git@lacie-cloudbox  "which git-upload-pack"

ntp

• Box loses regularly time synchronization, which then affects backuppc
• Workaround: log into https://lacie-cloudbox/, and reset ntp (uncheck / check ntp server).

slow performance, lot of swapping

The Lacie-Cloudbox only has 256MB physical memory, and some processes takes a lot of memory. When doing some unusual tasks, we can disable (stop) temporarily these processes, without apparently damage:

pkill -SIGSTOP twonky        # Usually consuming around 100MB+
pkill -f -SIGSTOP unicorn    # A python process running /usr/lib/unicorn/run.py (and consuming 100MB+)

To restart:

pkill -SIGCONT twonky
pkill -f -SIGCONT unicorn