Configuration LaCie-CloudBox: Difference between revisions

Revision as of 17:21, 24 April 2017

Reference information

Device: Lacie-CloudBox 3TB
Name: lacie-cloudbox
Lacie-CloudBox dashboard: http://lacie-cloudbox/
uname

uname -a
# Linux LaCie-CloudBox 2.6.31.14-svn7493 #1 Fri Oct 17 14:00:51 UTC 2014 armv5tel GNU/Linux

Mounting the drive in another pc, we see the following structure:

sudo parted -l

Model: ATA ST3000DM001-1CH1 (scsi)
Disk /dev/sdb: 3001GB
Sector size (logical/physical): 512B/4096B
Partition Table: gpt

Number  Start   End     Size    File system     Name         Flags
 1      1049kB  2097kB  1049kB                  grub_core    bios_grub
 2      2097kB  203MB   201MB   ext2            boot_rescue
 3      203MB   204MB   1049kB  ext2            nv_data
 4      204MB   1815MB  1611MB  ext2            root_1       raid
 5      1815MB  3426MB  1611MB  ext2            root_2       raid
 6      3426MB  4499MB  1074MB  ext3            var          raid
 7      4499MB  4768MB  268MB   linux-swap(v1)  swap         raid
 8      4768MB  3001GB  2996GB                  user_data    raid


Model: Linux Software RAID Array (md)
Disk /dev/md4: 1611MB
Sector size (logical/physical): 512B/4096B
Partition Table: loop

Number  Start  End     Size    File system  Flags
 1      0.00B  1611MB  1611MB  ext2


Model: Linux Software RAID Array (md)
Disk /dev/md5: 1611MB
Sector size (logical/physical): 512B/4096B
Partition Table: loop

Number  Start  End     Size    File system  Flags
 1      0.00B  1611MB  1611MB  ext2


Model: Linux Software RAID Array (md)
Disk /dev/md6: 1074MB
Sector size (logical/physical): 512B/4096B
Partition Table: loop

Number  Start  End     Size    File system  Flags
 1      0.00B  1074MB  1074MB  ext3


Model: Linux Software RAID Array (md)
Disk /dev/md7: 268MB
Sector size (logical/physical): 512B/4096B
Partition Table: loop

Number  Start  End    Size   File system     Flags
 1      0.00B  268MB  268MB  linux-swap(v1)

The disk is organized as follows :

/dev/sdb4 and /dev/sdb5 are raid volumes that contains two copies of the same root partition. The other copy is used when doing a firmware update (ping-pong approach, see [1]). We mount them with:

mdadm --assemble --run /dev/md4 /dev/sdb4       # Done automatically on Ubuntu
mdadm --assemble --run /dev/md5 /dev/sdb5       # Done automatically on Ubuntu
mount /dev/md4 /mount/md4
mount /dev/md5 /mount/md5

/dev/sdb6 contains another raid partition, mounted as an additional layer over root partition above. It contains:

/dev/md6
    0/
        etc/
        root/
        var/
    1/
        etc/
        root/
        var/
    lost+found/
    tmp/

Configuration

First install

Network share available at smb://lacie-cloudbox/.
Cloudbox dashboard available at http://lacie-cloudbox.local.
setup smtp access (smtp.scarlet.be, port 25)
IP Configuration — fixed IP address 172.19.100.98
Enable the netbackup-share (port 873, encrypted port 22)
Create user beq06659, with private share.

MAC	00:d0:4b:96:34:32
IP	172.19.100.98
hostname	lacie-cloudbox

Rooting

Guide: http://lacie.nas-central.org/wiki/Category:CloudBox#Enabling_Secure_Shell
Log: See Lacie NAS rooting for install log first rooting.

After firmware upgrade, we need to root it again.

Create telnetd.sh in a share root

cat > /smb/lacie-cloudbox/family/telnetd.sh <<- __END__
#!/bin/sh
/usr/sbin/telnetd -l /bin/sh
__END__

Get and build clunc:

# CLUNC Home: http://lacie-nas.org/doku.php?id=clunc
git clone http://git.lacie-nas.org/clunc.git
cd clunc
make

Run clunc, and reboot lacie:

./clunc -i 192.168.1.98

... Doesn't work anymore.

We mount the drive in another pc. There are 8 partitions, partitions /dev/sdb4 and above are raid.

sudo mdadm --assemble --run /dev/md0 /dev/sdb4
#mdadm: /dev/md0 has been started with 1 drive
sudo mount /dev/md0 /mnt/any

Enable SSH:

Edit /etc/initng/runlevel/default.runlevel.
Edit /etc/unicorn/unicorn_conf/unicorn.sharing.ssh.conf.
Edit /etc/ssh/ssd_config/.

Afterwards we unmount:

sudo umount /mnt/any
sudo mdadm --stop /dev/md0

... we reboot. And it doesn't work! :-(.

We changed the following file:

Edit file /etc/initng/runlevel/default.runlevel:

-#sshd
+sshd

Edit file /etc/unicorn/unicorn_conf/unicorn.sharing.ssh.conf

-enabled: false
+enabled: true

Edit file /etc/ssh/sshd_config

-PermitRootLogin no
-AllowUsers netbackup
+PermitRootLogin yes
+AllowUsers root netbackup git
+Match User root
+Match User git

Add to file /etc/environment:

PATH=/opt/bin:/opt/sbin

Edit file /etc/hosts
Edit file /etc/lighttpd/lighttpd.conf
Edit file /etc/passwd (for user git)

Install Git

To enable git, we must:

Enable git logging in /etc/passwd.
Enable git ssh logging in /etc/ssh/sshd_config

Git needs a lot of memory when compressing its database (occurs at each fetch!). To limit this, run as root:

git config --system pack.windowmemory 128m
git config --system pack.packsizelimit 1g
git config --system pack.threads 1
git config --system core.bigFileThreshold 16m

Then as user git, in all relevant projects (typically those with big files)

cd ~/git/minecraft_nosaves.git
git config --local core.bigFileThreshold 10m

Install rsync

We do via user git, so:

Set /bin/bash as shell for user git in /etc/passwd
Set PermitUserEnvironment yes in /etc/ssh/sshd_config
Create file /shares/git/.ssh/environment to add rsync into path:

PATH=/opt/bin:/opt/sbin:/bin:/usr/bin:/sbin:/usr/sbin

Startup script

Create the file /opt/etc/init.d/S80config (chmod 755):

#! /bin/sh

# Enable /bin/bash as shell for user git to also allow rsync + git in ssh login
sed -ri '/^git/{s_/home_/shares/git_; s_/bin/false_/bin/bash_}' /etc/passwd
# sed -ri '/^git/{s_/home_/shares/git_; s_/bin/false_/opt/bin/git-shell_}' /etc/passwd

# Change permission for /shares/git or sshd will complain (or add 'StrictModes off to /etc/ssh/ssh_config)
chmod go-w /shares/git

# Configure sshd
if ! grep -q git /etc/ssh/sshd_config; then 
    # Enable ssh login with user git
    sed -ri '/^AllowUsers/s/$/ git/;$aMatch User git' /etc/ssh/sshd_config
    # Enable user environment for user git (in order to set PATH)
    sed -ri $'/^PermitRootLogin/{a PermitUserEnvironment yes\n}' /etc/ssh/sshd_config
    # Restart sshd
    ngc --restart sshd
fi

# Add git to path
if ! grep -q /opt/bin /etc/environment; then echo "PATH=/opt/bin:/opt/sbin" >> /etc/environment; fi

# Add extra addresses to /etc/hosts
if ! grep -q "###hosts.inc###" /etc/hosts; then cat /opt/etc/hosts.inc >> /etc/hosts; fi

Create the hosts include file /opt/etc/hosts.inc:

# ###hosts.inc### additional hosts for lacie-cloudbox
172.19.3.1      local.gateway            # Static
172.19.3.2      manticore                # Static?
172.19.100.1	gryphon
172.19.100.2	minimoy                  # DHCP Static        00:24:54:02:98:8d
172.19.100.3	unicorn                  # obsolete
172.19.100.4	beqleunxp1nb103          # obsolete
172.19.100.5	nxl67002ux               # obsolete
172.19.100.6	griffin                  # DHCP Static        68:b5:99:f2:4d:99
172.19.100.7	lambada                  # obsolete
172.19.100.8	reddragon
172.19.100.9    leviathan                # obsolete
172.19.100.10   nxl67063                 # obsolete
172.19.100.11   mandala                  # DHCP Static        00:1b:24:fa:a2:9b
172.19.100.12   nxl67002                 # obsolete
172.19.100.13   nxl67063ux               # obsolete
172.19.100.14   nxl67170ux               # DHCP Static        b8:ca:3a:cc:07:bf
172.19.100.15   pc-112-597               # DHCP Static        b8:ca:3a:93:91:27
172.19.100.16   zyratastic
172.19.100.17   graphicdaemon
172.19.100.18   xbmcbuntu                # DHCP Static        00:01:2e:4e:46:05 00:01:2e:4e:46:06
172.19.100.19   zavcxl0005               # DHCP Static        48:0f:cf:27:ad:ad
# 172.19.100.98	lacie-cloudbox           # Static
172.19.100.99	mnemosyne                # Static
172.19.100.102	minimoy-wlan             # DHCP Static        00:26:5e:37:83:ea
172.19.100.105	nxl67002ux-wlan          # DHCP Static
172.19.100.106	griffin-wlan             # DHCP Static        00:24:d7:7d:4e:94
172.19.100.107	apple-pi                 # Static
172.19.100.111  mandala-wlan             # DHCP Static        00:1d:92:13:d2:7d
172.19.100.114  nxl67170ux-wlan          # DHCP Static        84:3a:4b:22:0d:0c
172.19.100.116  zyratastic-wlan
172.19.100.118  xbmcbuntu-wlan           # DHCP Static        0c:d2:92:54:c1:53
172.19.100.119  zavcxl0005-wlan          # DHCP Static        ac:fd:ce:40:c5:31
172.19.100.120	galaxy

Install Unison

ipkg provides unison 2.27.57. It can be installed via ipkg install unison.

ipkg list | grep unison
# unison - 2.27.57-2 - A cross-platform file-synchronization tool.

To get a more recent version, see Unison to compile from source. v2.40.102 is the same as on Ubuntu Trusty 14.04.

Netconsole server

On lacie-cloudbox:

First install netcat:

ipkg install netcat

Then start the server:

mkdir -p /shares/beq06659/netconsole
nc -l -p 6666 -u >/shares/beq06659/netconsole/zavcxl0005-netconsole&

On the client, at reboot/wake-up:

NETCON_TGTHOST=lacie-cloudbox
NETCON_PORT=6666
echo "[------------] Redocking on $(date +"%Y-%m-%d %H:%M:%S")..." | nc -w 1 -u -p 6665 $NETCON_TGTHOST $NETCON_PORT
if [ -n "$NETCON_TGTHOST" ]; then 
    NETCON_TGTIP=$(getent hosts $NETCON_TGTHOST|awk '{print $1}')
    ping -c 1 $NETCON_TGTIP >/dev/null
    NETCON_TGTMAC=$(arp -n $NETCON_TGTIP|awk '/ether/{print $3}')
    echo "Setting up netconsole for location '$LOCATION' ($NETCON_TGTHOST,$NETCON_TGTIP/$NETCON_TGTMAC)"
    sudo rmmod netconsole 2> /dev/null
    sudo modprobe netconsole netconsole=@/,$NETCON_PORT@$NETCON_TGTIP/$NETCON_TGTMAC
else 
    echo "netconsole - Unknown location $LOCATION"
fi

Shares

family

Mount with

sudo mount //lacie-cloudbox/beq06659 /mnt/any -o noperm,iocharset=utf8,sec=ntlm,username=public,password=

beq06659

Mount with

sudo mount //lacie-cloudbox/beq06659 /mnt/any -o noperm,iocharset=utf8,sec=ntlm,username=beq06659

Backup

Backup Mnemosyne

We use unison.

/root/.unison/default.prf (<file name="default.prf" tag="source">download</file>)

# Unison default preferences file

# Fastcheck - by default false on windows machine, and true on Unix machine.
# When true, uses modification date + file size as inone number. Otherwise do a full scan
fastcheck = true

# If any new preferences are added by Unison (e.g. 'ignore'
# preferences added via the graphical UI), then store them in the
# default profile file
addprefsto = default.prf

# Ignore file permissions (necessary on samba/CIFS fs)
# perms = 0

# Make sure that both roots are mounted
mountpoint = .

# Common Ignore specification
ignore = Name System Volume Information

/root/.unison/musics.prf (<file name="musics.prf" tag="source">download</file>)

# Common include
include default.prf

root = /shares/Family/musics
root = ssh://root@mnemosyne//volume1/musics

# Where to find unison on remote server
servercmd = /opt/bin/unison

# don't ask for non-conflicting changes
auto = true

# We *MUST* sync permissions because unison defaults would prevent access to files/folders
# via samba shares (since unison runs as root)
# perms = 0

/root/.unison/archives.prf (<file name="archives.prf" tag="source">download</file>)

# Common include
include default.prf

root = /shares/Family/archives
root = ssh://root@mnemosyne//volume1/archives

# Where to find unison on remote server
servercmd = /opt/bin/unison

# don't ask for non-conflicting changes
auto = true

# We *MUST* sync permissions because unison defaults would prevent access to files/folders
# via samba shares (since unison runs as root)
# perms = 0

BackupPC

See also Backuppc

Fix broken pipe errors

Backuppc requires lot of memory (rsync). To free as much as possible, I kill all python stuff (except ublock-daemon) and all media stuff:

pkill -SIGSTOP -f unicorn            # I stop it first so that it can't react
pkill -SIGKILL -f unicorn            # then kill it
sleep 1 && pkill -SIGSTOP -f unicorn # then stop it again because it respawns
pkill -SIGSTOP -f mt_daapd
pkill -SIGSTOP twonky
pkill -SIGSTOP php-cgi
pkill -SIGKILL -f mt-daapd           # media server stuff
pkill -SIGKILL twonky                # media server stuff
pkill -SIGKILL php-cgi               # php-cgi server launcher by lighttpd

Duplicity (Obsolete)

This section is OBSOLETE — I no longer use duplicity

Backup done with deja-dup, which use duplicity as back-end
Duplicity can't access samba share. Instead mount samba share locally, and use file:///smb/lacie-cloudbox/sharename instead. Note the triple slash.
View available backup:

duplicity collection-status file:///smb/lacie-cloudbox/beq06659

List file in current backup

duplicity list-current-files file:///smb/lacie-cloudbox/beq06659

Metadata information is stored in ~/.cache/duplicity. Attention, it seems this can grow quite large.

DHCP Server

install

# Install the service
ipkg install dhcp
mkdir /opt/var/run
# Edit the configuration
vi /opt/etc/dhcpd.conf
# Restart the service
/opt/etc/init.d/S56dhcpd

DHCP daemon configuration: /opt/etc/dhcpd.conf
Current leases are at /opt/etc/dhcpd.leases

Now I disabled the dhcpd, and will try again the one on vDSL+ router:

cp /opt/etc/dhcpd.conf /opt/etc/dhcpd.conf.bak     # Backup config if we want to reinstall
ipkg remove dhcp

DHCP server on the VDSL+ router is a huge pile of sh*te, so I enable it again:

ipkg install dhcp
cp /opt/etc/dhcpd.conf.bak /opt/etc/dhcpd.conf
/opt/etc/init.d/S56dhcpd

To do

Current backup only includes files that are readable by user beq06659. It should include all files.
Exclude some unnecessary folders. Organize these folders to ease future backup.
Backup / filesystem (at least /etc, /usr/local)

Backup solutions

deja-dup

Need one setup for each pc
No centralized management. Backup job could happen all at the same time. Same files between pc are stored multiple times.
Easy to setup
How to backup root filesystem?

duplicity
backuppc

How to install?
Will the NAS have enough memory?

bacula

Package readily available in OptWare

rsnapshot

See question on bacula/backuppc/rsnapshot at http://serverfault.com/questions/282228/bacula-vs-backuppc

Rooting NAS and install new app

TBC

Troubleshoot

sshd

See log at /var/log/messages
Check sshd config /etc/ssh/sshd_config:

Make sure user is authorized:

 AllowUsers root netbackup git

If user home is on /shares mount, then disable StrictModes

StrictModes no

Enable user environment

PermitUserEnvironment yes

then

env > ~/.ssh/environment
vi ~/.ssh/environment                # Remove useless variables

Edit /etc/passwd:

git:x:503:100:None:/shares/git:/opt/bin/git-shell

Restart sshd:

ngc --restart sshd

git

Check that environment is ok [2]

ssh git@lacie-cloudbox  "which git-upload-pack"

ntp

Box loses regularly time synchronization, which then affects backuppc
Workaround: log into https://lacie-cloudbox/, and reset ntp (uncheck / check ntp server).

slow performance, lot of swapping

The Lacie-Cloudbox only has 256MB physical memory, and some processes takes a lot of memory. When doing some unusual tasks, we can disable (stop) temporarily these processes, without apparently damage:

pkill -SIGSTOP twonky        # Usually consuming around 100MB+
pkill -f -SIGSTOP unicorn    # A python process running /usr/lib/unicorn/run.py (and consuming 100MB+)

To restart:

pkill -SIGCONT twonky
pkill -f -SIGCONT unicorn

Configuration LaCie-CloudBox: Difference between revisions

Revision as of 17:21, 24 April 2017

Contents

Reference information

Configuration

First install

Rooting

Install Git

Install rsync

Startup script

Install Unison

Netconsole server

Shares

Backup

Backup Mnemosyne

BackupPC

Duplicity (Obsolete)

DHCP Server

To do

Troubleshoot

sshd

git

ntp

slow performance, lot of swapping

Navigation menu

Configuration LaCie-CloudBox: Difference between revisions

Revision as of 17:21, 24 April 2017

Reference information

Configuration

First install

Rooting

Install Git

Install rsync

Startup script

Install Unison

Netconsole server

Shares

Backup

Backup Mnemosyne

BackupPC

Duplicity (Obsolete)

DHCP Server

To do

Troubleshoot

sshd

git

ntp

slow performance, lot of swapping

Navigation menu

Search