Configuration LaCie-CloudBox: Difference between revisions

From miki
Jump to navigation Jump to search
Line 3: Line 3:
* Name: '''lacie-cloudbox'''
* Name: '''lacie-cloudbox'''
* Lacie-CloudBox dashboard: '''http://lacie-cloudbox/'''
* Lacie-CloudBox dashboard: '''http://lacie-cloudbox/'''


* uname
* uname
<source lang=bash>
<source lang=bash>
Line 15: Line 13:
sudo parted -l
sudo parted -l
</source>
</source>
<source lang="c">
<source lang="text">
Model: ATA ST3000DM001-1CH1 (scsi)
Model: ATA ST3000DM001-1CH1 (scsi)
Disk /dev/sdb: 3001GB
Disk /dev/sdb: 3001GB
Line 79: Line 77:
<source lang="c">
<source lang="c">
/dev/md6
/dev/md6
0/
0/
etc/
etc/
root/
root/
var/
var/
1/
1/
etc/
etc/
root/
root/
var/
var/
lost+found/
lost+found/
tmp/
tmp/
</source>
</source>



Revision as of 17:21, 24 April 2017

Reference information

uname -a
# Linux LaCie-CloudBox 2.6.31.14-svn7493 #1 Fri Oct 17 14:00:51 UTC 2014 armv5tel GNU/Linux

Mounting the drive in another pc, we see the following structure:

sudo parted -l
Model: ATA ST3000DM001-1CH1 (scsi)
Disk /dev/sdb: 3001GB
Sector size (logical/physical): 512B/4096B
Partition Table: gpt

Number  Start   End     Size    File system     Name         Flags
 1      1049kB  2097kB  1049kB                  grub_core    bios_grub
 2      2097kB  203MB   201MB   ext2            boot_rescue
 3      203MB   204MB   1049kB  ext2            nv_data
 4      204MB   1815MB  1611MB  ext2            root_1       raid
 5      1815MB  3426MB  1611MB  ext2            root_2       raid
 6      3426MB  4499MB  1074MB  ext3            var          raid
 7      4499MB  4768MB  268MB   linux-swap(v1)  swap         raid
 8      4768MB  3001GB  2996GB                  user_data    raid


Model: Linux Software RAID Array (md)
Disk /dev/md4: 1611MB
Sector size (logical/physical): 512B/4096B
Partition Table: loop

Number  Start  End     Size    File system  Flags
 1      0.00B  1611MB  1611MB  ext2


Model: Linux Software RAID Array (md)
Disk /dev/md5: 1611MB
Sector size (logical/physical): 512B/4096B
Partition Table: loop

Number  Start  End     Size    File system  Flags
 1      0.00B  1611MB  1611MB  ext2


Model: Linux Software RAID Array (md)
Disk /dev/md6: 1074MB
Sector size (logical/physical): 512B/4096B
Partition Table: loop

Number  Start  End     Size    File system  Flags
 1      0.00B  1074MB  1074MB  ext3


Model: Linux Software RAID Array (md)
Disk /dev/md7: 268MB
Sector size (logical/physical): 512B/4096B
Partition Table: loop

Number  Start  End    Size   File system     Flags
 1      0.00B  268MB  268MB  linux-swap(v1)

The disk is organized as follows :

  • /dev/sdb4 and /dev/sdb5 are raid volumes that contains two copies of the same root partition. The other copy is used when doing a firmware update (ping-pong approach, see [1]). We mount them with:
mdadm --assemble --run /dev/md4 /dev/sdb4       # Done automatically on Ubuntu
mdadm --assemble --run /dev/md5 /dev/sdb5       # Done automatically on Ubuntu
mount /dev/md4 /mount/md4
mount /dev/md5 /mount/md5
  • /dev/sdb6 contains another raid partition, mounted as an additional layer over root partition above. It contains:
/dev/md6
    0/
        etc/
        root/
        var/
    1/
        etc/
        root/
        var/
    lost+found/
    tmp/

Configuration

First install

  • Network share available at smb://lacie-cloudbox/.
  • Cloudbox dashboard available at http://lacie-cloudbox.local.
  • setup smtp access (smtp.scarlet.be, port 25)
  • IP Configuration — fixed IP address 172.19.100.98
  • Enable the netbackup-share (port 873, encrypted port 22)
  • Create user beq06659, with private share.
MAC 00:d0:4b:96:34:32
IP 172.19.100.98
hostname lacie-cloudbox

Rooting

After firmware upgrade, we need to root it again.

  • Create telnetd.sh in a share root
cat > /smb/lacie-cloudbox/family/telnetd.sh <<- __END__
#!/bin/sh
/usr/sbin/telnetd -l /bin/sh
__END__
  • Get and build clunc:
# CLUNC Home: http://lacie-nas.org/doku.php?id=clunc
git clone http://git.lacie-nas.org/clunc.git
cd clunc
make
  • Run clunc, and reboot lacie:
./clunc -i 192.168.1.98
... Doesn't work anymore.
  • We mount the drive in another pc. There are 8 partitions, partitions /dev/sdb4 and above are raid.
sudo mdadm --assemble --run /dev/md0 /dev/sdb4
#mdadm: /dev/md0 has been started with 1 drive
sudo mount /dev/md0 /mnt/any
  • Enable SSH:
  • Edit /etc/initng/runlevel/default.runlevel.
  • Edit /etc/unicorn/unicorn_conf/unicorn.sharing.ssh.conf.
  • Edit /etc/ssh/ssd_config/.
  • Afterwards we unmount:
sudo umount /mnt/any
sudo mdadm --stop /dev/md0
  • ... we reboot. And it doesn't work! :-(.

We changed the following file:

  • Edit file /etc/initng/runlevel/default.runlevel:
-#sshd
+sshd
  • Edit file /etc/unicorn/unicorn_conf/unicorn.sharing.ssh.conf
-enabled: false
+enabled: true
  • Edit file /etc/ssh/sshd_config
-PermitRootLogin no
-AllowUsers netbackup
+PermitRootLogin yes
+AllowUsers root netbackup git
+Match User root
+Match User git
  • Add to file /etc/environment:
PATH=/opt/bin:/opt/sbin
  • Edit file /etc/hosts
  • Edit file /etc/lighttpd/lighttpd.conf
  • Edit file /etc/passwd (for user git)

Install Git

To enable git, we must:

  • Enable git logging in /etc/passwd.
  • Enable git ssh logging in /etc/ssh/sshd_config

Git needs a lot of memory when compressing its database (occurs at each fetch!). To limit this, run as root:

git config --system pack.windowmemory 128m
git config --system pack.packsizelimit 1g
git config --system pack.threads 1
git config --system core.bigFileThreshold 16m

Then as user git, in all relevant projects (typically those with big files)

cd ~/git/minecraft_nosaves.git
git config --local core.bigFileThreshold 10m

Install rsync

We do via user git, so:

  • Set /bin/bash as shell for user git in /etc/passwd
  • Set PermitUserEnvironment yes in /etc/ssh/sshd_config
  • Create file /shares/git/.ssh/environment to add rsync into path:
PATH=/opt/bin:/opt/sbin:/bin:/usr/bin:/sbin:/usr/sbin

Startup script

Create the file /opt/etc/init.d/S80config (chmod 755):

#! /bin/sh

# Enable /bin/bash as shell for user git to also allow rsync + git in ssh login
sed -ri '/^git/{s_/home_/shares/git_; s_/bin/false_/bin/bash_}' /etc/passwd
# sed -ri '/^git/{s_/home_/shares/git_; s_/bin/false_/opt/bin/git-shell_}' /etc/passwd

# Change permission for /shares/git or sshd will complain (or add 'StrictModes off to /etc/ssh/ssh_config)
chmod go-w /shares/git

# Configure sshd
if ! grep -q git /etc/ssh/sshd_config; then 
    # Enable ssh login with user git
    sed -ri '/^AllowUsers/s/$/ git/;$aMatch User git' /etc/ssh/sshd_config
    # Enable user environment for user git (in order to set PATH)
    sed -ri $'/^PermitRootLogin/{a PermitUserEnvironment yes\n}' /etc/ssh/sshd_config
    # Restart sshd
    ngc --restart sshd
fi

# Add git to path
if ! grep -q /opt/bin /etc/environment; then echo "PATH=/opt/bin:/opt/sbin" >> /etc/environment; fi

# Add extra addresses to /etc/hosts
if ! grep -q "###hosts.inc###" /etc/hosts; then cat /opt/etc/hosts.inc >> /etc/hosts; fi

Create the hosts include file /opt/etc/hosts.inc:

# ###hosts.inc### additional hosts for lacie-cloudbox
172.19.3.1      local.gateway            # Static
172.19.3.2      manticore                # Static?
172.19.100.1	gryphon
172.19.100.2	minimoy                  # DHCP Static        00:24:54:02:98:8d
172.19.100.3	unicorn                  # obsolete
172.19.100.4	beqleunxp1nb103          # obsolete
172.19.100.5	nxl67002ux               # obsolete
172.19.100.6	griffin                  # DHCP Static        68:b5:99:f2:4d:99
172.19.100.7	lambada                  # obsolete
172.19.100.8	reddragon
172.19.100.9    leviathan                # obsolete
172.19.100.10   nxl67063                 # obsolete
172.19.100.11   mandala                  # DHCP Static        00:1b:24:fa:a2:9b
172.19.100.12   nxl67002                 # obsolete
172.19.100.13   nxl67063ux               # obsolete
172.19.100.14   nxl67170ux               # DHCP Static        b8:ca:3a:cc:07:bf
172.19.100.15   pc-112-597               # DHCP Static        b8:ca:3a:93:91:27
172.19.100.16   zyratastic
172.19.100.17   graphicdaemon
172.19.100.18   xbmcbuntu                # DHCP Static        00:01:2e:4e:46:05 00:01:2e:4e:46:06
172.19.100.19   zavcxl0005               # DHCP Static        48:0f:cf:27:ad:ad
# 172.19.100.98	lacie-cloudbox           # Static
172.19.100.99	mnemosyne                # Static
172.19.100.102	minimoy-wlan             # DHCP Static        00:26:5e:37:83:ea
172.19.100.105	nxl67002ux-wlan          # DHCP Static
172.19.100.106	griffin-wlan             # DHCP Static        00:24:d7:7d:4e:94
172.19.100.107	apple-pi                 # Static
172.19.100.111  mandala-wlan             # DHCP Static        00:1d:92:13:d2:7d
172.19.100.114  nxl67170ux-wlan          # DHCP Static        84:3a:4b:22:0d:0c
172.19.100.116  zyratastic-wlan
172.19.100.118  xbmcbuntu-wlan           # DHCP Static        0c:d2:92:54:c1:53
172.19.100.119  zavcxl0005-wlan          # DHCP Static        ac:fd:ce:40:c5:31
172.19.100.120	galaxy

Install Unison

  • ipkg provides unison 2.27.57. It can be installed via ipkg install unison.
ipkg list | grep unison
# unison - 2.27.57-2 - A cross-platform file-synchronization tool.
  • To get a more recent version, see Unison to compile from source. v2.40.102 is the same as on Ubuntu Trusty 14.04.

Netconsole server

On lacie-cloudbox:

  • First install netcat:
ipkg install netcat
  • Then start the server:
mkdir -p /shares/beq06659/netconsole
nc -l -p 6666 -u >/shares/beq06659/netconsole/zavcxl0005-netconsole&

On the client, at reboot/wake-up:

NETCON_TGTHOST=lacie-cloudbox
NETCON_PORT=6666
echo "[------------] Redocking on $(date +"%Y-%m-%d %H:%M:%S")..." | nc -w 1 -u -p 6665 $NETCON_TGTHOST $NETCON_PORT
if [ -n "$NETCON_TGTHOST" ]; then 
    NETCON_TGTIP=$(getent hosts $NETCON_TGTHOST|awk '{print $1}')
    ping -c 1 $NETCON_TGTIP >/dev/null
    NETCON_TGTMAC=$(arp -n $NETCON_TGTIP|awk '/ether/{print $3}')
    echo "Setting up netconsole for location '$LOCATION' ($NETCON_TGTHOST,$NETCON_TGTIP/$NETCON_TGTMAC)"
    sudo rmmod netconsole 2> /dev/null
    sudo modprobe netconsole netconsole=@/,$NETCON_PORT@$NETCON_TGTIP/$NETCON_TGTMAC
else 
    echo "netconsole - Unknown location $LOCATION"
fi

Shares

family
  • Mount with
sudo mount //lacie-cloudbox/beq06659 /mnt/any -o noperm,iocharset=utf8,sec=ntlm,username=public,password=


beq06659
  • Mount with
sudo mount //lacie-cloudbox/beq06659 /mnt/any -o noperm,iocharset=utf8,sec=ntlm,username=beq06659

Backup

Backup Mnemosyne

We use unison.

BackupPC

Fix broken pipe errors

Backuppc requires lot of memory (rsync). To free as much as possible, I kill all python stuff (except ublock-daemon) and all media stuff:

pkill -SIGSTOP -f unicorn            # I stop it first so that it can't react
pkill -SIGKILL -f unicorn            # then kill it
sleep 1 && pkill -SIGSTOP -f unicorn # then stop it again because it respawns
pkill -SIGSTOP -f mt_daapd
pkill -SIGSTOP twonky
pkill -SIGSTOP php-cgi
pkill -SIGKILL -f mt-daapd           # media server stuff
pkill -SIGKILL twonky                # media server stuff
pkill -SIGKILL php-cgi               # php-cgi server launcher by lighttpd

Duplicity (Obsolete)

This section is OBSOLETE — I no longer use duplicity


  • Backup done with deja-dup, which use duplicity as back-end
  • Duplicity can't access samba share. Instead mount samba share locally, and use file:///smb/lacie-cloudbox/sharename instead. Note the triple slash.
  • View available backup:
duplicity collection-status file:///smb/lacie-cloudbox/beq06659
  • List file in current backup
duplicity list-current-files file:///smb/lacie-cloudbox/beq06659

Metadata information is stored in ~/.cache/duplicity. Attention, it seems this can grow quite large.

DHCP Server

  • install
# Install the service
ipkg install dhcp
mkdir /opt/var/run
# Edit the configuration
vi /opt/etc/dhcpd.conf
# Restart the service
/opt/etc/init.d/S56dhcpd
  • DHCP daemon configuration: /opt/etc/dhcpd.conf
  • Current leases are at /opt/etc/dhcpd.leases
  • Now I disabled the dhcpd, and will try again the one on vDSL+ router:
cp /opt/etc/dhcpd.conf /opt/etc/dhcpd.conf.bak     # Backup config if we want to reinstall
ipkg remove dhcp
  • DHCP server on the VDSL+ router is a huge pile of sh*te, so I enable it again:
ipkg install dhcp
cp /opt/etc/dhcpd.conf.bak /opt/etc/dhcpd.conf
/opt/etc/init.d/S56dhcpd

To do

  • Current backup only includes files that are readable by user beq06659. It should include all files.
  • Exclude some unnecessary folders. Organize these folders to ease future backup.
  • Backup / filesystem (at least /etc, /usr/local)
Backup solutions
  • deja-dup
  • Need one setup for each pc
  • No centralized management. Backup job could happen all at the same time. Same files between pc are stored multiple times.
  • Easy to setup
  • How to backup root filesystem?
  • duplicity
  • backuppc
  • How to install?
  • Will the NAS have enough memory?
  • bacula
  • Package readily available in OptWare
  • rsnapshot

See question on bacula/backuppc/rsnapshot at http://serverfault.com/questions/282228/bacula-vs-backuppc

Rooting NAS and install new app
  • TBC

Troubleshoot

sshd

  • See log at /var/log/messages
  • Check sshd config /etc/ssh/sshd_config:
  • Make sure user is authorized:
 AllowUsers root netbackup git
  • If user home is on /shares mount, then disable StrictModes
StrictModes no
  • Enable user environment
PermitUserEnvironment yes
then
env > ~/.ssh/environment
vi ~/.ssh/environment                # Remove useless variables
  • Edit /etc/passwd:
git:x:503:100:None:/shares/git:/opt/bin/git-shell
  • Restart sshd:
ngc --restart sshd

git

  • Check that environment is ok [2]
ssh git@lacie-cloudbox  "which git-upload-pack"

ntp

  • Box loses regularly time synchronization, which then affects backuppc
  • Workaround: log into https://lacie-cloudbox/, and reset ntp (uncheck / check ntp server).

slow performance, lot of swapping

The Lacie-Cloudbox only has 256MB physical memory, and some processes takes a lot of memory. When doing some unusual tasks, we can disable (stop) temporarily these processes, without apparently damage:

pkill -SIGSTOP twonky        # Usually consuming around 100MB+
pkill -f -SIGSTOP unicorn    # A python process running /usr/lib/unicorn/run.py (and consuming 100MB+)

To restart:

pkill -SIGCONT twonky
pkill -f -SIGCONT unicorn