Dumbledore: Difference between revisions

From miki
Jump to navigation Jump to search
 
(One intermediate revision by the same user not shown)
Line 51: Line 51:
We will use this pi4 as our main borg server.
We will use this pi4 as our main borg server.


* Add a user <code>borg</code>.
* Remove <code>pi</code> from <code>sudo</code> group.
* Remove <code>pi</code> from <code>sudo</code> group.
* Disable no password sudo rule ({{file|/etc/sudoers.d/010_pi-nopasswd}}).
* Disable no password sudo rule ({{file|/etc/sudoers.d/010_pi-nopasswd}}).
* Use strong password for user pi, borg and root.
* Install borg:
* Install borg:
<source lang="bash">
<source lang="bash">
Line 62: Line 64:
sudo pip3 install borgbackup
sudo pip3 install borgbackup
</source>
</source>
* Add to {{file|/etc/ssh/sshd_config}} [https://borgbackup.readthedocs.io/en/stable/usage/serve.html]:
* Add a user <code>borg</code>.
ClientAliveInterval 10
ClientAliveCountMax 30

;When using Wolverine NAS as storage:
* Mount borg share as separate autofs host ({{file|/smb/borg/borg}}), restricted to borg user:
* Mount borg share as separate autofs host ({{file|/smb/borg/borg}}), restricted to borg user:
<source lang="bash">
<source lang="bash">
Line 68: Line 74:
forcegid,file_mode=660,dir_mode=770,credentials=/etc/auto.master.d/smb.borg.borg.cred ://wolverine/homes/borg
forcegid,file_mode=660,dir_mode=770,credentials=/etc/auto.master.d/smb.borg.borg.cred ://wolverine/homes/borg
</source>
</source>

;Now, using USB as storage:
* Mount SSD in {{file|/etc/fstab}}:
<source lang="bash">
UUID=0a6dab24-a62c-42d3-b032-315c33f1a2ca /mnt/tera ext4 defaults,noatime 0 1
</source>
* Link it to {{file|/home/borg/repo}}
<source lang="bash">
sudo ln -sd /mnt/tera/borg /home/borg/repo
</source>
* Enable TRIM:
<source lang="bash">
sudo systemctl enable fstrim.timer
sudo systemctl start fstrim.timer
</source>
* Example of {{file|authorized_keys}}:
command="/usr/local/bin/borg serve --restrict-to-path /home/borg/repo/",restrict ssh-ed25519 AAA... xyz@abc


== Issues ==
== Issues ==
* When booting headless, the session is not the same as the one we get when we boot with hdmi connected.
* When booting headless, the session is not the same as the one we get when we boot with hdmi connected.
=== Samsung SSD 860 EVO through USB 3 ===
=== Samsung SSD 860 EVO through USB 3 ===
;USB3 vs USB2
* Can't get it to work with Advance USB 3 case (chipset ASMedia AS2105).
* Can't get it to work with Advance USB 3 case (chipset ASMedia AS2105).
* Work currently on USB 2 enclosure.
:* Work currently on USB 2 enclosure. Current perf on that enclosure
<source lang="bash">
hdparm -Tt /dev/sda
# /dev/sda:
# Timing cached reads: 1560 MB in 2.00 seconds = 780.65 MB/sec
# Timing buffered disk reads: 102 MB in 3.00 seconds = 33.96 MB/sec
</source>
* Trying new USB3 enclosure
:* It run fast on HP PC but slow on Pi4:
<source lang="bash">
# On HP PC
sudo hdparm -Tt /dev/sda
# /dev/sda:
# Timing cached reads: 11584 MB in 2.00 seconds = 5805.22 MB/sec
# Timing buffered disk reads: 1306 MB in 3.00 seconds = 434.93 MB/sec

# On Pi4:
hdparm -Tt /dev/sda
# /dev/sda:
# Timing cached reads: 1570 MB in 2.00 seconds = 785.51 MB/sec
# Timing buffered disk reads: 22 MB in 30.72 seconds = 733.25 kB/sec
</source>
:* See https://www.raspberrypi.org/forums/viewtopic.php?t=245931
:* Getting some bad messages in kernel log:
Dec 8 16:25:04 dumbledore kernel: [29345.602374] sd 0:0:0:0: [sda] tag#27 uas_eh_abort_handler 0 uas-tag 2 inflight: CMD IN
Dec 8 16:25:04 dumbledore kernel: [29345.602391] sd 0:0:0:0: [sda] tag#27 CDB: opcode=0x28 28 00 00 00 b0 00 00 04 00 00
Dec 8 16:25:04 dumbledore kernel: [29345.602658] sd 0:0:0:0: [sda] tag#24 uas_eh_abort_handler 0 uas-tag 1 inflight: CMD IN
Dec 8 16:25:04 dumbledore kernel: [29345.602670] sd 0:0:0:0: [sda] tag#24 CDB: opcode=0x28 28 00 00 00 b4 00 00 04 00 00
Dec 8 16:25:04 dumbledore kernel: [29345.642397] scsi host0: uas_eh_device_reset_handler start
Dec 8 16:25:04 dumbledore kernel: [29345.803363] usb 2-2: reset SuperSpeed Gen 1 USB device number 2 using xhci_hcd
Dec 8 16:25:04 dumbledore kernel: [29345.838322] scsi host0: uas_eh_device_reset_handler success
:* Adding <code>usb-storage.quirks=152d:1337:u </code> to {{file|/boot/cmdline.txt}}
:* Perf with the quirks:
<source lang="bash">
hdparm -Tt /dev/sda
# /dev/sda:
# Timing cached reads: 1606 MB in 2.00 seconds = 803.03 MB/sec
# Timing buffered disk reads: 814 MB in 3.00 seconds = 271.05 MB/sec
</source>

;TRIM
* Refer: https://wiki.debian.org/SSDOptimization
* Refer: https://wiki.debian.org/SSDOptimization
* Refer: http://blog.neutrino.es/2013/howto-properly-activate-trim-for-your-ssd-on-linux-fstrim-lvm-and-dmcrypt/
* Refer: http://blog.neutrino.es/2013/howto-properly-activate-trim-for-your-ssd-on-linux-fstrim-lvm-and-dmcrypt/

Latest revision as of 21:51, 8 December 2020

Dumbledore is the Raspberry Pi 4 that we use as backup server. See also Voldemort.

Information

  • CPU is a armv8, but as raspbian is 32-bit, it is reported as a 32-bit armv7l [1].
We need usual Debian ARM to get 64-bit apps running. But this may also not be as optimized as Raspbian.
lscpu
# Architecture:        armv7l
# Byte Order:          Little Endian
# CPU(s):              4
  • Ethernet: IP 192.168.1.94, link/ether dc:a6:32:69:da:4e
  • WiFi: IP DHCP, link/ether dc:a6:32:69:da:4f

Configuration

  • Install Raspbian Buster 32-bit (2020-08-20-raspios-buster-armhf-full.zip) using imager_1.4_amd64.deb.
  • Install NoMachine server (armv7 because 32-bit OS, also 64-bit is reported as not being optimized).
  • Enable hardware acceleration in server settings.
  • Add to /etc/NX/server/localhost/server.cfg [2]:
AvailableSessionTypes unix-remote,unix-console,unix-default,unix-application,shadow,unix-xsession-default,unix-gnome,unix-xdm,windows,physical-desktop
  • Restart the server /etc/NX/nxserver --restart.
  • Use static IP by editing /etc/dhcpcd.conf [3]:
+interface wlan0
+static ip_address=192.168.1.94/24
+#static ip6_address=fd51:42f8:caae:d92e::ff/64
+static routers=192.168.1.1
+static domain_name_servers=192.168.1.1 8.8.8.8 fd51:42f8:caae:d92e::1
  • Enable ssh server. The simplest is to [4]:
# Either touch a file and reboot the device
sudo touch /boot/ssh
sudo reboot
# Or enable through systemd
sudo systemctl enable ssh
sudo systemctl start ssh
  • Add to ~/.config/autostart/set_belgian_kbd.desktop:
[Desktop Entry]
Type=Application
Name=Set Belgian autostart
Comment=Set keyboard to Belgian layout
NoDisplay=true
Exec=sh -c '/usr/bin/setxkbmap be'
This should fix the wrong keyboard layout issue.

Borg server

We will use this pi4 as our main borg server.

  • Add a user borg.
  • Remove pi from sudo group.
  • Disable no password sudo rule (/etc/sudoers.d/010_pi-nopasswd).
  • Use strong password for user pi, borg and root.
  • Install borg:
sudo apt install cython cython3
sudo apt-get install python3 python3-dev python3-pip python-virtualenv \
    libssl-dev openssl \
    libacl1-dev libacl1 \
    build-essential
sudo pip3 install borgbackup
  • Add to /etc/ssh/sshd_config [5]:
  ClientAliveInterval 10
  ClientAliveCountMax 30
When using Wolverine NAS as storage
  • Mount borg share as separate autofs host (/smb/borg/borg), restricted to borg user:
borg       -fstype=cifs,vers=1.0,_netdev,perm,iocharset=utf8,sec=ntlm,uid=borg,forceuid,gid=borg, \
    forcegid,file_mode=660,dir_mode=770,credentials=/etc/auto.master.d/smb.borg.borg.cred      ://wolverine/homes/borg
Now, using USB as storage
  • Mount SSD in /etc/fstab:
UUID=0a6dab24-a62c-42d3-b032-315c33f1a2ca   /mnt/tera	ext4	defaults,noatime	0	1
  • Link it to /home/borg/repo
sudo ln -sd /mnt/tera/borg /home/borg/repo
  • Enable TRIM:
sudo systemctl enable fstrim.timer
sudo systemctl start fstrim.timer
  • Example of authorized_keys:
command="/usr/local/bin/borg serve --restrict-to-path /home/borg/repo/",restrict ssh-ed25519 AAA... xyz@abc

Issues

  • When booting headless, the session is not the same as the one we get when we boot with hdmi connected.

Samsung SSD 860 EVO through USB 3

USB3 vs USB2
  • Can't get it to work with Advance USB 3 case (chipset ASMedia AS2105).
  • Work currently on USB 2 enclosure. Current perf on that enclosure
hdparm -Tt /dev/sda
# /dev/sda:
#  Timing cached reads:   1560 MB in  2.00 seconds = 780.65 MB/sec
#  Timing buffered disk reads: 102 MB in  3.00 seconds =  33.96 MB/sec
  • Trying new USB3 enclosure
  • It run fast on HP PC but slow on Pi4:
# On HP PC
sudo hdparm -Tt /dev/sda
# /dev/sda:
#  Timing cached reads:   11584 MB in  2.00 seconds = 5805.22 MB/sec
#  Timing buffered disk reads: 1306 MB in  3.00 seconds = 434.93 MB/sec

# On Pi4:
hdparm -Tt /dev/sda
# /dev/sda:
#  Timing cached reads:   1570 MB in  2.00 seconds = 785.51 MB/sec
#  Timing buffered disk reads:  22 MB in 30.72 seconds = 733.25 kB/sec
   Dec  8 16:25:04 dumbledore kernel: [29345.602374] sd 0:0:0:0: [sda] tag#27 uas_eh_abort_handler 0 uas-tag 2 inflight: CMD IN 
   Dec  8 16:25:04 dumbledore kernel: [29345.602391] sd 0:0:0:0: [sda] tag#27 CDB: opcode=0x28 28 00 00 00 b0 00 00 04 00 00
   Dec  8 16:25:04 dumbledore kernel: [29345.602658] sd 0:0:0:0: [sda] tag#24 uas_eh_abort_handler 0 uas-tag 1 inflight: CMD IN 
   Dec  8 16:25:04 dumbledore kernel: [29345.602670] sd 0:0:0:0: [sda] tag#24 CDB: opcode=0x28 28 00 00 00 b4 00 00 04 00 00
   Dec  8 16:25:04 dumbledore kernel: [29345.642397] scsi host0: uas_eh_device_reset_handler start
   Dec  8 16:25:04 dumbledore kernel: [29345.803363] usb 2-2: reset SuperSpeed Gen 1 USB device number 2 using xhci_hcd
   Dec  8 16:25:04 dumbledore kernel: [29345.838322] scsi host0: uas_eh_device_reset_handler success
  • Adding usb-storage.quirks=152d:1337:u to /boot/cmdline.txt
  • Perf with the quirks:
hdparm -Tt /dev/sda
# /dev/sda:
#  Timing cached reads:   1606 MB in  2.00 seconds = 803.03 MB/sec
#  Timing buffered disk reads: 814 MB in  3.00 seconds = 271.05 MB/sec
TRIM
  • There are problem around NCQ (queued TRIM)...
  • Samsung SSD 840 and Samsung SSD 850 are blacklisted in libata-core.c.
  • However, there are patches telling that Queued TRIM is fully supported on Linux... or not [6], [7], [8], [9]
  • Enabled TRIM with
sudo systemctl enable fstrim.timer
sudo systemctl start fstrim.timer