Configuration Noekeon.org

From miki
Jump to navigation Jump to search

DNS zone

$TTL 3600
@	IN SOA dns105.ovh.net. tech.ovh.net. (2016090500 86400 3600 3600000 300)
                 IN NS     dns105.ovh.net.
                 IN NS     ns105.ovh.net.
                 IN MX 10  ober.noekeon.org.
                 IN MX 20  prime.immie.org.
                 IN A      91.134.133.203
                 IN TXT    "google-site-verification=PyZv_FR2vhqyGKz00ew_nnf_y_pprEgJt-G5SogqRyM"
             600 IN TXT    "v=spf1 a mx -all"
*                IN A      91.134.133.203
*                IN TXT    "v=spf1 -all"
ftp              IN A      62.182.63.46
imap             IN CNAME  ober.noekeon.org.
localhost        IN A      127.0.0.1
mail             IN CNAME  ober.noekeon.org.
ober             IN A      91.134.133.203
ober             IN TXT    "v=spf1 a -all"
pop              IN CNAME  ober.noekeon.org.
smtp             IN CNAME  ober.noekeon.org.

Original zone on OVH (with minimal entries): 

$TTL 3600
@          IN SOA    dns105.ovh.net. tech.ovh.net. (2016070301 86400 3600 3600000 300)
           IN NS     ns105.ovh.net.
           IN NS     dns105.ovh.net.
           IN MX 1   redirect.ovh.net.
           IN A      213.186.33.5
           IN TXT    "1|www.noekeon.org"
www        IN MX 1   redirect.ovh.net.
www        IN A      213.186.33.5
www        IN TXT    "3|welcome"
www        IN TXT    "l|fr"

PriorWeb Hints and Tips

  • MySQL
MySQL Host Name:
  • Use the generic name mysqlhost as MySQL host name in PHP scripts (as suggested by Priorweb's control panel)
$db_host="mysqlhost";
Create a new database:
Copy database:
  • First create the new database using Priorweb's Control Panel.
  • Second, go to Priorweb's phpMyAdmin page
  • Select database to backup
  • From menu above, chooose Operations, and then Copy Database To.
  • Uncheck CREATE DATABASE before copying, and check Add AUTO_INCREMENT value (no idea if that's necessary, but it is selected on the Export page), then click Go
Backup a database - Using phpMyAdmin:
  • Go to Priorweb's phpMyAdmin page
  • Select database to backup
  • From menu above, select EXPORT, select export in SQL format, export Structure and Data, choose a compression method (e.g. gzipped), click Go
Restoring a database - Using phpMyAdmin:
  • Go to Priorweb's phpMyAdmin page
  • Select database to restore
  • QUESTION: Do we have to delete the content of the database before importing???
Backup / Restore using command-line:
  • See procedure used to back-up this wiki here
  • Or see here

Miki's Kiwi Wiki

See dedicated page.

Miki's ToDoList

cd ~/kiwi.noekeon.org/miki/todo
unzip todolist.zip
mv todolist/* .
rmdir todolist
#Now remove garbage files...
find . -name _notes -exec rm -r {} \;            # Ignore errors on directory not found
find . -name __MACOSX -exec rm -r {} \;
  • Edit file dbconnection.php
$db_host="mysqlhost";
$db_name="mikido";
$username="miki";
$password="********";
username: miki
pwd: ********
database: mikido
DirectoryIndex todolist.php
  • Now you can visit https://kiwi.noekeon.org/miki/todo/. Create a user first before creating a task. Note that you must refresh the page (F5) first otherwise drop-down list is not refreshed and page generates an error.

Shell

  • In the following notes, ~ refers to original home directory /opt/www/daemenj/web.
  • bash shell resource file in ~/private/mip.bashrc:
#! /bin/bash

# Clever trick because we cannot write in /opt/www/daemenj/web...
# ... move HOME to private directory so that all commands fetch their configuration file there
export HOME=/opt/www/daemenj/web/private

# Ignore some controlling instructions
export HISTIGNORE="[   ]*:&:bg:fg:exit"

# Aliases
# #######

alias grep='grep --color'                     # show differences in colour

# Some shortcuts for different directory listings
eval `dircolors -b ~/.dircolors.cfg`
alias ls='ls -F --color=auto'					# classify files in colour
alias ll='ls -l'                                                # long list
alias la='ls -A'                                                # all but . and ..
alias l='ls -lA'                                                #
alias dir='ls --format=vertical'
alias vdir='ls --format=long'
  • Resource file for vim ~/private/.vimrc:
syntax enable
set bg=light
set number
nnoremap j h
nnoremap l j
"nnoremap k k
nnoremap m l
nnoremap h m
vnoremap j h
vnoremap l j
"vnoremap k k
vnoremap m l
vnoremap h m
  • Directory colors ~/private/.dircolors.cfg:
# Configuration file for dircolors, a utility to help you set the
# LS_COLORS environment variable used by GNU ls with the --color option.
# The keywords COLOR, OPTIONS, and EIGHTBIT (honored by the
# slackware version of dircolors) are recognized but ignored.
# Below, there should be one TERM entry for each termtype that is colorizable
TERM linux
TERM linux-c
TERM mach-color
TERM console
TERM con132x25
TERM con132x30
TERM con132x43
TERM con132x60
TERM con80x25
TERM con80x28
TERM con80x30
TERM con80x43
TERM con80x50
TERM con80x60
TERM cygwin
TERM dtterm
TERM mlterm
TERM putty
TERM xterm
TERM xterm-color
TERM xterm-debian
TERM rxvt
TERM rxvt-unicode
TERM screen
TERM screen-bce
TERM screen-w
TERM vt100
TERM Eterm
# Below are the color init strings for the basic file types. A color init
# string consists of one or more of the following numeric codes:
# Attribute codes:
# 00=none 01=bold 04=underscore 05=blink 07=reverse 08=concealed
# Text color codes:
# 30=black 31=red 32=green 33=yellow 34=blue 35=magenta 36=cyan 37=white
# Background color codes:
# 40=black 41=red 42=green 43=yellow 44=blue 45=magenta 46=cyan 47=white
NORMAL 00 # global default, although everything should be something.
FILE 00 # normal file
DIR 01;34 # directory
LINK 36 # symbolic link. (If you set this to 'target' instead of a
 # numerical value, the color is as for the file pointed to.)
FIFO 40;33 # pipe
SOCK 35 # socket
DOOR 35 # door
BLK 40;33;01 # block device driver
CHR 40;33;01 # character device driver
ORPHAN 40;31;01 # symlink to nonexistent file
# This is for files with execute permission:
EXEC 32
# List any file extensions like '.gz' or '.tar' that you would like ls
# to colorize below. Put the extension, a space, and the color init string.
# (and any comments you want to add after a '#')
# If you use DOS-style suffixes, you may want to uncomment the following:
#.cmd 32 # executables (bright green)
#.exe 32
#.com 32
#.btm 32
#.bat 32
.tar 01;31 # archives or compressed (bright red)
.tgz 01;31
.arj 01;31
.taz 01;31
.lzh 01;31
.zip 01;31
.z 01;31
.Z 01;31
.gz 01;31
.bz2 01;31
.deb 01;31
.rpm 01;31
.jar 01;31
# image formats
.jpg 35
.jpeg 35
.gif 35
.bmp 35
.pbm 35
.pgm 35
.ppm 35
.tga 35
.xbm 35
.xpm 35
.tif 35
.tiff 35
.png 35
.mov 35
.mpg 35
.mpeg 35
.avi 35
.fli 35
.gl 35
.dl 35
.xcf 35
.xwd 35
# audio formats
.flac 35
.mp3 35
.mpc 35
.ogg 35
.wav 35

Cron

This is the crontab file on noekeon.org. Install it with crontab -u daemenj crontab. 

# use /bin/bash to run commands, instead of the default /bin/sh
SHELL=/bin/bash
# mail any output to 'michael.peeters@noekeon.org', no matter whose crontab this is
MAILTO="michael.peeters@noekeon.org"
#
#
# m h dom mon dow    command   (dow=0|7 is sunday)
33 8 * * *            ~daemenj/private/changemonitor/monitor-all.sh >/dev/null 2>/dev/null

Server Install OVH

Spam filter

Candidates
  • SpamAssassin (SA)
The default standard.
Some installation guides [1].
  • DSpam
Excellent track record, but unfortunately no longer maintained.
Some installation guides [2], [3], [4]
  • Rspamd
A new system.

Mail server

master.cf
  • file master.cf tells which service are running
# ==========================================================================
# service type  private unpriv  chroot  wakeup  maxproc command + args
#               (yes)   (yes)   (yes)   (never) (100)
# ==========================================================================
smtp      inet  n       -       -       -       -       smtpd
  • smtp this is the usual smtp service on port 25. Used by other smtp server to send mail.
submission inet n       -       -       -       -       smtpd
 -o syslog_name=postfix/submission
 -o smtpd_tls_wrappermode=no
 -o smtpd_tls_security_level=encrypt
 -o smtpd_sasl_auth_enable=yes
 -o smtpd_recipient_restrictions=permit_mynetworks,permit_sasl_authenticated,reject
 -o milter_macro_daemon_name=ORIGINATING
 -o smtpd_sasl_type=dovecot
 -o smtpd_sasl_path=private/auth
  • submission this is the "secure" smtp service on port used by mail client to send new mail. Note that -o denotes configuration settings that override those in main.cf.
main.cf
  • Testing — add soft_bounce = yes when testing new configuration. This avoids rejecting mail permanently by changing REJECT into DEFER.
# Testing - Uncomment to DEFER instead of REJECT, hence not rejecting mail permanently when testing
soft_bounce = yes
  • Debugging — Add debug_peer_list to enable verbose logging.
# Debugging - Uncomment for verbose logging for connections from listed peers
debug_peer_list = 91.134.134.85
Setup SPF (Syntax, FAQ)
  • I add to the DNS zone:
noekeon.org.       IN  TXT  "v=spf1 a mx -all"
ober.noekeon.org.  IN  TXT  "v=spf1 a -all"
*.example.com.     IN  TXT  "v=spf1 -all"
First line says that all mails from @noekeon.org must come from one of the MX server of noekeon.org domain.
Second line says that mail from server with HELO identification "ober.noekeon.org" must be accepted only if from IP address of server "ober.noekeon.org".
Third rule says that no other sub-domain/server may send email.

Mail server - Rspamd spam filter

  • Install guide from rspam.com
  • Relevant Postfix documentation:
See example configuration.
Use soft_bounce = yes in main.cf to change a REJECT into DEFER, and hence to avoid losing mail when testing.
To Do
  • Use spamhaus black-list (see postfix example)
Rmilter
  • Guide [5]
  • Install with
   apt-get install rmilter
  • We keep the socket-based bind contrary to what guide suggests. So in main.cf:
# rmilter setup
smtpd_milters = unix:/var/run/rmilter/rmilter.sock
milter_default_action = accept
milter_protocol = 6
milter_mail_macros = i {mail_addr} {client_addr} {client_name} {auth_authen}
  • Check the socket. We see it is active and enabled (at boot so).
systemctl status rmilter.socket
# ● rmilter.socket - Another sendmail milter for different mail checks
#    Loaded: loaded (/lib/systemd/system/rmilter.socket; enabled)
#    Active: active (listening) since Mon 2016-07-18 18:49:22 CEST; 1 day 23h ago
#    Listen: /var/run/rmilter/rmilter.sock (Stream)
  • Check the service
systemctl status rmilter.service
# ● rmilter.service - Another sendmail milter for different mail checks
#    Loaded: loaded (/lib/systemd/system/rmilter.service; disabled)
#    Active: inactive (dead)
  • It is disabled, let's try to start it
systemctl start rmilter.service
  • It fails. We get
systemctl status rmilter.service -l
# ● rmilter.service - Another sendmail milter for different mail checks
#    Loaded: loaded (/lib/systemd/system/rmilter.service; disabled)
#    Active: failed (Result: exit-code) since Wed 2016-07-20 17:56:05 CEST; 1min 31s ago
#   Process: 24168 ExecStart=/usr/sbin/rmilter -c /etc/rmilter.conf -n (code=exited, status=233/RUNTIME_DIRECTORY)
#  Main PID: 24168 (code=exited, status=233/RUNTIME_DIRECTORY)

# Jul 20 17:56:05 ober.noekeon.org systemd[24168]: Failed at step RUNTIME_DIRECTORY spawning /usr/sbin/rmilter: File exists
# Jul 20 17:56:05 ober.noekeon.org systemd[1]: rmilter.service: main process exited, code=exited, status=233/RUNTIME_DIRECTORY
# Jul 20 17:56:05 ober.noekeon.org systemd[1]: Unit rmilter.service entered failed state.
  • Let's debug. Where is this runtime directory?
dlocate -L rmilter|xargs grep -id skip runtimedirectory
# /lib/systemd/system/rmilter.service:RuntimeDirectory=rmilter
# /lib/systemd/system/rmilter.service:RuntimeDirectoryMode=0755
  • Look for a fix on internet: Here [6], it suggests to add to /lib/systemd/system/rmilter.service:
 [Service]
 ExecStart=/usr/sbin/rmilter -c /etc/rmilter.conf -n
 ExecReload=/bin/kill -USR1 $MAINPID
 User=_rmilter
 RuntimeDirectory=rmilter
 RuntimeDirectoryMode=0755
+Restart=always
  • So let's try again
systemctl start rmilter.service
# Warning: Unit file of rmilter.service changed on disk, 'systemctl daemon-reload' recommended.
systemctl daemon-reload
systemctl status rmilter.service
# ● rmilter.service - Another sendmail milter for different mail checks
#    Loaded: loaded (/lib/systemd/system/rmilter.service; disabled)
#    Active: active (running) since Wed 2016-07-20 18:21:45 CEST; 53s ago
#  Main PID: 24895 (rmilter)
#    CGroup: /system.slice/rmilter.service
#            └─24895 /usr/sbin/rmilter -c /etc/rmilter.conf -n
# 
# Jul 20 18:21:45 ober.noekeon.org rmilter[24895]: main: starting rmilter version 1.8.6, listen on fd:3
# Jul 20 18:21:45 ober.noekeon.org rmilter[24895]: reload_thread: starting...
Rspamd
  • Install

apt-get install rspamd

  • ... it already fails
systemctl status rspamd.socket -l
# ● rspamd.socket - rapid spam filtering system
#    Loaded: loaded (/lib/systemd/system/rspamd.socket; enabled)
#    Active: failed (Result: resources)
#    Listen: [::]:11333 (Stream)
#            [::1]:11334 (Stream)
# 
# Jul 18 18:49:22 ober.noekeon.org systemd[1]: rspamd.socket failed to listen on sockets: Cannot assign requested address
# Jul 18 18:49:22 ober.noekeon.org systemd[1]: Failed to listen on rapid spam filtering system.
# Jul 18 18:49:22 ober.noekeon.org systemd[1]: Unit rspamd.socket entered failed state.
  • Actually problem comes from /etc/sysctl.d/local.conf, where we disabled ipv6. If we enable it back:
 # Disable IPv6
-net.ipv6.conf.all.disable_ipv6 = 1
-net.ipv6.conf.default.disable_ipv6 = 1
-net.ipv6.conf.lo.disable_ipv6 = 1
-net.ipv6.conf.eth0.disable_ipv6 = 1
+net.ipv6.conf.all.disable_ipv6 = 0
+net.ipv6.conf.default.disable_ipv6 = 0
+net.ipv6.conf.lo.disable_ipv6 = 0
+net.ipv6.conf.eth0.disable_ipv6 = 0

Now it works 

systemctl start rspamd.socket

Alternatively, if we want to keep ipv6 disabled, we need to edit systemd file /lib/systemd/system/rspamd.socket as follows 

--- rspamd.socket       2016-06-20 13:55:00.000000000 +0200
+++ rspamd.socket.old   2016-07-20 19:44:18.165848297 +0200
@@ -2,9 +2,10 @@
 Description=rapid spam filtering system
 
 [Socket]
-ListenStream=11333
-ListenStream=[::1]:11334
-BindIPv6Only=both
+BindIPv6Only=ipv6-only
+ListenStream=127.0.0.1:11333
+ListenStream=127.0.0.1:11334
+KeepAlive=true
 
 [Install]
 WantedBy=sockets.target

and reload rspamd: 

systemctl daemon-reload
systemctl start rspamd.socket
systemctl status rspamd.socket -l
# ● rspamd.socket - rapid spam filtering system
  • Note that rspamd.service is not started by default. It is started on demand. To start it:
rspamc -h ::1:11334 stat

Note that the following fails 

rspamc stat
rspamc -h 127.0.0.1:11334 stat

This is because rspamd only listen on ipv6 interface: 

netstat -lpn|grep 1133
# tcp6       0      0 :::11333                :::*                    LISTEN      1/init          
# tcp6       0      0 ::1:11334               :::*                    LISTEN      1/init          
cat /lib/systemd/system/rspamd.socket
#[Unit]
#Description=rapid spam filtering system
#
#[Socket]
#ListenStream=11333
#ListenStream=[::1]:11334
#BindIPv6Only=both
#
#[Install]
#WantedBy=sockets.target

To install rspamd webgui on https, we use apache2 as reverse proxy ([7],[8],[9])

Note
 [Socket]
-ListenStream=11333
-ListenStream=[::1]:11334
+ListenStream=127.0.0.1:11333
+ListenStream=0.0.0.0:11334

 <IfModule mod_ssl.c>
   <VirtualHost *:443>

+    # We don't need forward proxy, so let's disable it
+    ProxyRequests Off
+
+    <Location /rspamd/>
+      ProxyPass http://localhost:11334/
+      ProxyPassReverse http://localhost:11334/
+      Order deny,allow
+      Allow from all
+    </Location>

   </VirtualHost>
 </IfModule>

Note:

  • Module proxy_http is necessary to fix error
... AH01144: No protocol handler was valid for the URL /rspamd/. If you are using a DSO version of mod_proxy, make sure the proxy submodules are included in the configuration using LoadModule.
  • Mind the trailing slash in <Location /rspamd/>, to match the trailing one in ProxyPass and ProxyPassReverse address. A mismatch fails with an error (NULL) message.

Install the necessary modules and restart apache: 

a2enmod proxy
a2enmod proxy_http
a2enmod headers
service apache2 restart
  • To add HTTPS password:
 <IfModule mod_ssl.c>
   <VirtualHost *:443>

     <Location /rspamd/>
+      AuthType Basic
+      AuthName "mail.noekeon.org Rspamd webgui"
+      AuthUserFile /etc/apache2/mail.noekeon.org_rspamd.ssl.passwd
+      Require valid-user
    </Location>

   </VirtualHost>
 </IfModule>
Create the password, and don't forget to change ownership:
htpasswd -c -s /etc/apache2/mail.noekeon.org_rspamd.ssl.passwd rspamd
chown www-data:www-data /etc/apache2/*.passwd
chmod 644 /etc/apache2/*.passwd

Final configuration, everything works:

  • /lib/systemd/system/rmilter.socket
[Unit]
Description=Another sendmail milter for different mail checks

[Socket]
ListenStream=127.0.0.1:9900
SocketUser=_rmilter
SocketGroup=adm
SocketMode=660

[Install]
WantedBy=sockets.target
  • /lib/systemd/system/rspamd.socket
[Unit]
Description=rapid spam filtering system

[Socket]
ListenStream=127.0.0.1:11333
ListenStream=127.0.0.1:11334
BindIPv6Only=both

[Install]
WantedBy=sockets.target
  • /lib/systemd/system/rspamd.service
[Unit]
Description=rapid spam filtering system
After=nss-lookup.target

[Service]
ExecStart=/usr/bin/rspamd -c /etc/rspamd/rspamd.conf -f
User=_rspamd
RuntimeDirectory=rspamd
RuntimeDirectoryMode=0755
Restart=always

[Install]
WantedBy=multi-user.target
Better configuration
No need to edit /lib/systemd files. See issue #748 on GitHub.
Debugging
# testing new rmilter/rspam config. Edit systemd settings, then
# If changing /lib/systemd files, must do:
systemctl daemon-reload
# then
systemctl stop rspamd.socket rspamd.service rmilter.socket rmilter.service
systemctl start rmilter.socket rspamd.socket

# To disable ipv6:
echo '1' > /proc/sys/net/ipv6/conf/all/disable_ipv6
echo '1' > /proc/sys/net/ipv6/conf/default/disable_ipv6
echo '1' > /proc/sys/net/ipv6/conf/lo/disable_ipv6
echo '1' > /proc/sys/net/ipv6/conf/eth0/disable_ipv6
/etc/init.d/networking restart
# ... or via /etc/sysctl.d/local.conf
Logging

We have the choice between

  • console logging (sent to /var/log/syslog)
But this generates lot of spam for logcheck when log is at info level.
  • file logging (to /var/log/rspamd/rspamd.log)
But we lose the message timestamp.

To configure file logging, edit file /etc/rspamd.conf: 

@@ -24,7 +24,8 @@ options {
 }
 
 logging {
-    type = "console";
+    type = "file";
+    filename = "$LOGDIR/rspamd.log";
     systemd = true;
     .include "$CONFDIR/logging.inc"
     .include(try=true; priority=1,duplicate=merge) "$LOCAL_CONFDIR/local.d/logging.inc"

Currently, I go for logging to console, with a filter all rules in logcheck (basically ignore all messages). This may change pending update on GitHub issue #1496.

Mail server - SRS

Reference:

We must enable SRS to allow forwarding mail from domains that enabled SPF: 

apt-get install postsrsd

We check that it listens to the default ports (10001 and 10002) and uses the correct domain: 

systemctl status postsrsd
# ● postsrsd.service - SRS lookup table for Postfix
#    Loaded: loaded (/lib/systemd/system/postsrsd.service; enabled)
#    Active: active (running) since Mon 2016-09-05 11:53:27 CEST; 2h 11min ago
#  Main PID: 11273 (postsrsd)
#    CGroup: /system.slice/postsrsd.service
#            └─11273 /usr/sbin/postsrsd -f10001 -r10002 -s/etc/postsrsd.secret -X -unobody -c/var/lib/postsrsd -dnoekeon.org
ss -tupan|grep 1000[12]
# tcp    LISTEN     0      10             127.0.0.1:10001                 *:*      users:(("postsrsd",pid=11273,fd=4))
# tcp    LISTEN     0      10             127.0.0.1:10002                 *:*      users:(("postsrsd",pid=11273,fd=5))

Tell postfix to use SRS and reload: 

postconf -e "sender_canonical_maps = tcp:127.0.0.1:10001"
postconf -e "sender_canonical_classes = envelope_sender"
postconf -e "recipient_canonical_maps = tcp:127.0.0.1:10002"
postconf -e "recipient_canonical_classes = envelope_recipient,header_recipient"
postfix reload

Mail server - Gmail Postmaster tools

Account registered to night.moore.nm@gmail.com... but never got any data because our volume is probably too small. I can still use this account to test mail forwarding though and see how gmail sees our mail server.

Miscellaneous

  • Postfix complains that it is setuid root:
Jul 18 05:22:30 ober postfix/sendmail[27605]: warning: the Postfix sendmail command has set-uid root file permissions
Jul 18 05:22:30 ober postfix/sendmail[27605]: warning: or the command is run from a set-uid root process
Jul 18 05:22:30 ober postfix/sendmail[27605]: warning: the Postfix sendmail command must be installed without set-uid root file permissions
It is not necessary to run procmail as setuid root with postfix [10], so we do:
ls -l /usr/bin/procmail
# -rwsr-sr-x 1 root mail 88K Feb 11  2015 /usr/bin/procmail
chmod u-s /usr/bin/procmail
ls -l /usr/bin/procmail
# -rwxr-sr-x 1 root mail 88K Feb 11  2015 /usr/bin/procmail

Troubleshooting

Google GMail - 421-4.7.0 black listing

See Mail server.

Retrieved from "https://miki.immie.org/mediawiki/index.php?title=Configuration_Noekeon.org&oldid=9374"