Immie.org: Difference between revisions

From miki
Jump to navigation Jump to search
 
(29 intermediate revisions by the same user not shown)
Line 135: Line 135:


== Server Configuration ==
== Server Configuration ==
=== SSL certificate ===
=== Administration ===
{{note|Move this to some common page}}.
* Check no old packages:
<source lang="bash">
dpkg -l | grep -E "^rc" # Must be empty
</source>

* Check systemctl for failed services:
<source lang="bash">
systemctl
</source>

=== Upgrade from Jessie to Stretch ===
* See https://www.debian.org/releases/stretch/amd64/release-notes.en.txt
* Change {{file|/etc/apt/sources.list}} to:
<source lang="bash">
# cat /etc/apt/sources.list
# Stable = Stretch
# deb http://ftp.debian.org/debian/ stretch main
# deb http://security.debian.org/ stretch/updates main
deb http://archive.debian.org/debian/ stretch main
deb http://archive.debian.org/debian-security/ stretch/updates main

# # Testing
# deb http://ftp.debian.org/debian/ testing main
# # deb-src http://ftp.debian.org/debian/ testing main
# deb http://security.debian.org/ testing-security/updates main

# Unstable / Sid
# deb http://ftp.debian.org/debian/ sid main

# Backport
deb http://archive.debian.org/debian stretch-backports main
</source>
* Remove all pins from {{file|/etc/apt/preferences}}
* Disable all other sources from {{file|/etc/apt/sources.list.d}}

=== Guides ===
Guides I followed to install the server:
* http://www.pontikis.net/blog/debian-jessie-web-server-setup

I added '''testing''' and '''unstable''' repositories.

{{file|/etc/apt/sources.list}}:
<source lang=bash>
# Stable
deb http://ftp.debian.org/debian/ jessie main
deb http://security.debian.org/ jessie/updates main

# Testing
deb http://ftp.debian.org/debian/ testing main
deb http://security.debian.org/ testing/updates main

# Unstable / Sid
deb http://ftp.debian.org/debian/ sid main

# Backport
deb http://ftp.debian.org/debian jessie-backports main
</source>

{{file|/etc/apt/preferences}}:
<source lang=bash>
# cat /etc/apt/preferences
Package: *
Pin: release a=stable
Pin-Priority: 500

Package: *
Pin: release a=jessie-backports
Pin-Priority: 475

Package: *
Pin: release a=testing
Pin-Priority: 450

Package: *
Pin: release a=unstable
Pin-Priority: 400
</source>

Upgraded some packages from testing/unstable: [https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=819094]
<source lang=bash>
apt install debian-goodies=0.66 # Fix mysqld false positive in checkrestart
</source>

=== SSL - certbot (Let's Encrypt) ===
<source lang=bash>
<source lang=bash>
mkdir ca
mkdir ca
Line 194: Line 279:
;New - Using [https://letsencrypt.org/ Let's Encrypt]
;New - Using [https://letsencrypt.org/ Let's Encrypt]
* Follow [https://certbot.eff.org/#debianjessie-apache certbot] guide.
* Follow [https://certbot.eff.org/#debianjessie-apache certbot] guide.
* '''New''' &mdash; We install only version from jessie-backports:
<source lang=bash>
apt install python-certbot-apache=0.10.2-1~bpo8+1
# sudo apt-get install python-certbot-apache -t jessie-backports
certbot --authenticator=webroot --installer=apache
</source>
* '''{{red|TODO}}''' &mdash; See IMPORTANT NOTES below:
* '''{{red|TODO}}''' &mdash; See IMPORTANT NOTES below:
<source lang=text>
<source lang=text>
Line 218: Line 309:
SSLCertificateFile /etc/letsencrypt/live/www.immie.org/fullchain.pem
SSLCertificateFile /etc/letsencrypt/live/www.immie.org/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/www.immie.org/privkey.pem
SSLCertificateKeyFile /etc/letsencrypt/live/www.immie.org/privkey.pem

; Update - Add some issue with certbot
<pre>
Jun 12 00:38:12 prime certbot[9567]: File:
Jun 12 00:38:12 prime certbot[9567]: - Could not be found to be deleted /var/lib/letsencrypt/icJA1m-EoE3Gsf6HJlITR4GCBb_9wvlyYV4faqJ_aVk.pem - LE probably shut down unexpectedly
Jun 12 00:38:12 prime certbot[9567]: File:
Jun 12 00:38:12 prime certbot[9567]: - Could not be found to be deleted /var/lib/letsencrypt/icJA1m-EoE3Gsf6HJlITR4GCBb_9wvlyYV4faqJ_aVk.crt - LE probably shut down unexpectedly
Jun 12 00:38:13 prime certbot[9567]: Attempting to renew cert from /etc/letsencrypt/renewal/www.immie.org.conf produced an unexpected error: 'module' object has no attribute 'rand'. Skipping.
</pre>
* Uninstalled certbot completely, and removed all files ({{file|/etc/encrypt}}, {{file|/var/...}}).
* Reinstall with standalone server (did not try ''webroot'' as done before, maybe that would work) [https://github.com/certbot/certbot/issues/5405].
<source lang=bash>
certbot --authenticator standalone --installer apache --pre-hook "systemctl stop apache2" --post-hook "systemctl start apache2"
</source>

; Update - Yet more issue (on immie.org, but not on noekeon.org).
* See issue [https://github.com/certbot/certbot/issues/5111 #5111]. I get error <code>'module' object has no attribute 'rand'</code>.
* As a fix, revert back to <code>apache</code> plugin renewal, and removed pre-/post-hook. File
<pre>
[renewalparams]
authenticator = apache
account = 157b05fb0f012e07716c9dbdc3a9f923
installer = apache
</pre>
* Then temporarily fix {{file|/usr/lib/python2.7/dist-packages/acme/crypto_util.py}}. Changes:
<source lang=diff>
+import os

- cert.set_serial_number(int(binascii.hexlify(OpenSSL.rand.bytes(16)), 16))
+ cert.set_serial_number(int(binascii.hexlify(os.urandom(16)), 16))
</source>
* In fact I noticed that immie.org had a newer version of {{deb|python-openssl}}:
<source lang=bash>
ii python-openssl 16.0.0-1~bpo8+1 # On noekeon.org
ii python-openssl 18.0.0-1 # On immie.org
</source>
* So let's downgrade, and hope it will fix the problem:
<source lang=bash>
apt install python-openssl=16.0.0-1~bpo8+1
</source>
* ... still not fixed. We need to downgrade more packages (list obtained by comparing <code>dpkg</code> output on reference server):
<source lang="bash">
sudo apt install python-cffi-backend=1.9.1-2~bpo8+1 python-cryptography=1.7.1-3~bpo8+1 python-idna=2.0-3~bpo8+1 \
python-mock=2.0.0-3~bpo8+1 python-pkg-resources=33.1.1-1~bpo8+1 python-pyasn1=0.1.9-1~bpo8+1 \
python-setuptools=33.1.1-1~bpo8+1 python-six=1.10.0-3~bpo8+1 python-pbr=1.8.0-4.1~bpo8+1
</source>

; Update - 2019 Jan - tls-sni-01 challenge no longer supported
* Received mail for <code>immie.org</code> telling tls-sni-01 is no longer supported, and that I must switch to either <code>http-01</code> or <code>dns-01</code>. Checking with <code>certbot renew --dry-run</code> indeed confirm that I'm using tls-sni-01 as challenge method.
* Tried many things, but none seems to work. In fact, my certbot is too old
<source lang="bash">
certbot --version
# certbot 0.10.2 <--- too old, we need at least 0.28
</source>
* On Debian Jessie, the recommended install now is to use <code>certbot-auto</code> [https://certbot.eff.org/lets-encrypt/debianjessie-apache.html].
* First remove old certbot:
<source lang="bash">
apt remove certbot
</source>
* Install <code>certbot-auto</code>:
<source lang="bash">
cd /usr/local/bin
wget https://dl.eff.org/certbot-auto
chmod a+x certbot-auto
# Let's try
/usr/local/bin/certbot-auto renew --dry-run
</source>
:It works, so we change the cronjob task as follows:
<source lang="bash">
0 */12 * * * root test -x /usr/local/bin/certbot-auto -a \! -d /run/systemd/system && perl -e 'sleep int(rand(3600))' && /usr/local/bin/certbot-auto -q renew
</source>

; Update - 2019 Jan - urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization
* <code>certbot-auto</code> fails on noekeon.org, with error <code>The client lacks sufficient authorization</code>.
* We can debug the process with <code>--debug-challenges</code> (see [https://linux.m2osw.com/letsencrypt-client-lacks-sufficient-authorization this page] for details on how certbot updates apache2 config for the test).
<source lang="bash">
/usr/local/bin/certbot-auto certonly --dry-run --debug-challenges
# ...
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
# Challenges loaded. Press continue to submit to CA. Pass "-v" for more info about
# challenges.
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
</source>
* Now let's try to connect.
<source lang="bash">
wget http://keccak.team/.well-known/acme-challenge/VtSysECebfgnfMgpCbXMz4wzgyXqMc0qBAVd5DB2GJ8
# --2019-01-21 08:59:43-- http://keccak.team/.well-known/acme-challenge/VtSysECebfgnfMgpCbXMz4wzgyXqMc0qBAVd5DB2GJ8
# Resolving keccak.team (keccak.team)... 91.134.133.203
# Connecting to keccak.team (keccak.team)|91.134.133.203|:80... connected.
# HTTP request sent, awaiting response... 403 Forbidden
# 2019-01-21 08:59:43 ERROR 403: Forbidden.
</source>
* Checking Apache2 log {{file|/var/log/apache2/error.log}}, we get an <code>search permissions missing</code> error
<source lang="text">
... (13)Permission denied: [...] AH00035: access to /.well-known/acme-challenge/... denied (filesystem path '/var/lib/letsencrypt/http_challenges') because search permissions are missing on a component of the path
</source>
* Searching on the internet, we see this is due to wrong permissions on some directory in the path. Indeed, the culprit is {{file|/var/lib/letsencrypt}}:
<source lang="bash">
l /var/lib
# drwx------ 5 root root 4.0K Jan 21 08:06 letsencrypt/
</source>
* Fix the permissions to solve the issue:
<source lang="bash">
sudo chmod 755 /var/lib/letsencrypt/
</source>

;April 2019 &mdash;
* Doing a <code>certbot-auto renew --dry-run</code> fails on noekeon.org, with following error:
<source lang="text">
main: imap.noekeon.org
Type: connection
Detail: Fetching
http://www.noekeon.org.well-known/acme-challenge/tPzDt_Jjn802jApS1X7O7yrfEGcmGUlNxsi4AQltOVQ:
Invalid hostname in redirect target, must end in IANA registered
TLD
</source>
* As we see, it seems there is a missing slash <code>/</code> before <code>.well-known</code> in the URL.
* The problem is in the apache config, where the <code>selected text come here</code> file {{file|apache2/sites-available/noekeon.org.conf}}
<source lang="bash">
cd /etc
ag www.noekeon.org
# apache2/sites-available/000-default.conf
# 16: # Redirect everybody to http://www.noekeon.org
# 17: Redirect "/" "http://www.noekeon.org"
</source>
* We must add a trailing slash:
<source lang="diff">
- Redirect "/" "http://www.noekeon.org"
+ Redirect "/" "http://www.noekeon.org/"
</source>

;September 2023
* Upgrade to bookworm
* We revert to distribution's <code>certbot</code>.
<source lang="bash">
rm /usr/local/bin/certbot-auto
rm -rf /opt/eff.org/
apt install certbot python3-certbot-apache
</source>
* Upgrade all HTTP site to HTTPS.
<source lang="bash">
certbot -d radiogatun.noekeon.org --apache # Add certificate to a given domain (only)
</source>

;Note with <code>ServerAlias</code>
* If using server alias, make sure to request all certificates at the same time [https://community.letsencrypt.org/t/apache-using-serveralias-with-lets-encrypt/5130/2]:
<source lang="bash">
# <VirtualHost *:80>
# ServerName www.example.com
# ServerAlias example.com
#
# ServerAdmin webmaster@localhost
# DocumentRoot /var/www/www.example.com/public_html
# </VirtualHost>

./letsencrypt-auto --apache -d www.example.com -d example.com
</source>
* When doing this, we had an issue with a missing CAA field (see [https://docs.gandi.net/en/domain_names/faq/record_types/caa_record.html here] for more info) in the DNS configuration:
<source lang="bash">
certbot --apache -d ober.noekeon.org -d mail.noekeon.org -d imap.noekeon.org -d smtp.noekeon.org -d pop.noekeon.org
# Saving debug log to /var/log/letsencrypt/letsencrypt.log
# Requesting a certificate for ober.noekeon.org and 4 more domains
#
# Certbot failed to authenticate some domains (authenticator: apache). The Certificate Authority reported these problems:
# Domain: imap.noekeon.org
# Type: dns
# Detail: DNS problem: networking error looking up CAA for noekeon.org
#
# Hint: The Certificate Authority failed to verify the temporary Apache configuration changes made by Certbot. Ensure that the listed domains point to this Apache server and that it is accessible from the internet.
</source>
* To fix that, we add the following field in OVH DNS config (<code>@</code> means empty field, ie. configuration valid for all domain):
<source lang="text">
@ IN CAA 0 issue "letsencrypt.org"
</source>

;Certbot cron not running
* On noekeon.org, the cron service was not running.
* ... turns out the cron line is disabled when systemd is detected.
* ... turns out the certbot.timer was disabled in systemd as well ({{file|/etc/systemd/system/certbot.timer}} symlinked to null)
<source lang="bash">
systemctl enable certbot.timer
systemctl start certbot.timer
</source>


=== Apache ===
=== Apache ===
Line 223: Line 497:


=== Administration ===
=== Administration ===
* Use '''[http://www.webmin.com/ webmin]''' control panel: https://www.immie.org:10000/ (see [[Webmin]] page for usage).
* <s>Use '''[http://www.webmin.com/ webmin]''' control panel: https://www.immie.org:10000/ (see [[Webmin]] page for usage).</s>
I don't use '''webmin''' anymore (never used in fact). All administration is done via SSH command line.

;Upgrade
* Using [http://www.pontikis.net/tip/?id=35 needrestart] package to automatically tell when some services must be restarted.

=== MediaWiki Tuning ===
Following [https://www.mediawiki.org/wiki/Manual:Performance_tuning MediaWiki guide] and [https://www.mediawiki.org/wiki/User:Aaron_Schulz/How_to_make_MediaWiki_fast Aaron's guide]:
* Install {{deb|php-mbstring}}
<source lang=bash>
apt install php-mbstring
</source>

* Enable <code>$wgCacheDirectory</code>. In {{file|LocalSettings.php}}:
<source lang=bash>
$wgCacheDirectory = "$IP/cache";
</source>
Create the directory:
<source lang=bash>
cd /var/www/miki.immie.org/mediawiki
sudo -u www-data mkdir cache
chmod 700 cache
</source>

* Enable {{deb|memcached}} (see [https://www.mediawiki.org/wiki/Memcached MediaWiki memcached] page)

Install {{deb|memcached}}:
<source lang=bash>
apt-get install memcached php5-memcached
systemctl restart apache2.service
</source>

Add to {{file|LocalSettings.php}}
<source lang=bash>
$wgMainCacheType = CACHE_MEMCACHED;
$wgParserCacheType = CACHE_MEMCACHED; # optional
$wgMessageCacheType = CACHE_MEMCACHED; # optional
$wgMemCachedServers = array( "127.0.0.1:11211" );

$wgSessionsInObjectCache = true; # optional
$wgSessionCacheType = CACHE_MEMCACHED; # optional
</source>

* Enable '''Short URLs''' (URL like <code>https://miki.immie.org/wiki/Main_Page</code>). Follow [https://www.mediawiki.org/wiki/Manual:Short_URL/Apache this guide].

;Install FenTT
* Copy FenTT files to {{file|/var/lib/mediawiki/extensions/FenTT}}.
* Add to {{file|LocalSettings.php}}:
<source lang="php">
# Extension:FenTT
wfLoadExtension( 'FenTT' );
</source>

;Install PgnJS
* Copy PgnJS files to {{file|/var/lib/mediawiki/extensions/PgnJS}}.
* Add to {{file|LocalSettings.php}}:
<source lang="php">
# Extension:PgnJS
wfLoadExtension( 'PgnJS' );
</source>

;Install MathJax &mdash; OBSOLETE, replaced by SimpleMathJax
* Install mathjax library
<source lang="bash">
apt install libjs-mathjax
</source>
* Install extension
<source lang="bash">
git clone https://github.com/hbshim/mediawiki-mathjax /var/lib/mediawiki/extensions/MathJax
</source>
* Configure mathjax
** Set URL
** Disable <code>$...$</code> support (replace with <code>$$...$$</code>)
<source lang="diff">
--- a/MathJax.php
+++ b/MathJax.php
@@ -53,7 +53,7 @@ class MathJax_Parser {
static function ReplaceByMarkers(Parser &$parser, &$text )
{
$text = preg_replace_callback('/(\$\$)(.*?)(\$\$)/s', 'MathJax_Parser::Marker',$text);
- $text = preg_replace_callback('|(?<![\{\/\:\\\\])(\$)(.*?)(?<![\\\\])(\$)|s', 'MathJax_Parser::Marker', $text);
+ // $text = preg_replace_callback('|(?<![\{\/\:\\\\])(\$)(.*?)(?<![\\\\])(\$)|s', 'MathJax_Parser::Marker', $text);
$text = preg_replace_callback('/(\\\\\[)(.*?)(\\\\\])/s', 'MathJax_Parser::Marker', $text);
$text = preg_replace_callback('/(\\\\\()(.*?)(\\\\\))/s', 'MathJax_Parser::Marker', $text);
$text = preg_replace_callback('/(\\\begin{(?:.*?)})(.*?)(\\\end{(?:.*?)})/s', 'MathJax_Parser::Marker', $text);

--- a/mwMathJaxConfig.js
+++ b/mwMathJaxConfig.js
@@ -3,8 +3,8 @@ MathJax.Hub.Config({
showMathMenu: false,
extensions: ["fp.js"],
tex2jax: {
- inlineMath: [ ['$','$'], ["\\(","\\)"] ],
- displayMath: [ ['$$','$$'], ["\\[","\\]"] ],
+ inlineMath: [ ['$$','$$'], ["\\(","\\)"] ],
+ displayMath: [ ["\\[","\\]"] ],
processEscapes: true,
element: "content",
ignoreClass: "(tex2jax_ignore|mw-search-results|searchresults)", /* note: this is part of a regex, check the docs! */
@@ -161,5 +161,5 @@ MathJax.Hub.Config({
}
});
-MathJax.Ajax.loadComplete("_SUBSTITUTE YOUR URL___/mathjax/config/local/mwMathJaxConfig.js");
+MathJax.Ajax.loadComplete("https://miki.immie.org/mathjax/config/local/mwMathJaxConfig.js");
//]]>

</source>
* Enable and configure extension:
<source lang="php">
wfLoadExtension( 'MathJax' );
# MathJax location
$wgMathJaxJS = "/mathjax/MathJax.js";
# Configuration - see "http://docs.mathjax.org/en/latest/config-files.html"
$wgMathJaxProcConf = "TeX-AMS-MML_HTMLorMML-full";
# Local configuration file (excluding .js)
$wgMathJaxLocConf = "local/mwMathJaxConfig";
# Enabled memcached - This may cause issues with mathjax (see https://www.mediawiki.org/wiki/Extension_talk:MathJax)
# $wgParserCacheType = CACHE_NONE;
</source>

;Install SimpleMathJax
* Install extension (own fork):
<source lang="bash">
git clone https://github.com/xeyownt/SimpleMathJax.git /var/lib/mediawiki/extensions/SimpleMathJax
</source>
* Enable and configure extension:
<source lang="php">
wfLoadExtension( 'SimpleMathJax' );
</source>

;Allow upload extensions:
<source lang="php">
# Upload file types
$wgFileExtensions = array( 'png', 'gif', 'jpg', 'jpeg', 'pdf', 'zip', 'webm', 'mp4' );
</source>

=== Owncloud ===
Apt key expired on 2018-08-25 [https://github.com/owncloud/core/issues/32436]:
<source lang="bash">
apt-key list | grep -i owncloud -B 5 -A 2
# pub 2048R/479BC94B 2013-08-26 [expired: 2018-08-25]
# uid ownCloud build service <obsrun@localhost>
</source>

Update key with:
<source lang="bash">
apt-key del 47AE7F72479BC94B
wget -nv https://download.owncloud.org/download/repositories/production/Debian_9.0/Release.key -O Release.key && apt-key add - < Release.key
</source>


=== To Do ===
=== To Do ===
* Return ''error 403 - Forbidden'' when visiting https://miki.immie.org (server root).
* Return ''error 403 - Forbidden'' when visiting https://miki.immie.org (server root).
* Change immie password because we can brute-force it via webmin interface, or forbid immie.
* Change immie password because we can brute-force it via webmin interface, or forbid immie.

=== Firewall ===
We use '''iptable'''. Rules are defined in file {{file|/etc/iptables.up.rules}}.

=== Filesystem ===
* Force fsck every 15 reboots:
<source lang="bash">
tune2fs -c 15 /dev/vda1
tune2fs -l /dev/vda1 | grep -i "mount count"
# Mount count: 2
# Maximum mount count: 5
</source>

Latest revision as of 10:10, 16 July 2024

Links

Status

  • Mailpack

Domain Configuration (Gandi.net)

Managed via Gandi interface (https://www.gandi.net/).

Mail

  • Mailboxes
  • Email forwarding
  • Gandi Mail Pack: Activated 2 GB

Web forwarding

Contacts

Owner, Technical, Administrative, Billing: 

   MP4410-GANDI
   Michael Peeters
   peeters-ml1@noekeon.org

Name servers

   DNS1: a.dns.gandi.net
   DNS2: b.dns.gandi.net
   DNS3: c.dns.gandi.net

Zone

zone file - version 6
  • Currently in user - changed 28.06.2016, 18:12
  • Removed CNAME entries for noekeon.org migration tests.
@ 10800 IN A 91.134.134.85
prime 10800 IN A 91.134.134.85
blog 10800 IN CNAME blogs.vip.gandi.net.
imap 10800 IN CNAME access.mail.gandi.net.
miki 10800 IN CNAME prime
mip 10800 IN CNAME prime
owncloud 10800 IN CNAME prime
pop 10800 IN CNAME access.mail.gandi.net.
smtp 10800 IN CNAME relay.mail.gandi.net.
webmail 10800 IN CNAME agent.mail.gandi.net.
www 10800 IN CNAME prime
@ 10800 IN MX 50 fb.mail.gandi.net.
@ 10800 IN MX 10 spool.mail.gandi.net.
noekeon 10800 IN MX 10 prime
prime 10800 IN MX 10 prime
zone file - version 5
  • Currently in use - changed 20.06.2016, 11:34
  • Removed duplicate CNAME entries (immie) - was causing SERVFAIL in dig queries
; @          10800 IN A     91.134.134.85    ; Not sure I need this so disabled

; Mail server for @immie.org addresses (Gandi.net mail package)
@          10800 IN MX 50 fb.mail.gandi.net.
@          10800 IN MX 10 spool.mail.gandi.net.
blog       10800 IN CNAME blogs.vip.gandi.net.
imap       10800 IN CNAME access.mail.gandi.net.
pop        10800 IN CNAME access.mail.gandi.net.
smtp       10800 IN CNAME relay.mail.gandi.net.
webmail    10800 IN CNAME agent.mail.gandi.net.


; Our server at ovh
prime      10800 IN A     91.134.134.85
; Mail server for @prime.immie.org addresses
prime      10800 IN MX 10 prime

; Some virtual hosts at immie.org
www        10800 IN CNAME prime            ; host www.immie.org
miki       10800 IN CNAME prime            ; host miki.immie.org
owncloud   10800 IN CNAME prime            ; host owncloud.immie.org

; some aliases to prepare transition of domain noekeon.org
alongcil   10800 IN CNAME prime
gilles     10800 IN CNAME prime
gro        10800 IN CNAME prime
gva        10800 IN CNAME prime
heloise    10800 IN CNAME prime
jda        10800 IN CNAME prime
joan       10800 IN CNAME prime
keccak     10800 IN CNAME prime
ketje      10800 IN CNAME prime
keyak      10800 IN CNAME prime
kiwi       10800 IN CNAME prime
mip        10800 IN CNAME prime
radiogatun 10800 IN CNAME prime
sponge     10800 IN CNAME prime

; Mail for testing
noekeon    10800 IN MX 10 prime
Default Gandi zone file - version 1
Not used
@        10800  IN  A          217.70.184.38
blog     10800  IN  CNAME      blogs.vip.gandi.net.
imap     10800  IN  CNAME      access.mail.gandi.net.
pop      10800  IN  CNAME      access.mail.gandi.net.
smtp     10800  IN  CNAME      relay.mail.gandi.net.
webmail  10800  IN  CNAME      webmail.gandi.net.
www      10800  IN  CNAME      webredir.vip.gandi.net.
@        10800  IN  MX     50  fb.mail.gandi.net.
@        10800  IN  MX     10  spool.mail.gandi.net.

VPS Configuration (OVH)

This is done via OVH Manager (https://www.ovh.com/manager/).

Service name

This is the name of the server.

Service name prime.immie.org

Original name was vps282013.ovh.net.

Reverse DNS

In Advanced mode, click Modify the Reverse DNS.

IP 91.134.134.85
Name prime.immie.org

Original name was 85.ip-91-134-134.eu

Server Configuration

Administration

 ✐  Move this to some common page

.

  • Check no old packages:
dpkg -l | grep -E "^rc"  # Must be empty
  • Check systemctl for failed services:
systemctl

Upgrade from Jessie to Stretch

# cat /etc/apt/sources.list 
# Stable = Stretch
# deb http://ftp.debian.org/debian/ stretch main
# deb http://security.debian.org/ stretch/updates main
deb http://archive.debian.org/debian/ stretch main
deb http://archive.debian.org/debian-security/ stretch/updates main

# # Testing
# deb http://ftp.debian.org/debian/ testing main
# # deb-src http://ftp.debian.org/debian/ testing main
# deb http://security.debian.org/ testing-security/updates main

# Unstable / Sid
# deb http://ftp.debian.org/debian/ sid main

# Backport
deb http://archive.debian.org/debian stretch-backports main
  • Remove all pins from /etc/apt/preferences
  • Disable all other sources from /etc/apt/sources.list.d

Guides

Guides I followed to install the server:

I added testing and unstable repositories.

/etc/apt/sources.list: 

# Stable
deb http://ftp.debian.org/debian/ jessie main
deb http://security.debian.org/ jessie/updates main

# Testing
deb http://ftp.debian.org/debian/ testing main
deb http://security.debian.org/ testing/updates main

# Unstable / Sid
deb http://ftp.debian.org/debian/ sid main

# Backport
deb http://ftp.debian.org/debian jessie-backports main

/etc/apt/preferences: 

# cat /etc/apt/preferences 
Package: *
Pin: release a=stable
Pin-Priority: 500

Package: *
Pin: release a=jessie-backports
Pin-Priority: 475

Package: *
Pin: release a=testing
Pin-Priority: 450

Package: *
Pin: release a=unstable
Pin-Priority: 400

Upgraded some packages from testing/unstable: [1] 

apt install debian-goodies=0.66                    # Fix mysqld false positive in checkrestart

SSL - certbot (Let's Encrypt)

mkdir ca
cd ca
cp /usr/lib/ssl/misc/CA.pl .
sed -ri 's/365/3650/; s/1095/3650/' CA.pl
./CA.pl -newca

CA certificate filename (or enter to create)
Country Name (2 letter code) [AU]:BE
State or Province Name (full name) [Some-State]:BBW
Locality Name (eg, city) []:Brussels
Organization Name (eg, company) [Internet Widgits Pty Ltd]:immie.org
Organizational Unit Name (eg, section) []:
Common Name (e.g. server FQDN or YOUR name) []:immie.org.
Email Address []:

[...]
Serial Number: 14779988171032814905 (0xcd1d10ef0ee2e539)
Certificate is to be certified until May 20 17:25:05 2026 GMT (3650 days)

/usr/lib/ssl/misc/c_info demoCA/cacert.pem

demoCA/cacert.pem
subject= /C=BE/ST=BBW/O=immie.org/CN=immie.org.
issuer= /C=BE/ST=BBW/O=immie.org/CN=immie.org.
notAfter=May 20 17:25:05 2026 GMT

openssl x509 -text -fingerprint -sha1 -in demoCA/cacert.pem -out demoCA/cacert-immie.org.crt

Certificate:
 Serial Number: 14779988171032814905 (0xcd1d10ef0ee2e539)
 SHA1 Fingerprint=AD:5E:5C:8B:47:A6:E5:49:7B:E7:6F:F7:F2:E4:95:3B:EC:08:1C:06

./CA.pl -newreq-nodes

Country Name (2 letter code) [AU]:BE
State or Province Name (full name) [Some-State]:BBW
Locality Name (eg, city) []:Brussels
Organization Name (eg, company) [Internet Widgits Pty Ltd]:immie.org
Organizational Unit Name (eg, section) []:
Common Name (e.g. server FQDN or YOUR name) []:www.immie.org
Email Address []:

./CA.pl -sign

Sign the certificate? [y/n]:y
1 out of 1 certificate requests certified, commit? [y/n]y
New - Using Let's Encrypt
  • Follow certbot guide.
  • New — We install only version from jessie-backports:
apt install python-certbot-apache=0.10.2-1~bpo8+1
# sudo apt-get install python-certbot-apache -t jessie-backports
certbot --authenticator=webroot --installer=apache
  • TODO — See IMPORTANT NOTES below:
IMPORTANT NOTES:
 - Congratulations! Your certificate and chain have been saved at
   /etc/letsencrypt/live/www.immie.org/fullchain.pem. Your cert will
   expire on 2016-10-02. To obtain a new or tweaked version of this
   certificate in the future, simply run certbot again with the
   "certonly" option. To non-interactively renew *all* of your
   certificates, run "certbot renew"
 - If you lose your account credentials, you can recover through
   e-mails sent to m-certbot@immie.org.
 - Your account credentials have been saved in your Certbot
   configuration directory at /etc/letsencrypt. You should make a
   secure backup of this folder now. This configuration directory will
   also contain certificates and private keys obtained by Certbot so
   making regular backups of this folder is ideal.
 - If you like Certbot, please consider supporting our work by:

   Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
   Donating to EFF:                    https://eff.org/donate-le
  • Now all certificates are stored in a single file. Same for the key:
SSLCertificateFile      /etc/letsencrypt/live/www.immie.org/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/www.immie.org/privkey.pem
Update - Add some issue with certbot
Jun 12 00:38:12 prime certbot[9567]: File:
Jun 12 00:38:12 prime certbot[9567]: - Could not be found to be deleted /var/lib/letsencrypt/icJA1m-EoE3Gsf6HJlITR4GCBb_9wvlyYV4faqJ_aVk.pem - LE probably shut down unexpectedly
Jun 12 00:38:12 prime certbot[9567]: File:
Jun 12 00:38:12 prime certbot[9567]: - Could not be found to be deleted /var/lib/letsencrypt/icJA1m-EoE3Gsf6HJlITR4GCBb_9wvlyYV4faqJ_aVk.crt - LE probably shut down unexpectedly
Jun 12 00:38:13 prime certbot[9567]: Attempting to renew cert from /etc/letsencrypt/renewal/www.immie.org.conf produced an unexpected error: 'module' object has no attribute 'rand'. Skipping.
  • Uninstalled certbot completely, and removed all files (/etc/encrypt, /var/...).
  • Reinstall with standalone server (did not try webroot as done before, maybe that would work) [2].
certbot --authenticator standalone --installer apache --pre-hook "systemctl stop apache2" --post-hook "systemctl start apache2"
Update - Yet more issue (on immie.org, but not on noekeon.org).
  • See issue #5111. I get error 'module' object has no attribute 'rand'.
  • As a fix, revert back to apache plugin renewal, and removed pre-/post-hook. File
[renewalparams]
authenticator = apache
account = 157b05fb0f012e07716c9dbdc3a9f923
installer = apache
  • Then temporarily fix /usr/lib/python2.7/dist-packages/acme/crypto_util.py. Changes:
+import os

-    cert.set_serial_number(int(binascii.hexlify(OpenSSL.rand.bytes(16)), 16))
+    cert.set_serial_number(int(binascii.hexlify(os.urandom(16)), 16))
  • In fact I noticed that immie.org had a newer version of python-openssl:
ii  python-openssl                     16.0.0-1~bpo8+1       # On noekeon.org
ii  python-openssl                     18.0.0-1              # On immie.org
  • So let's downgrade, and hope it will fix the problem:
apt install python-openssl=16.0.0-1~bpo8+1
  • ... still not fixed. We need to downgrade more packages (list obtained by comparing dpkg output on reference server):
sudo apt install python-cffi-backend=1.9.1-2~bpo8+1 python-cryptography=1.7.1-3~bpo8+1 python-idna=2.0-3~bpo8+1 \
    python-mock=2.0.0-3~bpo8+1 python-pkg-resources=33.1.1-1~bpo8+1 python-pyasn1=0.1.9-1~bpo8+1 \
    python-setuptools=33.1.1-1~bpo8+1 python-six=1.10.0-3~bpo8+1 python-pbr=1.8.0-4.1~bpo8+1
Update - 2019 Jan - tls-sni-01 challenge no longer supported
  • Received mail for immie.org telling tls-sni-01 is no longer supported, and that I must switch to either http-01 or dns-01. Checking with certbot renew --dry-run indeed confirm that I'm using tls-sni-01 as challenge method.
  • Tried many things, but none seems to work. In fact, my certbot is too old
certbot --version
# certbot 0.10.2   <--- too old, we need at least 0.28
  • On Debian Jessie, the recommended install now is to use certbot-auto [3].
  • First remove old certbot:
apt remove certbot
  • Install certbot-auto:
cd /usr/local/bin
wget https://dl.eff.org/certbot-auto
chmod a+x certbot-auto
# Let's try
/usr/local/bin/certbot-auto renew --dry-run
It works, so we change the cronjob task as follows:
0 */12 * * * root test -x /usr/local/bin/certbot-auto -a \! -d /run/systemd/system && perl -e 'sleep int(rand(3600))' && /usr/local/bin/certbot-auto -q renew
Update - 2019 Jan - urn:ietf:params:acme:error:unauthorized
: The client lacks sufficient authorization
  • certbot-auto fails on noekeon.org, with error The client lacks sufficient authorization.
  • We can debug the process with --debug-challenges (see this page for details on how certbot updates apache2 config for the test).
/usr/local/bin/certbot-auto certonly --dry-run --debug-challenges
# ...
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
# Challenges loaded. Press continue to submit to CA. Pass "-v" for more info about
# challenges.
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
  • Now let's try to connect.
wget http://keccak.team/.well-known/acme-challenge/VtSysECebfgnfMgpCbXMz4wzgyXqMc0qBAVd5DB2GJ8
# --2019-01-21 08:59:43--  http://keccak.team/.well-known/acme-challenge/VtSysECebfgnfMgpCbXMz4wzgyXqMc0qBAVd5DB2GJ8
# Resolving keccak.team (keccak.team)... 91.134.133.203
# Connecting to keccak.team (keccak.team)|91.134.133.203|:80... connected.
# HTTP request sent, awaiting response... 403 Forbidden
# 2019-01-21 08:59:43 ERROR 403: Forbidden.
  • Checking Apache2 log /var/log/apache2/error.log, we get an search permissions missing error
... (13)Permission denied: [...] AH00035: access to /.well-known/acme-challenge/... denied (filesystem path '/var/lib/letsencrypt/http_challenges') because search permissions are missing on a component of the path
  • Searching on the internet, we see this is due to wrong permissions on some directory in the path. Indeed, the culprit is /var/lib/letsencrypt:
l /var/lib
# drwx------ 5 root      root     4.0K Jan 21 08:06 letsencrypt/
  • Fix the permissions to solve the issue:
sudo chmod 755 /var/lib/letsencrypt/
April 2019 —
  • Doing a certbot-auto renew --dry-run fails on noekeon.org, with following error:
main: imap.noekeon.org
Type:   connection
Detail: Fetching
http://www.noekeon.org.well-known/acme-challenge/tPzDt_Jjn802jApS1X7O7yrfEGcmGUlNxsi4AQltOVQ:
Invalid hostname in redirect target, must end in IANA registered
TLD
  • As we see, it seems there is a missing slash / before .well-known in the URL.
  • The problem is in the apache config, where the selected text come here file apache2/sites-available/noekeon.org.conf
cd /etc
ag www.noekeon.org
# apache2/sites-available/000-default.conf
# 16:  # Redirect everybody to http://www.noekeon.org
# 17:  Redirect "/" "http://www.noekeon.org"
  • We must add a trailing slash:
-       Redirect "/" "http://www.noekeon.org"
+       Redirect "/" "http://www.noekeon.org/"
September 2023
  • Upgrade to bookworm
  • We revert to distribution's certbot.
rm /usr/local/bin/certbot-auto
rm -rf /opt/eff.org/
apt install certbot python3-certbot-apache
  • Upgrade all HTTP site to HTTPS.
certbot -d radiogatun.noekeon.org --apache      # Add certificate to a given domain (only)
Note with ServerAlias
  • If using server alias, make sure to request all certificates at the same time [4]:
# <VirtualHost *:80>
#     ServerName www.example.com
#     ServerAlias example.com
#
#     ServerAdmin webmaster@localhost
#     DocumentRoot /var/www/www.example.com/public_html
# </VirtualHost>

./letsencrypt-auto --apache -d www.example.com -d example.com
  • When doing this, we had an issue with a missing CAA field (see here for more info) in the DNS configuration:
certbot --apache -d ober.noekeon.org -d mail.noekeon.org -d imap.noekeon.org -d smtp.noekeon.org -d pop.noekeon.org
# Saving debug log to /var/log/letsencrypt/letsencrypt.log
# Requesting a certificate for ober.noekeon.org and 4 more domains
# 
# Certbot failed to authenticate some domains (authenticator: apache). The Certificate Authority reported these problems:
#   Domain: imap.noekeon.org
#   Type:   dns
#   Detail: DNS problem: networking error looking up CAA for noekeon.org
# 
# Hint: The Certificate Authority failed to verify the temporary Apache configuration changes made by Certbot. Ensure that the listed domains point to this Apache server and that it is accessible from the internet.
  • To fix that, we add the following field in OVH DNS config (@ means empty field, ie. configuration valid for all domain):
@ IN CAA 0 issue "letsencrypt.org"
Certbot cron not running
  • On noekeon.org, the cron service was not running.
  • ... turns out the cron line is disabled when systemd is detected.
  • ... turns out the certbot.timer was disabled in systemd as well (/etc/systemd/system/certbot.timer symlinked to null)
systemctl enable certbot.timer
systemctl start certbot.timer

Apache

TBC

Administration

I don't use webmin anymore (never used in fact). All administration is done via SSH command line.

Upgrade
  • Using needrestart package to automatically tell when some services must be restarted.

MediaWiki Tuning

Following MediaWiki guide and Aaron's guide:

  • Install php-mbstring
apt install php-mbstring
  • Enable $wgCacheDirectory. In LocalSettings.php:
$wgCacheDirectory = "$IP/cache";

Create the directory: 

cd /var/www/miki.immie.org/mediawiki
sudo -u www-data mkdir cache
chmod 700 cache

Install memcached: 

apt-get install memcached php5-memcached
systemctl restart apache2.service

Add to LocalSettings.php 

$wgMainCacheType = CACHE_MEMCACHED;
$wgParserCacheType = CACHE_MEMCACHED; # optional
$wgMessageCacheType = CACHE_MEMCACHED; # optional
$wgMemCachedServers = array( "127.0.0.1:11211" );

$wgSessionsInObjectCache = true; # optional
$wgSessionCacheType = CACHE_MEMCACHED; # optional
Install FenTT
  • Copy FenTT files to /var/lib/mediawiki/extensions/FenTT.
  • Add to LocalSettings.php:
# Extension:FenTT
wfLoadExtension( 'FenTT' );
Install PgnJS
  • Copy PgnJS files to /var/lib/mediawiki/extensions/PgnJS.
  • Add to LocalSettings.php:
# Extension:PgnJS
wfLoadExtension( 'PgnJS' );
Install MathJax — OBSOLETE, replaced by SimpleMathJax
  • Install mathjax library
apt install libjs-mathjax
  • Install extension
git clone https://github.com/hbshim/mediawiki-mathjax /var/lib/mediawiki/extensions/MathJax
  • Configure mathjax
    • Set URL
    • Disable $...$ support (replace with $$...$$)
--- a/MathJax.php
+++ b/MathJax.php
@@ -53,7 +53,7 @@ class MathJax_Parser {
     static function ReplaceByMarkers(Parser &$parser, &$text )
     {
         $text = preg_replace_callback('/(\$\$)(.*?)(\$\$)/s',                         'MathJax_Parser::Marker',$text);
-        $text = preg_replace_callback('|(?<![\{\/\:\\\\])(\$)(.*?)(?<![\\\\])(\$)|s', 'MathJax_Parser::Marker', $text);
+        // $text = preg_replace_callback('|(?<![\{\/\:\\\\])(\$)(.*?)(?<![\\\\])(\$)|s', 'MathJax_Parser::Marker', $text);
         $text = preg_replace_callback('/(\\\\\[)(.*?)(\\\\\])/s',                     'MathJax_Parser::Marker', $text);
         $text = preg_replace_callback('/(\\\\\()(.*?)(\\\\\))/s',                     'MathJax_Parser::Marker', $text);
         $text = preg_replace_callback('/(\\\begin{(?:.*?)})(.*?)(\\\end{(?:.*?)})/s', 'MathJax_Parser::Marker', $text);

--- a/mwMathJaxConfig.js
+++ b/mwMathJaxConfig.js
@@ -3,8 +3,8 @@ MathJax.Hub.Config({
                    showMathMenu: false,
                    extensions: ["fp.js"],
                    tex2jax: {
-                   inlineMath: [ ['$','$'], ["\\(","\\)"] ],
-                   displayMath: [ ['$$','$$'], ["\\[","\\]"] ],
+                   inlineMath: [ ['$$','$$'], ["\\(","\\)"] ],
+                   displayMath: [ ["\\[","\\]"] ],
                    processEscapes: true,
                    element: "content",
                    ignoreClass: "(tex2jax_ignore|mw-search-results|searchresults)", /* note: this is part of a regex, check the docs! */
@@ -161,5 +161,5 @@ MathJax.Hub.Config({
                    }
                    });
 
-MathJax.Ajax.loadComplete("_SUBSTITUTE YOUR URL___/mathjax/config/local/mwMathJaxConfig.js");
+MathJax.Ajax.loadComplete("https://miki.immie.org/mathjax/config/local/mwMathJaxConfig.js");
 //]]>
  • Enable and configure extension:
wfLoadExtension( 'MathJax' );
# MathJax location
$wgMathJaxJS = "/mathjax/MathJax.js";
# Configuration - see "http://docs.mathjax.org/en/latest/config-files.html"
$wgMathJaxProcConf = "TeX-AMS-MML_HTMLorMML-full";
# Local configuration file (excluding .js)
$wgMathJaxLocConf = "local/mwMathJaxConfig";
# Enabled memcached - This may cause issues with mathjax (see https://www.mediawiki.org/wiki/Extension_talk:MathJax)
# $wgParserCacheType = CACHE_NONE;
Install SimpleMathJax
  • Install extension (own fork):
git clone https://github.com/xeyownt/SimpleMathJax.git /var/lib/mediawiki/extensions/SimpleMathJax
  • Enable and configure extension:
wfLoadExtension( 'SimpleMathJax' );
Allow upload extensions
# Upload file types
$wgFileExtensions = array( 'png', 'gif', 'jpg', 'jpeg', 'pdf', 'zip', 'webm', 'mp4' );

Owncloud

Apt key expired on 2018-08-25 [5]: 

apt-key list | grep -i owncloud -B 5 -A 2
# pub   2048R/479BC94B 2013-08-26 [expired: 2018-08-25]
# uid                  ownCloud build service <obsrun@localhost>

Update key with: 

apt-key del 47AE7F72479BC94B
wget -nv https://download.owncloud.org/download/repositories/production/Debian_9.0/Release.key -O Release.key && apt-key add - < Release.key

To Do

  • Return error 403 - Forbidden when visiting https://miki.immie.org (server root).
  • Change immie password because we can brute-force it via webmin interface, or forbid immie.

Firewall

We use iptable. Rules are defined in file /etc/iptables.up.rules.

Filesystem

  • Force fsck every 15 reboots:
tune2fs -c 15 /dev/vda1
tune2fs -l /dev/vda1 | grep -i "mount count"
# Mount count:              2
# Maximum mount count:      5
Retrieved from "https://miki.immie.org/mediawiki/index.php?title=Immie.org&oldid=17669"