My ideal journalling app

  • WYSIWYG front-end editor
  • Data stored as light-weight markup (Markdown, etc) 1
  • Data stored as plain text files in a directory structure 2
  • Historical versions saved
  • Cross platform, including at least Linux and Android
  • Able to publish in various formats to various destinations
  • Scriptable, to allow automatic indexing, export, etc

Actually, Zim seems to do all of this. I have used Zim in the past, and published a website <public.irons.nz> onto IPFS . But the IPFS node was a CPU and memory hog, so I stopped that experiment in 2022.

I had been wondering how to get the articles in Zim onto my blog. Perhaps I should turn that around:

  • Zim becomes the main thing
  • Save to a local version control system; it does this automatically
  • Backup to a remote version control server
  • Publish to blogs, etc
  • Publish as a website

And yet, somehow, it seems more of an effort to open a new program, rather than a new tab in a browser. Strange.


  1. This introduces compatibility issues, as there are many partly-compatible versions of almost every markup language ↩︎

  2. Easy to back up, version control, publish, edit externally ↩︎

The things I like to write about

I write sporadically about a few different things. Each of these fills a different purpose in my life, and I want to share them differently

Category Examples Medium Reason
Personal reflections Personal issues
People problems
Paper journal
Face-to-face conversation
If it is online, it will never remain private
Family activities Snaps of the grandchildren
Everyday activities
Private WhatsApp group Common denominator among the group
My interests Hobbies
Photographs
Blog Happy to share them widely
Professional matters Software and system problems
and their solutions
Blog Want to share them widely

Personal reflections are written in a paper journal; if they are shared at all, it is by face-to-face communication. My feeling is that if it is online anywhere, it will not stay private.

Everyday activities are shared on a private WhatsApp group consisting of parents, children, siblings, etc.

  • It is hard to know who to include and who to exclude.
  • Sometimes, there is something that might be of interest to a few more remote relatives or friends, but I don’t want to add them to the group. I have so solution for this. Perhaps an unlisted public group of some sort. This makes it relatively private and secure, provided that it does not matter if anything posted here becomes really public.
  • I am not a big fan of WhatsApp, but it is the common denominator for this group of people, and I don’t have the time or motivation to get everyone to use something better. This WhatsApp group is linked to a self-hosted messaging server so that I am confident that I have a record of everything if WhatsApp turns ugly.

Personal interests and professional matters that I feel can be made public, I write about on a blog linked to my own domain; the service is provided by a 3rd-party blog system.

  • I have configured the blog system to republish to various social networks.
  • I should also hook it up to my self-hosted messaging server so that I have control over the data.
  • At the moment, I use the blog provider’s web app to compose articles. This means that I need to be online; I have not found a suitable client application.

I self-host a Matrix server. Initially, this was an experiment to see if it is something that grandma could use rather than WhatsApp 1. I subsequently set up gateways to various other messaging services (WhatsApp, Facebook Messenger, Signal, Google Groups, SMS)

  • to give me ownership of a copy my data
  • to give me a single application for all of my chats

This is working pretty well, though I am not happy with my data backup solution.


  1. No. The server is reliable and stable. However, the client applications are confusing for non-geeks. ↩︎

micro.blog markdown summary

Structure

  • Heading 1: prefix with hash (#), or ==== on the next line
  • Heading 2: prefix with double hash (##) or ---- on the next line
  • Horizontal rule: four hyphens (----) on an isolated line
  • Block quote: prefix each line with greater-than sign (>)
  • Code blocks: three backticks with language (```python) is supported, but does not work in the preview. Alternatively, indent each line by 4 spaces.
  • Tables: use fairly obvious ASCII-art

Inline text

  • Emphasis: surround with underscore (_)
  • Strong emphasis: surround with double asterisk (**)
  • Strike-out: surround with double tilde (~~); does not work in the preview
  • Code: surround with backtick (`). Use more backticks than you want to include in the code. (```)
  • Highlight: surround with mark tags (<mark>, </mark>)
  • Subscript: surround with sub tags (<sub>, </sub>)
  • Superscript: surround with sup tags (<sup>, </sup>)

Lists

  • Unnumbered lists: prefix with asterisk (*)
  • Numbered list: prefix with a number, value not important (1)
  • Todo lists: use an unsorted list with square-brackets and square-brackets-x ([ ] and [x]) as the first text item. Does not show correctly in the preview.

Links and images

  • Links:
    • URL in angle brackets (<http://url>)
    • Display text in square brackets, URL in round brackets ([display text](http://url))
    • Index/display text in square brackets, empty square brackets ([display text][]). Then, elsewhere in the document, index definition, colon, URL ([display text]: http://url)
  • Images:
    • Exclamation mark, alt-text in square brackets, URL in round brackets, optional title enclosed in double-quotes (![alt text](http://url "title text"))
    • Exclamation mark, alt-text in square brackets, index reference in square brackets (![alt text][index ref]). Then, elsewhere in the document, index definition, colon, URL, title text ([index ref]: http://url "Title text")
    • It seems that title text is not shown
    • If you want to suggest the size, you have to fall back to the HTML <img> tag with width and/or height properties.
  • Ensure that link and image references are unique in the document.

Escapes

  • Markdown uses a number of special characters to indicate various markup. Examples are code spans (surrounded by backticks `) and emphasised text (surrounded by underscores _).
  • To include one of these special characters as themselves, escape it with a backslash \. So
    • This is an underscore ‘_’
    • This is a backtick ‘`’
    • This is an angle-bracket ‘<’
    • This is a backslash ‘\’
    • None of these are in a code-span.
  • The special symbols are
    • \ backslash
    • ` backtick
    • * asterisk
    • _ underscore
    • # hash symbol
    • + plus sign
    • - hyphen, minus sign
    • . dot, period
    • ! exclamation mark
    • { } curly brackets
    • [ ] square brackets
    • ( ) round brackets
    • < > angle brackets

Examples

Structure

Heading 1

Heading 2

This is normal text

This is a block-quote text of one line

Back to normal text, followed by a code block

code block
more code

Back to normal text. We can also use the syntax-highlighting style, though it does not show in the preview.

  • Python
    1
    2
    3
    4
    
    #! python3
    
    class Xdef:
        pass
    
  • Shell
    1
    2
    3
    
    #! /bin/sh
    
    i=0; while true; do echo $i; i=$(($i + 1)); done; then a lot more stuff; to show what happens when we overflow
    

Inline

  • This is emphasis with underscores and with asterisks
  • This is strong emphasis with double underscores and with double asterisks
  • This is strike-out with double tilde
  • This is code with lots of backticks code with 4 backticks ````
  • This is highlight surrounded mark-tags
  • Subscript H2O
  • Superscript a x2

Lists

  • Unordered list item 1
  • Unordered list item 2
    1. Ordered sub-list item 1
    2. Ordered sub-list item 2
  • Back to the original list
    • Unordered sub-list item 1
  • Back to the original list again

Here is something

  • Todo list is not supported
  • Or maybe it is


Images

  • Reverse tango hook alt text, with link and title

  • Reverse tango hook alt text, with reference

  • Reverse tango hook alt text, using img tag
  • Stephen Irons

The link definitions are below this line in the source, but invisible in the presentation.

Mechanical keyboard thoughts

In August 2020, I bought my first mechanical keyboard: a Keydous NJ68 from Aliexpress

  • 68 keys(5 rows typewriter, with arrows and a few editing keys)
  • PBT keycaps, with nice, simple lettering, Cherry profile
  • Cherry Brown switches
  • RGB backlight with lots of patterns, etc
  • Hot-swapable switches
  • Bluetooth or USB-C, with big battery

I am very pleased with the way it works. A few months later, I bought another, this time with Cherry Blue (noisy) switches. I use the noisy switches in my home office, and the Brown switches in my open-plan office at work.

About annually, I do a scout around of mechanical keyboards to see if anything interests me. Then I realise that, for me, a keyboard is a tool, not a hobby. I have plenty of other hobbies to consume my money.

Part of me would like to build a wooden housing for the keyboard. I have few nice chunks of wood (kwila, acacia, beech) and it would be an interesting winter project.

These are my thoughts about the decisions I would make if I were to choose again

  • Mechanical keys are nice
  • Clicky (blue) keys are great, but tactile (brown) are less offensive for others trying to sleep
  • TKL (84 keys) might be better than 68 keys
  • Bluetooth is not worth it if you have a BIOS password; unique keyboard dongle is nice as it keeps the desk clear.
  • RGB backlighting is pointless
  • Don’t worry too much about programmability, it is easy to learn new muscle memory – unless you have different keyboard that you use simultaneously

Details

  • I really do prefer the feel of the mechanical keys, and that is worth paying some money for.

  • I like both the clicky (Blue) and the feely (Brown) switches. I would probably get the Browns as the noise from the Blues does affect others at home.

  • It is nice and heavy, so it stays put on the desk.

  • I would probably choose a TKL (87 key) keyboard. This keyboard is big enough that there are not too many second-layer keys

    • I have learned where the most common editing keys are.
    • But the [`/~] key is on the right hand side so that the [Esc] key can be close to its normal position. This is unnatural.
    • The TKL layout is not much bigger, and I am not particularly limited in my desk space.
  • I would not choose Bluetooth.

    • I used to use a Logitech wireless keyboard, and do prefer to not have a cable lying on the desk.
    • I cannot use Bluetooth. For Various Reasons (TM), I have an encrypted disk and need to enter a password before the OS has booted. The BIOS on my PCs does not support Bluetooth, so I have to have either a wired keyboard, or one with a dongle that pretends to be a keyboard.
    • The battery lasts about a week, so I had to plug it in periodically anyway; I could get into the habit of hunting around for a USB cable to plug it in on Friday afternoons.
  • I really like the idea of swappable keycaps. However, in 2 years, I have not felt the need to change them.

    • I used to change keyboards when the bumps on [F] and [J] key wore off, usually after 3–4 years.
    • After 2 years, the [J] key is wearing down a bit; [F] is still ok.
    • I really do like some of the other colour options available.
    • I am not prepared to spend $100–$200 (or much, much more) on keycaps alone. This is not a hobby.
  • Don’t bother with hot-swap switches. It has been 2 years now, and I have not felt the need to change them. This is not a hobby.

  • Don’t bother with RGB backlighting.

    • I found the flashing patterns to be very distracting; I did rather like a dim cyan colour
    • I often accidentally hit the magic key-sequence to select another pattern, or to change the brightness, then I had to find the user guide to work out how to reset it
    • The light does not shine through the double-shot PBT keycaps; it only comes through the gaps between the keys
    • I have subsequently re-configured the keyboard to disable the pattern-change key-sequence.
  • Don’t bother with programmability (QMK, etc)

    • I keep the same keyboard long enough that it is easier to re-learn where the hidden keys are, rather than re-program the keyboard to suit my muscle memory.
    • This keyboard uses a proprietary, Windows-only application to re-program the keys.
    • I have only used it work around accidentally changing the backlighting; if there are no annoying colour patterns, and enough keys that second-layer keys are easy to learn, then there is no need.

Nice keyboard colour schemes

Despite having said that keycaps are too expensive, there are some really beautiful colour schemes out there that might be contenders when the [F] and [J] finally do wear out. My current favourites are

  • GMK Birch
  • GMK Green Tree

NZ Post scam text

I received a text today, purporting to come from NZ Post, saying that a parcel could not be delivered, with a link to click to arrange re-delivery.

Unfortunately, I missed the indications that it might be a scam:

  • It said they would arrange re-delivery the same day; normally, re-delivery happens the following day
  • Phone number is Austrilian (+61, rather than +64)
  • The link to click points to bit.ly, a link-shortening service, which also acts as a link-obfuscation service.
  • Sent using RCS rather than normal SMS; this is just a bit unusual

I clicked the link, and got a page that looked like a tracking page with a valid-looking tracking number, except

  • it had an unusual URL https://nztxpits.icu/?token=, nothing like NZ Post
  • the delivery events (pickup, out for delivery, etc) did not make any sense

My experience with your standard re-delivery web pages is that it is easier to do with a keyboard rather than on a phone, so I transferred over to the laptop.

Clicking the link again failed. Typing the tracking number into the NZ Post tracking service brought up a page with some data the same as the scam link, but slightly different details.

It was only at this point that I realised something was not right. Searching NZ Post website for current scams showed that this is a thing.

Lessons learned

  • I have stopped clicking on links in emails, unless I have explicitly requested something by logging in to the website, and the email appears in my inbox in the next few minutes.
  • I will have to stop clicking links on texts too.
  • Do not trust URL shorteners such as bit.ly, goo,gl, TinyURL, etc.
  • The bank says on its website that it no longer sends clickable links in emails, and requests that you log in manually; the same applies to all services.
  • If you do receive a link, even one that looks valid, do not click it. Instead, log in to the service manually, and type or copy-paste the identification number (tracking number, reference number, invoice number, etc) into the appropriate box on the website.
  • If a service provider does not provide enough information to do this, complain to the service provider.

Outcome

  • I clicked the link, so the scammer knows that my phone number is valid.
  • I did not click the re-delivery options button, so I did not enter any important information.
  • But this was by good luck, not by good judgement.
  • My house is weatherboard too.
  • Painted the same colour.
  • My gumboots are on the deck.

Move TLS certicates to a new host

Use ssh to tar a file on the remote host and pipe it to tar on the new host. This avoids the risk of leaving a file around with this sensitive data.

On the new host,

ssh user@other.host sudo -S tar -C /etc/letsencrypt -zvc live archive | sudo tar -C /etc/letsencrypt -zvx

Note that this runs sudo on the remote host; the -S argument uses stdin to read the remote user’s password. This means that the password is displayed on the terminal, but at least it is not in the command history. After running the command, consider clearing the terminal buffer using reset or similar.

Access home router web UI from a remote location

The home-router web UI is only accessible from the internal network, not from the outside world. This is to prevent the evil outsiders from try to break it, or break into it.

But sometimes I need to access it remotely. This is a multi-stage process involving multiple ssh port forwards, and some DNS and HTTP-client trickery.

Port forwarding

This needs a PC on the remote network (local to the router) and a server that is world-visible.

  • ssh into a PC on the remote network, then set up a local port-forward from a local port to the router

      remote-pc:~$ ssh -N -L 8123:192.168.1.1:80 stephen@localhost
    
  • ssh into the same remote PC, then set up a remote port-forward from the world-visible server to the

      remote-pc:~$ ssh -N -R 8124:localhost:8123 stephen@world.visible.server
    
  • from the local workstation, set up a local port-forward to the world-visible server

      local-pc:~$ ssh -N -L 8125:localhost:8124 stephen@world.visible.server
    

You can now use an HTTP client to connect to the local port (localhost:8125) and this is forwarded to the very remote router (192.168.1.1:80 on the remote network).

However, there is still a bit of HTTP trickery to do. The HTTP request include a header ‘Host:’. If the remote host (the router) is doing its job, it will verify that this hostname is correct. Normally, the HTTP client uses the hostname from the URL (localhost) as the ‘Host’ header.

In this case, the remote router accepts either the name configured in one of its settings, or one of its own IP address. In either case, either of those name does not resolve to the correct location.

If using curl, you can use the ‘-H’ option to set the hostname

curl -H 'Host: 192.168.1.1' http://localhost:8125

Using firefox, however, this is more tricky. You need an extension such as ‘Modify Header Value’ to change the header on a per-URL basis. I found it best to do the following:

  • add a new host-name to IP mapping to /etc/hosts, something like

      127.0.0.1    remote-router
    
  • add a new header modification to the firefox extension, something like

      remote-router  modify    'Host'   192.168.1.1
    

Self-hosted server: router/firewall configuration

The internet router is a residential model from Vodafone. It seems to do the following

  • Supports IPv6, including

    • full support for outgoing IPv6 connections
    • stateless address autoconfiguration (SLAAC)
  • Allows IPv4 incoming connections only.

    • By default, blocks all incoming connections (IPv4 only).
    • Opens individual ports forwarded to specified host:port (standard NAT stuff, IPv4 only).
    • Can set up a DMZ server, where all other incoming connections are routed to a single host (IPv4 only).
    • No way to configure IPv6 incoming connections.
  • It seems fairly reliable, but something drops out periodically.

    • The router itself
    • The far-end of the fibre connection
    • The local Wi-Fi extender (TP Link Deco system)

Setup

  • Ensure that IPv4 address remains fixed: requested from Vodafone, $5 per month

  • Ensure that IPv6 address remains fixed; might happen at the same time

  • Port configuration details

    • Leave the standard port-forwards as they are
    • Set up DMZ pointing to new server
    • Ensure that new server has firewall (ufw) enabled and stopping as much as possible

Self-hosted server: software required

Install and configure the following applications and services

  • Matrix homeserver

    • Server - synapse
    • Facebook messenger bridge - mautrix-facebook
    • Signal messaging bridge - mautrix-signal
    • Whatsapp bridge - mautrix-whatsapp
    • HTTP webhook - pushmatrix
    • Database - postgresql
    • HTTP server - nginx
    • TLS certificates - Let’s Encrypt
    • Move data from previous postgresql database
  • IoT platform

    • Server - Thingsboard
    • Database - postgresql
    • HTTP server - nginx
    • TLS certificates - Let’s Encrypt
    • Move data from previous postgresql database
  • Performance monitoring

    • Munin server
    • Munin node
    • HTTP server - nginx
    • TLS certficates - Let’s Encrypt
    • Move data from previous server – lots of RRD files
  • Backup storage - duplicity, rsync, etc

  • Data synchronisation - SyncThing

Self-hosted server: server hardware

I have an old HP EliteDesk G3 Mini PC

  • 4 core CPU
  • 8 GB RAM
  • 256 GB SSD (SSD)
  • Gigabit ethernet
  • Wi-Fi, etc

This is twice the specs of both of my current virtual servers added together, so it should do the trick.

Potential upgrades

  • RAM maxes out at 32 GB
  • Able to take 2 SSDs: one SATA and one NVMe

Things to consider

  • Increase RAM to 32 GB

  • Install large (2 TB? 4 TB?) SATA and NVMe SSDs

  • Set up RAID1 – save data on 2 different drives, allowing for continued operation even if one of them fails.

  • Backup system – to local and remote devices

  • UPS to allow continued operation in case of power-failure. In this case, it has to provide power for

    • Server
    • Router
    • Switch
    • Backup server
  • Performance monitoring

    • CPU usage and performance
    • RAM usage and performance
    • SSD usage and performance
    • Network usage and performance
    • CPU health – voltages and temperatures
    • SSD health – SMART monitoring, RAID status
    • Network health – errors, retries, etc

Self-hosted server

It seems wasteful to rent two virtual servers from a cloudy-provider when the same money would buy a real server with twice of everything (CPUs, RAM, SSD, etc) in 12 months. However, moving it in-house comes with some additional costs.

These are the things that run on the existing servers, in order of importance

  • Backup server for PCs and phones
  • Synchronisation server
  • Performance monitor
  • Matrix homeserver
  • IoT platform

However, the biggest bang-for-the-buck will come from moving the IoT platform, Thingsboard, running as iot.irons.nz on iolanthe.irons.nz.

  • The only service on iolanthe.irons.nz, so once it is moved, we can shut down

  • Very few incoming network connections

    • one or two incoming webhooks (Swarm, weather-station)
    • one incoming MQTT connection (weather-station)
    • for testing, one or two LwM2M devices

These are the things than need doing

We will tackle these one at a time.

Self-hosted Matrix server: move database to a new server

  • On both servers, create a ~/.pgpass file with the database usernames and passwords. Each line is the login credentials for one database, in the format hostname:port:database-name:username:password

  • On the old server, shut-down Synapse and the plugins

      sudo systemctl stop synapse
    
  • On the new server, shut-down Synapse and the plugins

       ssh stephen@constanza.irons.nz pg_dump -C -h localhost -U synapse synapse | psql -h localhost -U synapse synapse
    

    This took a few minutes; this is a small Synapse installation

    The databases are not exposed to the internet, so ssh is used to create a session on the remote side.

Self-hosted Matrix server: configure nginx and Let's Encrypt

Assuming that nxginx and Let’s Encrypt are installed

  • Create nginx configuration file /etc/nginx/sites-available/irons.nz

      server {
          listen 80;
          listen [::]:80;
    
          server_name irons.nz;
          return 301 https://$host$request_uri;
      }
    
      server {
          listen 443 ssl http2;
          listen [::]:443 ssl http2;
    
          listen 8448 ssl http2;
          listen [::]:8448 ssl http2;
    
          ssl_certificate /etc/letsencrypt/live/irons.nz/fullchain.pem; # managed by Certbot
          ssl_certificate_key /etc/letsencrypt/live/irons.nz/privkey.pem; # managed by Certbot
    
          server_name irons.nz;
    
          location / {
              return 404;
          }
    
          location /_matrix {
              proxy_pass http://localhost:8008;
              proxy_set_header X-Forwarded-For $remote_addr;
              proxy_set_header X-Forwarded-Proto $scheme;
              proxy_set_header Host $host;
    
              # Nginx by default only allows file uploads up to 1M in size
              # Increase client_max_body_size to match max_upload_size defined in homeserver.yaml
              client_max_body_size 50M;
          }
    
          location /_synapse/client {
              proxy_pass http://localhost:8008;
              proxy_set_header X-Forwarded-For $remote_addr;
              proxy_set_header X-Forwarded-Proto $scheme;
              proxy_set_header Host $host;
    
              # Nginx by default only allows file uploads up to 1M in size
              # Increase client_max_body_size to match max_upload_size defined in homeserver.yaml
              client_max_body_size 50M;
          }
      }
    
  • Temporarily comment-out the listen:443 lines and the certificate lines

  • Test the configuration, then restart nginx

    sudo nginx -t
    sudo systemctl restart nginx
    
  • Verify that it is running

    curl localhost:80
    
  • Update DNS record for https://irons.nz

  • Create certificate for https://irons.nz

      sudo certbot certonly --nginx
    

    This is an interactive command; there might be command-line options to specify the URL.

Install Let's Encrypt -- for nginx, using pip

Edit: this turned out to be too complicated. As much as I prefer to avoid a Snap package, I decided to go with it rather than jump through these hoops. When I have the system up and running, I might re-evaluation this decision.

  • Install dependencies

      sudo apt install python3 python3-pip libaugeas0
    
  • Remove any OS-installed version

      sudo apt remove cerbot
    
  • Set up a Python virtual environment with certbot

      sudo python3 -m venv /opt/certbot/
      sudo /opt/certbot/bin/pip install --upgrade pip
      sudo /opt/certbot/bin/pip install certbot certbot-nginx
    
  • Optionally, set up a symlink in the path

      sudo ln -s /opt/certbot/bin/certbot /usr/local/bin/certbot
    
  • Enable auto-renewal

      echo "0 0,12 * * * root /opt/certbot/bin/python -c 'import random; import time; time.sleep(random.random() * 3600)' && sudo certbot renew -q" | sudo tee -a /etc/crontab > /dev/null
    

Derived from Let’s Encrypt - pip

Self-hosted Matrix server: Install Synapse

  • Create a system-user

      SYNAPSE_USER_DIR=/var/lib/synapse
      sudo adduser --system synapse --home $SYNAPSE_USER_DIR/synapse
    
  • Install a Python3 virtual environment and Synapse

      SYNAPSE_DIR=$SYNAPSE_USER_DIR/synapse
      sudo -u synapse mkdir -p $SYNAPSE_DIR
      sudo -u synapse python3 -m venv $SYNAPSE_DIR/.venv
      sudo -u synapse $SYNAPSE_DIR/.venv/bin/pip install matrix-synapse
      sudo -u synapse $SYNAPSE_DIR/.venv/bin/pip install psycopg2-binary
    
  • Create and edit the configuration file. See https://matrix-org.github.io/synapse/develop/setup/installation.html for details.

      sudo -u synapse /.venv/bin/python3 -m synapse.app
    

    Incomplete

    Alternatively, copy the file from another installation and edit it

  • Create and install a systemd .service file

      (
      cat <<EOF
      [Unit]
      Description=Synapse Matrix server
      Documentation=https://www.matrix.org
      After=network.target
      #After=postgresql.service
    
      [Service]
      Type=notify
      NotifyAccess=main
      Restart=on-abort
    
      User=synapse
      Group=nogroup
      WorkingDirectory=$SYNAPSE_DIR
      ExecStart=$SYNAPSE_DIR/.venv/bin/python3 -m synapse.app.homeserver --config-path=$SYNAPSE_DIR/homeserver.yaml
      ExecReload=/bin/kill -HUP $MAINPID
      SyslogIdentifier=matrix-synapse
    
      [Install]
      WantedBy=multi-user.target
      EOF
      ) | sudo tee /etc/systemd/system/matrix-synapse.service          
    

    Derived from https://github.com/matrix-org/synapse/blob/master/contrib/systemd/matrix-synapse.service

  • Start service

      sudo systemctl start matrix-synapse
    
  • Verify that it has started

      systemctl status
      journalctl -u matrix-synapse
    
  • Set it to start automatically

      sudo systemctl enable matrix-synapse
    

Self-hosted Matrix server: set up PostgreSQL for Synapse

Assuming

  • PostgreSQL is installed.
  • Synapse and the various bridges will use the same database.

Do the following

  • Create a user

      sudo -u postgres createuser --pwprompt synapse
    

    This will prompt for a password; record the password: it will need to go into the synapse configuration file.

  • Create databases owned by this user as needed for

    • synapse

    • mautrix-facebook

    • mautrix-signal

    • mautrix-whatsapp

          sudo -u postgres createdb --encoding=UTF8 --locale=C --template=template0 --owner=synapse synapse
          sudo -u postgres createdb --encoding=UTF8 --locale=C --template=template0 --owner=synapse mautrix-facebook
          sudo -u postgres createdb --encoding=UTF8 --locale=C --template=template0 --owner=synapse mautrix-signal
          sudo -u postgres createdb --encoding=UTF8 --locale=C --template=template0 --owner=synapse mautrix-whatsapp
      

Yet another calendar-time special case

In the beginning, time was simple for computers. We just did everything in our local time. Sure, calendars were tricky, what with months with a different number of days, months that did not always have the same number of days, and so on.

But some places had this daylight saving thing, where the time changed twice a year, often by an hour in either direction. So, the intelligent among us decided to use UTC everywhere, and make adjustments to local time when needed.

However, real people use local time. For example, if I ask a customer when a problem happened, they will use local time. That meant that to convert from the computer time to local time, we needed to know which timezone was in use. So we used the combination (UTC, timezone name). But that did not solve the problem everywhere, because we need an up-to-date set of timezone data to calculate the local time; this is not always available, or is not always correct or up-to-date.

It then seemed that using (local time, timezone offset) or (UTC, timezone offset) would do it; these are semantically identical. I prefer, where possible, to use local time + timezone offset, for no justifiable reason.

That does just fine for times where the timezone offsets are correct – that is, for times in the past where the timezone database is correct. However, timezones are a political thing, and there are errors in everything, so the timezone database can change, either because a government decides to change the timezone rules, or to fix bugs in the database.

This means that you need to store 3 things about a timestamp

  • the actual time in either a local time or UTC
  • timezone offset from UTC at that time
  • the timezone name, in case the time is in the future, and the timezone offset changes, and we need to show the time in local time.

Self-hosted Matrix server: Overview

System

  • HP mini PC
  • 4 core, 3.1 GHz, 12 GB RAM, 128 GB SSD

Other context

  • A few other services running on the server
  • Domain name irons.nz already set up
    • A few subdomains using A, AAAA and CNAME records

OS setup

  • Ubuntu 22.04.2 LTS

  • postgresql (version 14)

  • nginx

  • Python3 (version

    • virtualenv
    • pip
  • git

Installation

  • postgreql
  • synapse
  • nginx
  • letsencrypt
  • mautrix-facebook
  • mautrix-signal
  • mautrix-whatsapp
  • pushmatrix

Using CANView on Ubuntu 22.04

CANView is a tool for viewing CAN bus data. It can view live data when connected to a suitable tool, or can view saved log files in a few formats.

It is a cross-platform tool, provided by Yacht Devices.

The latest version is 1.40, released on 5 August 2021.

On Ubuntu 22.04, it fails at startup, with the message

./CANView: error while loading shared libraries: libpng12.so.0: cannot open shared object file: No such file or directory

Investigation shows that, indeed, libpng12 is not installed; libpng16 is installed; and libpng12 is not available for installation.

A version of libpng12 is available for Ubuntu 22.04 from Launchpad, created by Linux Uprising.

I could not install this using dpkg, as it complained about a conflict. However, a .deb file is really just an archive file of some sort, and can be opened using an archive viewer. I used file-roller to extract libpng12.so.0.54.0 and a symlink libpng12.so.0

  • Download CANView

  • Unzip CANVIEW.zip somewhere

  • Change the executable flag on CANVIEW/Linux/CANView

  • Download libpng12

  • Extract libpng12.so.0.54.0 and the symlink libpng12.s0.0 into the same directory as the Linux executable

  • Run CANView, pointing the library loader to the directory that has libpng12. Assuming that libpng12 and CANView are in the current directory

      LD_LIBRARY_PATH=. ./CANView
    

Supported log file formats

  • .CAN,

Credit-card single-board computers

I have used the following credit-card sized single-board computers.

  • Raspberry Pi family
  • BeagleBone Black family
  • Rock64

My preference is for the BeagleBone Black family, with the Green Wireless from SeeedStudio as my favourite.

BeagleBone Green Wireless

  • All of the BeagleBone family have on-board eMMC for the main filesystem, rather than SD card
  • 4 USB ports
  • WLAN and Bluetooth, with on-board chip antenna and 2 u.FL connectors for external antennas
  • Connector for standard FTDI cable
  • Lots of GPIOs, various serial port, CAN, etc
  • No on-board HDMI framer (I thinik), so those pins can be used as GPIO; I don’t need video
  • Schematics, PCB layout and documentation for all of the devices available
  • No proprietary stuff
  • Modern Debian 11 (Bullseye), programmable via SD card socket

Disadvantages

  • Old components
  • Rather slow
  • Small RAM
  • No on-board DC power connector. Power comes from the micro USB socket, or you can supply power from a piggy-back board (‘cape’) via the 5 V pin.
  • Very few piggy-back boards available
  • Small community

I like the look of the BeagleBone Green Gateway (also from SeeedStudio), but I have not used it.

  • Similar to BBGW
  • On board DC power connector, with input range 5 – 28 V
  • Ethernet, WLAN, Bluetooth
  • 2 USB ports

Raspberry Pi

  • Huge community
  • Lots of add-on boards
  • SD card is not reliable as the primary file system
  • Proprietary main process; no documentation available
  • Ethernet via USB

Rock64

  • For its time, it was a very powerful processor
  • GB Ethernet
  • One ?? (fast) USB port
  • 4 standard USB ports
  • Proprietary processor; no documentation available
  • Small community, poor software support