Stephen Irons

The matrix-whatsapp bridge stopped transferring videos and images a few months ago, and stopped working completely a few weeks ago. It seems that Meta has changed things…

Assumptions

• Bridge mautrix-whatsapp is installed in it own directory as user synapse
• System services are controlled using systemd

Preparation

Download the new version of mautrix-whatsapp from [https://github.com/mautrix/whatsapp/releases], using whatever method you can. The bridge is a single executable; I prefer to append the version number to the binary and use a symlink to select between different versions

cd <bridge_dir>
sudo -u synapse mv mautrix-whatsapp-amd64 mautrix-whatsapp-v0.8.6
sudo -u synapse chmod +x mautrix-whatsapp-v0.8.6

Steps

In one shell, start watching the log-output to check for problems during shutdown and restart

cd <bridge_dir>
sudo tail -n 1000 -F logs/bridge.log

In a different shell, watch the service status

watch -n1 systemctl status mautrix-whatsapp

In a third shell, do the open-heart surgery

cd <bridge_dir>
sudo systemctl stop mautrix-whatsapp

sudo -u synapse ln -fs mautrix-whatsapp-amd64-v0.8.6 mautrix-whatsapp

sudo systemctl stop mautrix-whatsapp

###Notes

• The log-files are not world-readable.
• The name of the log-file has changed. It might be different in the future.

The matrix server has been running for many months now. It is time to update. The process was seamless.

###Assumptions

• python package matrix-synapse is installed using pip, in a virtual environment owned by user synapse. See [link-tbd] for details of the installation.
• services are controlled using systemd

###Steps

In one shell, start watching the log-output to check for problems during shutdown and restart

  cd <synapse_dir>
  tail -F homeserver.log

In a different shell, watch the service status

  watch -n1 systemctl status matrix-synapse

In a third shell, do the open-heart surgery

  sudo systemctl stop matrix-synapse

  cd <synapse_dir>
  sudo -u synapse  .venv/bin/pip freeze | sudo -u synapse tee pip-requirements-2022-06-17-1.txt
  sudo -u synapse  .venv/bin/pip install -U --upgrade-strategy eager matrix-synapse
  sudo -u synapse  .venv/bin/pip freeze | sudo -u synapse tee pip-requirements-2022-06-17-2.txt

  sudo systemctl start matrix-synapse

###Notes

• Keep a record of the pip-installed packages both before and after installation, to see what has changed.

Is it my imagination, or are ‘free’ phone apps and web apps much, much better than Open Source Linux installed apps?

• A spreadsheet is my favourite tool. Although I don’t like to support Google, I would rather use Google Sheets than Libre Office Calc, if only for the simple reason that I can refer to a column range from a specific top-most cell to the bottom of the column, whatever its row number is.
• A stick-and-box drawing tool is my second favourite tool. Visio was my favourite of these, but only on Windows, and I don’t have access to Visio any more. There is no native Linux app that is even close to the standard of Visio. Web-base draw.io is quite usable, and there is an installable Elektron version, so that it close.
• Words are an essential part of what I do; Libre Office Writer is ok, but is too complex, and tries to be bug-compatible with Microsoft Word. Once again, Google Docs does what I need, and can include diagrams from draw.io.
  • Actually, I found NewDeal Office to be even better for its time, back in the 2000s or so. The word-processor and spreadsheet were completely interoperable: a table in a word-processor document was a genuine spreadsheet, and each cell in a spreadsheet had all of the formatting capabilities of a word-processor.
  • I really don’t want to go back to LaTeX…
  • Those tools are acceptable with tables and pictures, but don’t handle line-drawings, snippets of music scores and other picture-like objects properly – you can import an image, but they don’t adjust properly for differing views (on-screen vs. printed, etc).
  • If I use a lIghtweight markup processor (Markdown, reStructuredText, AsciiDoc, NaturalDocs, etc, etc), it must handle those pcitueverything I want to put generally don’t a
• For photos, my phone has an adequate touch-up tool, for cropping, straightening the horizon, and adjusting exposure levels. Actually, there are some acceptable tools on Linux: shotwell and digiKam. I don’t often need to use the full GIMP experience, but sometimes want to remove unfortunate power-lines or expand the background to achieve a particular aspect ratio – Liquid Rescale to the rescue, probably available on my phone too by now.
  • But there is no easy way to mark up a photo, screenshot or map: to circle a feature in red, to point something out with an arrow or map-pin, to draw a line or add text.
• For audio editing, I use either Audacity (for trimming rubbish off the start and end) or Ardour (for cuts and fades, filters, etc. Audacity is generally ok, but I don’t like that it overwrites the original. Ardour is a bit finicky to set up for casual use, but once going, and you are used to it, works well.
• I use one of three Linux programs for processing videos – shotcut, flowblade and pitivi. They are actually not too bad for the things I need to do: taking full-length videos of a performance from multiplle camera and assembling them with cuts and cross-fades into a sensible video, synced with audio from a separate audio recording. They all work, and at least they don’t crash the way they used to. None of them are any good for making up an export-workflow to export to a number of different formats; they all assume that you will export your video once and upload it to YouTube; not what I do.
• For music score editing, MuseScore is wonderful. There is nothing to compare with on the phone. I have not used the big-name score editors much (Sibelius and Finale), but Musescore seems up there with them.

I started out thinking that Open Source Linux apps are much worse than phone apps and web apps, and wanted to rant about them. It turns out that it is only the office tools that suck. The photo, audio and video editing tools are fine; the music score editing is first class.

I will have to re-write this post as an impartial assessment of these tools.

Many years ago, I stopped tagging my photos with short (one-word) tags. Either the tools only allowed a limited number of tags, or they stored the tags in a separate database rather than in the photo file itself, or one of a number of other problems that I forget now.

This was the peak-tagging time: delic.io.us was the thing. Organisations were trying to standardise tag formats, and how to tag various types of data. Think IPTC, Dublin Core, XMP, Meta Content Framework, Resource Description Framework, etc.

Google was still a friendly startup that had revolutionised searching. I think everyone realised that tagging was limited, and full-text search was the real thing. Operating systems started scanning filesystems looking for tags, and indexing text, including descriptive text metadata within photos. Application were adding support for these search databases.

My idea was rather than adding a plethora of one-word tags, I would write a short text description in the format

• <who> is <where> doing <what> on <occasion>

To save time, I would usually add the same description to a groups of photos: a whole week of holiday pictures, all of the birthday-party snaps, etc.

Later, I might go back and rate the nicest photos 3, 4 or 5 stars, and add more detail to the description.

My assumption was that eventually the photo management tools would catch up and let me search; that assumption is now true, and my

However, I did not anticipate the growing and changing standards, and the way different tools implemented them.

There are mutliple ways of storing metadata within a photo file: Exif tag, XMP block, <something else that escapes me now>. Each of these has metadata items intended to describe the photo based on the ideas of ‘Title’, ‘Description’ and ‘Comment’, but with different names and slightly different meaning. Different tools can read some subset of these; they can write a smaller subset of those they can read. They use the descriptive text instead of file name.

I think the idea of ‘Title’, ‘Description’ and ‘Comment’ is valid

• ‘Title’ is an arbitrary name for the picture
• ‘Caption’
• ‘Subject’
• ‘Description’ is a text description of the content of the picture
• ‘Comment’

I now need a tool that can

• read tags from any of the metadata descriptive fields
• synchronise the content of equivalent tags from different standards, by offering to merge them, or selecting one and overwriting the others
• write the new content to my chosen subset of fields

Base Acro recently contracted a professional photographer to record what we have been up to. These are some of our favourites.

It is very different posing for photos compared with training, or dancing socially, or dancing competitively, or including aerials and lifts into your competitive dance. We could see how we looked, adjust things and try again. But it is hard to be satisfied once you start criticising yourself, and you get tired trying to hold a pose that normally would only last a few seconds.

Ansible seems like a useful tool to manage remote servers. Some things it might be useful for

• regular OS updates

    sudo apt update
    sudo apt upgrade
    sudo apt autoremove
    sudo reboot now
  

  But there are a few problems to solve

  • Some packages are ‘held back’; install them anyway.
  • Something pops up a warning about ‘Recommending reboot’; ignore, because we do reboot
  • Something pops up a warning about ‘Restaring services’; ignore, because we do a reboot.
  • Sometimes, a configuration file has changed (by hand or by a script), and needs merging with a new version from the upgrade. There is no automatic way to handle this. In future, use the common ‘conf.d’ structure to override defaults, rather than editing the main configuration file.

• ssh host- and user-certificate management, including

  • installing CA host-key signing certificates and CA user-key signing certificates

  • installing configuration snippets to use the certificates

  • regular certificate updates to ensure that they remain valid

    It is fairly easy to write ansible playbooks to do this. On the other hand, it is also relatively easy to write ansible playbooks that retrieve all authorized_key files from each host and scan them for keys they should be remove.

• Remove firefox snap

    sudo snap remove --purge firefox
  

• Add PPA from mozilla team

    sudo add-apt-repository ppa:mozillateam/ppa
  

• Ensure that the PPA is higher priority than the snap. Create the file /etc/apt/preferences.d/99mozillateampp with the following content

    Package: firefox*
    Pin: release o=LP-PPA-mozillateam
    Pin-Priority: 1001
  
    Package: firefox*
    Pin: release o=Ubuntu
    Pin-Priority: -1
  
  The first stanza marks the mozillateam PPA release as a very high priority; the second
  stanza marks the Ubuntu release as a very low priority.
  

• Install the .deb version of firefox using apt

    sudo apt update
    sudo apt install firefox
  
  Note: apt will warn that you are downgrading firefox. Ignore this, and go ahead with the 
  installation. The warning happens because Ubuntu snap has a version number that includes 
  an 'epoch' field, so that the snap version number is always greater than the .deb version number. 
  This warning only happens when you first install the .deb version.
  

Using ssh keys to login to remote servers provides a huge improvement in security over password login [1]. However, it creates a new problem – managing those keys. This becomes a real problem when a developer has installed a public key on a number of remote hosts, then leaves company. There is no easy way to find out which servers have that public key in any ‘authorized_keys’ file, so that it can be removed to revoke access.

OpenSSH certificates provide a partial solution to that problem.

A certificate consist of the public part of the key-pair signed by a certificate authority. The certificate is made available next to the ssh private key; the ssh client presents the certificate to the remote host instead of the public key; the remote host verifies the certificate against the certificate authority public key, then provides access.

The certificate can be restricted in certain ways: it can prevent logging in (used for tunneling only); can force the connection to run a fixed command; can restrict the validity of the certificate to a range of dates; and so on.

This ability to automatically expire the certificate means that a developer who leaves the company automatically has access revoked after a certain period. However, it does mean that the certificates have to be renewed periodically.

The problem has changed from one of revoking access in an exceptional case (someone leaves the company) to one of ensuring continued access over a period of time (every 6 months or year when the certificates expire).

It is possible to set up a remote-management task (using something like Ansible) to update certificates automatically. However, it would be just as easy to set up a remote-management task to remove no-longer-authorized keys from any authorized_keys files when it becomes necessary.