the corner office : tech blog

a tech blog, by Colin Pretorius

Sticky bits in octal

I learned an interesting thing today.

You can set sticky bits (in my case, setgid) using octal notation in chmod, but you can't unset sticky bits using octal, because it would have broken backwards compatibility.

For that, you need to use symbolic notation, eg. chmod g-s.

More

{2020.10.31 10:03}

Note Taking Apps

I've been looking at note-taking apps recently - this captures some of my research.

Motivation

A proper note-taking app appeals to me as an improvement to my current system of directories full of .txt files, and various subject-specific local blogs/sites built using my blog generation software. It "works" in many respects but is by no means optimal.

I have a vague notion that I'd like to end up with something like a collection-of-everything a la Evernote or OneNote, or a Zettelkasten-style repository for studies and research (and I know these aren't quite the same things, and I've not thought through how they might fit together, if indeed they should).

Things I want

Features which matter to me:

  1. Local/secure. No cloud.

  2. Open formats. No lock-in, and it being relatively easy to switch to other software in future. This means a format that's transportable (eg. markdown), and storage that's accesiible. I'm not averse to a bit of scripting, so I'm comfortable with file-system storage, or a relatively sane sqlite database structure.

  3. Version control, or a revision history. These days, if I can put something into git, I do.

  4. Ergonomics. A catch-all for "fast and easy to use, and not more painful than working in a typical text editor."

  5. Linking, structure, searching, tagging. The kinds of features which are clunky, difficult or impossible with disconnected text files.

While researching these I discovered various features which I haven't listed here (like network graphs, backlinks etc) which weren't on my original list of essentials, but once aware of them, seemed quite useful.

Worth mentioning that these apps have features I'm less interested in and don't mention/consider:

  1. cloud back-up

  2. multi-platform / mobile support

  3. attachment support (as in, features/usability for storing and working with attachments)

Apps

Joplin

Had I not come across a mention of Zettlr and the Zettelkasten method, I may have started using Joplin and remained unaware of the other apps: I'd seen it recommended when researching something else, and was all but ready to start using it.

Joplin is intended to be an open-source Evernote (which is mostly true, but not entirely, since it doesn't have things like pdf and OCR search).

GUI: Electron app, standard stuff with notebook/subnotebook/document structure on the left and a (configurable) split pane document editor viewer: raw markdown on the left and rendered on the right. It's the only app which has this and I really like the feature. Each document has an auto-generated guid as a unique id, as well as its title. Linking isn't via [[]] tags and is a bit clunky since you have to link the ids. Backlinks and tabbed editing have been on the feature request list for some time. The app keeps a revision history.

Storage: a sqlite db. However, Joplin supports automated, external sync with various providers (eg. Dropbox) as well as WebDAV, and a local file system. This means I was easily able to set it up to sync all content to markdown files in a directory. This gives you a directory full of documents with guids for filenames, but it's feasible to push them to git. Documents are markdown as well as some additional metadata (guids, parent guids, modification types, note types, etc). Worth mentioning that exported/synced notes can be encrypted so you store secure content in a DropBox folder, say.

Pros:

  • Split pane editing
  • Revision history
  • Sync to file system (or providers).

Cons:

  • no tabbed editing (just next/previous navigation through notes)
  • no [[]] internal link support, slightly clunky
  • no backlinks (or link graph)

Zettlr

As I mentioned above, I saw Zettlr mentioned when I was looking up something related to Joplin. The name suggests it's intended for use as a Zettelkasten but can also be used for simple markdown editing, and uniquely, is geared towards academic research with support for Pandoc, citations and bibliographies, etc. As best I understand it, the idea is you can do your research in Zettlr, and then easily convert your work into LaTeX or pdf articles.

GUI: the usual trees-on-left (but customisable to show documents, directories, etc) and a main editor which supports tabs. The editor is WYSIWYM with some markdown rendering, which is configurable. It supports themes but modifying CSS looks to be difficult. Internal linking is done by [[]] using ids, but name matching sort-of works. Search also encompasses tags and provides backlinks (of a sort). Documents support YAML front-matter (which is used for pandoc), but it's also how you specify a title for documents independent of the file name.

Other interesting features are a focus mode and a built-in pomodoro timer, as well as document structure navigation and readability metrics.

Annoyingly, editing a filename won't do wiki-style updates to incoming links.

Storage: on-disk markdown files. A result of its Zettelkasten association is that new documents get a timestamp-based ID by default, but you can name documents anything you like.

Pros:

  • flat files on disk
  • citation/bibliography/pandoc support

Cons:

  • search looks powerful but global search is relative to a chosen subdirectory, which could be a nuisance.
  • no crtl-tab or forward/backward navigation
  • no link map (but it's slated for release in a future version)
  • linking is intended to be id-based, and names in links will break if you rename files

Trilium

Trilium is an Electron app and was recommended to me by a colleague. It's the most powerful of the apps I looked at, with scripting support, multiple note types, attributes and tagging. The desktop app works as a standalone but can also sync with a self-hosted server and web app.

The GUI has the standard tree view on the left, and metadata and links on the right. It supports multi-tabbed editing and [[]] style linking. The editor is pure WYSIWIG with markdown commands immediately turned into rich text. The editor tracks a periodic revision history. Changing from the default themes required getting my hands dirty with custom css (but the mechanism for doing this is pretty powerful).

Storage: content is stored as HTML in sqlite. The FAQ gives a rationale for why Trilium doesn't use flat files including the ability to clone documents and showing them at different places in the tree. Fair enough. There's no default syncing but it's possible to manually export content as a zip file of html (or markdown) documents, so theoretically could be scripted to back up content into git. One downside is that I ran into CRC errors with the exported zip file. I'm sure it'll get fixed but is a reminder of why I value flat files + git.

Pros:

  • scriptability (also possibly a con)
  • version control (periodic diff-storing)

Cons:

  • raw format is HTML.
  • exporting is a manual step (and at the time of researching, generated a broken zip file when trying to export the full tree).

Obsidian

Obsidian is a new (post-lockdown) app by the developers of Dynalist (which I've seen mentioned various times, but never used), and still in beta (I tested v0.7.4). It's closed source but free for non-commercial use, and its data format is entirely accessible so it's zero-risk as far as lock-in is concerned.

GUI: the most interesting of them all. Editing is markdown WYSIWYG. It doesn't support multiple tabs but does support splitting and re-splitting the screen into panes, so that you can lay things out according to taste, and see multiple documents at once. Potentially intriguing, though tabs would be nice too. I've not played with it much but it does seem to have various shortcuts and tweaks, and is generally quite impressive. The network links and listed backlinks are also useful for Zettelkasten notes.

Storage: just markdown files in a directory so could (in theory) be used alongside another app like Zettlr.

Pros:

  • link graph and backlinks
  • polished/feature-rich editor
  • opens up over a directory of flat files
  • option to auto-correct incoming links if you rename files

Cons:

  • closed source
  • no multi-tabbed editing

VimWiki

VimWiki is, naturally, not another Electron app, it's a Vim plugin which allows you to edit your wiki in Vim.

I won't go into too many details, other than to say that it supports markdown and [[]] links, stores flat files on disk, and I'd probably consider it to be an excellent low-tech tool if I was happy to live purely in a console.

{2020.07.03 19:11}

Chromium on snap

Ubuntu is moving Chromium to be snap-only.

I'm not sure how strongly I feel about this (yet, since it hasn't hit 18.04 yet), but I'm partial to a not-snap version (there are various gripes/grumbles online about snap and what it means for Chromium use).

Useful pages:

{2020.06.03 16:32}

Ubuntu 19.10 VirtualBox VM Hanging on GUI Startup

tl;dr: upgrading an Ubuntu MATE VirtualBox VM to 19.10 caused it to hang on X startup. Increasing the VM video memory from 16MB to 32MB fixed the problem.

I have an Ubuntu MATE virtual machine which I use to test things before taking the plunge with my main system. (It was my experimental Linux desktop before I switched back to Linux on my PC).

I upgraded it to MATE 19.10, and after the first reboot, it went through the usual system startup, got to the point where it showed a text login, and then briefly flickered and ended up with a black screen and a non-blinking-cursor. I was able to ssh into the machine and reboot it, so it appeared to be a problem with X startup only.

A search came across VirtualBox, 3D acceleration & black screen - Solution. The page pointed to the VirtualBox Display settings tab for the VM, and mentioned disabling 3D acceleration as a possible cause. In my case the problem wasn't 3D graphics (that was already disabled), but after some experimenting with display settings, I discovered that it was because I no longer had enough video memory allocated to the VM.

Which is to say, it had 16MB of video memory before, and increasing it to 32 MB fixed the hang.

My VMs have had 16 MB by default for as long as I can remember. I haven't looked at what's changed to cause Ubuntu 19.10 to need more video memory, or experimented to see what the minimum requirement now is, but at least I could get on with my day.

{2019.12.23 05:49}

Swap

Not read all of this yet, but came across it while reading about Linux disk encryption (and a counter to a lot of "just disable swap or set swappiness to 0" advice)

Chris Downe - In defence of swap: common misconceptions

{2019.07.07 12:16}

zerofree

Links - zeroing out unused blocks (useful for backing up and compressing virtual drives)

{2019.06.28 21:29}

HTTPS

The blog's moved over to colinpretorius.org. I wrote about it on the main blog, but one thing I didn't mention was that the blog's also served via https.

I was initially unhappy about the idea of switching to https. It went against the idea of people flinging up web pages 20 years ago and them still being served up without much cost or intervention ever since.

I changed my mind when reading the EFF's Encrypting the Web. It'd be nice if we could just serve up http and be done with it, but content injection from ISPs and the like, are enough reason to say farewell to the old world.

So, I've switched to https using Let's Encrypt and hoping nothing breaks. And also hoping the renewal cron job kicks in later this year...

{2019.06.16 16:47}

Redirects

Wow. I'm doing the final bits of prep before switching over to the new blog location. One of the things on my list was "redirects". As in, check the apache docs for how to do them and be ready to switch over the server config. I knew I'd set it up before but was quite surprised when I went to check my apache config on the web server.

It had this:

# virtual host config for thecorneroffice.org
# 2006.03.22 CP

<VirtualHost *>

... and then a ton of redirects for all the various flavours of urls from the older Domino versions of the blog. And server aliasing and a lot more.

That config's been copied over from server to server, hosting provider to hosting provider, and there it is, 13 years later. I have no memory of putting together all that config, no recollection that I'd once known how to do it.

I wonder how many of those old redirects still get triggered. Just looking at access logs today, I saw some web crawler requests for site urls which disappeared 4 years ago. Nothing would surprise me.

{2019.06.14 21:50}

Linux

I was going to post about something else and then realised that while approximately 0% of the world's population won't care, the tiny fraction who do (me) would be quite surprised to see me posting on my tech blog about something else, where implicit in said topic would be the fact that I'm using Linux on my desktop at home. Noone likes a continuity error ruining the story.

When I bought my PC I installed Linux (XFCE), then soon after I switched to Windows. And then at some point late last year, I decided that I really didn't want to be running Windows, that I had a hankering for something a little more interesting (eg), a little less phone-homey, a little less annoying in various subtle ways.

And so I switched to Ubuntu MATE. Having switched away from Linux once before, I was rather cautious and methodical about it. I set up a VM, and over a few weeks, migrated all aspects of my computing life to the VM. I promised myself that if I was still happily using the VM in a month, I'd switch properly. A month passed, and I was still using the VM, so I took the plunge, wiped the hard drives and switched.

That was 6 months ago, and I have no urge to return to Windows.

Good things:

  • switching was a breeze. I copied across all dotfiles from my VM, found the right incantation to get MATE to pick 'em up on the new machine, and everything was set up exactly how I wanted it.
  • I couldn't say why (18.04 versus earlier 17.x versions?), but the issues I'd had with app performance and bluetooth and the like have been largely nonexistent. Also, I've enjoyed using MATE. I wouldn't say the experience is hugely different to using XFCE, but it looks good and I'm happy with it.
  • even with the arrival of WSL, and a pretty well-tweaked Cygwin set-up, it's just so much nicer doing unix-like things on a real unix-like system. Suddenly my headaches with borg backup and random git oddities and personal scripts for doing things just went away.
  • I can actually do what I'd spoken about doing when I got this machine and installed Linux on it - allow my son to be doing his thing on the PC, with me logged in remotely from a laptop upstairs and doing my thing at the same.

Niggles and things I miss:

  • pulseaudio (the sound daemon) is a bit rubbish. I regularly need to restart/killall the processes, especially if anyone's switched users on the machine. Also, weirdly enough, just opening windows in Chromium vs Firefox (I tend to have both open all the time, for different things), can cause sound to go fuzzy.
  • there are some Windows apps I miss. Notepad++ followed me via wine (there's a snap version, but that stopped working for me at some point, and I've stuck with a vanilla wine version). Also, sad to say, apps like TortoiseGit (as much as people love to hate it, there are some use cases like rebasing and cherry-picking where it's still the best tool for the job) and WinMerge don't really have decent Linux equivalents.

... and now I've forgotten whatever I was originally planning to write about.

{2019.05.19 14:43}

.Net 5

Microsoft has announced .Net 5.

I've been using C# and F# at work for years now, but it's taken some time for me to start favouring .Net at home. These days though it's solid on Linux, the tooling is decent, and as much as I prefer Kotlin over Java for my Java-based projects, I prefer F# (or falling back to C#) over Kotlin for new projects and tinkerings.

I was reading old posts on Miguel de Icaza's blog the other day and remembered how much the open source world hated .Net in the early 00's, and how distrustful everyone was of Microsoft. At the time that wasn't an irrational position. In the end Mono didn't end up being some evil destroy-Linux trojan, and these days .Net Core is open-sourced, everything happens on github, and nobody worries about evil Microsoft planning to shake everyone down eventually (cf. Java). It's ironic how much Microsoft aren't the bad guys anymore.

{2019.05.06 17:51}

« Older