Rusty Nu POSIX

A slight delay to this post, but happily I now have no more lecturing to do until 2027¹ :)

I switched a year or two ago to using NixOS as my daily driver following about 15 years as an increasingly irritated MacOS user. Shortly before I had become interested in Rust as a systems programming language that seemed to marry several things I like about OCaml with several other desirable things from C and Python.

I then more recently observed something that I thought was interesting: there seem to be a lot of recent replacements of what were once completely standard and changeless POSIX utilities. I’m thinking things like grep, find, ls and the like that I’ve been using uninterrupted, other than the occasional quibble over whether it was the original version or the GNU version, for about 30 years. Indeed, I have already raved (slightly) about just as a make replacement and its use with OCaml and LaTeX.

NixOS’ declarative configuration meant that I could actually see the list growing, all in one place – I suspect on other systems I wouldn’t have noticed in quite the same way because it would’ve been a much more incremental and diffuse process of change without a clear record of the choices made.

I thus find in my cli.nix config that describes the CLI tools I expect, to have the following collection:

      nu_posix = [
        bat # better cat
        bottom # btm ~ better top, htop, etc
        broot # interactive directory navigation
        chafa # terminal graphics viewer
        ctpv # terminal file previewer
        cyme # better `lsusb`
        delta # better syntax highlighting diff
        dua # disk usage, interactively
        eza # improved `ls`
        fd # `find` replacement
        fend # better CLI calculator
        hexyl # hex pretty printer
        htop # graphical top
        iotop # io top
        jujutsu # better git
        just # updated gnumake replacement
        procs # better ps
        ripgrep # rg ~ `grep` replacement
        sudo-rs # memory-safe `sudo`
        uutils-coreutils-noprefix # replaces GNU `coreutils`
        viddy # better watch
      ];

I think that most, if not all, of these are written in Rust: that particular language community seems to have a real enthusiasm for re-implementing long-standing tools but better, and I have to say I really appreciate it! When I say “better” I’m not particularly thinking of esoteric language features or development ideologies either. I mean better in two very particular senses:

1. Usability. Many of the older tools simply did not have great user interfaces and, when they were ok, they were not built using modern tooling. As a result getting documentation was somewhere between good and great if there was a decent man-page, with a range of potential switches for more short form help or for cases where the man-page was not installed – whether -h, --help, -help, -?, help, or something else. The short-form help would, of course, be formatted in arbitrary ways.

   The modern Rust-y replacements tend to use clap as a reasonably standard command-line parser. As a result, they are remarkably consistent in usage and format, typically producing something that looks a lot like man-page output in response to their -h|--help switch. In a world where man-pages are often an afterthought or, even worse, replaced by info documentation, I find this invaluable. They are also generally inclined to make greater use of modern terminal environments – eza as a replacement for ls is a good example of this.

2. Performance. Old tools were originally built for old computers in old languages (largely C) and, whether this is language ideology or just the practicalities of engineering long-standing widely-used codebases, tended not to be radically updated.

   Rust re-implementations, on the other hand, are from scratch – and Rust’s memory model appears to make it relatively easy for them to be made multi-threaded. On modern hardware this seems to make them startlingly higher performance than the alternatives. Tools I particularly appreciate for this include fd replacing find and ripgrep, rg, replacing grep.

Perhaps the most immediate example of the benefits of this that I’ve experienced is dua via dua i. Traditionally, when trying to clean up an uncomfortably full hard disk I would’ve ended up using some manual iterative application of either du -hS * or possibly something like find ... | xargs du. Or possibly written a Python script to do it for me. And it would’ve taken O(hours) for me to find where the space was being used and to do something about it. And I would’ve found it tedious and deeply irritating.²

In contrast, dua i gives me a TUI interface to navigate the filesystem from wherever I run it, the ability to cumulatively mark files and directories for trashing or immediate deletion, with subdirectory space summaries – and does so across ~850GB / 3 million files in about 10-15 seconds without using any form of caching, database, or other such thing. As far as I can tell, simply by being efficient and multi-threaded.

If this is the future, sign me up. (At least for the bits like this that are good.)