Migrating to NixOS

I’ve been running Fedora for the past 2 years, and while I think that of the mainstream distros, it is the most beginner-friendly and stable system to run, I have been running into some pains while using it. I realized that I have been suffering from configuration drift, which is basically when you fix some issue by configuring it so in some obscure, easy-to-forget configuration file in /etc/ or even some config file in ~/.config/, and when you reinstall your system, you have to do this all over again.

I’ve also been experimenting a lot with system defaults, and when I get stuff wrong it can also leave my system in a broken state for a while which is not ideal for situations when I need stuff to work and don’t have time to debug.

To solve this, I have migrated to NixOS.

What is NixOS?

NixOS is a Linux distro with a very, very unique philosophy: having a purely functional language (called Nix) be the lanugage in which you write all about your Linux system: your hardware (ie, what kernel modules and options you want), your software (everything from the bootloader to even the plugins inside your browser), and all their configuration options. You can:

• Write systemd services

• Add allowed SSH keys

• Create build instructions for software

• Erase your entire system and regenerate your system from scratch on boot

• And more!

NixOS seemed like an ideal way to manage my configs, both user and system-wide.

Pre-Install

First, I downloaded NixOS and tried to create an automated installer that would automatically partition my system and install NixOS with a custom configuration.nix that I ended up workshopping for a couple of days. (This file, by the way, basically houses all the system-wide configurations that NixOS uses to build the system, like what bootloader you want, what drives and partitions you have, the users you want premade on the system, user packages you want installed for that user, system services you want running, and so on and so forth)

The customizations I’ve done include:

• Use dnscrypt-proxy2 instead of systemd-resolved for DNS resolving. This way, I can use DNS over HTTPS instead of DNS over TLS, which is less likely to be blocked and allows me to make DNS requests in a secure fashion in more situations.

By default my config uses Cloudflare’s 1.1.1.1 DNS resolver, which will most likely not be blocked by good networks and is private enough for my uses.

• Use systemd-boot instead of GRUB for bootloader. This is just to try out something new, and since I hardly ever mess with bootloaders, I don’t really care either way for this preference.

• Download various command line utilities and graphical applications which I use. This includes compilers, LSPs, browsers, music players etc. In this way, I can keep track of everything on my system at all times and not have to resort to parse DNF transaction history every time I wish to do a clean install. It also allows me to delete any packages that I feel are a waste of space easily.

This is the section of the config that is most in flux, since I didn’t get all the packages that I use written down in one go. So, I’ve been adding to the list ever since depending on my need.

• Use zstd compression on BTRFS partition for easy space savings. zstd feels like it breaks the laws of physics sometimes, it achieves very good compression rates but takes negligible CPU cycles to decompress when the time comes to access the file.

I did debate using ZFS since NixOS’s unique nature would make it one of the easiest distros to use ZFS, but ultimately went with BTRFS since I felt there wasn’t really too much difference either way for a single disk, no RAID system and I already had an external hard drive formatted with BTRFS.

• PipeWire and Wireplumber for audio.

• Syncthing for easy file sync between devices and as a kind of backup for irreplaceable files.

Note: NixOS is all about reproducibility. However, that traditionally only extended to packages, not their versions etc. If you use my configuration.nix at two different times, you will most likely get different builds because the packages’ versions will be slightly different. Flakes allow you to “pin” the version numbers in a way, enabling true reproducibility. They’re also used in other utilities, like home-manager, which we’ll get to shortly. I don’t have my config migrated to flakes right now, but I will soon!

In the meantime, I also used rsync to backup my home folder to an external hard drive in order to keep my user configs (or at least, the ones I cared about) and data. (I tried using btrfs send/receive but I was running into some problems and I already had an rsync-based backup script)

The installer was jointly created by ChatGPT and tweaks from looking at blogs on NixOS, as well as the small customizations I made myself. However, it would never work. (My best guess is that I was using the wrong mountpoint for my EFI partition, but I was on a deadline to install NixOS due to other committments so I never tested this)

I wound up using the default installer that came with the KDE image of NixOS and pasting my configuration in the now live system. This time it worked, although I did have to workshop a couple things as well, including shifting to NixOS unstable by changing channels.

Home-Manager

Now that we have packages setup in a reproducible way, let’s do the same for config files. This was much harder than I expected.

First, we setup Home-Manager, which is a tool to manage the home folder configs. It allows you to specify user packages, write config files directly in a .nix file, and also point to read-only versions of configs for certain programs (like, say, Git).

Doing so is very easy. For example,

home.file = {
    ".config/gh" = {
      source = /home/gotlou/dotfiles/gh;
      recursive = true;
    };
}

tells home-manager to create a read-only symlink at ~/.config/gh to ~/dotfiles/gh and have it do the same recursively (ie, for every subfolder and file within ~/dotfiles/gh).

I used Impermanence’s list of files as well as casually glancing through my old ~/ for config files that seemed important, and added them to home-manager sequentially.

Syncthing

There was a small hiccup with Syncthing that bears mention. I just copied over the Syncthing config, which includes the database and the private key used for identification and authentication, but syncthing wouldn’t start.

In order to get it to work, you need to force Syncthing to rescan the files using the command syncthing --reset-database. Don’t worry, this won’t lose any data, but it will just “clean up” Syncthing a bit and it should work now.

Future work

Okay, so my time ran out to install NixOS. I had to get everything running under a busy weekend with other places to be and things to do, and this was the most I could do. Pretty good, all things considered.

I plan to get read-write permissions on the dotfiles using Impermanence (mentioned above) and Erase Your Darlings, which is just the nickname I give to erasing your entire system, save for a couple of folders, and rebuilding system at boot. I could also mount / on tmpfs, I expect it won’t eat much RAM since it is ultimately a bunch of text and some symlinks to stuff stored on the SSD, but I’ll have to investigate this a bit further

In this way, I will end up with a system whose entire configuration is defined in the contents stored in one single folder, my home folder will basically just contain personal files and dotfiles won’t really be stored in there, and one or two config files will contain the details about my system (services, users, packages etc)

Conclusion

I really really like NixOS, even if I haven’t set it up yet completely.

• It feels faster than my old Fedora install. Boot times are lower and program startup speed is higher too. Maybe this is “new computer smell” at work, but if I implement Erase Your Darlings I can pretty much ensure this

• The fact that I can boot into an older generation if nothing works on the latest update gives me immense peace of mind. The declarative nature of the config file means I can very easily experiment with new tech stacks on my system (like swapping out DNS resolvers) without really having to be in the following situation:

“OK, did I disable that old service? Does NetworkManager know about it? Is it still being run in memory? Why is my machine not booting now?”

• nix-shell allows you to specify libraries that you wish to use in your current shell. Basically, it is Python virtual environment but done better and for the entire shell. I use it to manage libraries needed by various Rust projects for example.

• nix-shell -p package allows you to get package temporarily installed in that shell only so you can try it out and use it without altering your configuration.nix. The required files are cached in /nix/store, so you can later use them, but they are not made available to general usage. For that, use nix-env -iA package, although this defeats the purpose of NixOS since this change is not recorded in your configuration.nix or any similar files.

• Stuff “just works” right now on my machine. If I boot into a certain generation, I can be guaranteed that that generation will perform the same as it did yesterday, so if an update borks something, I can just boot into an older generation as a workaround and wait for a fix or fix it myself depending on the issue.

I recommend playing with NixOS in a VM and building your config there, then installing it on real hardware and see the magic happen before your eyes: the same machine as in the VM, made using one or two files. Goodbye, tedious Linux install and setup time!

If you want to view my configuration, you can check out the repo and a mirror on GitHub. I tried to lay it out very very easily, and there is even a script which automates getting the system config into the stock KDE system you get after using the KDE GUI installer.

Thanks to Manmeet Singh and Priyanshu Tripathi for also doing the switch with me! They helped me set up everything, get started with NixOS and helped avoid any pain points they had thanks to their experience in setting it up as well.

This website was made using Markdown, Pandoc, and a custom program to automatically add headers and footers (including this one) to any document that’s published here.

Copyright © 2023 Saksham Mittal. All rights reserved. Unless otherwise stated, all content on this website is licensed under the CC BY-SA 4.0 International License