Using nixpkgs to quickly hack and patch some package source

Home | About | Items

12 September 2024

Today I decided I wanted to run some timings on some internal functions inside of the R source code. Come to think of it, since the code I was looking at is all byte-compiled, I could probably have just redefined it in my session and not gone to all this trouble.

But sometimes you still want to quickly hack some internet code just to see how something works, and in the past I've always gotten stuck with missing system requirements to build the dang thing after applying my hack. Nixpkgs promises to relieve this pain, so I thought to give it a go. Let's use my goal of making a custom version of R as an example.

The basic process couldn't be simpler: you fire up nix and tell it to unpack, configure, build, and install the package you're hacking. Don't be alarmed about the "install" step, in this context it just installs to the current directory, leaving the rest of your system unharmed.

Fetching source

First we tell nix to fetch the package from nixpkgs: 

nix develop github:nixos/nixpkgs#R

This will download the very latest commit from nixpkgs, which may not be what you want. So you can use different references, and as usual with Nix there are at least a dozen different ways to do the same thing. The nix develop manual describes a few of the ways (but not all), but I settled on this: 

nix develop github:nixos/nixpkgs?rev=574d1eac1c200690e27b8eb4e24887f8df7ac27c#R

which pulls nixpkgs from a particular commit hash.

It's important you don't do the simpler: 

nix develop nixpkgs#R

because you might run into this bug. It's possibly related to the flake registry on your system, or something else. If it was simple, the bug would have been fixed by now.

Unpacking

The previous step downloaded the R source and all the system requirements needed to build (a few gigabytes, thank you very much), and they're all in the nix store. It then dropped you into a new shell with the paths all set up, but hasn't touched your current directory.

You probably want to do this in an empty directory, say /tmp/r. 

cd /tmp/r
unpackPhase

Now take a look in your directory. You'll see a shiny new subdirectory containing the R source code.

Patch

Let's cd into the source directory and get ready to build. 

cd R-4.4.1
patchPhase

Configure, build, install

After patching, we can configure and build. First configure, 

configurePhase

and then build. 

buildPhase

Check (optional)

This step is optional if you're hacking, but you may want to run whatever checks are provided by the package. 

checkPhase

Install

Finally, we can install the artifacts and fix them up. 

installPhase

fixupPhase

This will create an outputs directory in the parent (of the source directory), so you can cd back there and have a look: 

cd ..
ls -l outputs

Edit, compile, repeat

Make some edits, then run the buildPhase, installPhase and fixupPhase again.

Wrapping up

This is just the basics, but it's already much more convenient than trying to follow the typical developers' own documentation about how to build their software. We all know the documentation never quite tells you everything.

But with nixpkgs, the 7,274 contributors as of this date have done the hard work of making it all actually work in a predictable, repeatable way, for over 100,000 packages. Thanks to them, we can build and hack pretty much any piece of open source software out there.

Next, I'd like to write up how to use an overlay to commit my hacks as a patchset to be applied automatically whenever I nix develop or nix-shell to resume work on this project. But that will have to wait a bit.

Before we go, a few minor tips.

Some tips

It's kind of interesting that nixpkgs ultimately creates a bunch of bash script functions and variables. We can take a look at some of them from inside our shell.

See all phases

typeset -F | grep Phase

See code for a single phase

This is really just for interest: 

typeset -f patchPhase

See the patches (if any) that would be applied

echo $patches

Contact: @mocom@mastodon.social. Published using C-c C-e P p. If you use this content commercially, kindly make an appropriate donation to Zig Software Foundation in my name.