running an external binary on NixOS
NOTE
TLDR: I installed
nix-ld
, so most binaries should work without any problem. If not, check below on how to configure it to make it work for your binary.Otherwise, there is also
nix-alien
andsteam-run
alternatives.
HA! Welcome to the dark side of NixOS. You want to run a binary you downloaded on the internet on NixOS? Nope, you can’t!
Well, it’s logical as NixOS philosophy is to have reproducible and immutable environments. So it’s logical some programs are not in the same place as the other Linux distribution.
Precompiled binaries that were not created for NixOS usually have a
so-called link-loader hardcoded into them. On Linux/x86_64 this is for example
/lib64/ld-linux-x86-64.so.2.
for glibc. NixOS, on the other hand,
usually has its dynamic linker in the glibc package in the Nix store and
therefore cannot run these binaries.
NixOS is not FHS compliant
There is something called the Filesystem Hierarchy Standard
(FHS) which is a reference describing the conventions used for the layout of
Unix-like systems, e.g. have the essential command binaries under /bin
, …
NixOS is NOT FHS compliant, and some programs you downloaded on the internet
will try to access hard-coded FHS file path like /usr/lib
or /opt
.
Moreover, most programs are using a hard-coded Executable and Linkable Format (ELF) path to be executed.
This format is a common standard file format for executable files, object code, shared libraries, and core dumps.
Generally, we write most programs in high-level languages such as C or C++. These programs cannot be directly executed on the CPU because the CPU doesn’t understand these instructions. Instead, we use a compiler that compiles the high-level language into object code. Using a linker, we also link the object code with shared libraries to get a binary file.
As a result, the binary file has instructions that the CPU can understand and execute. The binary file can adopt any format that defines the structure it should follow. However, the most common of these structures is the ELF format.
So you may encounter some error like this:
[ERROR][2024-06-13 14:07:39] .../vim/lsp/rpc.lua:734 "rpc" "/home/l-lin/.local/share/nvim/mason/bin/lua-language-server" "stderr" "Could not start dynamically linked executable: /home/l-lin/.local/share/nvim/mason/packages/lua-language-server/libexec/bin/lua-language-server\nNixOS cannot run dynamically linked executables intended for generic\nlinux environments out of the box. For more information, see:\nhttps://nix.dev/permalink/stub-ld\n"
So what can you do?
As it turns out, there are 10 different methods to run a non-nixos executable on Nixos! :scream:
Here are some methods that worked for me:
Patching ELF manually
# You can manually patch them, and if you're lucky, that's enough:
patchelf --set-interpreter $(patchelf --print-interpreter `which find`) lua-language-server
# You can now execute it like any other binary:
./lua-language-server
If that’s not enough, e.g. there’re some missing libraries:
$ ldd marksman | grep 'not found'
libz.so.1 => not found
libstdc++.so.6 => not found
In that case, you need to:
- find which packages provide those libraries
- you can use
nix-index
to find the packages - or you can use https://pkgs.org
libz.so.1
is provided byzlib
andlibstdc++.so.6
is part of the C/C++ compiler tool chain
- find the path to those packages in your Nix store
- if the package is not present in your Nix store, you will need to install it
$ # First generate the database index of all files in our channel (can be quite slow):
$ nix-index
+ querying available packages
+ generating index: 114066 paths found :: 29905 paths not in binary cache :: 00000 paths in queue
+ wrote index of 70,026,341 bytes
$ # We use the power of nix-locate to find the packages which contain the file:
$ nix-locate --minimal --top-level -w lib/libz.so.1
zlib.out
remarkable-toolchain.out
remarkable2-toolchain.out
libz.out
figma-linux.out
$ nix-locate --minimal --top-level -w lib/libstdc++.so.6
robo3t.out
remarkable-toolchain.out
remarkable2-toolchain.out
libgcc.lib
$ # You can find the package using the following command:
$ nix eval 'nixpkgs#zlib.outPath' --raw
/nix/store/lv6nackqis28gg7l2ic43f6nk52hb39g-zlib-1.3.1
$ nix eval 'nixpkgs#stdenv.cc.cc.lib.outPath' --raw
/nix/store/xvzz97yk73hw03v5dhhz3j47ggwf1yq1-gcc-13.2.0-lib
Now, patch the Rpath of the binary.
# By patching the RPATH, marksman is now aware of the missing
# libraries and works on NixOS
patchelf \
--set-rpath "$(nix eval nixpkgs#zlib.outPath --raw)/lib:$(nix eval nixpkgs#stdenv.cc.cc.lib.outPath --raw)/lib" \
marksman
rpath designates the run-time search path hard-coded in an executable file or library. Dynamic linking loaders use the rpath to find required libraries.
Specifically, it encodes a path to shared libraries into the header of an executable (or another shared library). This rpath header value (so named in the Executable and Linkable Format header standards) may either override or supplement the system default dynamic linking search paths.
As you can see, it’s quite tedious and error-prone. The following options may be better.
Running using nix-alien
nix-alien will help you run unpatched binaries without modifying them by setting the interpreter and linking the dynamic libraries needed.
First add nix-alien
in your home-manager configuration.
It should already be present at nix-alien:
{ inputs, systemSettings, ... }: {
home.packages = with inputs.nix-alien.packages.${systemSettings.system}; [ nix-alien ];
}
Then, you can run it like this:
# It will open an interactive form to choose where the
nix-alien ./marksman
It also have other options. I still did not explore them.
Using nix-ld
nix-ld provides a shim layer for these binaries.
It is installed in the same location where other Linux distributions install their
link loader, ie. /lib64/ld-linux-x86-64.so.2
and then loads the actual link loader
as specified in the environment variable NIX_LD
. In addition, it also accepts a
colon-separated path from library lookup paths in NIX_LD_LIBRARY_PATH
. This
environment variable is rewritten to LD_LIBRARY_PATH
before passing execution to
the actual ld
. This allows you to specify additional libraries that the executable
needs to run.
First add nix-ld
in your NixOS configuration.
It should already be present at nix-ld:
{ pkgs, ... }: {
programs.nix-ld = {
enable = true;
package = nix-ld-rs;
libraries = [
# ...
];
};
}
Now, you will be able to run any binary that only needs to have their interpreter patched! For example, most LSP servers will be able to run!
If not, use nix-index
with nix-locate
to find the package of the missing library:
$ nix-locate --minimal --top-level -w lib/libgobject-2.0.so.0
remarkable-toolchain.out
remarkable2-toolchain.out
glib.out
Then update unpatched-binaries.nix to include the package,
and apply the change with make nixos
.
Using steam-run
steam-run
is a tool in the Nix package repository that provides an environment
mimicking the traditional FHS, primarily intended for running the Steam gaming
client on NixOS. However, it can be used for other use cases (like this one).
First add steam-run
in your NixOS configuration (should already be present):
{ pkgs, ... }: {
# Run commands in the same FHS environment that is used for Steam: https://store.steampowered.com/
environment.systemPackages = with pkgs; [ steam-run ];
}
# Once steam-run is installed system-wide, you can run any program in the FHS environment:
steam-run your-program args
# Example running openfortivpn-webclient binary in current folder:
steam-run ./openfortivpn-webclient
References to run binaries in NixOS
- Packaging/Binaries - NixOS Wiki
- Patching Binaries for NixOS · Rootknecht.net
- Frequently Asked Questions — nix.dev documentation
- Running Downloaded Binaries on NixOS
- Different methods to run a non-nixos executable on Nixos
- How programs get run: ELF binaries
- How to make Mason works on NixOS
Some ways to create a FHS environment: