dryad is the first and only parallel, 64-bit ELF dynamic linker for GNU/Linux, written from scratch in Rust, and is:
- not parallel
- not ready for production
- a prototype
- doesn't really work
- in a massive state of flux
- parallel might be a) impossible, b) not performant, but it will be interesting to try
all most of these things will disappear in time!
NOTE: I am early adopting the new
rustup tool, and also now use a Makefile + experimental Cargo.
Of course, you will need your typical build tools on a linux system, essentially:
- an internet connection
- an x86-64 GNU/Linux box
Unfortunately, I currently do not support cross compiling at the moment (which is an unusual use case anyway), so you will need an x86-64 GNU/Linux machine, otherwise it will fail.
Once that's settled, you have two options, both fairly easy.
Setup - Easy
./setup.sh, which will download and install the latest rust nightly and musl target into
You can then proceed as normal:
./gen_tests.sh- builds the test binaries (do this once) (will add this as a make target soon)
dryad.so.1and copies it to
test/test- runs the test binary
make run- runs
./dryad.so.1, this should run correctly without segfaulting, please file a bug if it does not.
Setup - Use Nightly Rust, slightly less easier
If you don't want to use rustup, then setting up a build environment will require a little work: you will need a nightly rustc, a
x86_64-unknown-linux-musl target, and the ability to edit a Makefile to change the environment variables.
If the latest
rustc is not installed, just download and use the latest version of rustup script to install the latest nightly with the musl target enabled, i.e. something like:
curl -sSf https://static.rust-lang.org/rustup.sh > rustup.sh chmod +x rustup.sh sudo ./rustup.sh --channel=nightly --with-target=x86_64-unknown-linux-musl
And then edit the appropriate variables in the
Makefile to use
/usr/local as the
$PREFIX instead of
rust, and probably some other things I'm forgetting about.
You should be able to then compile and run as normal.
Compilation and Linking Requirements
Makefile does four things:
- compiles the x86-64 asm stubs which dryad needs (change the
clanghere if you like)
gcc -fPIC -c -o start.o src/arch/x86/asm.s
- compiles dryad into an object file:
rustc --target=x86_64-unknown-linux-musl src/lib.rs -g --emit obj -o dryad.o
- links the asm stubs with dryad and then the rust standard libs, and pthreads and libc and etc., and provides the very important linker flags such as
-soname dryad.so.1, etc.
- copies the resulting binary,
/tmp/dryad.so.1because that's what
PT_INTERPRETERis set to in the test binaries. In the future we'll obviously make this
/usr/lib/dryad.so.1, or wherever the appropriate place for the dynamic linker is (GNU's is called
3 from above is the problem in the cargo pipeline, which is why I still need to manually link. Additionally, rustc doesn't like to compile a musl binary as a shared object.
I believe some of these issues will go away if I transfer the start assembly into inline assembly in Rust source code (thereby potentially eliminating step 1), but the musl issue could be a problem.
The last step, running
test/test (or any of the other test binaries in
test), will output a ton of information and then segfault your machine, or perhaps not run at all, or really do any number of things --- I really can't say, since I've only tested on a single machine so far.
NOTE: if you're on Ubuntu or another linux distro which doesn't place
/usr/lib, you'll need to pass
LD_LIBRARY_PATH=/path/to/libc to your
LD_LIBRARY_PATH=/path/to/libc test/test. Furthermore, if
libc doesn't have symbolic links for the
soname pointing to the actual binary, or the actual binary is installed as the
soname, then it also won't work. We need
ld.so.cache reader and parser for this - feel free to work on it!
dryad is almost capable of interpreting a (simple) binary (like
test/test) which uses
Specifically, this means is that
dryad at a high level does the following:
- relocates itself
- loads and
mmap's all binaries in the flattened dependency list
- relocates every loaded binary (technically, relocates a subset of the most common relocation symbols)
- sets up each binary's GOT with its runtime symbol resolution function (
_dryad_resolve_symbol), and its "rendezvous" data structure
- resolves GNU ifuncs, and if
LD_BIND_NOWis set, prebinds all function symbols.
- passes control to the executable
- (optionally, if
LD_BIND_NOWis not set) lazily binds function calls
There are several major, and many minor tasks that need to be finished to be even remotely "complete". The first and most major one is properly setting up TLS. Currently, it hacks it about by just calling the musl symbol
__init_tls so we don't segfault on
fs:0 accesses and their ilk.
But it really needs to be properly setup, as it's a delicate procedure.
This is easily the least documented part of the entire dynamic linking process I have come across, so work is slow going. Also there are some questions about how this will work exactly, which I'll detail at some other time, or in a blog post.
dryad should be capable of interpreting itself, which you can verify by invoking
./dryad.so.1 (yes, dryad is it's own program interpreter).
1. Documenting a Dynamic Linker
The primary goal of this project is to completely document:
- the dynamic linking process on an GNU/Linux ELF x86-64 system
- an implementation of such a process
The current state of documentation and information on this subject is an embarassment, and I'm continually appalled at the lack of materials, documentation, etc. I've jokingly told people I'm worried what will happen when all the old C programmers die - but I'm not really joking.
Code is not documentation. If it were, then this project would have been easy and finished some time ago.
As such, I hope to thoroughly document the implementation, the process, and maybe even my experiences.
I will be updating this section with more content shortly, please bear with me.
2. Implementing a Dynamic Linker
The current target implementation for dryad is an ELF x86-64 GNU/Linux system.
This is important to note:
- The ELF loader only supports the 64-bit variant
- The asm assumes an x86-64 instruction set
- The linker currently targets Linux, although this need not be set in stone.
I would like to have a working implementation for an ELF x86-64 GNU/Linux target before/if beginning work on other architectures or systems.
That being said, in particular I'm not very interested in porting dryad to work on 32-bit Linux systems, because:
- 32-bit systems are in obsolescence in my opinion
- Will significantly complicate the ELF target in the source code, as cfg flags would be needed depending on what target we want to switch at build time, etc.
- 32-bit ELF dynamic linking is much better documented, and I want to document a 64-bit dynamic linker
Contributions wholeheartedly welcome! Let's build a production dynamic linker in Rust for use in x86-64 GNU/Linux systems (and beyond)! Or not, that's cool too.
If you don't know anything about dynamic linking on x86-64 GNU systems for ELF, that's totally OK, because as far as I can tell, no one really does anymore. Here are some random resources if you're curious:
- The ELF specification
- x86-64 System V Application Binary Interface
- ELF TLS spec
- google's bionic dynamic linker source code
- glibc dynamic linker source code
- musl dynlink.c code
- sco dynamic linking document
- iecc dynamic linking article
- ELF loading tutorial
- Info on the GOT - GOT values
man ld-sofor dynamic linking basics
man dlopenfor runtime dynamic linking basics
man 3 getauxvalfor information on the auxiliary vector passed by the kernel to programs
- I'll also hopefully add a couple articles on some of my _mis_adventures on my essentially defunct blog
Here are some major todos off the top of my head
- MAJOR: properly init dynamic linker's TLS: it's the final countdown.
dlfcn.himplementation and shared object bindings for runtime dynamic loading support
/etc/ld.so.cacheloader and parser
- better documentation
- fix any number of the todos littered across the code
- make unsafe code safer with rust best practices; rust experts definitely needed!
- add profiling configs
- add tests
- actually implement dynamic linking without segfaulting
- x all the things
Be excellent to each other