Merge the load_elf() and commit_elf() functions into a single
load_main_executable() function that takes care of both things.
Also split "stage 3" into two separate stages, keeping the lazy
relocations in stage 3, and adding a stage 4 for calling library
initialization functions.
We also make sure to map the main executable before dealing with
any of its dependencies, to ensure that non-PIE executables get
loaded at their desired address.
Instead of having a special case in the dynamic loader where we ignore
TM-related GCC symbols, just stub them out in LibC like we already do
for various other things we don't support.
(...and ASSERT_NOT_REACHED => VERIFY_NOT_REACHED)
Since all of these checks are done in release builds as well,
let's rename them to VERIFY to prevent confusion, as everyone is
used to assertions being compiled out in release.
We can introduce a new ASSERT macro that is specifically for debug
checks, but I'm doing this wholesale conversion first since we've
accumulated thousands of these already, and it's not immediately
obvious which ones are suitable for ASSERT.
When performing a global symbol lookup, we were recomputing the symbol
hashes once for every dynamic object searched. The hash function was
at the very top of a profile (15%) of program startup.
With this change, the hash function is no longer visible among the top
stacks in the profile. :^)
This logging mode was unusable anyway since it spams way too much.
The dynamic loader is in a pretty good place now anyway, so I think
it's okay for us to drop some of the bring-up debug logging. :^)
Also, we have to be careful with dbgln_if(FOO_DEBUG, "{}", foo())
where foo() is something expensive, since it might get evaluated
even if !FOO_DEBUG.
Let's use a stronger type than void* for this since we're talking
specifically about a virtual address and not necessarily a pointer
to something actually in memory (yet).
It was very confusing how these functions used the "undefined" state
of Symbol to signal lookup failure. Let's use Optional<T> to make things
a bit more understandable.
The dynamic loader will now mark RELRO segments read-only after
performing relocations. This is pretty cool!
Note that this only applies to main executables so far,.
RELRO support for shared libraries will require some reorganizing
of the dynamic loader.
For a data segment that starts at a non-zero offset into a 4KB page and
crosses a 4KB page boundary, we were failing to pad the VM allocation,
which would cause the memcpy() to fail.
Make sure we round the segment bases down, and segment ends up, and the
issue goes away.
Using the text segment for the VM reservation ran into trouble when
there was a discrepancy between the p_filesz and p_memsz.
Simplify this mechanism and avoid trouble by making the reservation
as a MAP_PRIVATE | MAP_NORESERVE throwaway mapping instead.
Fixes#5225.
load_from_image() becomes map() and link(). This allows us to map
an object before mapping its dependencies.
This solves an issue where fixed-position executables (like GCC)
would clash with the ASLR placement of their own shared libraries.
Validation was happening in two steps, some in the constructor, and then
some later on, in load_from_image().
This made no sense so just move all the validation to the constructor.
Refactor DynamicLoader construction with a try_create() helper so that
we can call mmap() before making a loader. This way the loader doesn't
need to have an "mmap failed" state.
This patch also takes care of determining the ELF file size in
try_create() instead of expecting callers to provide it.
Previously regions were stored in a vector and then a pointer to
regions in this vector were taken and stored. The problem is the vector
were still appended after pointers were taken, if enough regions were
present the vector would grow so large that it needed a resize, this
cause his memory to moved and now the previous pointers are now
pointing to old memory we just freed.
Fixes#5160
To support upcoming W^X changes in the kernel, the dynamic loader needs
to be careful about the order in which permissions are added to shared
library text segments.
We now start by mapping text segments read-only (no-write, no-exec).
If relocations are needed, we make them writable, and then finally,
for all text segments, we finish by making them read+exec.
Use mmap() with the new MAP_RANDOMIZED flag to load shared libraries at
random addresses in each process.
To avoid address space collisions, we start by doing a large chunk mmap
that covers enough VM for both text and data, then we unmap and remap
the data segment separately, once we know everything will fit.
This is pretty cool! :^)
