We currently only supported Variant II which is used by x86-64.
Variant I is used by both AArch64 (when using the traditional
non-TLSDESC model) and RISC-V, although with small differences.
The TLS layout for Variant I is essentially flipped. The static TLS
blocks are after the thread pointer for Variant I, while on Variant II
they are before it.
Some code using ELF TLS already worked on AArch64 and RISC-V even though
we only support Variant II. This is because only the local-exec model
directly uses TLS offsets, other models use relocations or
__tls_get_addr().
These changes are compatible with clang-format 16 and will be mandatory
when we eventually bump clang-format version. So, since there are no
real downsides, let's commit them now.
We currently expect that the relocation type numbers are unique across
all architectures. But RISC-V and x86_64 use the same numbers for
different relocation types (R_X86_64_COPY = R_RISCV_JUMP_SLOT = 5).
So create a generic reloc type enum which maps to the arch-specific
reloc types instead of checking for all arch reloc types individually
everywhere.
This commit un-deprecates DeprecatedString, and repurposes it as a byte
string.
As the null state has already been removed, there are no other
particularly hairy blockers in repurposing this type as a byte string
(what it _really_ is).
This commit is auto-generated:
$ xs=$(ack -l \bDeprecatedString\b\|deprecated_string AK Userland \
Meta Ports Ladybird Tests Kernel)
$ perl -pie 's/\bDeprecatedString\b/ByteString/g;
s/deprecated_string/byte_string/g' $xs
$ clang-format --style=file -i \
$(git diff --name-only | grep \.cpp\|\.h)
$ gn format $(git ls-files '*.gn' '*.gni')
It's possible for a malloc inside load_program_headers() to steal the
reserved memory space we created for the program headers. Remove the
reservation later in the method.
We currently don't call any DT_FINI_ARRAY functions, so change that.
The call to `_fini` in `exit` is unnecessary, as we now call the
function referenced by DT_FINI in `__call_fini_functions`.
MAP_FILE is not in POSIX, and is simply in most LibCs as a "default"
mode. Our own LibC defines it as 0, meaning "no flags". It is also not
defined in some OS's, such as Haiku. Let's be more portable and not use
the unnecessary flag.
This is a prerequisite for upstreaming our LLVM patches, as our current
hack forcing `-ftls-model=initial-exec` in the Clang driver is not
acceptable upstream.
Currently, our kernel-managed TLS implementation limits us to only
having a single block of storage for all thread-local variables that's
initialized at load time. This PR merely implements the dynamic TLS
interface (`__tls_get_addr` and TLSDESC) on top of our static TLS
infrastructure. The current model's limitations still stand:
- a single static TLS block is reserved at load time, `dlopen()`-ing
shared libraries that define thread-local variables might cause us to
run out of space.
- the initial TLS image is not changeable post-load, so `dlopen()`-ing
libraries with non-zero-initialized TLS variables is not supported.
The way we repurpose `ti_module` to mean "offset within static TLS
block" instead of "module index" is not ABI-compliant.
As IFUNC resolvers may call arbitrary functions though the PLT, they can
only be called after the PLT has been populated. This is true of the
`[[gnu::target_clones]]` attribute, which makes a call to
`__cpu_indicator_init`, which is defined in `libgcc_s.so`, through the
PLT.
`do_plt_relocation` and `do_direct_relocation` are given a parameter
that controls whether IFUNCs are immediately resolved. In the first
pass, relocations pointing to IFUNCs are put on a worklist, while all
other relocations are performed. Only after non-IFUNC relocations are
done and the PLT is set up do we deal with these.
No functional changes intended. This is in preparation of a commit that
overhauls how IFUNCs are resolved.
This commit lets us move the implementation of PLT patching from
`DynamicObject` to `DynamicLoader` where all other relocation code
lives. For this, got[2] now stores the loader's address instead of the
object's.
This is the AArch64 equivalent of `R_X86_64_IRELATIVE`, which specifies
a symbol whose address is determined by calling a local IFUNC resolver
function.
This commit adds the used relocation types to elf.h, and handles the
types in DynamicLoader and DynamicObject. No new functionalitty has to
be added, as the same code can be reused between aarch64 and x86_64.
This will make it easier to support both string types at the same time
while we convert code, and tracking down remaining uses.
One big exception is Value::to_string() in LibJS, where the name is
dictated by the ToString AO.
We have a new, improved string type coming up in AK (OOM aware, no null
state), and while it's going to use UTF-8, the name UTF8String is a
mouthful - so let's free up the String name by renaming the existing
class.
Making the old one have an annoying name will hopefully also help with
quick adoption :^)
This automatically fixes an issue where we were accidentally copying
garbage data from beyond the TLS segment as uninitialized data isn't
actually stored inside the image.
Each of these strings would previously rely on StringView's char const*
constructor overload, which would call __builtin_strlen on the string.
Since we now have operator ""sv, we can replace these with much simpler
versions. This opens the door to being able to remove
StringView(char const*).
No functional changes.
Previously we would just tightly pack the different libraries' TLS
segments together, but that is incorrect, as they might require some
kind of minimum alignment for their TLS base address.
We now plumb the required TLS segment alignment down to the TLS block
linear allocator and align the base address down to the appropriate
alignment.
The original heuristic of "a library being in `s_global_objects` means
that it was fully initialized already" doesn't hold up anymore since we
changed the loading order. This was causing us to skip parts of the
initialization of dependency libraries when running dlopen (since it was
the only user of that setting).
Instead, set a flag after we run stage 4 (which is the "run the global
initializers" stage) and check that flag when determining unfinished
dependencies. This entirely replaces the `skip_global_objects` logic.
This keeps us from needlessly allocating storage via `malloc` as part
of the `Vector`s that early, which we might conflict on while reserving
memory for the main executable.
.text sections of objects that contain textrels have to be writable
during the relocation procedure. Because of this, we would segfault if
we tried to execute IFUNC resolvers defined in them. Let's print a
meaningful error message instead.
Additionally, a warning is now printed when we load objects with
textrels, as in the future, additional security mitigations might
interfere with them being loaded.
IFUNC is a GNU extension to the ELF standard that allows a function to
have multiple implementations. A resolver function has to be called at
load time to choose the right one to use. The PLT will contain the entry
to the resolved function, so branching and more indirect jumps can be
avoided at run-time.
This mechanism is usually used when a routine can be made faster using
CPU features that are available in only some models, and a fallback
implementation has to exist for others.
We will use this feature to have two separate memset implementations for
CPUs with and without ERMS (Enhanced REP MOVSB/STOSB) support.
IFUNC resolvers depend on the resolved function's address having been
relocated by the time they are called. This means that relative
relocations have to be done first.
The linker is kind enough to put R_*_RELATIVE before R_*_IRELATIVE in
.rel.dyn, but .relr.dyn contains relative relocations too.
The DT_RELR relocation is a relatively new relocation encoding designed
to achieve space-efficient relative relocations in PIE programs.
The description of the format is available here:
https://groups.google.com/g/generic-abi/c/bX460iggiKg/m/Pi9aSwwABgAJ
It works by using a bitmap to store the offsets which need to be
relocated. Even entries are *address* entries: they contain an address
(relative to the base of the executable) which needs to be relocated.
Subsequent even entries are *bitmap* entries: "1" bits encode offsets
(in word size increments) relative to the last address entry which need
to be relocated.
This is in contrast to the REL/RELA format, where each entry takes up
2/3 machine words. Certain kinds of relocations store useful data in
that space (like the name of the referenced symbol), so not everything
can be encoded in this format. But as position-independent executables
and shared libraries tend to have a lot of relative relocations, a
specialized encoding for them absolutely makes sense.
The authors of the format suggest an overall 5-20% reduction in the file
size of various programs. Due to our extensive use of dynamic linking
and us not stripping debug info, relative relocations don't make up such
a large portion of the binary's size, so the measurements will tend to
skew to the lower side of the spectrum.
The following measurements were made with the x86-64 Clang toolchain:
- The kernel contains 290989 relocations. Enabling RELR decreased its
size from 30 MiB to 23 MiB.
- LibUnicodeData contains 190262 relocations, almost all of them
relative. Its file size changed from 17 MiB to 13 MiB.
- /bin/WebContent contains 1300 relocations, 66% of which are relative
relocations. With RELR, its size changed from 832 KiB to 812 KiB.
This change was inspired by the following blog post:
https://maskray.me/blog/2021-10-31-relative-relocations-and-relr
